Data Operations Engineer
Role details
Job location
Tech stack
Job description
- Design, develop, and maintain scalable data infrastructure to support our BI/MI workloads.
- Manage and optimize data pipelines and ETL/ELT processes.
- Ensure high availability, performance, and reliability of our data platforms and services.
- Conduct code reviews, mentoring others, and enforce best practices in data engineering.
- Diagnose and resolve data quality issues, ensuring accuracy, efficiency, and security.
- Maintain the confidentiality, integrity and availability of LMAX information assets Essential Experience
Requirements
You are comfortable architecting, building and supporting end-to-end solutions that are practical and performant. Throughout your career, you have developed a broad set of knowledge and skills and are comfortable wearing different hats. You have acquired the know-how around the tools and technologies required to build and solve complex issues in analytical and data pipelines. You can collaborate with engineers and key personnel from other departments (Product Managers, Data Scientists, Platform Engineers, and Operational Business teams) to design scalable, efficient, and reliable data solutions that drive LMAX Group forward., * 3+ years in data engineering or backend software development
-
Strong Python skills for data retrieval and pipeline development
-
Infrastructure-as-Code tooling for deployment and automation (Terraform, Ansible, etc.)
-
Working knowledge of Kubernetes for container orchestration
-
Hands-on experience with several of the following:
-
ETL/ELT orchestration and pipeline tooling (Airflow, Meltano, DBT, etc.)
-
Real-time streaming platforms (Kafka, Redpanda, Active/RabbitMQ, etc.)
-
SQL databases, particularly MySQL Desired Experience:
-
Experience with data warehousing tools and platforms (Snowflake, Iceberg, etc.)
-
Familiarity with Java
-
Experience with cloud-based services, particularly AWS
-
Proven ability to manage stakeholders, their expectations and explain complex problems or solutions in a manner suitable for the audience
-
Knowledge of data governance and metadata management
-
Knowledge of data security best practices and compliance requirements Success Looks Like:
-
Safely implement controls to enable self-service of Data for other teams such as Data Scientists and the Business
-
Be a driving force within the team, leveraging automation and tooling
-
A measurable increase of useable data sources fed into reports
-
Demonstrate performant and resilient data infrastructure that can scale with the needs of the Group
-
Receive positive feedback from technology teams and other business stakeholders
Benefits & conditions
- 25 days of holiday
- Bonus
- Pension contribution
- Private medical, dental, and vision coverage
- Life assurance
- Critical illness cover
- Wellness contribution program with access to ClassPass
- Plumm Platform
- Five volunteering days
- Give as You Earn initiative
- Learning and development programs
- Electric Vehicle Scheme
- Cycle to Work Scheme
- Season Ticket Loan