DataOps Engineer
Role details
Job location
Tech stack
Job description
Data Engineering & Delivery Design develop and test data engineering solutions aligned to business requirements and quality standards. Build and support end-to-end data pipelines covering ingestion transformation and consumption layers. Apply engineering best practices to ensure scalability reliability and performance of data solutions.
DataOps & Automation Drive automation-first approaches across data pipelines and operational processes reducing manual intervention. Identify opportunities to improve process maturity tooling and operational efficiency. Cloud & Infrastructure Develop and manage cloud-based data platform components using infrastructure-as-code techniques (eg Terraform). Deploy and maintain services within AWS-based data ecosystems (eg S3 Glue Redshift).
CI/CD & Engineering Practices Contribute to CI/CD pipelines ensuring reliable and repeatable deployments. Follow and enhance Git-based version control and release management practices.
Monitoring Support & Troubleshooting Investigate and resolve data pipeline failures performance issues and data quality concerns. Use monitoring and logging tools to identify and address operational issues. Provide support for production systems including occasional out-of-hours support where required.
Governance Assurance & Quality Ensure adherence to engineering standards security controls and approved design principles. Participate in peer reviews testing and assurance activities to maintain solution quality. Identify risks issues and defects and escalate where appropriate.
Collaboration & Stakeholder Engagement Work effectively with cross-functional teams and third-party suppliers to deliver solutions. Communicate clearly with both technical and non-technical stakeholders. Support and mentor team members where required.
Requirements
Strong experience in data engineering development within a commercial environment. Hands-on experience with AWS data services including S3 Glue and Redshift (or comparable cloud technologies). Strong programming capability in SQL Python PySpark or Scala. Proven experience using Terraform for infrastructure as code and Git for version control. Experience with CI/CD pipelines and deployment workflows. Strong understanding of data pipeline design and implementation. Experience in data quality assurance and testing within pipelines. Demonstrated ability to troubleshoot and resolve technical issues in production environments.
Desirable Experience with tools such as Databricks Informatica Qlik Replicate/Compose SAS or SSIS. Knowledge of data modelling methodologies (eg Kimball Data Vault Lakehouse). Experience in financial services data environments. Exposure to Agile delivery methodologies. Experience industrialising or supporting machine learning models., Degree in a relevant discipline or equivalent experience in data engineering or software engineering. Professional certifications in cloud platforms or DevOps practices are advantageous.
Key Capabilities & Behaviours Strong focus on quality governance and engineering best practices. Automation-first mindset with the ability to improve process maturity and efficiency. Ability to articulate technical solutions design decisions and pipeline flows clearly. Proactive approach to continuous improvement and learning. Ability to work collaboratively and influence within a cross-functional delivery environment.