Data Platform Engineer - Cloud

• Support the migration of data pipelines, notebooks, and data processing workloads between cloud environments
• Analyse existing data engineering solutions and identify migration requirements
• Develop and maintain data processing pipelines using modern cloud data platform technologies
• Support testing, validation, and reconciliation of migrated datasets
• Work with structured and semi-structured data stored in cloud environments
• Troubleshoot and resolve data processing issues across development, test, and production environments
• Support the operation and enhancement of cloud-based data platforms
• Assist with onboarding datasets, users, and workloads onto target platforms
• Collaborate with platform and architecture teams to implement scalable data solutions
• Support platform optimisation, monitoring, and operational readiness activities
• Develop and maintain Terraform code supporting data platform deployments
• Support deployment of data platform components, jobs, permissions, and environment configurations
• Assist with automation of deployment and release processes
• Maintain infrastructure consistency across environments
• Support cloud migration initiatives and platform transformation programmes
• Contribute to platform standards, engineering practices, and reusable deployment patterns
• Produce technical documentation and implementation guides

Technologies:

• AWS
• CI/CD
• Cloud
• Databricks
• DevOps
• ETL
• Git
• IAM
• Support
• Python
• PySpark
• SQL
• Terraform
• Security, We are hiring a Senior Data Platform Engineer for a hybrid role based in Leeds or London, working three days a week on a major cloud data modernisation programme. You will help migrate legacy Databricks pipelines and workloads to a modern AWS infrastructure, building automated, large-scale data solutions that secure and transform critical healthcare data assets. The salary is £70,000 to £80,000 plus depending on experience. This role requires SC clearance or eligibility, and candidates must be able to meet UK sponsorship and residency requirements for SC clearance.

• 5+ years of data engineering experience
• Experience delivering cloud-based data platforms
• Experience supporting migration of data workloads between environments
• Experience working with Infrastructure as Code using Terraform
• Experience working within regulated or enterprise-scale environments
• Strong skills in Python, PySpark, SQL, data pipeline development, ETL/ELT design, data modelling fundamentals, and data quality/validation
• Experience with AWS, S3, IAM, and cloud-based data platforms
• Experience with Terraform, infrastructure automation, and Git
• Understanding of CI/CD concepts, Agile delivery, and DevOps principles
• Desirable: Databricks, Palantir Foundry, Foundry Pipeline Builder, ontology concepts, Delta Lake, healthcare or NHS data experience, Secure Data Environment experience, and cloud migration programmes
• SC Clearance or eligibility is mandatory
• UK citizen or have the right to work in the UK and have resided in the UK for at least 5 years