Data Engineer (Databricks)

We are seeking an experienced Data Engineer with strong expertise in Databricks, Apache Spark, Delta Lake, Python, and SQL to take a lead role in delivering innovative data projects.

You will design and build scalable, cloud-based data pipelines on platforms such as AWS, Azure, or GCP, and provide technical leadership within delivery squads. Working closely with both clients and internal teams, you'll translate business requirements into powerful, production-grade data solutions., * Design, build, and optimise large-scale data pipelines using Databricks and Spark.

• Implement and maintain Delta Lake architectures and data governance best practices.
• Deliver end-to-end solutions across cloud platforms (AWS, Azure, or GCP).
• Provide technical leadership and mentor junior engineers within project teams.
• Collaborate with business stakeholders to translate requirements into technical designs.
• Apply modern engineering practices including CI/CD and automated testing.

• Proven experience as a Data Engineer working in cloud environments.
• Expert-level knowledge of Databricks, Apache Spark, and Delta Lake.
• Advanced Python and SQL programming skills.
• Strong understanding of CI/CD pipelines, automated testing, and data governance.
• Excellent communication and stakeholder engagement skills.

• Leadership opportunities on high-impact data projects.
• Continuous learning with funded certifications and access to a vibrant Databricks community.
• Clear progression pathways and ongoing professional development.
• 25 days annual leave plus bank holidays.
• Hybrid working model 2-3 days per week in the Central London office.

Omnis Partners is proud to represent a leading data and AI consultancy that is driving transformation across industries. Our client empowers organisations to harness the full potential of their data, reducing costs, mitigating risk, and uncovering new revenue opportunities through cutting-edge solutions and world-class talent. Working with major technology partners including Databricks, Snowflake, and Collibra, they deliver projects that revolutionise how data is used across sectors such as financial services, pharmaceuticals, energy, retail, healthcare, and manufacturing.