Data Engineer

What You’ll Be DoingDesign, develop and maintain data pipelines and data engineering solutions using DatabricksBuild scalable data processing solutions using PySpark and SQLWork across Databricks Lakehouse environments, supporting implementation, optimisation and ongoing improvementDevelop and optimise batch and data processing workflowsContribute to technical solution design and data platform architectureWork closely with technical and business stakeholders to understand requirements and deliver effective solutionsSupport testing, deployment and production implementation activitiesTroubleshoot performance, reliability and data quality issuesFollow and contribute to engineering best practices across Databricks and cloud data platformsSupport clients through different stages of their data platform and lakehouse transformation journeys

What We’re Looking ForStrong commercial experience in Data EngineeringHands-on experience with Databricks and/or Apache SparkStrong PySpark and SQL development skillsExperience designing, building and supporting data pipelinesExperience working with Azure or AWSGood understanding of modern data platforms and Lakehouse architecturesStrong analytical and problem-solving abilitiesExcellent communication skills, with the ability to work effectively with both technical and non-technical stakeholders

Desirable ExperienceExperience with any of the following would be highly beneficialDelta LakeUnity CatalogMLfowDatabricks optimisation and performance tuningAI, Machine Learning or Generative AI initiativesAI-assisted development tools such as Claude, GitHub Copilot or CursorConsultancy, professional services or client-facing delivery experienceActive or previous SC Clearance

The OpportunityA specialist data and AI consultancy is growing its Databricks capability and is looking for a Senior Databricks Consultant to join its delivery team.This is an opportunity to work on modern cloud data platforms and Databricks Lakehouse programmes, helping enterprise clients design, build and optimise scalable data engineering solutions.You’ll work closely with experienced data architects, engineers and client stakeholders, with Databricks at the centre of the role. The position would suit someone who enjoys hands-on engineering while also contributing to technical solution design and client delivery.