Data Engineer

Our client is a recruitment consultancy specialising in IT, Finance, HR, and Digital Technology. Our client is seeking an experienced Data Engineer specializing in Databricks and Treasury Data Lake environments to join a fully remote project. The successful candidate will be responsible for designing, developing, maintaining, and optimizing end-to-end data pipelines within a modern Lakehouse architecture, ensuring reliable, scalable, and high-performance data solutions. This role requires strong hands-on experience with Databricks, DBT, advanced SQL, and enterprise data integration, alongside a solid understanding of data governance, automation, and CI/CD practices. RESPONSIBILITIES o Design, build, and maintain end-to-end data ingestion pipelines integrating multiple source systems. o Develop and optimize data transformations across bronze, silver, and gold layers within a Databricks Lakehouse architecture. o Implement automated data quality controls, monitoring frameworks, and alerting mechanisms to ensure data integrity and reliability. o Design and maintain data models and transformation logic using DBT and advanced SQL. o Leverage Databricks Lakehouse Federation and Lakeflow Connect to support efficient data integration and accessibility. o Develop and maintain CI/CD pipelines using GitHub and Azure DevOps, ensuring controlled deployments, testing, and version management. o Identify and resolve performance bottlenecks, improving scalability, maintainability, and efficiency. o Collaborate with business and technical stakeholders to translate requirements into robust data engineering solutions. o Support data governance standards, documentation, troubleshooting, and release management activities. o Contribute to analytics and reporting capabilities by delivering reliable, high-quality data products.

o Proven professional experience as a Data Engineer in enterprise data environments. o Strong hands-on expertise with Databricks and modern Data Lake/Lakehouse architecture. o Practical experience developing data transformations using DBT (Data Build Tool). o Advanced SQL skills, including query optimization and complex data transformations. o Experience designing and managing data pipelines across bronze, silver, and gold layers. o Knowledge of Databricks Lakehouse Federation and Lakeflow Connect. o Experience implementing data quality frameworks, monitoring, and automated alerting. o Familiarity with GitHub, Azure DevOps, and CI/CD processes. o Understanding of data governance, data modeling, and enterprise integration practices. o Knowledge of artificial intelligence-related concepts and their application within modern data ecosystems. o Strong analytical, troubleshooting, documentation, and communication skills. o Ability to work independently and collaborate effectively with distributed teams.