Senior Data Engineer - Azure Databricks - Banking

We are looking for an experienced Senior Data Engineer to join a high-performing engineering team responsible for delivering a modern, enterprise-scale Azure data platform. This is an exciting opportunity to work on large-scale data engineering initiatives, building robust, cloud-native solutions that power analytics, AI, machine learning, and business-critical applications.

Working within a collaborative Agile environment, you will design, develop and optimise high-performance data pipelines while helping shape the future of a modern Azure Lakehouse platform.

This role would suit someone who enjoys solving complex engineering challenges, building scalable solutions and working with the latest Microsoft Azure and Databricks technologies., * Design, develop and maintain scalable data engineering solutions using Microsoft Azure and Azure Databricks.

• Build robust ETL/ELT pipelines using Python, PySpark, SQL and Spark technologies.
• Develop batch and streaming data processing applications using Delta Lake and Lakehouse architecture.
• Design and optimise high-performance Spark workloads for scalability, reliability and cost efficiency.
• Develop reusable frameworks, libraries and engineering components to accelerate data delivery.
• Build and maintain CI/CD pipelines using Azure DevOps.
• Implement automated testing, monitoring, logging and data quality controls.
• Work closely with Architecture, Cloud, DevOps and AI teams to deliver enterprise-grade data solutions.
• Mentor junior engineers and contribute to engineering standards, best practices and code quality.

We're looking for someone with strong hands-on experience across most of the following:

• 5+ years of Data Engineering experience
• Azure Databricks
• Python
• PySpark
• SQL
• Apache Spark
• Delta Lake
• Lakehouse Architecture
• Azure Data Factory
• Azure Data Lake Storage
• Azure DevOps
• ETL / ELT development
• CI/CD pipelines
• Performance tuning and Spark optimisation
• Data modelling and transformation
• Version control using Git
• Agile delivery environments

Desirable Experience

Experience with any of the following would be highly beneficial:

• Scala
• Azure Event Hubs
• Streaming data pipelines
• Terraform or Infrastructure as Code
• Databricks Unity Catalog
• Machine Learning or AI data platforms
• Data governance, metadata and lineage
• MLOps
• Microsoft Fabric
• Enterprise-scale cloud migrations

Preferred Certifications

• Microsoft Certified: Azure Data Engineer Associate (DP-203)
• Databricks Certified Data Engineer Associate or Professional
• Azure Developer Associate
• Azure DevOps Engineer Expert

What We're Looking For

You'll be someone who enjoys writing clean, maintainable code, solving challenging technical problems and working collaboratively with engineering teams. You'll have a passion for modern cloud technologies, continuous improvement and delivering high-quality, scalable data solutions that make a real business impact.

Excellent communication skills and the ability to work across technical and business teams are essential.

This is an opportunity to work on a modern Azure data platform using some of the latest technologies in the Microsoft ecosystem. You'll be involved in enterprise-scale engineering, collaborate with highly experienced architects and engineers, and play a key role in delivering innovative data solutions that support analytics, AI and digital transformation initiatives. If you're passionate about Azure, Databricks and building world-class data engineering solutions, we'd love to hear from you.