Senior Data Engineer/Data Platform Engineer (Databricks)

Designing and building end-to-end data pipelines using Databricks (PySpark/SQL/Delta Lake) Developing and managing robust, scalable Lakehouse architectures Implementing Delta Live Tables (DLT) for reliable, testable, production-grade pipelines

Optimising performance using: Working with Unity Catalog to implement Centralised governance Leveraging Databricks Workflows/Jobs for orchestration Building CI/CD pipelines for data & platform deployments (Terraform, GitHub Actions, Azure DevOps etc.) Enabling Real Time and streaming pipelines (Structured Streaming, Auto Loader) Contributing to MLOps workflows, integrating models into production pipelines Collaborating with Data Scientists, Analysts, and Platform teams to improve developer experience and platform usability

Strong experience building pipelines with PySpark/Spark SQL Deep understanding of Delta Lake Databricks Platform Expertise Hands-on experience with: Unity Catalog (governance & security), Spark Declarative Pipelines, Databricks Workflows/Jobs Strong understanding of Lakehouse architecture principles Performance & Optimisation Platform/DevOps Engineering, Infrastructure as Code, CI/CD pipelines Strong experience in at least one - Azure, AWS

Very Nice to Have:

Databricks certifications (Data Engineer Associate/Professional) Experience with Feature Store/MLflow Exposure to Databricks AI/Mosaic AI/Model Serving