> Markdown version of [/jobs/ext/4533074-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/4533074-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Insight
- **Location:** Halifax, UK
- **Experience:** Expert
- **Salary:** £75,624.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Airflow, Amazon Web Services, Data Analysis, Microsoft Azure, Big Data, BigQuery, Cloud Database, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Data Mart, Data Transformation, Dataspaces, Data Vault Modeling, Data Warehousing, IBM InfoSphere DataStage, Python (Programming Language), Data Mesh, SQL Databases, Data Streaming, Talend, Google Cloud, Informatica Powercenter, Snowflake, Apache Spark, Generative AI, Infrastructure as Code (IaC), Git, Data Lakes, Pyspark, Kubernetes, Data Analytics, Star Schema, Apache Kafka, Data Management, Data Lakehouse, Terraform, Azure Synapse Analytics, Data Pipelines, Docker, Amazon Redshift, Databricks
- **Published:** November 3, 2026
- **Apply:** https://www.adzuna.co.uk/jobs/details/5950861648

## About the Role

Proficient in: SQL, Python, PySpark, Spark

 - Data modeling (Star Schema, Snowflake, Data Vault)
 - ETL/ELT tools: Airflow, DataStage, Informatica, Talend, DBT
 - Cloud data warehouses: BigQuery, Snowflake, Synapse, Redshift
 - Event streaming: Kafka, Pub/Sub, Event Hub
 - DevOps: Git, CI/CD, Terraform, Docker, Kubernetes
 - Cloud platforms: AWS, Azure, GCP

Preferred Skills

 - Experience with Data Lakehouse (Databricks, Delta Lake)
 - Master Data Management (MDM)
 - Data Mesh/Data Product Architecture
 - AI/ML & Generative AI data platforms
 - Domain experience in BFSI, Insurance, Retail, or Healthcare

## Description

Lead and drive data modernization, transformation, governance, and analytics consumption initiatives.

 - Build and manage scalable ETL/ELT pipelines for diverse data types (structured, semi-structured, unstructured).
 - Develop batch and real-time data processing solutions using modern cloud and big data technologies.
 - Design and implement data models, data marts, and analytics-ready datasets for BI, AI, and ML.
 - Lead cloud data engineering projects across AWS, Azure, or GCP.
 - Ensure data quality, governance, lineage, security, and compliance throughout the data ecosystem.
 - Collaborate with business stakeholders, data architects, data scientists, and BI teams to deliver data-driven business value.
 - Modernize legacy data warehouses and support data lakehouse architectures and platform transformations.
 - Establish and enforce engineering best practices, CI/CD pipelines, Infrastructure as Code (IaC), and automation.
 - Mentor and lead data engineering teams; conduct architecture reviews and provide technical governance.
 - Optimize platform performance, scalability, reliability, and manage cloud costs.