Lead Data Engineer

ARC IT Recruitment
Charing Cross, United Kingdom
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Charing Cross, United Kingdom

Tech stack

Architectural Patterns
Azure
Cloud Computing
Continuous Integration
Data as a Services
Data Architecture
Information Engineering
Data Governance
Data Infrastructure
ETL
Data Systems
Data Warehousing
Database Development
DevOps
Disaster Recovery
Python
Meta-Data Management
Power BI
SQL Databases
Data Ingestion
Azure
Spark
Microsoft Fabric
Data Lake
PySpark
Data Management
Azure
Data Pipelines
Databricks

Job description

Lead Data Engineer is sought by expanding central data platform team within a prestigious global wealth management organisation. This role offers the opportunity to help shape enterprise-wide data engineering standards, contribute to platform strategy, and build modern, scalable data solutions that support business-critical decision-making. Essential to this role are strong python skills, expertise in MS Fabric, SQL, data warehousing, Data Lake and Azure technologies., As the Lead Data Engineer, you will be responsible for designing, developing, and maintaining modern data solutions using Microsoft Fabric and Azure technologies. You will contribute to the development of engineering standards, optimise platform performance, and support the adoption of secure, scalable, and observable data architectures.

Working closely with stakeholders, product owners, analysts, data scientists, and platform specialists, you will play a key role in delivering high-quality data products and promoting best practices across federated data teams., * Define and implement data engineering standards, frameworks, and best practices.

  • Design, build, and maintain secure, scalable, and observable data pipelines.
  • Develop data solutions using Microsoft Fabric, Lakehouse, and Data Warehouse architectures.
  • Implement and manage Medallion Architecture patterns for data ingestion and transformation.
  • Ensure data quality, completeness, integrity, and governance across the platform.
  • Optimise data processing workflows, pipelines, and storage solutions for performance and cost efficiency.
  • Collaborate with Data Scientists, Analysts, Product Owners, and business stakeholders to deliver high-value data products.
  • Support CI/CD implementation and engineering automation practices.
  • Contribute to platform roadmap planning and adoption of emerging Microsoft Fabric capabilities.
  • Design monitoring, disaster recovery, and business continuity strategies for critical data services.
  • Promote effective data governance, metadata management, and catalogue standards.
  • Mentor engineers and provide technical leadership across multiple data initiatives.

Requirements

  • 5 years' experience in Data Engineering within cloud-based environments.
  • Proven expertise with Microsoft Fabric and modern data platform technologies.
  • Advanced Python and SQL development skills.
  • Experience building and optimising ETL/ELT pipelines.
  • Strong understanding of Lakehouse, Data Warehouse, and Data Lake architectures.
  • Experience with Azure Data Factory, Azure Synapse, and Apache Spark.
  • Knowledge of data modelling, semantic models, and enterprise reporting solutions.
  • Experience implementing data governance, security, and quality frameworks.
  • Strong understanding of modern data engineering practices, including observability and monitoring.
  • Experience working with CI/CD pipelines and DevOps practices.
  • Ability to engage effectively with both technical and non-technical stakeholders.

What We're Looking For

The successful candidate will be a proactive and commercially minded data engineering leader who combines strong technical capability with excellent communication skills. You will have experience delivering complex data platform initiatives, influencing stakeholders at all levels, and helping organisations realise the value of modern data technologies.

If you are passionate about building scalable data platforms, driving engineering excellence, and shaping the future of enterprise data capabilities, we would love to hear from you. Skills

  • Azure Data Factory
  • CI/CD (DevOps)
  • Data Warehouse Development & Management
  • Databricks (Data Engineering)
  • Lakehouse (Data Architecture)
  • Microsoft Fabric (Data & Service)
  • Power BI
  • PySpark
  • Python
  • SQL

Apply for this position