Senior Azure Data Engineer

Stott and May
Coventry, United Kingdom
5 days ago

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Coventry, United Kingdom

Tech stack

Azure
Big Data
Cloud Database
ETL
Data Transformation
Data Migration
Data Systems
JSON
Python
SQL Azure
Cloud Services
Standard Sql
Azure
SQL Stored Procedures
SQL Databases
T-SQL
Data Ingestion
Azure
Spark
Change Data Capture
Data Lake
PySpark
Semi-structured Data
Azure
Data Pipelines

Job description

We are seeking an experienced Senior Azure Data Engineer to support a large-scale data migration programme, designing and delivering robust, scalable and high-performing data solutions within the Microsoft Azure ecosystem. This role will focus on building enterprise-grade data pipelines, migration frameworks and transformation processes using Azure Synapse Analytics, Azure Data Factory (ADF), PySpark, Python and SQL. You will work closely with technical and business stakeholders to ensure successful delivery of data migration and integration initiatives., * Design and deliver enterprise-scale data migration solutions and payload architectures.

  • Build and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF).
  • Develop Change Data Capture (CDC) solutions to support incremental and delta data loads.
  • Create data transformation and cleansing processes using PySpark and Spark DataFrames.
  • Develop Python-based utilities and automation scripts to support migration and validation activities.
  • Design and optimise data models, SQL queries, stored procedures and analytical views within Azure Synapse Analytics.
  • Manage data ingestion, transformation and storage across Azure Data Lake Storage (ADLS) and Azure SQL environments.
  • Support processing of structured and semi-structured data formats, including Delta Lake and JSON.
  • Collaborate with stakeholders to define requirements and deliver high-quality data solutions.
  • Provide technical leadership, mentoring and support to junior team members.

Requirements

  • Strong hands-on experience with Azure Synapse Analytics.
  • Extensive experience building solutions with Azure Data Factory (ADF).
  • Advanced Python and PySpark development skills.
  • Strong SQL and T-SQL scripting experience.
  • Experience designing and implementing large-scale ETL/ELT pipelines.
  • Experience with Change Data Capture (CDC) and data migration frameworks.
  • Knowledge of Azure Data Lake Storage (ADLS) and Azure SQL Database.
  • Experience working with Delta Lake and JSON data formats.
  • Minimum 5 years' experience within cloud-based data engineering environments.
  • Strong stakeholder management, communication and problem-solving skills.

Desirable Skills

  • Microsoft DP-203: Azure Data Engineer Associate certification.
  • Experience leading data migration or transformation projects.
  • Previous mentoring or team leadership experience.
  • Exposure to enterprise-scale cloud data platforms and modern analytics

Apply for this position