Senior Data Engineer

Join a growing data function within an organisation dedicated to helping people across the UK improve their financial well-being. In this role, you’ll be responsible for designing and leading the implementation of data flows that connect operational systems for analytic, primarily within an Azure environment., * Data Pipelines and ETL Processes: Design, develop, and implement ETL (extract, transform, load) processes and meta-data driven data pipelines using tools like Azure Data Factory and Apache Spark.

• Monitoring and Failure Recovery: Create monitoring procedures to detect failures or unusual data profiles and establish recovery processes to ensure data integrity.
• Azure Services: Use Azure's suite of services, including cloud-based data technology platforms and Data Lake-based storage, to ensure data solutions are scalable and reliable.
• Collaboration and Leadership: Partner with data scientists, analysts, and other stakeholders to deliver solutions that meet their data needs. You will also provide technical leadership and mentorship to junior data engineers.
• Data Governance and Quality: Enforce data governance policies, maintain high standards of data quality and security, and ensure only thoroughly tested, high-quality code and architecture are delivered.

• Proven experience as a Data Engineer, with a focus on Azure services.
• Strong expertise in designing and implementing ETL processes.
• Experience using SQL to query and manipulate data.
• Proficiency in Azure data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Lake Storage/One Lake, and Azure Data Factory.
• Experience with monitoring and failure recovery in data pipelines.
• Understanding of data extraction from vendor REST APIs.
• Strong communication, collaboration, and problem-solving skills.
• Knowledge of modern data architecture principles like Data Lakes, External tables, and medallion architecture is an asset.
• Experience with Spark/PySpark or Python is a bonus, or a willingness to develop these skills.

• Pension scheme: a salary sacrifice scheme with a 2:1 employer match on contributions up to 10% of your salary.
• Annual Leave: up to 30 days of annual leave plus bank holidays.
• Life Assurance Plan: a lump sum payment of up to 4 times your salary.
• Financial Benefits: includes bi-annual pay reviews, interest-free season ticket and parking permit loans, a Cycle to Work scheme, and a “Give as You Earn” scheme for charitable donations. You can also get one-on-one financial wellbeing sessions and access discounts.
• Health and Wellbeing: Health Cash Plan, discounted gym memberships, funded eye tests, and seasonal flu vouchers. You'll also have access to a confidential Employee Assistance Programme (EAP) with 24/7 counselling and a Mental Health First Aider Network.
• Flexible working options are available on a hybrid basis, with a minimum of two days per week in the office.
• Career and Development includes an award-winning Management & Leadership Academy, funding for professional and personal development, and access to a digital learning platform.