Data Engineer

You will be joining a small data team in this area of the business which will work with external partners to build out the organisations data capability offering. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining robust, scalable, and secure data pipelines and platform that enable them to make data -driven decisions.

You'll work closely with the 'Head of Data Engineering' to grow out this data function with the recruitment of further data engineering colleagues. You'll be exposed to a wide range of projects that include internal and external suppliers., * Lead on the introduction of foundational data management capabilities to improve trust, accessibility, and efficiency in an organisation that has limited data management capability, lacks data management practices, including governance, metadata standards, and quality controls.

• Design, implement, and optimise physical data models that align with pipeline architecture, by using the approach that ensures efficient query performance, scalable storage, and robust integration and delivers adaptable and resource -efficient data processing, meeting the organisation's evolving analytical and operational demands.
• Work closely with data analysts, architects, devops Engineers, and business stakeholders through regular communication and collaborative planning to ensure data solutions are closely aligned with business objectives and effectively meet user needs.
• Transform raw data into meaningful insights by developing and maintaining tailored ETL (Extract, Transform, Load) processes enabling customised processes, empowering stakeholders to make informed decisions based on high-quality, processed information
• Managing the aspirations of a variety of stakeholders to enable successful project delivery can be challenging, especially when their priorities may differ or even conflict and require reconciliation to meet business and project needs.

• Proven experience as a Data Engineer
• Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles.
• Experience of using programming languages such as Python, Scala and SQL.
• Demonstrable knowledge of data modelling and data warehousing within platforms such as Azure.
• Practical experience with Microsoft Azure services, including Azure Data Lake (Gen2), Synapse, Event Hubs, and Cosmos DB, within scalable cloud -based architectures.
• Experience in using Git, Azure DevOps, or GitHub Actions for version control, CI/CD, and collaborative data delivery.
• Robust understanding of data governance, data quality, and metadata management.
• Experience of communicating technical information and data to a non-technical audience and working collaboratively with analysts, architects, and product owners to deliver data solutions that meet user and organisational needs.

• Expected 6-month project (with potential for extension or even temp to perm for the right candidate)
• To be based in their modern offices 2 days per week with the wider team
• A chance to truly contribute to large scale digitalisation projects within Wales.
• Start date asap
• Interviews likely to commence w/c 15th onwards
• IR35 status to be determined