Data Engineer
Role details
Job location
Tech stack
Job description
Effective use of data is critical to the successful delivery of the Open University's strategic goals. Reporting to the Data Technology Director within Digital Services, this key Data Engineer role is responsible for building, operating, and evolving the Data Technology capability for the Open University.
The successful candidate will be an experienced data engineering professional with strong expertise in modern data platforms, preferably Microsoft Azure. You will bring hands-on experience with data engineering tools, programming languages, architectures, integration patterns, and operational best practices, and demonstrate the ability to implement these effectively at enterprise scale.
This role is responsible for the technical development of the Azure data platform and will act as the primary point of contact for data processing related to Tuition Services. As part of the Chief Data Office, the role will work closely with analytics and data teams to deliver a comprehensive and high-quality data service to the University., * Acting as an individual contributor across the wider technology team while serving as a specialist in data engineering.
- Designing and delivering end-to-end data transformation pipelines, supporting both batch and streaming data ingestion.
- Implementing resilient, self-healing, and observable data ingestion and consumption infrastructure.
- Taking technical ownership of workstreams, acting as the subject matter expert, and proactively managing risks and issues.
- Defining, developing, and maintaining technical designs, documentation, and user artefacts, while driving continuous improvement within an agile delivery model.
- Identifying and resolving technical problems and delivery roadblocks, working collaboratively with peers to drive issues to closure.
- Performing performance tuning and code optimisation, and reviewing peers' code to ensure adherence to best practices.
- Mentoring and supporting the development of junior members of the data engineering team.
- Working closely with the Data Engineering Lead and Analysts to ensure timely delivery of solutions that meet business and technical requirements.
- Supporting technology innovation initiatives across the Open University through proofs of concept and exploratory work.
During the initial phase, the role will focus on:
- Delivering highly critical data engineering solutions that enable the Open University to achieve its key strategic programmes.
- Working closely with the Tuition Services team on current and future data processing, reporting, and analytics solutions.
- Defining and delivering a roadmap for data platform uplift, aligned with Microsoft Azure's technology roadmap, to meet future analytics needs in a cost-effective and scalable manner., You can view your progress and application communications when you are logged into our recruitment system. Please check your spam/junk folders if you do not receive associated email updates.
Requirements
Do you have experience in Spark?, * Communicating effectively with stakeholders at all levels, managing expectations, and facilitating discussions in high-risk, complex, or time-constrained environments.
- Designing and delivering enterprise-scale data integration solutions across the full data development lifecycle, with strong expertise in cloud-based data engineering (preferably Microsoft Azure).
- Leading and motivating teams to ensure the reliable and efficient delivery of enterprise data services.
- Implementing both on-premises and cloud-based data engineering solutions, with hands-on experience in relational databases, ETL tools, and programming languages such as SQL, Python, PySpark, and Java.
- Defining and enforcing data engineering standards, architectural patterns, and industry best practices.
- Advising on and shaping future technology roadmaps to deliver long-term business value.
- Investigating emerging data technologies and trends, conducting horizon scanning, and introducing innovative approaches and ways of working.
Desirable:
- Azure certification as a Cloud Architect or Cloud Data Engineer.
- Practical experience in cloud-based development on the Microsoft Azure platform.
- Strong hands-on experience with Azure Data Factory, Data Lake, Synapse, SQL Database, and Microsoft Fabric.
- Experience with Azure Databricks and Spark cluster management.
- Experience building Azure Data Factory pipelines and configuring CI/CD pipelines using Git.
Benefits & conditions
We are open to discussions about flexible working. Whether it's a job share, part time, compressed hours or another working arrangement. Please reach out to us to discuss what works best for you.
It is anticipated that a hybrid working pattern can be adopted for this role, where the successful candidate can work from home and the office. However, as this role is contractually aligned to our Milton Keynes office it is expected that some attendance in the office will be required when necessary and in response to business needs. We'd expect this to be approximately once per month. Next steps in the Recruitment process
We anticipate that interviews for this role will be taking place online via Microsoft Teams during the week commencing 16 March 2026. Early closing date notification
