> Markdown version of [/jobs/ext/2794973-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/2794973-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer

- **Company:** Response Informatics
- **Location:** Salford, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Microsoft Azure, Big Data, Computer Programming, Continuous Integration, Information Engineering, Data Systems, Distributed Computing Environment, Job Scheduling, Python (Programming Language), Metadata, Query Optimization, Data Processing, Azure Data Factory, Apache Spark, Git, Pandas, Deployment Automation, Data Pipelines, Databricks
- **Published:** September 8, 2026
- **Apply:** https://www.collegerecruiter.com/job/2840674842-senior-data-engineer

## About the Role

* Azure Data Factory (ADF): Expertise in building and orchestrating data pipelines.
 * Databricks: Hands-on experience with notebooks, clusters, and job scheduling.
 * Pandas: Advanced data manipulation and transformation skills.
 * SparkSQL: Strong knowledge of distributed data processing and query optimization.
 * CI/CD: Experience with Azure DevOps, Git, or similar for automated deployments.
 * Metadata-driven architecture: Proven experience designing and implementing metadata frameworks.
 * Programming: Proficiency in Python and/or Scala for data engineering tasks.

## Description

* Design, develop, and maintain metadata-driven data pipelines using Azure Data Factory and Databricks.
 * Build and implement end-to-end metadata frameworks, ensuring scalability and reusability.
 * Optimize data workflows leveraging SparkSQL and Pandas for large-scale data processing.
 * Collaborate with cross-functional teams to integrate data solutions into enterprise architecture.
 * Implement CI/CD pipelines for automated deployment and testing of data solutions.
 * Ensure data quality, governance, and compliance with organizational standards.
 * Provide technical leadership and take complete ownership of assigned projects.


## Related Videos

- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Advanced Typing in TypeScript](https://www.wearedevelopers.com/videos/496-advanced-typing-in-typescript)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Getting to Know Your Legacy (System) with AI-Driven Software Archeology](https://www.wearedevelopers.com/videos/1437-getting-to-know-your-legacy-system-with-ai-driven-software-archeology)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)