> Markdown version of [/jobs/ext/2841873-data-engineer](https://www.wearedevelopers.com/jobs/ext/2841873-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Cmspi - **Location:** Manchester, UK - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Clean Code Principles, Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Automation of Tests, Microsoft Azure, Software Quality, Continuous Integration, Data Architecture, Information Engineering, Extract Transform Load (ETL), Data Security, Data Systems, DevOps, Apache Hadoop, Python (Programming Language), Object-Oriented Software Development, Windows PowerShell, Scrum Methodology, Azure Data Lake, Scala (Programming Language), Selenium, YAML, File Transfer Protocol (FTP), Azure Data Factory, Apache Spark, Data Lakes, Kubernetes, Information Technology, Playwright, Bicep, Data Management, Terraform, Data Pipelines, Serverless Computing, Docker, Key Vault, Databricks - **Published:** September 11, 2026 - **Apply:** https://www.careerjet.co.uk/jobad/gb9bfce4fc99fb6e637fbda8348adf9a19 ## About the Role You are a great match if: * 2+ years' professional experience as a Data Engineer or DevOps Engineer. * Experience building ETL/ELT workflows for analytics using Spark or Hadoop. * Strong Microsoft Azure experience across DevOps/CI/CD, Azure Data Factory, Azure Data Lake, Azure Functions, Key Vault, monitoring, networking, identity, and cloud deployment services. * Databricks experience, including workflows/jobs, notebooks, Delta Lake, Unity Catalog and other Databricks-native tools. * Advanced Python skills, including efficient functional and object-oriented code, reusable Python libraries, and practical experience with packages such as requests, Selenium, and Playwright. * Strong problem-solving, analytical, communication and cross-functional collaboration skills. * Bachelor's degree in Computer Science, Mathematics or another analytical/numerical subject. * Highly desirable: experience with Agile squads, payments, Terraform/Bicep, Docker/Kubernetes, Scala, PowerShell or YAML. ## Description As a Data Engineer, you will help shape the evolution of CMSPI's payments intelligence platform. You will work across the full data engineering lifecycle, designing secure data retrieval and ingestion capabilities, building reliable analytics workflows and supporting client onboarding. Working closely with a cross-functional development squad, you will turn business requirements into maintainable solutions while learning and applying modern engineering practices and technologies. Requirements JOB ROLE Key accountability for this role: Deliver innovative, scalable and secure data solutions for CMSPI's payments intelligence platform. Key responsibilities will include: * Design scalable data architectures and retrieval solutions across APIs, web scrapers and SFTP, with security and performance at the centre. * Deliver prompt, efficient onboarding of CMSPI client data and contribute to new data features that create continuous business value. * Collaborate with engineers, agile delivery leads and business stakeholders to define, refine and deliver requirements in an agile squad. * Maintain high software quality through clean code, collaborative code and architecture reviews, automated testing and continuous integration. * Respond to stakeholder issues and queries, troubleshoot production problems and drive practical improvements to data solutions. * Help shape CMSPI's next generation data platform using powerful tools such as Databricks AI Gateway, enabling governed, agent-powered capabilities that support self-building and self-healing data pipelines. * Participate in sprint planning and daily stand-ups, keeping stories and tasks current and raising critical blockers early. * Learn, adopt and promote best practices in data engineering, including reusable Python libraries and maintainable functional and object-oriented code. * Stay aware of emerging technology trends and research ideas that can improve CMSPI's data platforms and engineering practices. * Work collaboratively as part of an Agile development squad and contribute to a culture of continuous learning and delivery. ## Related Videos - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Back(end) to the Future: Embracing the continuous Evolution of Infrastructure and Code](https://www.wearedevelopers.com/videos/440-back-end-to-the-future-embracing-the-continuous-evolution-of-infrastructure-and-code) - [CI/CD with Github Actions](https://www.wearedevelopers.com/videos/856-ci-cd-with-github-actions) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The 12 Best Jobs for Software Engineers](https://www.wearedevelopers.com/magazine/401-the-12-best-jobs-for-software-engineers) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)