> Markdown version of [/jobs/ext/3100392-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/3100392-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Hackajob Ltd - **Location:** London, UK (Remote available) - **Experience:** Expert - **Salary:** £95,000.0 - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Microsoft Azure, Cloud Computing, Cloud Database, Continuous Integration, Data Integration, Data Transformation, Data Systems, Data Warehousing, DevOps, Github, Apache Hadoop, JSON, Machine Learning, Scrum Methodology, Cloud Services, DataOps, Data Streaming, UML, Data Processing, Google Cloud, Cloud Platform System, Apache Spark, Git, SC Clearance, Data Lakes, Data Analytics, Extreme Programming, Data Management, Data Pipelines, Docker, Databricks - **Published:** September 27, 2026 - **Apply:** https://www.adzuna.co.uk/jobs/details/5899589619 ## About the Role * We are looking for candidates with a range of skills and experience, and we encourage applications even if you do not meet every criterion. * Proficiency in Git, including GitHub Actions, and the ability to explain the benefits of different branch strategies. * Strong experience with Infrastructure as Code and the ability to guide infrastructure deployment across different environments. * Knowledge of handling and transforming data types such as JSON and CSV using Apache Spark, Databricks, or Hadoop. * Good understanding of modern data system architectures, including Data Warehouses, Data Lakes, and Data Meshes. * Ability to create data pipelines in cloud environments and integrate error handling within them. * Ability to create reusable libraries to encourage consistency across multiple data pipelines. * Ability to document and present end-to-end diagrams for data processing systems in cloud environments, with some knowledge of C4, UML, or similar diagramming approaches. * Enthusiasm for learning and self-development. * Experience working on agile delivery-led projects and applying practices such as Scrum, XP, and Kanban. * Ability to own the cloud infrastructure underpinning data systems through a DevOps approach. * Experience designing and implementing efficient data transformation processes at scale in both batch and streaming use cases. * Experience delivering data platforms as a skilled Data Engineer. * Knowledge of SOLID, DRY, and TDD principles and how to apply them practically in projects. * Ability to demonstrate a commercial mindset and grow accounts organically with senior stakeholders. * Experience improving resilience by checking for software vulnerabilities and implementing appropriate testing strategies, including unit, integration, and data quality testing. * Ability to offer guidance on implementing a robust DevOps approach in a data project. * Comfort discussing tools and approaches used in DataOps, including orchestration, data integration, and data analytics. * Eligibility for SC security clearance, which requires 5 years of UK residency and 5 years of employment history, or equivalent full-time education history, with limited time spent outside the UK in the last five years. ## Description * Play a pivotal role in helping public sector organisations become truly data-led by equipping them with robust data platforms. * Join our data team in spreading data knowledge and skills beyond silos and into delivery teams. * Help implement efficient data pipelines and storage. * Define, shape, and improve data strategies across central and local government. * Help public sector teams understand the value of their data and use it more effectively. * Establish yourself as a trusted advisor in data-driven approaches using public cloud services such as AWS, Azure, and GCP. * Contribute to our recruitment efforts. * Take on line management responsibilities. * Design and implement efficient data transformation processes at scale. * Produce documentation and diagrams that explain cloud-based data processing systems. Technologies: * AWS * Azure * Cloud * Databricks * DevOps * GCP * Git * GitHub * Hadoop * Support * JSON * Kanban * Security * Slack * Spark * TDD * UML * XP * Architect * CI/CD * Data Warehouse * Docker * Embedded * LESS * Machine Learning * Network More: We are Made Tech, a technology company on a mission to use technology to improve society for everyone. We help organisations transform, deliver, and manage world-class digital products and services, with a strong focus on public sector work. Our team culture is built around a happy, inclusive, and diverse workforce, with communities of practice, Slack communities, learning spaces, and support for mental health, neurodiversity, and connection. We offer benefits including 30 days of paid annual leave, flexible parental leave, part-time remote working, paid counselling, financial and legal advice, a 7% employer-matched pension, and a flexible benefits platform with options such as Smart Tech, Cycle to Work, and a health care cash plan. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [ChatGPT and Java: A Match Made in Heaven or Hell?](https://www.wearedevelopers.com/videos/536-chatgpt-and-java-a-match-made-in-heaven-or-hell) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) - [Beyond UML: Making Sense of AI-Generated Code through Visual Architecture](https://www.wearedevelopers.com/videos/2102-beyond-uml-making-sense-of-ai-generated-code-through-visual-architecture) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)