> Markdown version of [/jobs/ext/3024805-lead-data-engineer-aws-ai-sql](https://www.wearedevelopers.com/jobs/ext/3024805-lead-data-engineer-aws-ai-sql). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - AWS / AI / SQL - **Company:** Adria Solutions ltd - **Location:** Manchester, UK - **Experience:** Expert - **Salary:** £70,000.0 - £90,000.0 - **Contract:** Permanent contract - **Skills:** Sql Server Data Tools (SSDT), Artificial Intelligence, Amazon Web Services, Amazon S3, Data Analysis, Automation of Tests, Big Data, Continuous Integration, Customer Data Management, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Data Warehousing, Python (Programming Language), Machine Learning, Metadata, Microsoft SQL Server, Power BI, Standard Sql, SQL Databases, SQL Server Integration Services, Data Processing, Data Server Interface, Data Layers, Event Driven Architecture, Pyspark, Apache Flink, AWS Data Analytics, Real Time Data, Data Management, Video Streaming, Terraform - **Published:** September 22, 2026 - **Apply:** https://find.jobs/jobs-near-me/apply/ats-redirect/?id=2983182265-2 ## About the Role We re looking for an experienced Data Engineer who has built and operated complete data platforms rather than solely delivering individual components. You ll need experience across: * Batch data processing, ELT and orchestration. * Modern streaming technologies such as Flink or comparable platforms. * AWS data services, including S3, Lambda, Glue, Athena, Kinesis and RDS. * Infrastructure as code using Terraform. * CI/CD and automated testing. * Lakehouse architecture and open table formats such as Iceberg. * Medallion architecture and dimensional data modelling. * Strong SQL and Python. * SQL Server, SSIS/SSDT and Power BI. * Semantic layers, common metrics and data glossaries. * Data governance, lineage, retention, classification and auditability. * Managing and developing a small technical team. You should also be comfortable using advanced AI coding assistants, such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience * Regulated environment. * PySpark for large-scale data transformations. * Migrating legacy data estates to lakehouse or metadata-driven architectures. * Customer Data Platforms or event-driven systems. * Data-quality tools such as dbt tests or Great Expectations. * MCP servers or governed natural-language data interfaces. * Financial crime, fraud or regulatory-reporting data. * Power BI administration and semantic modelling. ## Description An established organisation is looking for a hands-on Lead Data Engineer to take ownership of its existing data estate and lead the development of a modern, scalable data platform. This is an excellent opportunity to build a platform supporting Business Intelligence, Data Science and Applied AI. You ll introduce modern batch and real-time data-processing capabilities, automate inefficient processes and make trusted data more accessible across the organisation. Alongside remaining technically hands-on, you ll manage and develop a small team of Data Engineers. Key responsibilities * Own the reliability, security and ongoing development of the data platform. * Modernise the existing SQL Server, SSIS/SSDT and Power BI environment. * Develop scalable batch and real-time data-processing solutions. * Lead the design of a Customer Data Platform. * Build and evolve a lakehouse architecture using AWS technologies. * Create governed semantic models, consistent metrics and clear data definitions. * Enable secure, self-service data analysis using modern AI tooling. * Automate manual, repetitive or fragile data processes. * Support BI, Data Science, Machine Learning and Applied AI teams. * Take ownership of data governance, security, monitoring and platform costs. * Line-manage, develop and help grow the Data Engineering team. * Work closely with technical, operational and senior business stakeholders. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Beyond Dashboards: Fixing Text-to-SQL with Semantic RAG](https://www.wearedevelopers.com/videos/2036-beyond-dashboards-fixing-text-to-sql-with-semantic-rag) - [A Data Mesh needs Open Metadata](https://www.wearedevelopers.com/videos/505-a-data-mesh-needs-open-metadata) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)