> Markdown version of [/jobs/ext/2795637-senior-lead-data-engineer-ai-focused](https://www.wearedevelopers.com/jobs/ext/2795637-senior-lead-data-engineer-ai-focused). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior / Lead Data Engineer (AI-Focused) - **Company:** PaymentGenes - **Location:** Greater London, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Airflow, BigQuery, Cloud Database, Code Review, Encodings, Continuous Integration, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Software Design Patterns, DevOps, Python (Programming Language), Performance Tuning, DataOps, Management of Software Versions, Workflow Management Systems, Feature Engineering, Sql Optimization, System Availability, Delivery Pipeline, Large Language Models, Snowflake, Data Layers, Machine Learning Operations, Terraform, Api Management, Docker, Amazon Redshift - **Published:** September 8, 2026 - **Apply:** https://www.collegerecruiter.com/job/2840675606-senior--lead-data-engineer-ai-focused ## About the Role * Advanced SQL & Python * Workflow orchestration (Dagster preferred) * Cloud data warehouses (Snowflake, BigQuery, Redshift, etc.) * Data modelling for analytics and AI use cases * API integrations and ingestion design patterns AI / ML Infrastructure * Feature engineering architecture * ML pipeline and deployment workflows * Experience supporting production ML systems * Familiarity with embeddings, vector databases, LLM orchestration (desirable) * Data observability and model monitoring Platform & DevOps * CI/CD for data workflows * Docker / containerisation * Infrastructure-as-code (e.g., Terraform) * Monitoring and alerting systems * 6-10+ years in data engineering or related disciplines * Proven experience architecting and scaling modern data platforms * Experience enabling ML/AI production workflows * Demonstrated technical leadership and mentoring * Ability to influence senior stakeholders, * Architectural thinking with long-term vision * The ability to balance innovation with reliability * Clear communication across technical and non-technical audiences * A strong ownership mindset and accountability for outcomes, * A scalable, reliable data platform powering AI and analytics growth * Reduced ML time-to-production * High levels of data quality, observability, and governance maturity * Improved cost-performance efficiency across the data stack * A strong, growing data engineering capability within the team This is a high-impact leadership role within a forward-thinking technology environment where AI and data are core to the business strategy. ## Description * Design scalable ELT/ETL frameworks using DBT and cloud data warehouses * Establish orchestration standards (Dagster or equivalent) * Drive decisions across batch, streaming, and real-time pipelines * Champion data modelling standards, semantic layers, and metric governance, * Architect data foundations supporting the ML lifecycle * Design feature stores, embedding pipelines, and AI-ready datasets * Enable MLOps workflows (data versioning, monitoring, retraining triggers) * Support production inference (batch and real-time) * Evaluate and integrate emerging AI tooling where strategically valuable * Set best practices for testing, documentation, lineage, and observability * Lead code reviews and mentor data & analytics engineers * Drive CI/CD and infrastructure-as-code adoption * Own platform reliability, performance optimisation, and cost efficiency * Establish SLAs for data freshness and quality * Partner with Data Science, Product, and Engineering leadership * Translate business strategy into scalable data solutions * Influence KPI and metric governance across teams * Act as technical escalation point for complex data challenges ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Developer Experience, Platform Engineering and AI powered Apps](https://www.wearedevelopers.com/videos/990-developer-experience-platform-engineering-and-ai-powered-apps) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)