> Markdown version of [/jobs/ext/2019711-principal-ai-data-engineer](https://www.wearedevelopers.com/jobs/ext/2019711-principal-ai-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Principal AI Data Engineer

- **Company:** Certain Advantage
- **Location:** London, UK
- **Contract:** Temporary contract
- **Skills:** Artificial Intelligence, Automation of Tests, Microsoft Azure, Computer Programming, Data Architecture, Information Engineering, Extract Transform Load (ETL), DevOps, Github, Python (Programming Language), SQL Databases, Azure Data Factory, Large Language Models, Prompt Engineering, Generative AI, Data Lakes, Qlikview, Spark Streaming, Machine Learning Operations, Virtual Agents, GPT, Data Pipelines, Databricks
- **Published:** August 11, 2026
- **Apply:** https://www.totaljobs.com/job/principal-ai-engineer/certain-advantage-job107823602

## About the Role

* Strong expertise with Databricks, including Delta Lake, Unity Catalog, DLT and MLflow.
 * Deep knowledge of Large Language Models including GPT, Llama, Claude and Mistral.
 * Experience building and deploying GenAI, RAG and Agentic AI solutions in enterprise environments.
 * Strong programming skills in Python, SQL and/or Scala.
 * Experience with Azure cloud technologies including Azure OpenAI and Azure AI Foundry.
 * Knowledge of Spark Structured Streaming, Autoloader and modern data engineering practices.
 * Experience with ETL/ELT, data modelling and scalable data architectures.
 * Familiarity with Azure Data Factory, Qlik Replicate and data ingestion pipelines.

## Description

This role is an initial contract until the end of the year and is hybrid, requiring 3 days per week onsite in London.

You'll be responsible for:

 * Developing AI, GenAI and Agentic AI prototypes using Azure AI Foundry, Copilot Studio, Databricks Mosaic AI and MLflow.
 * Building, optimising and evaluating Retrieval-Augmented Generation (RAG) solutions using prompt engineering and embedding models.
 * Designing and deploying AI agents using frameworks including LangChain, LangGraph and AutoGen.
 * Delivering production-ready GenAI applications for commercial teams within Trading & Supply.
 * Building scalable AI workflows on Databricks using Mosaic AI, MLflow, Genie and AgentBricks.
 * Applying DevOps best practices through CI/CD pipelines, GitHub workflows and automated testing.
 * Designing evaluation frameworks to benchmark, test and improve GenAI system performance and reliability.


## Related Videos

- [Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More](https://www.wearedevelopers.com/videos/869-inside-the-ai-revolution-how-microsoft-is-empowering-the-world-to-achieve-more)
- [ Evaluating AI models for code comprehension](https://www.wearedevelopers.com/videos/1462-evaluating-ai-models-for-code-comprehension)
- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [Beyond GPT: Building Unified GenAI Platforms for the Enterprise of Tomorrow](https://www.wearedevelopers.com/videos/1525-beyond-gpt-building-unified-genai-platforms-for-the-enterprise-of-tomorrow)
- [Speak, Code, Deploy: Transforming Developer Experience with Voice Commands](https://www.wearedevelopers.com/videos/1159-speak-code-deploy-transforming-developer-experience-with-voice-commands)

## Related Articles

- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [The State of WebDev AI 2025 Results: What Can We Learn?](https://www.wearedevelopers.com/magazine/581-the-state-of-webdev-ai-2025-results-what-can-we-learn)
- [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)