AI Platform & Data Engineer - Databricks / MLOps

JCW is partnering with a global financial institution to hire an AI Platform & Data Engineer to build and run the data foundation behind its enterprise GenAI platform. AI is only as good as the data feeding it, and this role owns that layer. You will build the Databricks pipelines that prepare, govern and deliver trusted data to AI models, RAG applications and agents. You will also make sure those AI systems run reliably in production. You will work closely with Architecture, Technology, Data and business stakeholders, and partner with Azure, Databricks and other infrastructure providers. This is a hands-on engineering seat for someone who is strong in data engineering on Databricks and has worked on getting AI into production.

What You’ll Do

• Build and maintain scalable data pipelines on Databricks and Azure that ingest, clean, transform and deliver data for AI and GenAI use cases
• Prepare data for RAG and AI applications, including document ingestion, chunking, embeddings and managing vector indexes with Databricks Vector Search
• Own data quality across the AI platform, defining quality checks and monitoring for data drift, freshness and completeness
• Implement end-to-end data lineage, versioning and governance across data, prompts, models and responses using Unity Catalog and MLflow, delivering audit-ready artifacts for regulatory review
• Build CI/CD and infrastructure-as-code using Databricks Asset Bundles to promote data pipelines, models and AI workflows across DEV/QA/REL/PROD
• Support the MLOps/LLMOps lifecycle, including model deployment, monitoring for performance and grounding, and cost visibility
• Apply security and privacy controls to sensitive data used in AI, including access management, secrets management and protection of data and embeddings
• Support day-2 operations through runbooks, incident response and SLAs, and help educate wider teams on data and AI platform best practices

• Bachelor’s degree in Computer Science, Data Engineering, Data Science or a related field
• 5+ years of hands-on data engineering experience, with recent work supporting AI/ML or GenAI use cases in production
• Strong Python, PySpark and SQL skills
• Deep hands-on Databricks experience, including Delta Lake, Unity Catalog, Workflows and Asset Bundles, along with Azure data services
• Experience preparing data for GenAI, such as document processing, chunking, embeddings and vector databases (e.g., Databricks Vector Search, Azure AI Search)
• Hands-on experience with MLflow and MLOps/LLMOps practices
• Experience building data quality checks, monitoring and lineage in a production environment
• Experience with CI/CD, infrastructure-as-code and containerization
• Familiarity with data governance and security controls in a regulated environment, ideally financial services
• Excellent communication skills, with the ability to partner with technical and non-technical stakeholders

JCW is proud to be an equal opportunity recruiter. xrbllit We are committed to fostering an inclusive recruitment process and ensuring equal consideration for all applicants regardless of gender, age, disability, religion, sexual orientation, marital status, or race.