> Markdown version of [/jobs/ext/3975122-senior-ai-data-engineer](https://www.wearedevelopers.com/jobs/ext/3975122-senior-ai-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior AI Data Engineer

- **Company:** Huxley Associates
- **Location:** Boston, MA, United States
- **Experience:** Expert
- **Salary:** $250,000.0
- **Contract:** Permanent contract
- **Skills:** LangGraph Framework, Unity 3d, Agile Methodology, Artificial Intelligence, Software Applications, Continuous Integration, Information Engineering, Data Governance, Distributed Computing Environment, Machine Learning, Performance Tuning, OpenAI, Search Technologies, Pinecone, LangChain, Retrieval-Augmented Generation, Large Language Models, Multi-Agent Systems, Prompt Engineering, Llamaindex, Pgvector, Data Lakes, Pyspark, Data Management, Data Pipelines, Automation Anywhere, ChromaDB, Databricks
- **Published:** October 27, 2026
- **Apply:** https://www.huxley.com/en-gb/job/senior-ai-data-engineer/4061889

## About the Role

* 5+ years of experience in Data Engineering, AI Engineering, or Machine Learning Infrastructure roles.
 * Hands-on experience building and supporting production RAG systems at scale.
 * Strong expertise integrating LLM APIs (OpenAI, Anthropic, Google, or similar) into production environments.
 * Experience with vector databases and retrieval technologies such as Pinecone, pgvector, Chroma, Databricks Vector Search, or similar platforms.
 * Advanced Python development skills and experience building production-quality software.
 * Experience with modern cloud data platforms, data pipelines, CI/CD workflows, and data governance practices.
 * Strong understanding of AI observability, monitoring, evaluation, and performance optimization.
 * Experience working in Agile development environments., * Experience with Databricks, Delta Lake, Unity Catalog, or modern lakehouse architectures.
 * Familiarity with LangChain, LangGraph, LlamaIndex, or similar AI orchestration frameworks.
 * Exposure to agentic AI architectures, tool-calling frameworks, and multi-step AI workflows.
 * Knowledge of large-scale distributed processing using PySpark or similar technologies.
 * Experience building AI solutions in highly regulated, enterprise, or document-intensive environments., The ideal candidate has deep experience building reliable AI systems in production and understands what it takes to move beyond proof-of-concept solutions. They're comfortable operating at the intersection of data engineering, machine learning infrastructure, and AI application development, with a strong focus on scalability, observability, reliability, and business impact.

## Description

* Design, build, and maintain end-to-end RAG pipelines, including document ingestion, chunking, embedding generation, vector indexing, and retrieval optimization.
 * Develop and manage vector search infrastructure, ensuring highly accurate and contextually relevant retrieval for AI-powered applications.
 * Build scalable embedding pipelines for both structured and unstructured data sources.
 * Implement and optimize LLM-powered features, including prompt engineering, output validation, latency management, fallback strategies, and cost optimization.
 * Develop observability frameworks to monitor AI system performance, retrieval quality, token usage, response quality, and application reliability.
 * Collaborate closely with application engineers to integrate AI capabilities into user-facing products.
 * Build and maintain data pipelines that support AI workloads while adhering to governance, security, and data quality standards.
 * Evaluate emerging AI technologies and recommend best practices for production adoption.
 * Contribute to architecture decisions, engineering standards, CI/CD processes, testing frameworks, and operational excellence initiatives.