Senior AI Data Engineer

• Design, build, and maintain end-to-end RAG pipelines, including document ingestion, chunking, embedding generation, vector indexing, and retrieval optimization.
• Develop and manage vector search infrastructure, ensuring highly accurate and contextually relevant retrieval for AI-powered applications.
• Build scalable embedding pipelines for both structured and unstructured data sources.
• Implement and optimize LLM-powered features, including prompt engineering, output validation, latency management, fallback strategies, and cost optimization.
• Develop observability frameworks to monitor AI system performance, retrieval quality, token usage, response quality, and application reliability.
• Collaborate closely with application engineers to integrate AI capabilities into user-facing products.
• Build and maintain data pipelines that support AI workloads while adhering to governance, security, and data quality standards.
• Evaluate emerging AI technologies and recommend best practices for production adoption.
• Contribute to architecture decisions, engineering standards, CI/CD processes, testing frameworks, and operational excellence initiatives.

• 5+ years of experience in Data Engineering, AI Engineering, or Machine Learning Infrastructure roles.
• Hands-on experience building and supporting production RAG systems at scale.
• Strong expertise integrating LLM APIs (OpenAI, Anthropic, Google, or similar) into production environments.
• Experience with vector databases and retrieval technologies such as Pinecone, pgvector, Chroma, Databricks Vector Search, or similar platforms.
• Advanced Python development skills and experience building production-quality software.
• Experience with modern cloud data platforms, data pipelines, CI/CD workflows, and data governance practices.
• Strong understanding of AI observability, monitoring, evaluation, and performance optimization.
• Experience working in Agile development environments., * Experience with Databricks, Delta Lake, Unity Catalog, or modern lakehouse architectures.
• Familiarity with LangChain, LangGraph, LlamaIndex, or similar AI orchestration frameworks.
• Exposure to agentic AI architectures, tool-calling frameworks, and multi-step AI workflows.
• Knowledge of large-scale distributed processing using PySpark or similar technologies.
• Experience building AI solutions in highly regulated, enterprise, or document-intensive environments., The ideal candidate has deep experience building reliable AI systems in production and understands what it takes to move beyond proof-of-concept solutions. They’re comfortable operating at the intersection of data engineering, machine learning infrastructure, and AI application development, with a strong focus on scalability, observability, reliability, and business impact.

Compensation: Target base salary up to approximately $250K+, depending on experience and level., In addition to base pay, direct-hire employees may be eligible for client offered benefits such as medical, dental, and vision coverage, and paid leave where required by applicable law. Eligibility may vary based on factors such as location and hire date and is subject to change.