AI Data Engineer

n \n

• Strong hands-on experience with SQL and at least one programming language: Python or Scala. \n

• Experience working with cloud-based data platforms. \n

• Hands-on experience with data orchestration tools such as Airflow and dbt. \n

• Understanding of embedding models, vector search, and RAG architectures. \n

• Experience processing structured and unstructured data. \n

• Familiarity with ML lifecycle concepts, including model training and inference.

n \n

• Design and develop data pipelines to ingest, clean, and transform structured and unstructured data, including documents, PDFs, transcripts, and logs. \n

• Build and maintain vector embedding pipelines and integrate with vector databases. \n

• Develop feature stores and data pipelines for ML model training and inference. \n

• Implement data pipelines supporting RAG architectures, including chunking strategies, embedding refresh cycles, and metadata tagging. \n

• Collaborate with ML Engineers and Data Scientists to understand data requirements for model training and evaluation. \n

• Ensure data lineage, quality, and governance across pipelines supporting AI systems. \n

• Optimize data pipelines for cost and performance, including Dynamic Tables and streaming ingestion. \n

• Support MLOps/LLMOps workflows, including dataset versioning and tracking data used for model training to ensure reproducibility and compliance. \n

• Implement data entitlement and access control mechanisms to ensure AI systems access only authorized data. \n

\n

\n