Data Operations Engineer / DataOps Specialist

We are seeking an experienced Senior Data Operations Engineer (DataOps Specialist) with 10+ years of experience in designing, building, and managing modern data platforms and AI-enabled data pipelines. The ideal candidate will have strong expertise in Python, LangChain, LangGraph, Vector Databases, LLM applications, ETL/ELT frameworks, cloud platforms, and DataOps best practices.

This role is responsible for building scalable data pipelines, ensuring data quality, automating workflows, monitoring production systems, and supporting AI/GenAI data platforms across enterprise environments., * ETL Tools

• Apache Spark
• Hadoop Ecosystem
• AWS / Azure / GCP
• Data Modeling
• Data Architecture
• REST APIs
• Data Integration
• Git
• CI/CD Pipelines
• Docker
• Kubernetes
• Jenkins

• Terraform (preferred), Data Pipeline Development

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build automated workflows for data ingestion, transformation, and processing.
• Support both batch and real-time data processing frameworks.
• Develop reusable Python components for enterprise data operations.
• Integrate structured and unstructured data sources., * Build Retrieval-Augmented Generation (RAG) pipelines using LangChain and LangGraph.
• Develop AI workflows utilizing Vector Databases.
• Integrate enterprise applications with Large Language Models.
• Optimize prompt engineering strategies and LLM orchestration.
• Manage embeddings, document indexing, and semantic search pipelines., * Implement automated data validation and reconciliation processes.
• Develop data quality monitoring frameworks.
• Ensure consistency, integrity, and reliability of enterprise datasets.
• Support metadata management and data lineage.
• Enforce enterprise data governance standards., * Monitor production data pipelines and workflows.
• Detect and resolve pipeline failures.
• Perform root cause analysis for data issues.
• Implement alerting and observability solutions.
• Ensure SLA compliance and operational stability.

Database Extract Transform and Load (ETL), Artificial Intelligence (AI), Python Programming/Scripting Language, Data Management, Best Practices, Cloud Computing, Data Quality, Scalable System Development, Production Systems, Production Control, * 10+ years of experience in Data Engineering, Data Operations, or DataOps.

• Strong experience building enterprise-scale ETL/ELT pipelines.
• Hands-on experience with Python-based data engineering and automation.
• Experience working in Agile and DevOps environments., * Python
• LangChain
• LangGraph
• Large Language Models (LLMs)
• Vector Databases (Pinecone, Milvus, Weaviate, ChromaDB, FAISS)
• Prompt Engineering
• AI Data Pipelines