Data Engineer with Python Spark Databricks and AIML

TekCommands Inc

Dallas, TX, United States

about 1 month ago

Role details

Contract type

Temporary to permanent

Employment type

Full-time (> 32 hours)

Compensation

$41,600.0 - $124,800.0

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Microsoft Azure Cloud Database Continuous Integration Information Engineering Data Systems Python (Programming Language) PostgreSQL Machine Learning Enterprise Data Management Data Ingestion

+17 more

Retrieval-Augmented Generation Large Language Models Snowflake Prompt Engineering Apache Spark Generative AI Build Management Data Lakes Pyspark HuggingFace Data Management Virtual Agents Artificial Intelligence Markup Language (AIML) Data Pipelines Automation Anywhere Docker Databricks

Job description

As a Computational Engineering Expert, you will leverage your specialized knowledge to enhance the training of next-generation AI systems. Your contributions will directly influenc…

1 month ago

Requirements

We are seeking an experienced Data Engineer with expertise in Python, Spark, Databricks, and AI/ML technologies to build scalable data platforms and Generative AI solutions. The ideal candidate should have experience developing data pipelines, LLM applications, AI agent workflows, and cloud-based data engineering solutions. Key Responsibilities Design and build scalable data pipelines using Python, Spark, Databricks, and Delta Lake. Develop and manage data solutions on Azure, Snowflake, and PostgreSQL. Build and optimize LLM/GenAI pipelines, including data ingestion, prompt engineering, fine-tuning, and deployment. Develop AI workflows using LangChain, LangGraph, AI Agents, and RAG architectures. Integrate with OpenAI, Azure OpenAI, Hugging Face, and other LLM APIs. Collaborate with data scientists, AI engineers, and business teams. Ensure data quality, governance, security, and performance across enterprise data platforms. Required Skills 7 12 years of Data Engineering experience. Strong expertise in Python, PySpark, and Apache Spark. Hands-on experience with Databricks and Delta Lake. Experience with Snowflake and PostgreSQL. Experience building LLM/GenAI applications and pipelines. Strong knowledge of LangChain, LangGraph, AI Agents, and RAG (Retrieval-Augmented Generation). Experience with OpenAI, Azure OpenAI, or Hugging Face APIs. Good understanding of Azure Cloud architecture and scalable data solutions. Strong analytical, problem-solving, and communication skills. Preferred Skills Telecom domain experience. Experience with Model Context Protocol (MCP). AI Agent orchestration. Docker. CI/CD pipelines. Azure or GCP cloud platforms. Top Skills Python & PySpark Databricks & Apache Spark LLM/GenAI (LangChain, LangGraph, RAG, OpenAI/Azure OpenAI) Snowflake & Delta Lake Azure Cloud Nice to Have Telecom industry experience MCP (Model Context Protocol) Docker & CI/CD GCP Cloud Experience