> Markdown version of [/jobs/ext/1068622-data-ai-automation-engineer](https://www.wearedevelopers.com/jobs/ext/1068622-data-ai-automation-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data & AI Automation Engineer - **Company:** IT America - **Location:** Austin, TX, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Automation of Tests, Microsoft Azure, Software Documentation, Computer Programming, Continuous Delivery, Continuous Integration, Data Validation, Information Engineering, Github, Python (Programming Language), Meta-Data Management, Standard Sql, Software Engineering, SQL Databases, Strategies of Testing, Enterprise Data Management, Data Processing, Large Language Models, Snowflake, Multi-Agent Systems, Prompt Engineering, Event Driven Architecture, Data Lakes, Pyspark, Kubernetes, Information Technology, Data Lineage, Data Management, Virtual Agents, Software Version Control, Serverless Computing, Databricks - **Published:** June 9, 2026 - **Apply:** https://www.dice.com/job-detail/6137f3d6-a544-4eca-a943-4cee1f4ed720 ## About the Role * 10+ years of experience in Data Engineering, Software Engineering, or a related technical discipline. * Extensive hands-on experience with modern cloud data platforms such as Azure Databricks, Snowflake, or similar technologies. * Proven background building AI-driven automation solutions utilizing LLMs, prompt engineering methodologies, and agent orchestration frameworks. * Strong programming expertise in Python and advanced SQL development within production environments. * Practical experience working with Azure Databricks, including notebooks, workflows, jobs, Delta Live Tables (DLT), and related services. * Experience implementing CI/CD and Continuous Testing (CT) pipelines using tools such as GitHub Actions and Azure DevOps. * Demonstrated success creating automated testing frameworks, data validation solutions, or quality assurance processes. * Previous experience supporting highly regulated, compliance-driven, or risk-sensitive environments is advantageous. Technical Skills: * Advanced Python development skills with a strong foundation in software engineering principles, testing methodologies, documentation standards, and source control practices. * Expertise in SQL and PySpark performance tuning and optimization. * Hands-on experience integrating LLM services, AI APIs, prompt engineering techniques, and agent frameworks such as Databricks Agent Framework, LangChain, AutoGen, OpenAI, or Azure OpenAI. * Deep understanding of Azure Databricks, including Delta Lake, workflows, asset bundles, and enterprise-scale data processing. * Experience building workflow automation solutions using orchestration platforms, serverless technologies, or event-driven architectures. * Strong analytical skills with the ability to break down complex business and technical processes into scalable automation components. * Excellent communication skills with the ability to explain technical designs, AI agent architectures, and automation strategies to both technical and non-technical stakeholders. Education: * Bachelor's degree in Computer Science, Software Engineering, Data Science, Information Technology, or a related field. * Master's degree preferred. ## Description We are seeking an experienced Data & AI Automation Engineer to lead the development of intelligent automation solutions within an Azure Databricks ecosystem. This role focuses on leveraging Large Language Models (LLMs), AI agents, and advanced automation techniques to enhance data engineering productivity, accelerate software delivery, and improve data quality processes. The ideal candidate will combine strong data engineering expertise with AI-driven automation capabilities to build scalable solutions that streamline development, testing, governance, and operational activities across the data platform. Working closely with architects, engineers, and platform teams, this individual will help drive innovation through intelligent agent frameworks and automated workflows., * Architect and develop AI-powered agents using LLMs, rule-based logic, or hybrid approaches to automate data engineering operations within Azure Databricks. * Create automation solutions that generate, optimize, and refactor PySpark and SQL code to improve development efficiency and maintainability. * Design intelligent testing and validation agents to automate data quality assessments, reconciliation processes, and QA activities. * Implement automated governance capabilities including metadata management, lineage tracking, compliance monitoring, and policy enforcement. * Develop autonomous CI/CD functions such as test creation, deployment verification, release validation, and recovery/rollback procedures. * Build monitoring and diagnostic agents capable of detecting anomalies, identifying performance issues, and supporting root-cause investigations. * Establish prompt engineering standards, reusable templates, and agent orchestration patterns tailored to enterprise data platforms. * Partner with data engineering and architecture teams to identify automation opportunities and prioritize high-value initiatives. * Define and enforce best practices for AI agent lifecycle management, including development, testing, deployment, observability, and maintenance. * Produce technical documentation, workflow diagrams, operational procedures, and knowledge-transfer materials. * Evaluate emerging AI technologies, agent frameworks, and automation tools to support continuous platform improvement. ## Related Videos - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [How Cisco embraced a DevOps culture within its network engineering team](https://www.wearedevelopers.com/videos/99-how-cisco-embraced-a-devops-culture-within-its-network-engineering-team) - [Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure](https://www.wearedevelopers.com/videos/1532-agentic-ai-from-theory-to-practice-developing-multi-agent-ai-systems-on-azure) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) - [Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact](https://www.wearedevelopers.com/videos/1541-azure-ai-foundry-for-developers-open-tools-scalable-agents-real-impact) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts) - [From Prototype to Production: Build AI Agents with This Free 4-Course Learning Path](https://www.wearedevelopers.com/magazine/655-from-prototype-to-production-build-ai-agents-with-this-free-4-course-learning-path)