> Markdown version of [/jobs/ext/3395067-data-software-engineer-ai-agents](https://www.wearedevelopers.com/jobs/ext/3395067-data-software-engineer-ai-agents). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Software Engineer/ AI Agents

- **Company:** EPAM Systems, Inc.
- **Location:** United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Microsoft Azure, Data as a Services, Information Engineering, Data Transformation, Data Security, Apache Hive, NoSQL, Scala (Programming Language), GitHub Copilot, Large Language Models, Apache Spark, Microsoft Fabric, Pyspark, Data Programming, Data Analytics, Cosmos DB, Software Coding, Azure Synapse Analytics, Data Pipelines, Databricks
- **Published:** September 24, 2026
- **Apply:** https://www.dice.com/job-detail/f411fbbe-8d6d-49a0-8545-2e8fc108831d

## About the Role

We are looking for an experienced Senior Data Software Engineer with strong hands-on expertise in Azure and the Spark ecosystem , primarily focused on building and maintaining data transformation pipelines. The candidate should be comfortable with any Spark-adjacent technology (PySpark, Spark SQL, Scala Spark, Databricks, Synapse, etc.) rather than being locked into one specific flavor. Experience with Microsoft Fabric is a strong plus but not a requirement. The ideal candidate combines great technical skills with leadership capability, contributing to architecture, design, development, and mentoring of engineering teams - preferably in complex enterprise or financial services environments. Responsibilities Lead the design, development, and optimization of scalable data engineering solutions on Azure, using Spark-based processing (PySpark, Spark SQL, Scala, or equivalent) Own end-to-end data transformation pipelines, including ingestion, transformation, storage, and analytics Work with, and governance across large-scale platforms Capability to contribute to solution architecture and technical decision-making Competency in mentoring engineering teams and setting coding standards and best practices Understanding of high-performance data access patterns using Cosmos DB (NoSQL API) English proficiency at an Upper-Intermediate level (B2) or higher Nice to have Hands-on experience with Microsoft Fabric and OneLake (Delta / OpenLake) Familiarity with financial instruments and financial services data Exposure to AI-assisted development tools such as GitHub Copilot and awareness of industry-standard LLMs Knowledge of Data Science fundamentals and collaboration experience with DS teams

## Description

Azure-native data services such as Data Factory, Databricks, and Synapse Support high-performance data access patterns using Cosmos DB (NoSQL API) where applicable Collaborate with data scientists, AI engineers, and product stakeholders to enable data-driven analytics and insights Mentor and guide junior engineers, setting coding standards and best practices Ensure data quality, security, governance, and performance across platforms Contribute to technical decision-making and solution architecture discussions Requirements 3+ years of experience in data engineering roles, preferably within complex enterprise or financial services environments Expertise in Azure cloud data services, including Data Factory, Databricks, and Synapse Proficiency in the Spark ecosystem, including PySpark, Spark SQL, and Scala Spark Background in designing and maintaining end-to-end data transformation pipelines covering ingestion, transformation, storage, and analytics Skills in ensuring data quality, security


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis)
- [Developer Tools for Microsoft Azure](https://www.wearedevelopers.com/videos/450-developer-tools-for-microsoft-azure)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)