> Markdown version of [/jobs/ext/3543543-sr-data-engineer](https://www.wearedevelopers.com/jobs/ext/3543543-sr-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Sr Data Engineer

- **Company:** Optum, Inc
- **Location:** Eden Prairie, MN, United States (Remote available)
- **Experience:** Expert
- **Salary:** $91,700.0 - $163,700.0
- **Contract:** Permanent contract
- **Skills:** Agile Methodology, Artificial Intelligence, Business Analytics Applications, Unit Testing, Microsoft Azure, Code Review, Continuous Delivery, Continuous Integration, Data Architecture, Data Validation, Information Engineering, Data Governance, Data Integration, Extract Transform Load (ETL), Data Transformation, Github, Information Lifecycle Management, Python (Programming Language), Machine Learning, Scrum Methodology, Cloud Services, Standard Sql, Azure Data Lake, Software Engineering, PL-SQL, Data Streaming, Azure Service Bus, Feature Store, Azure Data Factory, Apache Spark, Data Lakes, Pyspark, Kubernetes, Information Technology, Apache Kafka, Spark Streaming, Data Pipelines, Docker, Databricks
- **Published:** October 1, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=97fa3ea752ece8fd

## About the Role

* 4+ years of experience in Data Engineering developing with Python and PySpark
 * 4+ years of experience building scalable ETL/ELT data pipelines using Databricks
 * 4+ years of experience working within Agile/Scrum project environments
 * 4+ years of hands-on experience with Azure Databricks and Delta Lake architecture
 * 4+ years of experience implementing CI/CD pipelines using GitHub Actions or Azure DevOps
 * 3+ years of hands-on experience with Azure cloud services, including Azure Data Factory (ADF) or Azure Data Lake Storage (ADLS)
 * 2+ years of experience with unit testing, data validation, and pipeline testing frameworks, * Bachelor's or higher degree in Computer Science, Engineering, Information Technology, or a related field
 * Experience building, operationalizing, or deploying machine learning models, feature stores, and AI-powered data pipelines
 * Experience with real-time and streaming analytics frameworks (e.g., Spark Structured Streaming, Event Hubs, Kafka)
 * Knowledge of enterprise data governance, security masking, access control policies, and data quality frameworks
 * Experience building reusable automation frameworks, Docker containers, or Kubernetes orchestration workflows
 * Strong self-starter with a proven track record of managing technical deliverables independently and taking end-to-end ownership
 * 4+ years of experience writing complex SQL / PL-SQL queries and data transformations
 * 2+ years of experience working with Databricks Delta Live Tables (DLT)
 * All Telecommuters will be required to adhere to UnitedHealth Group's Telecommuter Policy.

## Description

As a Senior Data Engineer on the (Input Required) team at Optum Technology, you will lead the design and development of scalable, cloud-native data pipelines and machine learning data architectures on Azure and Databricks. In this role, you will build data integration routines using PySpark, Delta Lake, and Azure Data Factory, while collaborating with Data Science and AI teams to operationalize ML models, feature stores, and streaming analytics frameworks. You will leverage enterprise-approved AI tools to automate workflows, establish reusable engineering accelerators, and ensure data quality, security, and responsible AI implementation across high-volume healthcare data ecosystems., * Design and develop scalable ETL/ELT data pipelines using Databricks, Apache Spark, and Python on Azure cloud platforms

 * Build cloud-native data solutions leveraging Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), and Delta Lake architecture
 * Collaborate with Data Science and AI teams to operationalize machine learning models, feature stores, model input pipelines, and real-time/streaming frameworks
 * Design, develop, and deploy AI-powered solutions to address complex business challenges with an emphasis on responsible use of AI
 * Build reusable frameworks, engineering accelerators, and automation scripts to improve engineering efficiency and delivery quality
 * Ensure data quality, governance, security, and compliance across the complete data lifecycle
 * Optimize the performance, scalability, and cost-efficiency of batch and streaming data workflows, including Databricks Delta Live Tables (DLT)
 * Participate in Agile development practices (sprint planning, code reviews, CI/CD automation with GitHub Actions or Azure DevOps), and provide production support/on-call coverage to ensure system stability
 * Leverage enterprise-approved AI tools to streamline software engineering workflows, automate tasks, and drive continuous operational improvement
 * Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI

You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models)
- [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)