Sr Data Engineer

As a Senior Data Engineer on the (Input Required) team at Optum Technology, you will lead the design and development of scalable, cloud-native data pipelines and machine learning data architectures on Azure and Databricks. In this role, you will build data integration routines using PySpark, Delta Lake, and Azure Data Factory, while collaborating with Data Science and AI teams to operationalize ML models, feature stores, and streaming analytics frameworks. You will leverage enterprise-approved AI tools to automate workflows, establish reusable engineering accelerators, and ensure data quality, security, and responsible AI implementation across high-volume healthcare data ecosystems., * Design and develop scalable ETL/ELT data pipelines using Databricks, Apache Spark, and Python on Azure cloud platforms

• Build cloud-native data solutions leveraging Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), and Delta Lake architecture
• Collaborate with Data Science and AI teams to operationalize machine learning models, feature stores, model input pipelines, and real-time/streaming frameworks
• Design, develop, and deploy AI-powered solutions to address complex business challenges with an emphasis on responsible use of AI
• Build reusable frameworks, engineering accelerators, and automation scripts to improve engineering efficiency and delivery quality
• Ensure data quality, governance, security, and compliance across the complete data lifecycle
• Optimize the performance, scalability, and cost-efficiency of batch and streaming data workflows, including Databricks Delta Live Tables (DLT)
• Participate in Agile development practices (sprint planning, code reviews, CI/CD automation with GitHub Actions or Azure DevOps), and provide production support/on-call coverage to ensure system stability
• Leverage enterprise-approved AI tools to streamline software engineering workflows, automate tasks, and drive continuous operational improvement
• Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI

You’ll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.

• 4+ years of experience in Data Engineering developing with Python and PySpark
• 4+ years of experience building scalable ETL/ELT data pipelines using Databricks
• 4+ years of experience working within Agile/Scrum project environments
• 4+ years of hands-on experience with Azure Databricks and Delta Lake architecture
• 4+ years of experience implementing CI/CD pipelines using GitHub Actions or Azure DevOps
• 3+ years of hands-on experience with Azure cloud services, including Azure Data Factory (ADF) or Azure Data Lake Storage (ADLS)
• 2+ years of experience with unit testing, data validation, and pipeline testing frameworks, * Bachelor’s or higher degree in Computer Science, Engineering, Information Technology, or a related field
• Experience building, operationalizing, or deploying machine learning models, feature stores, and AI-powered data pipelines
• Experience with real-time and streaming analytics frameworks (e.g., Spark Structured Streaming, Event Hubs, Kafka)
• Knowledge of enterprise data governance, security masking, access control policies, and data quality frameworks
• Experience building reusable automation frameworks, Docker containers, or Kubernetes orchestration workflows
• Strong self-starter with a proven track record of managing technical deliverables independently and taking end-to-end ownership
• 4+ years of experience writing complex SQL / PL-SQL queries and data transformations
• 2+ years of experience working with Databricks Delta Live Tables (DLT)
• All Telecommuters will be required to adhere to UnitedHealth Group’s Telecommuter Policy.

Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you’ll find a far-reaching choice of benefits and incentives. The salary for this role will range from $91,700 to $163,700 annually based on full-time employment. We comply with all minimum wage laws as applicable.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, and data they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits, and career development opportunities. Come make an impact on the communities we serve as you help us advance health equity on a global scale. Join us to start Caring. Connecting. Growing together.