> Markdown version of [/jobs/ext/1327646-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/1327646-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Resource Optimization & Innovation, L.L.C. - **Location:** San Francisco, CA, United States - **Experience:** Expert - **Salary:** $165,000.0 - $220,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Airflow, Amazon Web Services, Amazon S3, Apache HTTP Server, Clinical Data Repository, Data as a Services, Information Engineering, Data Infrastructure, Python (Programming Language), PostgreSQL, Standard Sql, Software Deployment, Large Language Models, Apache Spark, Electronic Medical Records, Fastapi, Pyspark, Kubernetes, Information Technology, Performance Monitor, Machine Learning Operations, Vertica, Data Pipelines - **Published:** July 18, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=3b04521ddf59dfb3 ## About the Role * Bachelors degree in Computer Science, Mathematics, Statistics, or a related field, or equivalent practical experience * 5+ years of experience in data engineering roles * 3+ years of experience using PySpark to build data pipelines * 3+ years of experience in public cloud provider technologies (AWS tooling such as S3, EMR, or Athena) * Strong proficiency in Python and SQL * Hands-on experience across the full data stack, with particular depth in data modeling and pipeline design * Practical experience with LLM-assisted development, with an understanding of its capabilities and limitations * Willingness to participate in on-call operational support for owned systems, * Experience with one or more of the following technologies: Apache Iceberg, Dagster, Clickhouse, PostgreSQL, FastAPI, Metabase * Experience working with healthcare data, including HIPAA compliance, data de-identification, and familiarity with open data standards such as OMOP CDM * Experience building and supporting data pipelines for ML workflows, including model training, validation, deployment, and ongoing performance evaluation ## Description As a Senior Data Engineer at Regard, you will own the design, development, and production deployment of the data services that power the Regard platform. From ingesting and standardizing clinical data across health systems to making it reliably available for downstream product, analytics, and machine learning workflows, you'll build and evolve the infrastructure that enables the platform. This includes analyzing and tuning Spark workloads and partitioning strategies to control costs, adapting to upstream breaking changes, and enforcing rigorous data quality standards so our analytics are as dependable as our application code. We prioritize transparent, code-driven systems over black-box services, and you'll help architect the data platform that supports that philosophy. About Regard Our mission is to bring world-class healthcare to everyone. Regard is an AI-powered Proactive Documentation platform that advances how care is delivered by reviewing all patient data in the EHR to recommend diagnoses and surface clinical evidence. Regard drafts a note even before the physician sees the patient, enabling an approach that gets documentation right at the point of care - we call it Proactive Documentation. This improves quality of care, reduces physician burden, and improves hospital finances. We are excited by challenges, mission-oriented work, and meaningful relationships. We work closely with some of the top health systems in the country and are leading the change that healthcare - one of the largest and most inefficient industries in the world - needs. We want you to join us. Our Tech Stack: * Data: S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, Clickhouse, PostgreSQL, FastAPI, Metabase, * Collect, model, and consolidate data into the data platform to support analytics, ML development, and research initiatives * Design, build, and evolve data models and pipelines that reliably transform and deliver data to downstream consumers * Own data quality in collaboration with engineering teams, ensuring datasets are trustworthy and production-ready * Partner closely with product to deliver analytics and actionable insights to internal and external stakeholders * Own the reliability and day-to-day operation of the data platform and its pipelines through proactive monitoring, alerting, and operational management ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)