> Markdown version of [/jobs/ext/91105-data-engineer-iii](https://www.wearedevelopers.com/jobs/ext/91105-data-engineer-iii). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer III - **Company:** Robert Half - **Location:** San Francisco, CA, United States - **Experience:** Experienced - **Salary:** $104,000.0 - $153,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Automation of Tests, Big Data, Software Quality, Code Review, Encodings, Continuous Integration, Data as a Services, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Security, Data Systems, Data Virtualization, Data Warehousing, DevOps, Distributed Computing Environment, Python (Programming Language), Machine Learning, Systems Development Life Cycle, Salesforce.Com, Search Technologies, SQL Databases, Feature Engineering, Sql Optimization, Large Language Models, Apache Spark, Cloudformation, Build Management, Data Lakes, Heroku, AI Platforms, Kubernetes, Information Technology, Real Time Data, Data Management, Machine Learning Operations, Virtual Agents, Terraform, Data Pipelines, Docker, Databricks - **Published:** May 24, 2026 - **Apply:** https://www.juju.com/job/00000000g279l7 ## About the Role + Bachelor's in Computer Science, Engineering, or related field (Master's preferred) + 5+ years with Python and SQL in data engineering for big data ML/analytics workloads + 5+ years designing, building, and troubleshooting scalable ETL/ELT pipelines for business-critical production systems + 3+ years with cloud data services (AWS), container orchestration (Docker, Kubernetes), and IaC (Terraform, CloudFormation) + 3+ years architecting ML workflows and data platforms with CI/CD, automated testing, and distributed processing (Spark) + 3+ years collaborating cross-functionally with Data Science, MLOps, Platform Engineering, and DevOps teams + 3+ years implementing data quality testing and optimizing SQL/Python for cost/performance in the cloud + Understanding of the full Data Science SDLC, and experience mentoring engineers Strongly Preferred - Databricks & AI Platform + 2+ years hands-on with Databricks (Delta Lake, Unity Catalog, Databricks SQL) + Experience with MLflow experiment tracking and model registry workflows + Experience designing pipelines that serve AI/ML inference - real-time feature engineering, embedding generation, and context retrieval for LLM-based systems + Understanding of how data engineering supports Agentic AI: agent-accessible data services, low-latency retrieval, and pipelines enabling autonomous multi-step workflows + Familiarity with Databricks Mosaic AI, Vector Search, and/or Feature Store + FinOps awareness - compute cluster optimization, cost attribution by workload Nice to Have + Familiarity with Salesforce/Heroku data infrastructures + Experience with data virtualization (e.g., Dremio) + Understanding of Platform Engineering concepts and internal developer platforms + Experience migrating from legacy data warehouse/lake to unified lakehouse architecture + Familiarity with Odaseva data security and management ## Description + Define technical approach for data engineering initiatives, mentor less-senior engineers, and set standards for code quality through leadership and code reviews + Design and build data foundations that enable AI/ML capabilities - feature stores, embedding pipelines, vector search indexes, and model training datasets + Align data engineering solutions with business strategy, including support for Agentic AI workloads Data Infrastructure & Platform + Own health, scalability, and modernization of data infrastructure with Databricks as the strategic platform - including workload migration, compute optimization, and Unity Catalog adoption + Optimize pipeline performance (Delta Lake table layouts, clustering, Z-ordering) and establish monitoring/alerting best practices with clear SLAs + Build data infrastructure supporting Agentic AI systems - real-time data access layers, context retrieval pipelines, and agent-accessible data services + Collaborate cross-functionally with DevOps, Platform Engineering, and MLOps roles to integrate data solutions into the broader technology environment and shared AI infratstructure - Mlflow registries, feature stores, and agent orchestration layers + Provide consultation to Senior Leadership on complex projects and drive continuous improvement initiatives Data Quality, Governance & Collaboration + Champion data governance at all layers for data, models, and AI assets + Implement data quality strategies (master data management, validation rules, Delta Live Tables expectations) to ensure trust in enterprise data + Serve as liaison across data engineering, AI engineering, and business teams; promote data literacy and stewardship ## Related Videos - [Coffee with Developers - Adam Wiggins](https://www.wearedevelopers.com/videos/919-coffee-with-developers-adam-wiggins) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)