> Markdown version of [/jobs/ext/3558879-staff-software-engineer-ml-datasets-data-pipelines](https://www.wearedevelopers.com/jobs/ext/3558879-staff-software-engineer-ml-datasets-data-pipelines). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Staff Software Engineer, ML Datasets & Data Pipelines

- **Company:** WAABI PARTNERS
- **Location:** San Francisco, CA, United States (Remote available)
- **Experience:** Expert
- **Salary:** $148,000.0 - $260,000.0
- **Contract:** Permanent contract
- **Skills:** Airflow, Amazon Web Services, Extract Transform Load (ETL), Data Flow Control, Apache Hadoop, MapReduce, Python (Programming Language), Machine Learning, Tensorflow, Google Cloud, Pytorch, Apache Spark, Deep Learning, Kubernetes, Information Technology, Machine Learning Operations, Lidar, Data Pipelines, Apache Beam
- **Published:** October 2, 2026
- **Apply:** https://www.juju.com/job/16_d3d5b43e6

## About the Role

* BS or MS in Computer Science, Machine Learning, or a related technical field, with 4+ years of industry experience.
 * Proficiency in Python and strong software engineering fundamentals, including experience with deep learning frameworks such as PyTorch, TensorFlow, or JAX.
 * Hands-on experience building distributed ETL and data processing pipelines.
 * Direct experience managing ML pipelines, including dataset management, dataloading, and optimization.
 * Strong understanding of cloud job orchestration, monitoring, and instrumentation best practices.
 * A collaborative, open-minded approach with a passion for tackling hard problems in autonomous technology and a strong willingness to mentor others.

Bonus Points:

 * Experience with optimizing large scale distributed training pipelines and/or highly optimized ML inference pipelines.
 * Experience with MapReduce (Apache Hadoop/Spark) or orchestration frameworks (Apache Airflow, Apache Beam, Google Cloud Dataflow, AWS Step Functions).
 * Experience solving data challenges specific to autonomous driving.
 * Familiarity with linear algebra (projections, transforms) and 3D geometry.
 * Experience working with multimodal sensor data (e.g., LiDAR, RADAR, camera).

## Description

As a Senior/Staff Software Engineer embedded within our Autonomy & Algorithms team, you will build the scalable ML data pipelines necessary to train and evaluate Waabi's autonomous driving platform. Working closely with world-renowned scientists and engineers, you will solve complex data challenges to accelerate our launch of fully driverless vehicles.

You Will..

 * Design and implement data pipelines using real-world driving data and Waabi World (our high-fidelity simulator) to train and evaluate deep learning models.
 * Optimize data formats, caching, and dataloading to drive highly efficient ML training and evaluation at scale.
 * Improve data sampling and composition for deep data introspection to track model performance and uncover critical edge-case scenarios.
 * Champion engineering excellence by writing high-quality, well-structured, and rigorously tested code.
 * Help drive project roadmap planning, prioritization, and delivery.


## Related Videos

- [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow)
- [How to develop an autonomous car end-to-end: Robotic Drive and the mobility revolution](https://www.wearedevelopers.com/videos/22-how-to-develop-an-autonomous-car-end-to-end-robotic-drive-and-the-mobility-revolution)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Photonic Computing: Programming a New Class of AI Accelerators (incl. Live Coding)](https://www.wearedevelopers.com/videos/100196-photonic-computing-programming-a-new-class-of-ai-accelerators-incl-live-coding)
- [Remote Driving on Plant Grounds with State-of-the-Art Cloud Technologies](https://www.wearedevelopers.com/videos/251-remote-driving-on-plant-grounds-with-state-of-the-art-cloud-technologies)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this)