> Markdown version of [/jobs/ext/3998314-staff-machine-learning-performance-engineer](https://www.wearedevelopers.com/jobs/ext/3998314-staff-machine-learning-performance-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Staff Machine Learning Performance Engineer

- **Company:** XPeng Motors
- **Location:** Santa Clara, United States
- **Experience:** Expert
- **Salary:** $244,140.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, C++ (Programming Language), Nvidia CUDA, Data Mining, Data Structures, Python (Programming Language), Machine Learning, Performance Tuning, High Performance Computing, Pytorch, Large Language Models, Agentic-AI, Free and Open-Source Software, Machine Learning Operations
- **Published:** October 28, 2026
- **Apply:** https://startup.jobs/senior-staff-machine-learning-performance-engineer-xpeng-motors-10591589

## About the Role

* Master's or PhD in CS, CE or EE, or equivalent practical experience, with 8+ years of industry experience, the majority of it in performance engineering.
 * Hands-on experience optimizing GPU training or inference workloads in production, at multi-node scale.
 * A track record of diagnosing performance problems in real production workloads and landing the fixes - often in systems owned by other teams - with the gain measured and verified in production.
 * In-depth understanding of computer architecture and high-performance computing.
 * Strong C++ and Python, with a solid command of data structures and algorithms.

Preferred Skill Requirements:

 * Hands-on experience writing and tuning CUDA or Triton kernels.
 * Fluent in PyTorch, with concrete insight into what makes PyTorch code fast or slow - graph breaks, host-side overhead, synchronization points and memory layout.
 * Training at the scale of thousands of accelerators, including multi-node failure modes and long-running job stability.
 * Building profiling, tracing or benchmarking infrastructure that other teams depend on - CUPTI, Perfetto, or equivalent.
 * Using LLM agents to automate parts of an engineering workflow.
 * Autonomous driving, robotics, or another domain with hard latency, power and memory constraints.
 * Publications or open-source contributions in ML systems or performance.

## Description

The Mission: We are building the next generation of end-to-end autonomous driving on physical AI foundation models - vision-language-action policies and world models - and the same training stack carries our robotics program. These training runs span thousands of accelerators and days of wall-clock - how fast we train them is how fast we learn. As a Senior Staff Machine Learning Performance Engineer, you will own that speed: working with the modeling teams on their hardest bottlenecks, setting the technical direction for performance, and turning one-off wins into defaults - including the agentic workflows that find the next round of them automatically., * Training Performance at Scale: Own step time, scaling efficiency and long-run stability for foundation-model training across thousands of accelerators - from the data path to kernels, parallelism and communication.

 * Cross-Team Technical Leadership: Work directly with the modeling, infrastructure, simulation, data-mining and robotics teams on their hardest bottlenecks, and set the performance direction those teams build against.
 * Reusable by Design: Turn one-off wins into modular components that become defaults across most models and workloads, rather than a patch that helps a single job.
 * Observability at Scale: Extend performance tooling and metric coverage across the stack, while keeping instrumentation overhead low enough to stay on for production jobs fleet-wide.
 * Agentic Automation: Build agents that reproduce any job on demand, verify numerical parity and separate a real regression from run-to-run noise - so that most of the optimizations that land are found and fixed by AI rather than by hand.