> Markdown version of [/jobs/ext/3606794-senior-deep-learning-engineer-accuracy-evaluation](https://www.wearedevelopers.com/jobs/ext/3606794-senior-deep-learning-engineer-accuracy-evaluation). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Deep Learning Engineer, Accuracy Evaluation

- **Company:** NVIDIA Ltd.
- **Location:** Santa Clara, CA, United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Code Generation, Statistical Hypothesis Testing, Machine Learning, Regression Analysis, Open Source Technology, Large Language Models, Deep Learning, Discretization, Build Management, Information Technology, Slurm, Machine Learning Operations
- **Published:** October 7, 2026
- **Apply:** https://arc.dev/remote-jobs/j/redirect/ppldlsvsc5

## About the Role

* BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field
 * 6+ years of hands-on experience with LLMs, designing and running evaluations for large language models or multimodal AI systems, including experience with agentic, multi-turn, or reasoning-heavy settings.
 * Strong statistical foundations: experimental design, significance testing, regression analysis, and the ability to distinguish signal from noise in benchmark results at scale.
 * Proven experience building evaluation infrastructure (pipelines, benchmark harnesses, reproducible CI systems) not just consuming existing benchmarks.
 * Clear, precise communicator who can translate quantitative evaluation results into decisions for researchers, product teams, and senior leadership.

Ways To Stand Out From The Crowd

 * Deep familiarity with open-source evaluation frameworks
 * Experience designing evaluations for agentic systems: tool use, multi-turn reasoning, environment-based benchmarks (SWE-bench, GAIA, WebArena-style), or interactive evaluation settings.
 * Track record of publishing or contributing to evaluation research, new benchmark design, methodology papers, or reproducibility analyses that shaped how the field measures model capability.
 * Experience measuring model accuracy under low-precision inference (FP8, INT4, quantization-aware settings) and understanding how calibration and sparsity interact with benchmark results.
 * Comfort running large-scale workloads on HPC/Slurm clusters, including reproducible experiment management (MLflow, W&B) and compute cost optimization across hundreds of benchmark runs.

## Description

* Design and build decision-grade evaluation environments for NVIDIA's frontier models spanning reasoning, multimodal, long-context, and agentic systems, producing auditable accuracy signals that gate every major model release.
 * Research and develop novel evaluation methodologies for emerging model families and capability domains (low-precision numerics, multi-turn agentic tasks, code generation) where established benchmarks don't yet exist or don't generalize.
 * Build and operate the evaluation infrastructure and pipelines including benchmark environments, regression CI systems, and statistical analysis tooling, used by model, product, and applied research teams across NVIDIA.
 * Partner with model research, training, and customer teams to translate evaluation signals into concrete decisions: release go/no-go, training iteration direction, and competitive positioning against external frontier models.