> Markdown version of [/jobs/ext/2719107-research-engineer-judgment-systems](https://www.wearedevelopers.com/jobs/ext/2719107-research-engineer-judgment-systems). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Research Engineer, Judgment Systems - **Company:** Variance, Inc. - **Location:** San Francisco, CA, United States - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Computer Programming, Performance Tuning, Reinforcement Learning, Machine Learning Operations - **Published:** September 4, 2026 - **Apply:** https://startup.jobs/research-engineer-judgment-systems-variance-8137619 ## About the Role * Strong programming skills and comfort working in research-heavy codebases * Familiarity with LLMs, agent systems, post-training, reinforcement learning, retrieval, or adjacent areas * Ability to design clean experiments and draw reliable conclusions from noisy results * Strong engineering judgment and a bias toward building * Interest in fraud, risk, trust and safety, compliance, or other regulated and adversarial domains Our culture We believe in ownership, urgency, and craft. We enjoy spirited debate, wild ideas, and building things we're proud of. We're fully in-person in San Francisco. ## Description * Care deeply about protecting people from fraud, scams, and abuse * Have strong opinions about model quality, evaluation, and experimental rigor * Want to work on core model and agent behavior * Are excited to train, fine-tune, and improve models for hard real-world judgment tasks * Think in tight research loops: hypothesis, experiment, evaluation, failure analysis, iteration * Thrive in ambiguous, fast-moving environments where the path is not obvious and the feedback loop is short * Are motivated by the challenge of making AI systems work in adversarial, regulated, and high-consequence settings * Want to help define what trustworthy AI means in real-world use cases What you'll do * Train, fine-tune, and improve models for fraud, scams, abuse, and other high-stakes judgment workflows * Own research threads focused on improving agent capability, reliability, and decision quality * Build proprietary benchmarks, datasets, and evals that reflect real customer workflows, regulatory constraints, and real failure modes * Design and run experiments across post-training, retrieval, tool use, planning, memory, and long-horizon agent behavior * Study where models break, why they break, and how to make them more robust * Prototype new training strategies, agent architectures, and evaluation methods, then turn the best ideas into production systems * Work closely with founders and engineering to translate research advances into deployed product capabilities * Push the boundary of what AI agents can do in regulated industries What success looks like * Our models get materially better at making hard judgment calls in production * Our models are trusted at scale * We develop evals and training loops that compound over time * We understand failure modes more clearly and improve system behavior faster * New research ideas turn into real product capabilities quickly ## Related Videos - [Practical performance tuning for Serverless Java on AWS](https://www.wearedevelopers.com/videos/2075-practical-performance-tuning-for-serverless-java-on-aws) - [Photonic Computing: Programming a New Class of AI Accelerators (incl. Live Coding)](https://www.wearedevelopers.com/videos/100196-photonic-computing-programming-a-new-class-of-ai-accelerators-incl-live-coding) - [On the straight and narrow path - How to get cars to drive themselves using reinforcement learning and trajectory optimization](https://www.wearedevelopers.com/videos/205-on-the-straight-and-narrow-path-how-to-get-cars-to-drive-themselves-using-reinforcement-learning-and-trajectory-optimization) - [You are not an AI developer](https://www.wearedevelopers.com/videos/1148-you-are-not-an-ai-developer) - [The LLM Evolution: From Sequence Imitation to Verifiable Reasoning](https://www.wearedevelopers.com/videos/100343-the-llm-evolution-from-sequence-imitation-to-verifiable-reasoning) - [Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated](https://www.wearedevelopers.com/videos/100099-tour-de-force-open-source-llm-inference-optimization-from-simple-to-sophisticated) ## Related Articles - [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security](https://www.wearedevelopers.com/magazine/659-dev-digest-196-ai-killed-devops-llm-political-bias-ai-security) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)