Senior Software Engineer, Open Harness Engineering

This role will focus on building and improving evaluation infrastructure for agentic systems, with an emphasis on reliable measurement, reproducible experimentation, and scalable execution. You will work across engineering teams to diagnose failures, analyze benchmark results, and strengthen the tooling needed to assess performance in complex, nondeterministic environments. The position also includes contributing reusable capabilities upstream and helping create maintainable documentation that supports broader adoption., * Define and improve the approach for evaluating agent harnesses across multiple frameworks.

• Build and manage scalable evaluation systems that incorporate CI and sandboxed execution.
• Diagnose and analyze agent failures from logs and benchmarks.
• Design controlled experiments to identify genuine improvements beyond noise and anomalies.
• Deliver focused improvements, including upstream contributions and maintainable documentation.
• Collaborate with cross-functional teams to address optimization and runtime needs through reusable capabilities.

• Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Applied Math, or a related field, or equivalent experience.
• 8+ years of hands-on software engineering experience with proven leadership in architecture design.
• Expert Python skills and proficiency in Rust, Go, C++, or TypeScript.
• Experience building or extending large language model agents, evaluation infrastructure, and developer tools.
• Solid grasp of asynchronous execution, networking, and distributed compute frameworks.
• Experience crafting reproducible experiments and performance investigations under nondeterministic conditions.
• Record of contributions to open-source software with strong testing and documentation practices.

• Comprehensive benefits package for employees and their families.
• Equity eligibility as part of compensation.
• Highly competitive salaries within the industry.

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification-your background could be exactly what this team needs.