AI System Performance & Benchmaking Engineer -- W2 Profiles -- Onsite

• Review AgentSysPerf’s architecture end to end: benchmark adapters, measurements, analyzers, storage, replay, reporting and dashboards.
• Review state persistence, container lifecycle, concurrency and the AgentInvoker protocol implementation.
• Integrate up to 4 new agentic benchmarks (e.g., τ-bench, BrowseComp, GAIA, BFCL). This includes configuration, CLI execution, model/runtime integration, metrics capture and results persistence.
• Measure latency, throughput and resource use on Intel hardware.
• Improve run variance, repeatability, fault handling and SQLite write concurrency during large sweeps.
• Add new schema fields (policy hashes, topology IDs, route decisions, resource placement) to the storage and telemetry pipelines.

• Python, Linux internals, Docker, cgroups and Linux perf tools.
• Hardware profiling.
• SQLite and OTLP/OpenTelemetry exporters.
• Benchmarking LLMs or agents, including inference engines and runtime configurations.

Nice to have:

• Xeon platform tuning.
• Terminal-Bench.
• Experience with AgentSysPerf itself.