Software Engineer (Model Evaluation & Benchmarking)
Role details
Job location
Tech stack
Job description
We are hiring Engineers focused on AI Model Evaluation to build the systems that ensure multimodal AI behaves reliably, consistently, and predictably as it moves from research into production. This position focuses on evaluating generative and vision-based models through automated benchmarking, dataset-driven testing, and performance validation pipelines.
You will work at the intersection of applied science, infrastructure, and product - helping define how we measure realism, consistency, and quality across image, video, and multimodal AI systems.
Why This Role Exists
Modern AI evaluation extends beyond pass/fail testing. Multimodal generative systems require:
- benchmarking across visual realism, pose consistency, and identity preservation,
- automated regression detection across model checkpoints,
- scalable evaluation pipelines integrated into continuous deployment workflows.
We are building evaluation systems where research velocity and product reliability must coexist. This role is for engineers interested in defining how quality is measured in generative AI systems.
What you'll do
- Build automated evaluation pipelines for multimodal AI models.
- Benchmark diffusion models, vision systems, and generative workflows.
- Validate model checkpoints and detect regressions across versions.
- Develop evaluation metrics for realism, consistency, and performance.
- Integrate evaluation tooling into CI/CD workflows.
- Collaborate with ML researchers and infrastructure teams to ensure production readiness.
- Analyze failure modes and propose evaluation strategies.
Requirements
Do you have experience in Research?, * LLM, VLM, or Stable Diffusion model evals
- Image/Video benchmarking techniques
- Multimodal evaluation frameworks
- dataset-driven testing workflows
- research experiment validation pipelines, * Degree in Computer Science, AI, Engineering, or comparable combination of education and practical experience.
- Strong programming skills in Python.
- Familiarity with object-oriented programming (C++, Java, Python, or similar).
- Strong data structures and algorithms fundamentals.
- Understanding of machine learning experimentation workflows., * Experience evaluating vision or generative models.
- Familiarity with HuggingFace ecosystem or open-source ML toolkits.
- Experience building automated test frameworks or benchmarking tools.
- Knowledge of diffusion models or multimodal architectures.
Experience with data analysis tools (NumPy, Pandas, visualization libraries).