> Markdown version of [/jobs/ext/2946788-data-infra-signals-and-evaluations-ic](https://www.wearedevelopers.com/jobs/ext/2946788-data-infra-signals-and-evaluations-ic). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Infra - Signals and Evaluations (IC)

- **Company:** INTELLIGENCE MATTERS PARTNERS, LLC
- **Location:** San Francisco, CA, United States
- **Contract:** Permanent contract
- **Skills:** Python (Programming Language), Machine Learning, Software Engineering, Reinforcement Learning, Pytorch, Indexer, Industrial Software
- **Published:** September 16, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=7459b90fec158fb7

## About the Role

* Experience in ML evaluation, dataset engineering, reinforcement-learning environments, simulation, scientific ML, test infrastructure, or production AI systems.
 * Strong foundations in statistics, experimental design, benchmark validity, distribution shift, calibration, reward design, and evidence-based acceptance.
 * Fluency in Python and modern machine-learning frameworks, with the software engineering discipline to build tested modules, data contracts, or services.
 * Experience converting qualitative model or agent failures into reproducible datasets, tests, or environment scenarios.
 * Ability to reason across the complete data path and identify how acquisition, transformation, indexing, orchestration, and presentation affect correctness., * Experience with dataset curation, benchmark platforms, deep reinforcement learning, world models, synthetic data, multimodal evaluation, or scientific validation.
 * Experience with PyTorch or JAX, distributed evaluation, experiment tracking, environment frameworks, or model and agent observability.
 * Experience with uncertainty quantification, calibration, out-of-distribution detection, conformal methods, or selective prediction.
 * Experience evaluating models deployed to aircraft, satellites, robotics, industrial systems, or other constrained environments.

## Description

Matter is hiring a Signal, Data, and Evaluation Engineer to build the datasets, benchmarks, environments, and evidence that make our models and AI systems scientifically defensible. Reporting to Ignacio Cases Martin, this individual contributor will work across data curation, simulation and learning environments, model and agent evaluation, and release-quality evidence., * Build versioned dataset workflows for collection, curation, filtering, labeling, ground truth, synthetic data, hard-negative mining, contamination controls, and train-evaluation isolation.

 * Design benchmark suites across perception, multimodal reasoning, physics-informed prediction, retrieval, planning, tool use, world modeling, and customer tasks.
 * Create simulation and environment infrastructure for reinforcement learning, imitation learning, offline learning, model-based learning, and agent training.
 * Define repeatable evaluation methods across accuracy, calibration, generalization, robustness, safety, latency, cost, and operational outcomes.
 * Turn field, mission, and customer failures into versioned datasets, benchmark cases, adversarial tests, and environment scenarios.
 * Build automated promotion gates and communicate clearly what has been tested, what remains unproven, and what evidence supports release., * Model and agent releases are supported by reproducible datasets, benchmarks, and clearly stated evidence.
 * Failures from experiments and deployments become durable test cases that improve future systems.
 * Evaluation results preserve scientific meaning and make capability, uncertainty, and residual risk legible to the team.