Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

4 years minimum

Compensation

$177,000.0 - $247,000.0

Working hours

Regular working hours

Job source

www.juju.com

Tech stack

Artificial Intelligence Data Analysis Computer Engineering Query Languages Python (Programming Language) Mathematical Software SQL Databases Scripting Multi-Agent Systems Prompt Engineering Model Validation Information Technology

Job description

• Measure abuse and harms: develop statistical telemetry and measurement frameworks to detect, monitor, and understand policy violations or emergent harms.
• Scale model evaluation: create a scalable evaluation framework that adapts to changing agentic model capabilities and the safety or risk landscape, grounded in statistics.
• Evaluate safeguards: scale offline and online performance evaluation of safeguards using human-in-the-loop methods and active learning.
• Design experiments and analyses: run controlled experiments and rollouts to measure the impact of policy or risk definition changes and safety mitigations.
• Build reporting and intelligence: develop signals, pipelines, dashboards, and real-time monitoring mechanisms to support interventions and safety planning.

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, Mathematics, or Statistics (or equivalent practical experience).
• Minimum 6 years of work experience in analytics (minimum 4 years with a Ph.D.).
• Experience with data querying languages such as SQL, scripting languages such as Python, and/or statistical or mathematical software such as R., * Experience in frontier AI products or risks, including navigating online adversarial environments in Trust & Safety or Fraud/Risk/Security contexts, including model evals, threat modeling, actor telemetry, and human-in-the-loop review systems.
• Comfort working in fast-paced, high-ambiguity, cross-functional environments, including explaining risk dimensions clearly to policy stakeholders.
• Background working with ambiguous and sparse data to operationalize harm prevalence measurement, causal inference, and root-cause analysis, supported by strong quantitative and statistical foundations.
• Master’s or Ph.D. degree in a quantitative field.
• Experience implementing responsible and ethical AI practices such as risk assessment, bias mitigation, and quality and accuracy reviews.
• Demonstrated ability to integrate AI tools to optimize or redesign workflows and deliver measurable impact, such as efficiency or quality improvements.
• Demonstrated ongoing AI skill development, including prompt/context engineering and agent orchestration, and staying current with emerging AI technologies.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.juju.com

Prepare application

• Draft this with your agent
• Open in Claude
• Open in ChatGPT
• Copy prompt