LLM Red Team Specialist for AI Model Evaluation

Design and execute adversarial, multi-step probes that reveal where frontier language models appear competent but quietly fail. You will create short, focused challenge tasks, run experiments to reproduce failures, and collaborate closely with researchers to convert findings into robust evaluation benchmarks. Key Responsibilities

• Probe models, exploring behavior on coding, machine learning, and analytic tasks to find subtle errors and failure modes.
• Design challenge tasks that surface weaknesses, are difficult for models, and remain fair to grade.
• Run reproducible experiments, capture evidence, and write clear, actionable findings that others can reproduce.
• Work with task authors to close loopholes, eliminate shortcuts, and tighten grading criteria.
• Share insights with researchers and colleagues to iteratively improve benchmarks and evaluations.

• MSc or PhD in a STEM field, or equivalent practical experience in a research-heavy role involving data analysis and coding.
• At least 1 year of experience in research, research engineering, security, or AI evaluation.
• Proven ability to identify vulnerabilities, edge cases, or failure modes in large language models or ML systems, through red teaming, adversarial testing, security research, or rigorous model evaluation.
• Working proficiency in Python and Git, with the ability to script probes and analyses independently.
• Strong familiarity with LLM capabilities, limitations, and common evaluation techniques.
• Preferred experience in AI model training, model evaluation, or benchmark and task authoring.
• High attention to detail, creativity in finding what others miss, strong written communication, and ability to work independently on ambiguous, open-ended problems.
• Capacity to engage reliably for approximately 35 hours per week.

Work Terms

• Employment type, pay frequency, and compliance are handled via W-2 employment through an employer-of-record that administers payroll, benefits, and onboarding.

• Role is full-time, structured, and integrated into the client laboratory team, not a freelance or task-based gig.
• Position is fully remote within the United States.
• Typical engagement is approximately 35 hours per week.
• Individual evaluation tasks generally represent one to two days of continuous, focused effort.
• Placement is as part of the client lab’’s extended workforce and involves close collaboration with internal researchers.

Compensation

• Hourly rate: $60.00 to $90.00 per hour.

Eligibility

• Candidates must be eligible for W-2 employment within the United States, as hiring and payroll are administered by the employer-of-record.

• The employer is an equal opportunity employer and provides reasonable accommodations for qualified individuals with disabilities., + $40.00-55.00 per hour In this role, you’‘ll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world inp…

• 1 month ago +