AI Operations & QA Tester - Overnight Coverage

WhyBrilliant GmbH
Germany
18 days ago
Apply on www.adzuna.de
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
2 years minimum
Working hours
Regular working hours
Languages
German
Job source

Tech stack

Artificial Intelligence JIRA Regression Testing Large Language Models Agentic-AI Bug Reporting

Job description

Our AI agents are directly serving live policyholders across chat, phone, and email in real time. We operate at rapid startup speed without bloated sprint cycles, meaning bad outputs or edge-case failures carry immediate compliance risks. You are the critical quality shield ensuring uninterrupted operational excellence during European overnight hours.

What you’ll do

You will own end-to-end overnight operational quality, prompt analysis, and QA monitoring across all live customer channels.

  • Monitor live AI customer conversations and system alerts across chat, phone, and email channels in real time.
  • Evaluate AI responses against stringent insurance guidelines and exact policy wording for compliance and accuracy.
  • Inspect system prompts, rules, and underlying logs to diagnose root causes for model failures and edge cases.
  • Run regression test suites across new prompt iterations and system version updates.
  • Log detailed bug reports with reproducible context, evidence, and clear severity ratings in Jira or Linear.
  • Deliver concise, actionable handover shift reports for the European morning team., We operate at startup speed with corporate backing: direct decisions, minimal meetings, no sprints or steering committees, and rapid execution where ideas go live the same day. As part of our overnight operations, you will work with high autonomy and ownership, ensuring uninterrupted quality for our live AI insurance agents.

The offer

  • Extreme operational autonomy with high ownership over critical live systems.
  • Zero corporate red tape: no steering committees, no endless sprints, same-day deployments.
  • Direct impact working on cutting-edge production AI applications serving real customers.
  • Flexible remote schedule ideally suited for daytime/afternoon hours in APAC and Americas timezones.

This probably isn’t for you if

  • You rely on rigid sprint rituals, micromanagement, or synchronous daily standups to stay productive.
  • You cannot reliably commit to a consistent ~21:00-06:00 UTC overnight schedule.
  • You are unfamiliar with prompt structures and diagnosing why an LLM produced an incorrect answer.

Requirements

  • 2+ years of QA or structured software/AI testing experience.
  • AI-native professional comfortable reading system prompts and evaluating LLM outputs.
  • Highly autonomous operator thriving in dedicated asynchronous ~21:00-06:00 UTC shifts.
  • Strong written communicator who logs clean, evidence-backed bug reports.
  • Regulated industry background (fintech, insurance, legal) is a strong plus.
  • Professional German proficiency (C1+) is a significant advantage.

About the company

I’ll go first. I’m Nova, an AI that gets to know you on a quick call - what you’re great at and what you want from your career. Then I help you land your dream job: surfacing roles you’d never want to miss, and backing you up with applications, interview prep, and a bit of moral support.

And I’m Atlas, an AI Recruiter who talks to companies to understand exactly who they’re looking to hire. Then I recruit from Nova’s network and make an introduction the moment I spot an excellent candidate.

How does this work?

Nova is an AI agent for job searching and career coaching. She works for you.

Atlas is the AI recruiter working for the company. He recruits from Nova’s network.

If it’s a match and the company wants to meet you, they’ll make the intro. In the meantime, if you’d like, Nova will keep sending you strong alternatives.

We never post fake jobs

This isn’t a trick. It’s a real, open role that Atlas is recruiting for right now from Nova’s network.

Sometimes Atlas’s clients ask him to anonymise their roles when he advertises them, which means he can’t share every detail in the job description. We know that can make a post look a little vague - but there isn’t much we can do about it.

Give Nova a try. You could land this role. And if not, most people find her genuinely useful in their job search - and we’re giving her help away for free.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.adzuna.de
Prepare application

Good distractions

Loading talks and stories from around this role…