> Markdown version of [/jobs/ext/3520783-ai-ml-data-scientist](https://www.wearedevelopers.com/jobs/ext/3520783-ai-ml-data-scientist). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# AI/ML Data Scientist

- **Company:** Ai-native
- **Location:** London, UK (Remote available)
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** A/B Testing, Artificial Intelligence, Amazon Web Services, Cloud Computing, Relational Databases, Python (Programming Language), PostgreSQL, NumPy, Next.js, SQL Databases, Large Language Models, Information Technology, Atlassian Tools, Free and Open-Source Software, Machine Learning Operations
- **Published:** October 1, 2026
- **Apply:** https://www.apply4u.co.uk/jobs/aiml-data-scientist/49378011

## About the Role

failures end to end: find where the AI falls down on notation, working, partial credit, or a specific exam board, and turn that into a fix - Work with our learning developers and teachers to turn pedagogical judgement into something measurable - Get hands-on with our product data: spot patterns, surface insights, and shape what we build next - Contribute to the wider codebase where it makes sense (Python, Postgres, AWS, some Next.js) What we're looking for - At least 2 years in a data science, ML, or applied research role, post-graduation - Hands-on experience evaluating or benchmarking AI/ML systems: you've built evals, designed metrics, or run structured model comparisons, and you can talk about where your own metrics misled you - Strong Python for data work (pandas, numpy, notebooks, and the ability to write code other people can run) - Confident with SQL and relational databases - Solid grounding in statistics and experiment design: sampling, significance,
confidence intervals, and knowing when a result doesn't mean what it looks like - Working knowledge of LLMs and how they're trained, evaluated, prompted, and fine-tuned - A degree in Computer Science, Maths, Physics, Statistics, Engineering, Data Science, or a similarly quantitative field - Genuine curiosity about how AI systems behave in the wild, not just on a leaderboard - Experience with eval frameworks and LLM-as-judge setups, including their failure modes - Fine-tuning, RLHF, DPO, or preference data collection - Built annotation or human-evaluation pipelines, and managed the people doing the labelling - Worked on AI products in production where quality was subjective and hard to measure - Any teaching, tutoring, or exam-marking experience, or familiarity with the UK curriculum (GCSE, A-Level, AQA/Edexcel/OCR) - Published research, open-source contributions, or writing about evaluation - Cloud infrastructure experience (AWS) - Something
you've built in your own time that you'd love to show us Right to work You must have the right to work in the UK. We're not able to offer visa sponsorship for this role. Who thrives here - You care about quality and take pride in work that holds up under scrutiny - You're comfortable being the person who says the model got worse, with the evidence to back it - Curious about how things work, and willing to dig into other people's code and data to find out - You take initiative. If something looks off, you flag it or fix it rather than waiting to be told - You can explain a technical result to someone who isn't technical, and be trusted on it - Comfortable in a startup environment. Things move very quickly and priorities shift often What you'll get - Real ownership of how Medly measures AI quality, on a product used by hundreds of thousands of students - Direct work with founders and senior engineers with experience at Meta, Atlassian, and
beyond - Hybrid setup: 4 days in our central London office, one day work from home - A team that takes your ideas seriously and ships them - Room to grow into ownership of Medly's wider AI and data function #J-18808-Ljbffr

## Description

exam board. Right now, judging whether a change made that better or worse is the hardest problem we have. You'll build and improve the benchmarks, evals, and experiments that answer it, and you'll be the reason we can ship quickly without breaking what works. You'll have real ownership from day one. This is not a role where you'll be handed a spec. What you'll do - Own our AI benchmarking: help improve and design subject-specific evals that measure tutoring, marking, and content generation quality against what a good teacher would actually say - Improved the eval harnesses and offline test suites that let us ship model and prompt changes with confidence, and catch regressions before students do - Run experiments and A/B tests on live AI features, and make the call on what the results actually mean - Build and curate high-quality datasets for fine-tuning and evaluation, including designing labelling schemes and rubrics that other people can apply consistently - Investigate model