Senior Machine Learning Engineer

• LLM QA - the core technology behind Toloka’s automated quality-check mechanism. Every annotation flowing through Self-Service is reviewed by an LLM agent we design, train, and operate.
• Model distillation and fine-tuning - adapting frontier and open-source models to Toloka’s tasks to hit the right quality at the right cost.
• Evaluation, benchmarking, cost modeling, and model selection across providers.

We own the full chain. The same team designs the ML solution, ships it to production, keeps it running 24/7, analyzes the results coming back from real projects, and feeds that signal into the next iteration. No hand-off between research, engineering, and operations - it’s all us.

About the Position

As a Senior ML Engineer, you will design, train, and deploy the AI agents that drive our core products. You will focus on end-to-end ML tasks-including fine-tuning and Reinforcement Learning (RL)-to build resilient agentic workflows in Python. You will own the full lifecycle of your models, closing the loop from research and benchmarking to production scaling and monitoring.

What you’ll do

• Train, fine-tune, and distill ML models (including RL approaches) to power autonomous AI agents.
• Build and operate agentic workflows in Python, handling complex reasoning and hybrid human-expert interactions.
• Own evaluation and benchmarking, selecting foundational models and establishing cost models.
• Manage the full ML lifecycle: design solutions, ship to production, and monitor real-time signals.
• Implement observability metrics tailored for agent logic, model performance, and system reliability.

• 3+ years of experience in ML: Strong background in model training, fine-tuning, and Reinforcement Learning (RL);
• 1+ year in Agent Development: Practical experience building, evaluating, and launching autonomous AI agents;
• Agentic Frameworks: Proven experience working with frameworks such as LangChain, LlamaIndex, or AutoGen;
• Open-Source LLMs: Practical knowledge of model distillation and adapting open-source models (e.g., Llama, Mistral);
• Python Mastery: Advanced proficiency in Python with a drive to apply disciplined software engineering standards to ML;
• End-to-End Ownership: Ability to work across the entire chain, from research to production operations;
• Language: Fluency in English (B2 or above).

• You will be part of an international, dynamic environment that drives innovation and sets new standards in the AI and technology sector.
• Competitive compensation package including base salary, bonus, and ESOP.
• Paid PTO and benefits will vary depending on location.
• We offer a full remote or hybrid model (if you are based in NL or Serbia).
• IT setup and home office allowances.

About Toloka

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains-from doctors and lawyers to physicists and engineers-and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients including Anthropic, Amazon, Microsoft, Poolside, Recraft, and Shopify.

Recently, we secured strategic investment led by Bezos Expeditions and Nebius Group with participation from Mikhail Parakhin, CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands, Serbia, and more.

About the Team

We are the ML team inside Toloka - we build the machine-learning products that power the platform itself, so every project running on Toloka is faster, cheaper, and more reliable.

A few examples of what we own