WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

Member of Technical Staff (Integration/RL Team, Research Engineer)

Cohere
Paris, France
11 days ago
Apply on www.welcometothejungle.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source
www.welcometothejungle.com

Tech stack

Clean Code Principles Data Infrastructure Software Debugging Python (Programming Language) Machine Learning Tensorflow Software Engineering Test-Driven Development (TDD) Pytorch Large Language Models Kubernetes Production Code
+1 more
MLIR (Multi-Level Intermediate Representation)

Job description

  • The integration team is responsible for developing and scaling machine learning algorithms and infrastructure for LLM post-training, with a focus on large-scale, distributed RL methods
  • We strive for excellence in both engineering and science by meticulously designing experiments and design docs
  • While tasks are assigned according to everyone’s expertise, there is a global team effort to write production code and support the team research efforts, depending on individual interests and organizational needs
  • In particular, this role aims to enhance the global quality of the post-training codebase by implementing new tools to ease and support research, optimizing post-training algorithms, and scaling distributed RL to unprecedented levels
  • Design and write high-performing and scalable software for training models
  • Develop new tools to support and accelerate research and LLM training
  • Coordinate with other engineering teams (Infrastructure, Efficiency, Serving) and the scientific teams (Agent, Multimodal, Multilingual, etc.) to create a strong and integrated post-training ecosystem
  • Craft and implement techniques to improve performance and speed up our training cycles, both on SFT, offline preference, and the RL regime
  • Research, implement, and experiment with ideas on our cluster and data infrastructure
  • Collaborate, Collaborate, and Collaborate with other scientists, engineers, and teams!

Requirements

  • [Bonus] Experience in ML, LLM and RL academic research
  • Extremely strong software engineering skills
  • Value test-driven development methods, clean code, and strive to reduce technical debts at all levels
  • [Bonus] Experience with distributed training infrastructures (Kubernetes) and associated frameworks (Ray)
  • Proficiency in Python and related ML frameworks such as JAX, Pytorch and/or XLA/MLIR
  • [Bonus] Hands-on experience with the post-training phase of model training, with a strong emphasis on scalability and performance
  • Experience using and debugging large-scale distributed training strategies (memory/speed profiling)
  • Have a deep passion for quality work
  • Enjoy tuning and optimising large LLM models
  • Comfortable diving into complex ML codebases to identify and resolve issues, ensuring the smooth operation of our systems
  • Comfortable working with people with different levels of software engineering skills, from beginner to more advanced
  • Thrive in a fast-paced, technically challenging environment, where you can contribute your innovative ideas and solutions

Benefits & conditions

  • Six weeks’ paid vacation
  • Equity / stock options
  • RRSP, 401(k), and Pension Scheme contributions
  • Coverage for 100% of your insurance premiums across health, dental, vision, and travel
  • Additional coverage for accessing mental health providers/services
  • Six months of fully paid parental leave, including adoption and surrogacy
  • Financial support for egg freezing and IVF in Canada and the UK
  • A monthly fitness and wellness allowance
  • Globally dispersed company that supports a remote work culture
  • A $2,000 annual education benefit for professional development
  • A weekly stipend for meals when working remotely and catered lunch when working from one of our global offices
  • A monthly arts and culture allowance
  • A monthly quality time allowance

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.welcometothejungle.com
Prepare application

Good distractions

Loading talks and stories from around this role…