Senior Machine Learning Engineer, Inference

United States Agency For International Development
Mountain View, CA, United States
15 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$250,000 - $300,000
Working hours
Regular working hours
Languages
English

Tech stack

Artificial Intelligence Systems Engineering C++ (Programming Language) Nvidia CUDA Distributed Systems Python (Programming Language) Machine Learning Open Source Technology Software Deployment System Programming Rust (Programming Language) Graphics Processing Unit (GPU)
+9 more
Load Balancing Large Language Models Caching Backend Discretization Kubernetes Information Technology Machine Learning Operations Model Inference

Job description

For our client, a fast-growing AI company building realtime voice and LLM technology, we are looking for a Senior Machine Learning Engineer with strong expertise in ML inference optimization to join its inference team.

In this role, you will own inference optimization end-to-end - taking models from research to production at consumer scale and solving latency and throughput problems for realtime AI products used by hundreds of millions of people.

Our client is a well-funded, venture-backed AI research lab and realtime inference provider with a small, senior engineering team.

Team

Inference: The team owns how our client’s models run in production - inference engines, GPU efficiency, model acceleration and the distributed serving infrastructure behind realtime voice and LLM products.

This is an opportunity for senior engineers who want to work deep inside modern inference engines on problems at real consumer scale.

Responsibilities

  • Own inference optimization end-to-end, from research models to production deployment;
  • Optimize latency and throughput of realtime voice and LLM inference at consumer scale;
  • Work inside inference engines such as vLLM, TRT-LLM and SGLang to improve performance;
  • Apply model acceleration techniques - quantization, distillation, speculative decoding, caching, continuous batching and paged attention;
  • Write high-performance code in C++, CUDA, Rust or optimized Python for GPU workloads;
  • Build and scale distributed multi-GPU and multi-node serving with Kubernetes, Ray and custom load balancing for high concurrency;
  • Collaborate with research and product teams to bring new models into production., * Stage 2: Pre-Screening. A short Q/A session to assess your experience and technical background. You can choose between an automated Q/A video session (recommended for faster feedback) or a recruiter interview.
  • Stage 3: Shortlisting. Candidates whose experience best matches the role will be shortlisted and presented to the hiring manager., * We aim to keep candidates informed throughout the process and provide updates after each completed stage.

Requirements

Machine Learning, Machine Learning, Artificial Intelligence (AI), Problem Solving Skills, Research Laboratory, Inference Engine, GPU (Graphics Processing Unit), Voice Products, Performance Management, Caching, * 5+ years of experience as an ML inference or systems engineer, at senior level (Google L5 / Meta E5 or equivalent);

  • Hands-on ML inference optimization inside vLLM or TRT-LLM (engine-level work, not only deployment);
  • Experience building backend/ML systems and serving models in production;
  • Systems programming and GPU optimization skills: C++, CUDA, Rust or optimized Python;
  • Experience with model acceleration: quantization, distillation, speculative decoding, caching, continuous batching, paged attention;
  • Distributed systems experience: Kubernetes, Ray, load balancing, multi-GPU/multi-node inference, high concurrency;
  • Based in the US, Canada, UK, Germany, Switzerland or Serbia;
  • Excellent communication skills in English.

Nice to have

  • A track record of fast promotions;
  • Public technical work - open-source inference engine contributions or technical write-ups;
  • PhD in Computer Science, Physics or Mathematics, or a degree from a top CS program;
  • Ability to commute to Mountain View, CA.

About the company

AIDA Recruitment - a partner for technology-sector companies. We recruit IT specialists from all around the world for our clients. We use digital technology to work faster, be more efficient, and provide the best solutions for both clients and candidates.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…