> Markdown version of [/jobs/ext/4360778-sr-machine-learning-engineer-foundation-models-inference-cloud-os-inference](https://www.wearedevelopers.com/jobs/ext/4360778-sr-machine-learning-engineer-foundation-models-inference-cloud-os-inference). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Sr. Machine Learning Engineer, Foundation Models Inference - Cloud OS & Inference

- **Company:** Apple Inc.
- **Location:** New York, NY, United States
- **Experience:** Expert
- **Salary:** $175,000.0 - $308,500.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, C++ (Programming Language), Cloud Computing, Nvidia CUDA, Python (Programming Language), Machine Learning, OpenAI, Tensorflow, Software Deployment, Systems Architecture, Private Cloud Environment, Pytorch, Large Language Models, Deep Learning, Kubernetes, Information Technology, SGLang, TensorRT, VLLM, Model Inference, Docker
- **Published:** October 3, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=597eb75a99c78eaf

## About the Role

Experience building productions systems in Go or Python.

Strong knowledge of deep learning architectures including Transformers, encoder/decoder models, and multimodal variants.

Experience with inference optimization frameworks such as TensorRT-LLM, vLLM, SGLang, TGI, or Nvidia Triton Server.

Experience authoring custom CUDA kernels using CUDA C++ or OpenAI Triton.

MS in Computer Science, Machine Learning, Artificial Intelligence, Data Science, or a related field.

Minimum Qualifications

5+ years of experience leading complex, ambiguous technical projects from end to end.

Hands-on experience with LLM inference stacks.

Working knowledge of GPU or TPU programming concepts.

Proficiency with PyTorch, JAX, or TensorFlow.

Experience building and operating high-throughput services at large distributed scale.

Proficiency deploying applications on cloud platforms (AWS, GCP, or equivalent) using Kubernetes and Docker.

BS in Computer Science, Machine Learning, Artificial Intelligence, Data Science, or a related field.

## Description

You will work at the intersection of research and production, partnering closely with the Foundation Model Research team and our external partners to bring cutting-edge model architectures from prototype to planetary-scale deployment. You will own hard problems in inference efficiency, hardware/software codesign, systems architecture, and tooling, and help set the technical direction for the engineers around you.

This role sits within CloudOS and Private Cloud Compute (PCC) - Apple's purpose-built, privacy-preserving cloud infrastructure for AI workloads. PCC represents a first-of-its-kind approach to running foundation models in the cloud with verifiable privacy guarantees, and CloudOS is the systems foundation that makes it possible. You will be building and optimizing inference systems on top of this infrastructure, working closely with platform and security teams to deliver both performance and trust at scale.","responsibilities":"Partner with the Foundation Model Research team and our external partners to optimize inference for the latest model architectures across language, vision, and speech.

Design and ship production-grade inference systems serving millions of customers in real time.

Build profiling tools and simulators to identify and resolve performance bottlenecks across different hardware configurations and use cases.

Drive technical decisions on high-throughput, low-latency serving at supercomputing scale.

Mentor and grow engineers across the organization.