> Markdown version of [/jobs/ext/3960514-staff-software-engineer-foundation-model-inference](https://www.wearedevelopers.com/jobs/ext/3960514-staff-software-engineer-foundation-model-inference). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Staff Software Engineer- Foundation Model Inference

- **Company:** Databricks
- **Location:** San Francisco, CA, United States
- **Experience:** Expert
- **Salary:** $190,000.0 - $265,000.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Distributed Systems, Design of User Interfaces, Monitoring of Systems, Human-Computer Interaction, OpenAI, Azure Machine Learning, Service-Oriented Architecture, Software Engineering, Virtual Machines, Cloud Platform System, Pytorch, Delivery Pipeline, Large Language Models, Qwen, Backend, SGLang, Machine Learning Operations, Google Gemini, Restful APIs, GPT, Automation Anywhere, Databricks
- **Published:** October 27, 2026
- **Apply:** https://www.careerbuilder.com/job-details/staff-software-engineer-foundation-model-inference-san-francisco-ca--26636d92-ff42-4d54-9b32-bb3e0b800cb1

## About the Role

Software Engineering, Artificial Intelligence (AI), User Interface/Experience (UI/UX), Virtual Machine (VM), Process Improvement, Use Cases, Business Growth, Product Development, Power Generation, Application Programming Interface (API), We are looking for high-agency engineers who are excited to work on powering model inference at enterprise scale., * 8+ years of experience in backend or infrastructure engineering

 * Experience with distributed systems, scalable APIs, or cloud-native infrastructure
 * Experience with real-time serving, ML infrastructure, or GPU orchestration
 * Familiarity with service-oriented architecture, deployment pipelines, and system observability

## Description

* Build LLM infrastructure powering large-scale inference workloads for customers through partner models (OpenAI, Anthropic, Gemini) and self-hosted models (Qwen, GPT-OSS, Llama)
 * Improve reliability, latency, and efficiency of distributed AI workloads
 * Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences
 * Shape how developers and data scientists build and interact with AI on Databricks, * Exposure to platforms like SageMaker, Vertex AI, or Azure ML
 * Contributions to OSS projects like MLflow, PyTorch, Ray, vLLM, SGLang
 * Built developer platforms or internal tools supporting AI workflows

Pay Range Transparency