> Markdown version of [/jobs/ext/2719048-software-engineer-baseten-inference-stack](https://www.wearedevelopers.com/jobs/ext/2719048-software-engineer-baseten-inference-stack). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Software Engineer - Baseten Inference Stack - **Company:** Baseten, Inc - **Location:** San Francisco, CA, United States - **Contract:** Permanent contract - **Skills:** Application Release Automation, Continuous Integration, Software Debugging, Distributed Systems, Routing, Open Source Technology, Datadog, Autoscaling, Large Language Models, Model Validation, Backend, Kubernetes, Information Technology, Low Latency, Machine Learning Operations, TensorRT - **Published:** September 4, 2026 - **Apply:** https://startup.jobs/software-engineer-baseten-inference-stack-baseten-8029708 ## About the Role * Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field * Strong background in distributed systems, backend infrastructure, or platform engineering * Experience building and operating production systems where reliability, latency, and scale are first-class concerns * Strong sense of developer experience: you think about how systems are used, not just how they work * Motivated and willing to learn new languages, frameworks, and systems as needed * Ability to debug complex systems across multiple layers of the stack * Genuine interest in inference engineering. You don't need to have hands on experience but are willing to learn * Excellent communication and collaboration skills BONUS * Experience with Kubernetes, including concepts like operators and custom resources * Prior work on Dynamo, vLLM, SGLang, TensorRT-LLM, or similar inference frameworks * Experience with distributed scheduling, autoscaling, or service orchestration * Experience operating GPU workloads in production * Familiarity with observability tooling, CI/CD systems, or release automation * Experience contributing to open-source infrastructure or ML systems ## Description Baseten's Inference Stack team builds the distributed runtime that powers large-scale LLM inference across our platform. We operate at the intersection of distributed systems, model performance, infrastructure, and developer experience. We enable customers to deploy and operate cutting-edge LLM models with industry-leading performance, scalability, reliability, and ease of use. As a Software Engineer on the Inference Stack team, you'll work across the stack - from the developer experience customers use to deploy models, the libraries used for features like tool calling and reasoning, all the way down to the systems we use to orchestrate deployments in Kubernetes and route traffic efficiently. This is an ideal role for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users., * Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference * Work across the stack, from customer-facing features to low-level infrastructure components * Build platform capabilities related to routing, autoscaling, scheduling, observability, and runtime management * Improve the reliability, scalability, and usability of our inference stack * Collaborate closely with Model Performance engineers to make new inference optimizations broadly available to customers and easy to configure * Help define best practices around testing, release automation, benchmarking, and operational excellence * Debug complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads * Make thoughtful engineering tradeoffs balancing performance, reliability, operational simplicity, and developer experience * Own projects end-to-end: from architecture and implementation through deployment, monitoring, and iteration based on customer feedback ## Related Videos - [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops) - [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock) - [Debugging in the Dark](https://www.wearedevelopers.com/videos/1658-debugging-in-the-dark) - [Creating a routing app with Google Maps API from scratch](https://www.wearedevelopers.com/videos/831-creating-a-routing-app-with-google-maps-api-from-scratch) - [Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated](https://www.wearedevelopers.com/videos/100099-tour-de-force-open-source-llm-inference-optimization-from-simple-to-sophisticated) - [Nest.js - TypeScript in the backend can also be clean](https://www.wearedevelopers.com/videos/1033-nest-js-typescript-in-the-backend-can-also-be-clean) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)