> Markdown version of [/jobs/ext/2709366-software-engineer-hosted-model-infrastructure](https://www.wearedevelopers.com/jobs/ext/2709366-software-engineer-hosted-model-infrastructure). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Software Engineer - Hosted Model Infrastructure - **Company:** Palantir Technologies - **Location:** New York, United States (Remote available) - **Experience:** Experienced - **Salary:** $145,000.0 - $200,000.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Artificial Intelligence, Audit Trail, User Authentication, C++ (Programming Language), Concurrency Controls, Software Debugging, Github, Gradle, Monitoring of Systems, Python (Programming Language), Machine Learning, Open Source Technology, Software Engineering, Systems Integration, Rust (Programming Language), Delivery Pipeline, Backend, Rate Limiting, Containerization, Kubernetes, Information Technology, Machine Learning Operations, Software Coding, Docker, Golang - **Published:** September 4, 2026 - **Apply:** https://startup.jobs/software-engineer-hosted-model-infrastructure-palantir-8009070 ## About the Role * High empathy for customer needs and drive to deliver reliable, easy-to-use models * Ability to work effectively across multiple languages and layers of the stack, from backend services and ML tooling to container orchestration and deployment configuration * Strong debugging skills and motivation to trace problems from application code through containers, orchestration, and hardware * Curiosity about emerging AI capabilities and the ability to quickly evaluate and integrate new models and technologies as the landscape evolves * Active US Security clearance, or eligibility and willingness to obtain a US Security clearance is beneficial, but not necessary, * 4+ years of professional software engineering experience building and operating production systems * Engineering background in Computer Science, Mathematics, Software Engineering, Physics, or similar field * Strong coding skills with demonstrated proficiency in programming languages, such as Java, C++, Python, Rust, or similar languages. Familiarity with the Python ML ecosystem is valuable. * Experience with containers, Kubernetes, and deploying backend services in production environments * Strong written and verbal communication skills and ability to iterate quickly with teammates, incorporating feedback and holding a high bar for quality ## Description We are a software engineering team with expertise in enabling ML models in production. We deploy AI models to run in variety of environments: air-gapped government networks, forward-deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements. Our customers rely on us for frontier AI capabilities running on hardware they control, often with constrained GPU resources and limited direct access. Rising to that challenge and meeting those expectations is what Palantir's excels at. We treat models like any other software: continuously tested, continually delivered, packaged for reproducible deployment, and built for long-term maintainability. You will own services end-to-end, and work across the full stack, from inference engines, GPU scheduling to deployment pipelines, observability, and integration with Palantir's platform. The goal is to deliver new models and capabilities quickly and continuously. Join us if you want to solve problems at the intersection of infrastructure and machine learning that directly enable critical customers. Technologies We Use * Different backend languages, including Java, Rust, Python and Go * Model serving engines for GPU-accelerated inference * Docker and Kubernetes for containerization and orchestration * Industry-standard build tooling, including Gradle and GitHub Core Responsibilities * Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines * Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access * Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments * Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution * Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly * Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production * Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need * Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems ## Related Videos - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops) - [Go with the Flow: Stop the Leaks Before Your Memory's a Waterfall!](https://www.wearedevelopers.com/videos/100073-go-with-the-flow-stop-the-leaks-before-your-memory-s-a-waterfall) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Agentic employees in world's most downloaded FinTech app](https://www.wearedevelopers.com/videos/100123-agentic-employees-in-world-s-most-downloaded-fintech-app) - [Scoring 2000 Products per Request: Performance Pitfalls in Golang](https://www.wearedevelopers.com/videos/2073-scoring-2000-products-per-request-performance-pitfalls-in-golang) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)