> Markdown version of [/jobs/ext/3449526-senior-software-engineer-inference](https://www.wearedevelopers.com/jobs/ext/3449526-senior-software-engineer-inference). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Software Engineer - Inference

- **Company:** Shadowgate Partners, Inc.
- **Location:** Annapolis Junction, MD, United States
- **Experience:** Expert
- **Salary:** $260,000.0 - $285,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Continuous Integration, Python (Programming Language), Octopus Deploy, Reliability Engineering, Cloud Services, Prometheus, Large Language Models, Grafana, Containerization, Kubernetes, Docker, Programming Languages
- **Published:** September 1, 2026
- **Apply:** https://www.clearancejobs.com/jobs/9132953/senior-software-engineer-inference

## About the Role

* Experience with Python and/or other modern programming languages.
 * Experience with Argo CD and/or other CI/CD frameworks.
 * Experience with Kubernetes and Helm.
 * Experience with AWS or other cloud service providers.
 * Strong communication skills and the ability to mentor other engineers.
 * Excellent leadership and stakeholder management skills.
 * Ability to balance hands-on engineering with leadership and coordination responsibilities.

Desired Skills

 * Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
 * Experience with other LLM hosting frameworks and practices.
 * Experience supporting production software using Site Reliability Engineering (SRE) best practices.
 * Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.
 * Experience with Docker and containerization.
 * Experience in traffic shaping and quality-of-service engineering.
 * Knowledge of and interest in hosting AI capabilities., * Twelve (12) or more years of relevant professional experience.
 * B.S. in a technical discipline, or an additional four (4) years of experience in lieu of a degree.

## Description

* Lead the evaluation, configuration, and deployment of inference models for end users.
 * Translate high-level stakeholder goals into engineering outcomes.
 * Mentor engineers and grow development best practices and standards for the team.
 * Develop in-house services and techniques to ensure continual, high-quality inference service.
 * Partner with other teams across the organization to establish solid infrastructure for services and integrate LLM-powered tools that meet user needs.
 * Collaborate with teammates on short-term, high-priority efforts to meet urgent inference demands.