Senior Software Engineer - Inference

• Lead the evaluation, configuration, and deployment of inference models for end users.
• Translate high-level stakeholder goals into engineering outcomes.
• Mentor engineers and grow development best practices and standards for the team.
• Develop in-house services and techniques to ensure continual, high-quality inference service.
• Partner with other teams across the organization to establish solid infrastructure for services and integrate LLM-powered tools that meet user needs.
• Collaborate with teammates on short-term, high-priority efforts to meet urgent inference demands.

• Experience with Python and/or other modern programming languages.
• Experience with Argo CD and/or other CI/CD frameworks.
• Experience with Kubernetes and Helm.
• Experience with AWS or other cloud service providers.
• Strong communication skills and the ability to mentor other engineers.
• Excellent leadership and stakeholder management skills.
• Ability to balance hands-on engineering with leadership and coordination responsibilities.

Desired Skills

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
• Experience with other LLM hosting frameworks and practices.
• Experience supporting production software using Site Reliability Engineering (SRE) best practices.
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.
• Experience with Docker and containerization.
• Experience in traffic shaping and quality-of-service engineering.
• Knowledge of and interest in hosting AI capabilities., * Twelve (12) or more years of relevant professional experience.
• B.S. in a technical discipline, or an additional four (4) years of experience in lieu of a degree.

Salary Range: $260k-$285k per year with an additional 25% ($65k-$71k) in immediately vested company 401(k) contributions