Junior Software Engineer - Inference

We’re seeking a software engineer to support our AI infrastructure team, helping build and maintain the foundation for advanced AI capabilities while ensuring access to high-quality LLMs., * Procure, configure, and test new inference models, preparing them for release to users.

• Develop in-house services and techniques to ensure continuous high-quality inference service.
• Collaborate with model vendor teams to create reliable pipelines for closed-source model usage.
• Support surge efforts to address short-term, high-priority inference needs.
• Engage with other teams to establish solid infrastructure for AI services and integrate LLM-powered tools.

• Experience with Python and/or modern programming languages.
• Familiarity with Argo CD or other CI/CD frameworks.
• Experience with Kubernetes/Helm.
• Familiarity with AWS or another cloud service provider.

Nice to Have:

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
• Experience supporting production software using SRE best practices.
• Knowledge of Elastic, Grafana/Prometheus, and observability practices.

Experience Required: 3 years with Bachelor’s degree in a technical field, or 7 years without degree