> Markdown version of [/jobs/ext/2311989-senior-software-developer-llm-infrastructure](https://www.wearedevelopers.com/jobs/ext/2311989-senior-software-developer-llm-infrastructure). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Software Developer, LLM Infrastructure

- **Company:** WEALTHSIMPLE US, LTD.
- **Location:** Toronto, United States (Remote available)
- **Experience:** Expert
- **Salary:** $152,900.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Core Foundation, Data Infrastructure, Distributed Systems, Python (Programming Language), PostgreSQL, Machine Learning, Open Source Technology, Software Engineering, Data Streaming, Large Language Models, Snowflake, Backend, Fastapi, Kubernetes, Apache Kafka, Machine Learning Operations, Hardware Infrastructure, Terraform, Amazon Redshift
- **Published:** August 30, 2026
- **Apply:** https://arc.dev/remote-jobs/j/redirect/pg4dps15qm

## About the Role

* 6+ years of software engineering experience building production-ready distributed systems or data platform infrastructure.
 * Strong proficiency in Python and infrastructure tooling like Kubernetes and Terraform.
 * Proven experience designing and operating highly performant, observable backend systems.
 * Practical experience or deep familiarity with the modern AI ecosystem (e.g., serving open-source models, vLLM, Ray, model routing, or evaluation frameworks).
 * Track record of owning technical projects end-to-end with high autonomy.

Our stack includes:

 * Container & Infra: Kubernetes, Terraform, AWS GPU infrastructure
 * Languages & APIs: Python, FastAPI
 * LLM Tooling & Serving: LiteLLM, vLLM, Ray, Bedrock
 * Data & Streaming: Kafka, Postgres, Redshift, Snowflake

## Description

The Machine Learning Infrastructure team builds the core foundation powering AI and GenAI initiatives across Wealthsimple. As our AI footprint expands, our priority is building scalable, cost-effective infrastructure that enables high-performance inference using open-source models alongside vendor solutions. Our responsibilities are to:

 * Build and maintain infrastructure to host and serve open-source LLMs reliably on cloud/GPU systems.
 * Implement intelligent model-routing frameworks to optimize performance, cost, and latency.
 * Develop evaluation and benchmarking tooling (Evals) to test, validate, and compare model performance across use cases.
 * Set the technical direction for operating scalable LLM infrastructure company-wide., As a Senior Software Engineer on the ML Infrastructure team, you will design and implement the core backend and infrastructure powering our AI systems. You will work closely with data scientists, platform engineers, and engineering leaders to scale our GenAI footprint efficiently and securely.

In this role, you will have the opportunity to:

 * Build model-routing architecture (e.g., LiteLLM integration) to dynamically direct requests across internal and external models.
 * Provision and manage high-performance GPU infrastructure for serving open-source LLMs.
 * Build evaluation frameworks (Evals) that enable engineers and data scientists to benchmark model quality against production requirements.
 * Drive key cost-optimization and infrastructure efficiency initiatives without sacrificing system accuracy or reliability.


## Related Videos

- [Unveiling the Magic: Scaling Large Language Models to Serve Millions](https://www.wearedevelopers.com/videos/1619-unveiling-the-magic-scaling-large-language-models-to-serve-millions)
- [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon)
- [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock)
- [Intro to FastAPI](https://www.wearedevelopers.com/videos/462-intro-to-fastapi)
- [How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge](https://www.wearedevelopers.com/videos/1328-how-to-avoid-llm-pitfalls-mete-atamel-and-guillaume-laforge)
- [Implementing Feature Environments with AWS and Terraform](https://www.wearedevelopers.com/videos/531-implementing-feature-environments-with-aws-and-terraform)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [The 7 Most Popular Backend Frameworks for Developers](https://www.wearedevelopers.com/magazine/403-the-7-most-popular-backend-frameworks-for-developers)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)