> Markdown version of [/events/conference-india-2026/sessions/2040-bypassing-the-gpu](https://www.wearedevelopers.com/events/conference-india-2026/sessions/2040-bypassing-the-gpu). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Bypassing the GPU Tax: Smart Layer-7 Routing with Small Language Models (SLMs)


- **Event:** Conference India 2026

## Description

Sending every single user prompt, formatting request, or minor sub-agent reasoning task directly to massive, expensive frontier models running on heavy GPU clusters is a massive financial anti-pattern. Production data shows that up to 40% of intermediate sub-agent steps can be handled flawlessly by highly efficient Small Language Models (SLMs under 8B parameters). The trick to unlocking these massive savings isn't rewriting your application; it's building intelligent routing directly into your application delivery plane.

This session walks through architecting a three-layer intelligent routing network using standard cloud-native infrastructure. We will demonstrate how to intercept prompt traffic at the Layer-7 Ingress layer, analyze prompt complexity or user intent using an embedded lightweight classifier, and dynamically route simple operations to CPU-bound SLM clusters while reserving prime, expensive GPU space for complex reasoning chains. 

We will step away from cloud-provider lock-in and look at how to build this framework using open-source infrastructure tools like Envoy Proxy, the Kubernetes Gateway API, and vLLM/Ollama. You’ll walk away with a ready-to-use routing schema and architectural blueprints that can slice token delivery costs by half, drastically lower average request latency, and make your distributed AI applications financially viable at scale.

## Speaker

### [Sagar Utekar](https://www.wearedevelopers.com/@sagar-utekar)

SRE@Omnissa | Kubestronaut | CNCF Ambassador | Docker Captain


## Related talks at this congress

- [How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design](https://www.wearedevelopers.com/events/conference-india-2026/sessions/2003-how-8-gpus-became-16) — Purnima Kumari
- [Your AI Agents Are Bleeding Tokens: Building a Runtime Cost Control Plane for LLMs](https://www.wearedevelopers.com/events/conference-india-2026/sessions/2066-your-ai-agents-are) — Aakriti Aggarwal, Bhavishya Pandit
- [React Under Fire: Rendering Performance for Streaming Agent UIs](https://www.wearedevelopers.com/events/conference-india-2026/sessions/1992-react-under-fire) — Abhirupa Mitra
- [Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination](https://www.wearedevelopers.com/events/conference-india-2026/sessions/1976-architecting-trusted) — Pushkar Mishra

## Watch remotely

Can’t make it to Bengaluru? Watch this session live with Pro. You also get:

- All full videos, bookmarks, and playlists
- World Congress livestreams

[See pricing](https://www.wearedevelopers.com/pricing)

## Links

- [Get tickets](https://www.wearedevelopers.com/conference-india/tickets)