WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2024 • Aug 20, 2024 • Session details

Chatbots are going to destroy infrastructures and your cloud bills

Stan Girard

Is your new RAG chatbot secretly bankrupting your cloud budget? Monolithic architectures bottleneck servers and skyrocket costs. Learn how decoupling heavy GPU tasks saves your infrastructure.

Pause
Mute Enter Fullscreen
#1 about 1 min

Introduction to chatbot infrastructure and cloud challenges

How the rapid growth of an open-source chatbot project highlights the importance of avoiding structural infrastructure mistakes.

#2 about 2 min

Contrasting web developers and data scientists

Differences in resource constraints, scaling models, and deployment environments separate traditional software and data engineering roles.

#3 about 2 min

The new paradigm of AI engineers

AI engineering merges practices from both web and data science, introducing unique optimization and computing constraints.

#4 about 2 min

Understanding basic retrieval-augmented generation architectures in chatbots

A breakdown of RAG workflows entails document chunking, GPU-bound embeddings, and sequential model generation calls.

#5 about 2 min

Scaling bottlenecks in generative AI applications

The challenge of managing parallel execution, slow response times, and sequential steps complicates user-facing AI deployments.

#6 about 2 min

Building a lean initial chatbot prototype application

Using a minimal tech stack creates an efficient and lightweight artificial intelligence container application.

#7 about 2 min

Bloat and complexity from adding GPU-bound tasks

Introducing OCR and advanced embeddings drastically increases container image sizes and subsequent infrastructure scaling costs.

#8 about 2 min

Refactoring application architectures for complex technical tasks

Separating fast user-facing business logic from slow background workers handles complex orchestrations and integrations effectively.

#9 about 3 min

Infrastructure challenges of on-premise chatbot deployments

Hosting monolithic, multi-model AI applications on local organizational servers poses massive server tuning and scaling challenges.

#10 about 1 min

Architecting generative AI systems by targeted user load

Guidelines for splitting application components apply directly to managing continuous or spiky application traffic patterns.

#11 about 3 min

Best practices for utilizing large language models in production

Strategies favoring managed APIs and service-oriented architectures reduce the immense overhead of self-hosting models.

#12 about 2 min

Overlooked AI infrastructure and operational deployment barriers

Managing complex deployments like configuring GPUs on Kubernetes requires dedicated pipelines for backups and non-deterministic testing.

#13 about 3 min

Evaluating large language models using real-time video games

A hackathon approach to measuring model capability proves that smaller setups often outperform larger architectures in speed-sensitive tasks.

Matching moments

1:22 min

Leveraging the comprehensive generative artificial intelligence stack

Duan Lightfoot Duan Lightfoot · World Congress 2024

1:32 min

Architectural patterns for developing robust generative AI applications

Julián Duque Julián Duque · World Congress 2025

2:25 min

Progressing gracefully from generic chatbots to agentic workflows

Anshul Jindal Anshul Jindal +1 · World Congress 2026 Europe

3:25 min

Constructing scalable AI solutions using LangChain and LangGraph

Julián Duque Julián Duque · World Congress 2025

1:20 min

Engineering lessons for scaling high-performance AI infrastructure

Wayne Liu Wayne Liu · World Congress 2026 North America

4:20 min

Evolution and challenges of building AI applications

Roberto Carratalá Roberto Carratalá · World Congress 2025

More from World Congress 2024

Beyond the Hype: Real-World AI Strategies Panel thumbnail
24:07
about 1 year ago

Beyond the Hype: Real-World AI Strategies Panel

Mike Butcher, Jürgen Müller, Katrin Lehmann, Tobias Regenfuss

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation thumbnail
21:17
almost 2 years ago

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation

Carl Lapierre

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI! thumbnail
31:50
about 2 years ago

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging thumbnail
29:43
about 2 years ago

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging

Sue Quackenbush

Livecoding with AI thumbnail
31:39
about 2 years ago

Livecoding with AI

Rainer Stropek

Dapr in Practice thumbnail
24:08
about 2 years ago

Dapr in Practice

marcklefter

Upcoming sessions on this topic

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Open session

React Under Fire: Rendering Performance for Streaming Agent UIs

  • Abhirupa Mitra

    Slack

    Senior Member of Technical Staff

Open session

Beware of Strangers Bearing Code: Open Source Trust in the Agent Era

  • Vikram Vaswani

    Consultant

Open session

AI Is the Easy Part: Why Faster Engineers Don't Make Faster Businesses

  • Bharat Sharma

    Booking

    Senior Engineering Manager

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Related videos

Supercharge your cloud-native applications with Generative AI thumbnail
32:13
World Congress 2024 • about 2 years ago

Supercharge your cloud-native applications with Generative AI

Cedric Clyburn

Performant Architecture for a Fast Gen AI User Experience thumbnail
25:47
World Congress 2024 • about 2 years ago

Performant Architecture for a Fast Gen AI User Experience

Nathaniel Okenwa

How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge thumbnail
42:26
Coffee With Developers • over 1 year ago

How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge

Meta Atamel, Guillaume Laforge

Make it simple, using generative AI to accelerate learning thumbnail
26:54
World Congress 2024 • about 2 years ago

Make it simple, using generative AI to accelerate learning

Duan Lightfoot

WeAreDevelopers LIVE - Is AI replacing developers?, Stopping bots, AI on device & more thumbnail
1:01:03
LIVE • over 1 year ago

WeAreDevelopers LIVE - Is AI replacing developers?, Stopping bots, AI on device & more

Chris Heilmann, Daniel Cranney, Sebastian Gingter

Bringing the power of AI to your application. thumbnail
32:26
World Congress 2024 • about 2 years ago

Bringing the power of AI to your application.

Krzysztof Cieślak

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.