Lessons Learned Building a GenAI Powered App

Our GenAI app’s output was unpredictable. We solved it by using a second LLM as a validator, achieving 94% accuracy.

#1about 2 minutes

The limitations of a static trivia database

The original quiz application was built on the Open Trivia API, which resulted in significant limitations like a fixed set of topics, formats, and languages.

#2about 5 minutes

Reimagining the quiz app with generative AI

A live demonstration showcases how generative AI can create dynamic quizzes on any topic, in any language, and even generate a relevant cover image.

#3about 6 minutes

The technical architecture of the GenAI quiz app

The application uses Flutter for the multi-platform UI, Cloud Run for hosting, Firestore for real-time data, and Vertex AI for accessing Google's generative models.

#4about 3 minutes

Navigating the inconsistency and uncertainty of LLMs

While GenAI makes complex tasks seem easy, achieving consistent and high-quality results is difficult due to the inherent non-deterministic nature of LLMs.

#5about 2 minutes

Knowing when not to use a large language model

For tasks like fuzzy string matching or simple image editing, traditional libraries and tools can be more effective, reliable, and cheaper than using an LLM.

#6about 4 minutes

Effective prompting and defensive coding for LLMs

Write clear but not overly detailed prompts, manage them like code with versioning, and code defensively to handle failures, malformed data, and empty results from the LLM.

#7about 2 minutes

Applying frameworks and engineering principles to LLM development

Using higher-level frameworks like LangChain can simplify development, while standard software engineering practices like caching and parallel calls are crucial for performance and cost management.

#8about 4 minutes

The challenge of ensuring quality and accuracy in LLMs

While it's easy to test the format of an LLM's output, verifying its quality and factual accuracy is much harder and may require using another LLM as a validator.

#9about 1 minute

Improving LLM accuracy with grounding techniques

To increase factual accuracy and reduce hallucinations, ground the model's responses in reliable data sources using tools like Google Search or a custom knowledge base via Vertex AI Search.

#10about 1 minute

How GenAI unblocks features but introduces new challenges

Generative AI can rapidly expand an application's capabilities, but this introduces a new class of problems related to accuracy, consistency, and validation that require new engineering solutions.