From Hallucination to Justification: Hands-On Explainability for LLMs

Human beings are biased and often wrong. Artificial intelligence learns from human-created data. Therefore, artificial intelligence is biased and often wrong. This has been a critical problem across machine learning applications in the last years. To break open the black box of AI models, and understand how they make decisions, the concept of explainability was introduced.

Then, LLMs entered the chat. They answer our questions confidently and with a beautiful prose, even when they are making up data. Explainability then becomes essential to trust -or not- their output. But when the existing explainable AI methods cannot be directly applied to these models, what do we do?

In this workshop, we will delve into the topic of explainable AI, and its importance in the current context of LLMs and agents. Starting from traditional machine learning to then focus on LLMs, we will cover the different methods that can be implemented, from well-known ones to novel proposals stemming from our internal research. We will also introduce research-proven prompting strategies, tips, and tricks to integrate explanations on third-party LLM services that are not natively explainable.

Through guided exercises, you will get to peek under the hood of AI models, LLMs’ behavior, and agents reasoning, by trying out these different techniques, and seeing their benefits and limitations first-hand. You will experience the risks and challenges that generative AI and agentic AI bring when implementing explainability, and learn practical ways to tackle them.

By the end of the workshop, you will leave with a mental toolkit you can apply immediately to know:

• When to trust (or distrust) LLMs
• Which explainability capabilities to implement (and how) on your RAG systems and LLM-based workflows (or which ones to look for when choosing a third-party service), and
• How to make LLMs’ behavior more predictable, transparent, and ultimately safe

Prerequisites: before the workshop, make sure that you have installed both Ollama and Python 3.10 or higher on your laptop, as well as an IDE that can run Jupyter notebooks e g. VSCode. On Ollama, make sure to pull the model qwen3:0.6b – the model is 500MB, so the download might take some time