Tutorials often show a simplistic happy path, but real-world agents require robust systems to handle diverse and unexpected user queries.
Productionizing an agent requires moving beyond basic prompts to include evaluations, golden datasets, tracing, user feedback loops, and security guardrails.
Unlike deterministic software, LLM-based agents can be tricked through new attack vectors like prompt injection, requiring specialized security considerations.
Recommended learning materials include understanding LLM fundamentals, reading "Agentic Design Patterns," and focusing on building proper evaluations and golden datasets.
Developers are shifting from writing all code to acting as tech leads for their own teams of agents, requiring skills in review, direction, and oversight.
Giving agents like OpenClau unrestricted access to a local machine is risky; it's better to use sandboxed or containerized environments for experimentation.
Hallucinations and sycophancy persist in agents, but they can be mitigated through a combination of model-level improvements and system-level guardrails.
A case study highlights the importance of building a comprehensive golden dataset, fostering deep collaboration with SMEs, and using a tight user feedback loop for continuous improvement.
While models now have built-in capabilities like Chain of Thought, adding simple, domain-specific sentences to a system prompt can still dramatically improve performance.
Agents are primarily used for intelligent search across large datasets and for automating complex, reproducible human workflows with human-in-the-loop oversight.
38:01Julia Kordick
32:55Dr. Damir Dobric
56:27Luke Hinds
45:09Gunnar Grosch
57:43Lee Faus
30:48Max Tkacz
27:11Alfonso Graziano
42:26Meta Atamel & Guillaume Laforge
Jobs that call for the skills explored in this talk.
Wolters Kluwer
Alphen aan den Rijn, Netherlands
autonomous-teaming
München, Germany
Collaboration Betters The World GmbH
