AI demos create unrealistic expectations by using curated inputs and showing only happy paths, which does not reflect real-world user interaction.
User churn is often caused by poor workflow integration and lack of trust, not by the underlying model's quality or performance on benchmarks.
AI features fail when they act as "bolt-on" tools that add steps, instead of being "built-in" to existing user workflows to reduce friction.
Building user trust requires providing verifiable sources and ensuring that AI mistakes are cheap and easy to revert.
AI systems must gracefully hand off to humans for ambiguous or out-of-scope requests instead of failing silently or confidently giving wrong answers.
AI features should start by suggesting actions and only move to autonomous acting for low-risk, reversible tasks to maintain user control.
Continuously improve AI features by instrumenting user interactions and building an evaluation harness with golden sets and "LLM as a judge" into the CI pipeline.
The TRUST framework (Task fit, Recovery, User control, Signals, Trust calibration) provides a structure for building reliable AI systems.
Use confidence scores to gate actions between suggesting and acting, and integrate golden sets with an "LLM as a judge" into your CI pipeline for automated testing.
An automation agent built on Slack uses a multi-step process involving intent gating, user plan confirmation, and a separate execution environment to ensure safety and control.
Use the TRUST framework as a pre-ship checklist and start by creating messy golden cases, implementing a confidence gate, and logging user overrides.
The success of an AI feature depends on the system around the model, earning user trust, and designing for long-term, repeated usage.
Jobs that call for the skills explored in this talk.