Simon Says "Format Drive": Convenience Meets Consequences

AI assistants are starting to control things we care about: our files, our browsers, our passwords — sometimes even our entire computer. From an IT security perspective, this is a big change. We are giving systems that operate on natural language the same level of access that used to be reserved for carefully hardened software.

In this talk, an experienced red teamer* and an AI professor have worked together to show what that means in practice.

Using real tools such as Clawdbot, we demonstrate live how easily modern AI assistants can be abused by everyday users. No hacking tools, no technical background, no complex exploits. In many cases, a few well-chosen instructions are enough to bypass safeguards and influence an AI into performing actions it was never meant to perform.

These attacks work because language itself becomes the attack surface. When security relies on instructions, those instructions can be manipulated — by anyone. This creates a gap between traditional security models and how AI systems actually behave once they are deployed with broad permissions.

We explain why these failures occur, how easily users expose themselves by trusting AI with too much access, and why familiar security assumptions break down in this context. The talk combines hands-on demonstrations with practical guidance for developers and product builders, focusing on how to limit damage, design safer systems, and avoid repeating old security mistakes in a new form.

The goal is awareness through understanding: showing the flaws, explaining the risks, and helping people prevent them — before convenience becomes consequence.