> Markdown version of [/videos/715-a-hundred-ways-to-wreck-your-ai-the-in-security-of-machine-learning-systems?t=435](https://www.wearedevelopers.com/videos/715-a-hundred-ways-to-wreck-your-ai-the-in-security-of-machine-learning-systems?t=435). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# A hundred ways to wreck your AI - the (in)security of machine learning systems

Are your machine learning models secretly harboring critical vulnerabilities? Discover how to defend your AI infrastructure against novel data poisoning attacks and traditional remote code execution exploits.

- **Speakers:** Balázs Kiss
- **Event:** World Congress 2023
- **Published:** September 27, 2023
- **Duration:** 24:23
- **URL:** https://www.wearedevelopers.com/videos/715-a-hundred-ways-to-wreck-your-ai-the-in-security-of-machine-learning-systems

## Summary

The rush to adopt artificial intelligence often overlooks a fundamental truth: AI models are still just software, and 'if it has bugs, we can exploit them.' This presentation explores the expanding attack surface introduced by machine learning, bridging the gap between novel AI vulnerabilities and traditional software engineering flaws. While generative AI can rapidly write code, it frequently produces insecure logic, relies on outdated practices, or fabricates dependencies that attackers weaponize as supply chain attacks. Securing AI requires understanding two distinct categories of threats. First, ML-specific architectures face novel attacks like evasion and data poisoning. Threat actors use adversarial samples—imperceptible pixel shifts that push inputs past a model's decision boundary—to alter classifications, or reverse-engineer models to extract sensitive training data. Second, traditional vulnerabilities continue to plague ML infrastructure. Frameworks built on low-level languages harbor critical flaws like buffer overflows. To build robust environments, developers must mature their threat modeling using resources like the OWASP Machine Learning Security Top 10 and the Adversarial Robustness Toolbox. A critical immediate focus must be secure model parsing. The widespread reliance on Python's pickle format for saving models introduces severe remote code execution risks, as pickled objects can define their own deserializers. Transitioning to secure file formats like HDF5 and treating AI models as untrusted input are essential steps in preventing attackers from compromising ML systems.

**Keywords:** AI security, machine learning vulnerabilities, adversarial machine learning, data poisoning attacks, model evasion techniques, adversarial robustness toolbox, OWASP machine learning security top 10, python pickle RCE, pytorch serialization risks, secure AI coding, model extraction attacks, AI supply chain risks, algorithmic threat modeling, HDF5 format, deepfake impersonation, catastrophic forgetting

## Chapters

1. **Understanding the landscape of AI capabilities and risks** (00:03) — How widespread AI adoption introduces new security challenges and failure modes.
1. **Addressing security flaws in AI-generated code** (03:11) — Why relying on AI for code generation can introduce outdated or flawed validation logic.
1. **Fundamental problems in machine learning model training** (04:18) — How overfitting, catastrophic forgetting, and bad input data compromise model reliability.
1. **Misusing AI for malware generation and physical evasion** (05:39) — How attackers leverage AI to automate exploitation, mutate malware, and bypass image recognition.
1. **Corrupting AI models and software supply chains** (07:15) — How malicious training data and compromised dependencies inject backdoors into AI applications.
1. **Utilizing industry threat models for AI security** (08:59) — Navigating frameworks from OWASP, NIST, and MITRE to understand and mitigate machine learning vulnerabilities.
1. **Categorizing threats across the machine learning lifecycle** (10:20) — Differentiating between novel machine learning threats and traditional software vulnerabilities affecting AI systems.
1. **Exploiting models through adversarial evasion samples** (12:21) — How imperceptible modifications to input data force machine learning models to make incorrect decisions.
1. **Poisoning data to manipulate model behavior** (13:18) — Injecting malicious data during training phases to alter decision boundaries and trigger targeted failures.
1. **Stealing confidential data through model extraction attacks** (14:48) — Using model inversion and excessive requests to extract sensitive training data or replicate proprietary models.
1. **Generating adversarial samples against digit recognition models** (16:47) — A live demonstration using the Adversarial Robustness Toolbox to trick an MNIST model into misreading financial digits.
1. **Preventing remote code execution in PyTorch models** (19:08) — Why loading unverified PyTorch models stored in the Pickle format exposes systems to arbitrary code execution.
1. **Securing developer wrapper code around machine learning models** (21:43) — Addressing traditional software bugs like exposed API keys and command injection in open-source AI wrappers.
1. **Replacing the Pickle format with secure serialization alternatives** (23:40) — Migrating to standard HDF5 or static serialization formats to prevent arbitrary code execution vulnerabilities.

## Related Moments

- [Top security vulnerabilities for AI applications](https://www.wearedevelopers.com/videos/1637-delay-the-ai-overlords-how-oauth-and-openfga-can-keep-your-ai-agents-from-going-rogue) (from "Delay the AI Overlords: How OAuth and OpenFGA Can Keep Your AI Agents from Going Rogue")
- [Emerging risks and attack vectors in AI systems](https://www.wearedevelopers.com/videos/1948-building-trustworthy-ai-in-industry-beyond-traditional-cybersecurity) (from "Building Trustworthy AI in Industry: Beyond Traditional Cybersecurity")
- [Core AI security risks and data poisoning](https://www.wearedevelopers.com/videos/1217-can-machines-dream-of-secure-code-emerging-ai-security-risks-in-llm-driven-developer-tools) (from "Can Machines Dream of Secure Code? Emerging AI Security Risks in LLM-driven Developer Tools")
- [Core principles of manipulating artificial intelligence models](https://www.wearedevelopers.com/videos/824-hacking-ai-how-attackers-impose-their-will-on-ai) (from "Hacking AI - how attackers impose their will on AI")
- [Introduction to machine learning security and robustness](https://www.wearedevelopers.com/videos/627-machine-learning-promising-but-perilous) (from "Machine Learning: Promising, but Perilous")
- [Understanding data poisoning and model bias risks](https://www.wearedevelopers.com/videos/1563-prompt-injection-poisoning-more-the-dark-side-of-llms) (from "Prompt Injection, Poisoning & More: The Dark Side of LLMs")

## Related Articles

- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Exploring AI: Opportunities and Risks for Developers](https://www.wearedevelopers.com/magazine/522-exploring-ai-opportunities-and-risks-for-developers)
- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)

## Related Jobs

- [Staff Developer Advocate, GitHub Security Lab](https://www.wearedevelopers.com/jobs/ext/1921051-staff-developer-advocate-github-security-lab) at **GitHub**
- [Security Architect - AI](https://www.wearedevelopers.com/jobs/ext/1581899-security-architect-ai) at **ZEISS Group**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1355348-machine-learning-engineer) at **TWILIO**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [Principal Software Engineer, Enterprise AI Platform](https://www.wearedevelopers.com/jobs/ext/1467292-principal-software-engineer-enterprise-ai-platform) at **GitHub**