> Markdown version of [/videos/627-machine-learning-promising-but-perilous?t=754](https://www.wearedevelopers.com/videos/627-machine-learning-promising-but-perilous?t=754). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Machine Learning: Promising, but Perilous

A simple sticker on a stop sign can completely blind your AI perception model. Uncover the hidden security perils of transfer learning and how to fortify your ML pipelines.

- **Speakers:** Nura Kawa
- **Event:** World Congress 2023
- **Published:** August 11, 2023
- **Duration:** 27:19
- **URL:** https://www.wearedevelopers.com/videos/627-machine-learning-promising-but-perilous

## Summary

Machine learning's growing accessibility, particularly through transfer learning, has dramatically lowered the barrier to entry for developers but simultaneously introduces severe security vulnerabilities. While taking a pre-trained "teacher" model—like BERT or LLaMA—and fine-tuning it into a "student" model saves computational resources and data, it inadvertently transfers the foundational model's risks directly into new applications. This interconnectedness means that an attack designed for a widespread open-source model can easily compromise any downstream system built upon it.

Adversarial machine learning operates as a continuous cat-and-mouse game, primarily consisting of inference, extraction, evasion, and poisoning attacks. Evasion attacks, or adversarial examples, use slight data perturbations—such as adding a sticker to a stop sign or altering pixels so an image of a duck registers as a horse—to reliably fool perception models. Meanwhile, poisoning introduces subtle backdoors into massive training datasets, allowing attackers to exploit physical triggers to bypass smart security without detection. Because transfer learning relies heavily on third-party data, detecting these embedded backdoors before deployment becomes nearly impossible for downstream engineering teams.

Mitigating these risks requires integrating security into every phase of the ML Ops workflow through ML SecOps, beginning with strict threat modeling and risk assessments before a single line of code is written. Development teams must scrutinize the transparency of external data sources and strategically obscure which specific foundational modules they use to block targeted exploits. By leveraging techniques like adversarial training to build robust teacher models, fine-tuning all model layers instead of just the final classification head to differentiate the student architecture, and utilizing frameworks like the Adversarial Robustness Toolbox, organizations can fortify their deployments against evolving attacks. Adopting these proactive defenses is increasingly critical as upcoming compliance standards—like the EU AI Act—will legally mandate certified robustness for safety-critical applications.

**Keywords:** machine learning security, transfer learning vulnerabilities, adversarial machine learning, model poisoning attacks, evasion attacks, backdoor insertion, adversarial robustness toolbox, ml secops, model extraction, inference attacks, teacher-student models, adversarial training, ai threat modeling, training data perturbations, eu ai act compliance

## Chapters

1. **Introduction to machine learning security and robustness** (02:56) — The very traits making machine learning accessible and powerful inherently create severe security vulnerabilities.
1. **Driving productivity with complex machine learning models** (04:42) — Massive parameter models enable semantic and instance context recognition to dramatically increase developer productivity.
1. **Overcoming data limitation barriers in automated applications** (06:29) — Organizations lacking resources or sufficient annotated data leverage accessible frameworks and machine learning as a service.
1. **Leveraging transfer learning to train efficient student models** (08:08) — Using feature extraction layers from massive teacher models allows rapid task learning without huge computational burdens.
1. **Uncovering the inherent security risks of transfer learning** (10:46) — Relying on generic teacher architectures allows adversaries to leverage known vulnerabilities against derivative models.
1. **Exploring inference, extraction, and the adversarial threat landscape** (12:34) — Security defenses continually adapt against foundational attack vectors designed to infer private dataset occurrences or steal proprietary algorithms.
1. **Evading detection through targeted adversarial example perturbations** (14:26) — Minimal input optimizations force perception systems to misclassify critical targets like road signs or audio profiles.
1. **Compromising datasets through backdoor and data poisoning attacks** (16:09) — Injecting rare trigger patterns into minimal training instances grants unauthorized system access without affecting regular functionality.
1. **Integrating threat modeling into machine learning security operations** (18:23) — Prioritizing use-case risks and environment impacts prior to deployment establishes standardized machine learning security workflows.
1. **Evaluating transparency across external teacher model resources** (20:53) — Validating outsourced training processes and withholding architecture details minimizes supply chain risks for deployed models.
1. **Developing fail-safes and defenses through adversarial training** (22:21) — Minimizing prediction error across adversarially perturbed datasets inoculates critical applications against straightforward input manipulation.
1. **Stress testing and fortifying dependent student machine models** (23:54) — Fine-tuning foundational layers and tracking deployed model behaviors helps identify domain shifts or emerging adversarial actions.
1. **Security frameworks and resources for developing robust models** (25:18) — Standardizing defenses with established cybersecurity guidelines and robust testing toolboxes operationalizes enterprise machine learning safety.

## Related Moments

- [Utilizing industry threat models for AI security](https://www.wearedevelopers.com/videos/715-a-hundred-ways-to-wreck-your-ai-the-in-security-of-machine-learning-systems) (from "A hundred ways to wreck your AI - the (in)security of machine learning systems")
- [Establishing core machine learning fundamentals and robust guardrails](https://www.wearedevelopers.com/videos/829-multimodal-generative-ai-demystified) (from "Multimodal Generative AI Demystified")
- [Generating adversarial patterns and defending consumer machine learning models](https://www.wearedevelopers.com/videos/824-hacking-ai-how-attackers-impose-their-will-on-ai) (from "Hacking AI - how attackers impose their will on AI")
- [Dual usage of machine learning in cybersecurity](https://www.wearedevelopers.com/videos/770-skynet-wants-your-passwords-the-role-of-ai-in-automating-social-engineering) (from "Skynet wants your Passwords! The Role of AI in Automating Social Engineering")
- [Understanding data poisoning and model bias risks](https://www.wearedevelopers.com/videos/1563-prompt-injection-poisoning-more-the-dark-side-of-llms) (from "Prompt Injection, Poisoning & More: The Dark Side of LLMs")
- [Unique privacy vulnerabilities in language model architectures](https://www.wearedevelopers.com/videos/1218-data-privacy-in-llms-challenges-and-best-practices) (from "Data Privacy in LLMs: Challenges and Best Practices")

## Related Articles

- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [How machine learning can help us tell fact from fiction](https://www.wearedevelopers.com/magazine/509-how-machine-learning-can-help-us-tell-fact-from-fiction)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)

## Related Jobs

- [Staff Developer Advocate, GitHub Security Lab](https://www.wearedevelopers.com/jobs/ext/1921051-staff-developer-advocate-github-security-lab) at **GitHub**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1355348-machine-learning-engineer) at **TWILIO**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [Security Architect - AI](https://www.wearedevelopers.com/jobs/ext/1581899-security-architect-ai) at **ZEISS Group**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1597388-machine-learning-engineer) at **ZEISS Group**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1377841-machine-learning-engineer) at **Almedia**