> Markdown version of [/videos/770-skynet-wants-your-passwords-the-role-of-ai-in-automating-social-engineering?t=1482](https://www.wearedevelopers.com/videos/770-skynet-wants-your-passwords-the-role-of-ai-in-automating-social-engineering?t=1482). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Skynet wants your Passwords! The Role of AI in Automating Social Engineering

Could your team spot a phishing lure dynamically generated by an autonomous LLM? Learn why passkeys are your only defense when AI makes malicious content indistinguishable from reality.

- **Speakers:** Wolfgang Ettlinger, Alexander Hurbean
- **Event:** World Congress 2023
- **Published:** October 6, 2023
- **Duration:** 31:19
- **URL:** https://www.wearedevelopers.com/videos/770-skynet-wants-your-passwords-the-role-of-ai-in-automating-social-engineering

## Summary

The rapid advancement of artificial intelligence and machine learning is fundamentally degrading the trust placed in everyday digital communications. As tools capable of generating hyper-realistic audio, text, and images become increasingly accessible, bad actors are beginning to leverage these technologies to automate sophisticated social engineering campaigns. By deploying autonomous agents powered by large language models (LLMs), attackers can scale highly targeted engagements—such as voice cloning fraud and personalized phishing—removing the manual labor bottleneck that historically limited cybercriminal operations.

To illustrate the mechanics of this imminent threat, security researchers demonstrated an educational proof of concept utilizing open-source LLMs and service enumerators to craft convincing, autonomous phishing workflows. By querying target DNS records to uncover corporate software stacks, automated attack systems can dynamically generate highly contextual lures impersonating trusted enterprise platforms. Furthermore, while leading commercial models feature usage safeguards, they remain vulnerable to prompt jailbreaking techniques, and a vast ecosystem of uncensored, open-weight models exists that can be trained to iteratively refine malicious outputs without restriction.

Defending against this automated threat landscape requires shifting away from relying on user intuition or currently unreliable AI text detection algorithms. Because AI-generated malicious content is becoming indistinguishable from genuine communication, organizations must adopt robust technical controls like passwordless authentication and passkeys to render traditional credential harvesting obsolete. Ultimately, establishing strict verification protocols, such as configuring S/MIME and PGP for corporate messaging or utilizing shared out-of-band passphrases for personal interactions, is critical for maintaining operational trust when sensory verification fails.

**Keywords:** automated social engineering, LLM autonomous agents, voice cloning fraud, AI-generated phishing campaigns, open-source LLM vulnerabilities, DNS service enumeration, AI prompt jailbreaking, passwordless authentication passkeys, S/MIME email verification, PGP communication signatures, prompt injection countermeasures, deepfake detection limitations, credential harvesting prevention

## Chapters

1. **Future realities of AI in social engineering** (00:04) — How artificial intelligence could automate phishing and scale social engineering attacks in the near future.
1. **Dual usage of machine learning in cybersecurity** (01:03) — How AI tools can be leveraged for both detecting threats and developing automated malicious campaigns.
1. **Eroding trust through artificial media generation** (02:34) — How highly realistic, artificially generated images and audio complicate the verification of digital identities.
1. **Real-world voice cloning and targeted fraud cases** (04:07) — How attackers use cloned voices for CEO fraud and highly targeted family emergency scams.
1. **Scaling malicious tasks with autonomous AI agents** (05:57) — How agents like AutoGPT can be repurposed to perpetually scale scams, phishing, and fake chats.
1. **Designing an automated phishing attack pipeline** (09:35) — The theoretical steps for building a system that gathers target intelligence to perpetually distribute personalized phishing.
1. **Bypassing safety filters with conversational jailbreaks** (11:40) — Exploiting prompt injection and community-found jailbreaks to override restrictions on commercial AI platforms.
1. **Architecture for automated targeted phishing campaigns** (14:11) — Designing a system that uses large language models alongside service enumerators to craft convincing email payloads.
1. **Configuring local models and service enumerators** (17:21) — Utilizing an open-source text generation web UI to orchestrate prompts and identify target infrastructure.
1. **Demonstrating the automated targeted phishing software** (21:39) — Observing a proof of concept automatically generate tailored lures based on a victim's specific domain records and software usage.
1. **Defensive strategies against AI-driven social engineering** (24:42) — Enhancing security postures through passwordless authentication, verified digital signatures, and prompt injection counter-attacks.

## Related Moments

- [Enhancing social engineering and phishing with generative AI](https://www.wearedevelopers.com/videos/926-wwc24-chris-wysopal-helmut-reisinger-and-johannes-steger-fighting-digital-threats-in-the-age-of-ai) (from "WWC24 - Chris Wysopal, Helmut Reisinger and Johannes Steger - Fighting Digital Threats in the Age of AI")
- [Automating compromised roles through AI agents and backup scammers](https://www.wearedevelopers.com/videos/100057-synthetic-insiders-the-new-ai-risk-to-your-org) (from "Synthetic Insiders: The New AI Risk to Your Org")
- [Understanding AI chatbot vulnerabilities and stateful attacks](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems) (from "Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems")
- [Open-source voice cloning and deepfake vulnerabilities](https://www.wearedevelopers.com/videos/1794-wearedevelopers-live-from-javascript-to-webassembly-high-performance-charting-and-more) (from "WeAreDevelopers LIVE – From JavaScript to WebAssembly, High-Performance Charting and More")
- [Bypassing language model safeguards utilizing contextual prompt injection attacks](https://www.wearedevelopers.com/videos/1286-wearedevelopers-live-browser-extensions-honey-scam-jailbreaking-llms-and-more) (from "WeAreDevelopers Live: Browser Extensions, Honey Scam, Jailbreaking LLMs and more")
- [Distinguishing AI-assisted users from experienced security professionals](https://www.wearedevelopers.com/videos/100331-fighting-the-next-wave-of-cybercrime) (from "Fighting the Next Wave of Cybercrime")

## Related Articles

- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)
- [Exploring AI: Opportunities and Risks for Developers](https://www.wearedevelopers.com/magazine/522-exploring-ai-opportunities-and-risks-for-developers)
- [Dev Digest 134 - Where pixels sing?](https://www.wearedevelopers.com/magazine/477-dev-digest-134-where-pixels-sing)
- [How machine learning can help us tell fact from fiction](https://www.wearedevelopers.com/magazine/509-how-machine-learning-can-help-us-tell-fact-from-fiction)

## Related Jobs

- [Security Architect - AI](https://www.wearedevelopers.com/jobs/ext/1581899-security-architect-ai) at **ZEISS Group**
- [Security Engineer](https://www.wearedevelopers.com/jobs/ext/1574416-security-engineer) at **Twilio**
- [Engineer, Offensive Security Organization](https://www.wearedevelopers.com/jobs/ext/1992296-engineer-offensive-security-organization) at **Twilio**
- [Staff Developer Advocate, GitHub Security Lab](https://www.wearedevelopers.com/jobs/ext/1921051-staff-developer-advocate-github-security-lab) at **GitHub**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Security Engineer, Incident Response](https://www.wearedevelopers.com/jobs/ext/1249908-security-engineer-incident-response) at **Twilio**