WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Jul 17, 2025

Fireside Chat with Cloudflare's Chief Strategy Officer, Stephanie Cohen (with Mike Butcher MBE)

Stephanie Cohen , Mike Butcher

Stephanie Cohen declares that uncompensated AI scraping is unsustainable. Learn how Cloudflare is driving a permission-based web by blocking AI training crawlers by default.

Pause
Mute Enter Fullscreen
#1 about 2 min

Blocking AI training crawlers by default on networks

Enabling website owners to block AI crawlers or charge for content scraping.

#2 about 3 min

Balancing content creator incentives and AI training needs

High-quality AI development demands sustainable models that compensate original content authors.

#3 about 2 min

Accessing crawler management tools through a free proxy

Routing traffic through specific gateways enables detailed visibility and immediate control over web scrapers.

#4 about 2 min

Integrating AI management tools directly into developer platforms

Infrastructure optimized for AI environments enables scalable agent deployment through distributed physical networks.

#5 about 2 min

Verifying bot identities through cryptographic authentication protocols

Cryptographically signing bot traffic prevents malicious actors from bypassing indexing rules or spoofing legitimate agents.

#6 about 3 min

Optimizing web infrastructure for machine traffic and agents

Adapting the internet for programmatic access requires robust host interfaces and specialized software development kits.

#7 about 3 min

Defending networks against massive distributed denial of service attacks

Increasing automated bot volume amplifies the urgency for exact threat identification techniques at edge locations.

#8 about 4 min

Adapting to changes in traditional web search and traffic

The shift from standard search indexing to generative summaries fundamentally alters established internet navigation pathways.

#9 about 2 min

Verifying digital content authenticity through open validation standards

Implementing tracking standards like C2PA ensures transparent tracing of media origination and generative alterations.

#10 about 5 min

Engaging globally and expanding infrastructure operations engineering teams

Opening direct feedback loops and recruiting across new international tech hubs accelerates the deployment of novel capabilities.

Matching moments

1:24 min

Monetizing automated web scraping with Cloudflare pay per crawl

Brian Whippo Brian Whippo · World Congress 2025

5:14 min

Analyzing the surge in internet bot and crawler traffic

Chris Heilmann +2 · LIVE

2:18 min

Implementing anti-bot protocols with Privacy Access Control Tokens

Chris Heilmann +2 · LIVE

53 sec

Identifying technical hurdles with aggressive AI web crawlers

Klaus-M. Schremser Klaus-M. Schremser · World Congress 2026 Europe

2:18 min

Managing security risks introduced by autonomous AI agents

Christian Heilmann Christian Heilmann · World Congress 2025

4:56 min

Q&A on platform discovery, security, and prompt injection

Rishabh Budhiraja Rishabh Budhiraja +1 · World Congress 2026 Europe

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33
11 months ago

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28
about 1 year ago

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel, Laura Moritz, Madalina Florean

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12
about 1 year ago

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe, Konstantin Vinogradov, Nils Eiteneyer, Stephan Wirris

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19
about 1 year ago

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler, Jasmin Kaiser, Jeff Hausmann, Sonja Alvarez

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09
about 1 year ago

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher, Jan Oberhauser, Keno Dreßel, Mirko Novakovic

AI Sovereignty: What Does It Take? thumbnail
29:13
about 1 year ago

AI Sovereignty: What Does It Take?

Boris Hecker, Clemens Wasner, Robin Hermann, Sonja Alvarez

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 24, 2026 · 17:30–18:00

Stage 5

Securing AI Agent Infrastructure: Identity, Attestation, and Trust at Scale

Abdel Fane

Founder of OpenA2A

Abdel Fane
Open session

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Tech Leaders Stage

AI Governance & Compliance: Who Is Responsible When the Bots Start Acting?

Kumesh Aroomoogan, Paul Anthony Claxton, Eric Osiakwan

Kumesh Aroomoogan
Paul Anthony Claxton
Eric Osiakwan
Open session

World Congress 2026 North America

September 24, 2026 · 14:50–15:20

Stage 1

The Era of Machine-Driven Defense is Here: Headless Security

Loris Degioanni

Founder & CTO of Sysdig

Loris Degioanni
Open session

World Congress 2026 North America

September 24, 2026 · 14:50–15:20

Stage 7

When Agents Became Users: Rearchitecting Identity and Permissions for AI at Scale

Yoav Gal

Product Lead

Yoav Gal
Open session

World Congress 2026 North America

September 24, 2026 · 16:50–17:20

Mainstage

Building AI Products vs. Building With AI

Aparna Dhinakaran, Rukmini Reddy, Tamar Bercovici

Aparna Dhinakaran
Rukmini Reddy
Tamar Bercovici
Open session

World Congress 2026 North America

September 25, 2026 · 09:00–09:30

Stage 7

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel

Related videos

Web Scraping, AI Agents, and the Future of Open Source - Kevin Lewis (Apify) thumbnail
40:29
Coffee With Developers • 3 months ago

Web Scraping, AI Agents, and the Future of Open Source - Kevin Lewis (Apify)

Fighting the Next Wave of Cybercrime thumbnail
30:27
World Congress 2026 Europe • 3 months ago

Fighting the Next Wave of Cybercrime

Michele Zuccala, Misha Bragin, Ross Kukulinski, Frank Schlesinger, Tomislav Tipurić

Exploring AI: Opportunities and Risks in Development thumbnail
35:01
LIVE • almost 2 years ago

Exploring AI: Opportunities and Risks in Development

Angie Jones, Kent C Dobbs, Liran Tal, Chris Heilmann

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
LIVE • 2 months ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE – AI vs the Web & AI in Browsers thumbnail
53:46
LIVE • 11 months ago

WeAreDevelopers LIVE – AI vs the Web & AI in Browsers

Chris Heilmann, Daniel Cranney, Raymond Camden

WeAreDevelopers LIVE – AI and Privacy, On-Device AI and More thumbnail
1:00:55
LIVE • 9 months ago

WeAreDevelopers LIVE – AI and Privacy, On-Device AI and More

Chris Heilmann, Daniel Cranney, Tarek Ziadé

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.