Cloudflare has issued a new policy requiring AI companies to distinguish between web crawlers used for search indexing and those used for AI training by September 15. Failure to comply will result in these AI crawlers being blocked by default on many publisher sites protected by Cloudflare's services.
Background
This policy reflects the growing tension between AI developers needing large-scale data for training and publishers seeking compensation or control over how their content is used.
- Source
- TechCrunch
- Published
- Jul 2, 2026 at 01:48 AM
- Score
- 7.0 / 10