OpenAI, Google, Microsoft, and Anthropic have agreed to adopt a new IETF-standardized opt-out mechanism that lets publishers control whether their content is used for AI training. The policy token will live in robots.txt files, giving publishers a unified way to set scraping preferences across multiple AI companies rather than negotiating individually.
Background
AI companies have faced growing criticism over the unauthorized scraping of copyrighted content from publishers and creators for training large language models.
- Source
- The Verge
- Published
- Oct 31, 2026 at 04:26 AM
- Score
- 7.0 / 10