Digital Marketing
Cloudflare Strips AI Opt-Outs From 118 Sites
HasData re-ran 10,894 domains after Cloudflare's September 15 update and found 118 sites lost their managed AI-training opt-outs in robots.txt.
GPTBot.
SEO
Robots.txt in 2026: Crawling Rules, AI Opt-Outs and Gaps
Robots.txt is voluntary, narrow and now central to AI training opt-outs. Here's what SEO and performance marketers need to know about RFC 9309, GPTBot blocks and crawler gaps.
