Beehiiv’s new Cloudflare partnership gives indie journalists a new level of control over AI crawlers

Beehiiv announced a partnership with Cloudflare on Tuesday, granting its independent journalist creators beta access to Cloudflare's AI Crawl Control services. This integration, available through Beehiiv dashboards, allows creators to granularly manage AI agent access to their newsletters, enabling them to opt in or out of crawling and view data on which AI agents are accessing their content, which are being blocked, and the referral traffic generated by AI products. Creators can individually allow or block specific AI agents, with Cloudflare automatically updating these settings for new agents from the same companies. This partnership offers Beehiiv creators a significant level of on-platform control over AI content ingestion, surpassing the limited robots.txt file options available on platforms like Substack and Ghost. Cloudflare's AI Crawl Control is part of a broader initiative to provide publishers with greater control over AI model access to their work and to develop new revenue streams around these controls. Last year, Cloudflare made blocking AI crawlers a default setting for all customers and introduced a "pay-per-crawl" marketplace, allowing customers to monetize AI crawler access to their content.