You’ve heard, of course, of pay-per-view. And we are used to streaming revenue on a pay-per basis from the likes of Audible and Spotify. This week has seen the launch (admittedly at the moment in beta) of possibly the most transformative source of pay-per revenue of all. Cloudflare is now offering what it calls Pay Per Crawl, which will enable people to charge AI companies for scraping content from their sites.
ALLi News Editor Dan Holloway
Why does this matter? I will confess that I was today years old when I discovered that, in the words of TechCrunch, Cloudflare is “a cloud infrastructure provider that serves 20 percent of the web.” In other words, it’s a big operation. Cloudflare claims that its AI control tools are currently deployed on a million websites.
Control and Compensation for AI Crawling
Pay Per Crawl builds on a series of AI blockers and an AI dashboard that Cloudflare introduced a few months ago. It will allow publishers to charge AI companies for using their content, at the same time as providing data on how many AI crawlers are trying to access their site—an interesting way of testing market size. And that would include us as indies, if our sites are powered by Cloudflare.
In another interesting development, all new sites set up with Cloudflare will now have AI blocking as the default setting, meaning no crawlers will be allowed to scrape sites unless specific permission is granted.
Publishers and news companies have been at the forefront in welcoming this new level of control—unsurprisingly, as they are at the forefront of the lawsuits against AI companies. But as the rollout continues, there will be the possibility for everyone to benefit.
Meanwhile, at Amazon…
Another story that caught my eye as an “and finally” item is the latest from Amazon. The behemoth has, it claims, now deployed its one millionth robot. And it has also launched a new generative AI to control its robot fleet. So there we are!
Thoughts or further questions on this post or any self-publishing issue?
If you’re an ALLi member, head over to the SelfPubConnect forum for support from our experienced community of indie authors, advisors, and our own ALLi team. Simply create an account (if you haven’t already) to request to join the forum and get going.
Non-members looking for more information can search our extensive archive of blog posts and podcast episodes packed with tips and advice at ALLi's Self-Publishing Advice Center.
