I think we're aiming the machine-access fight at the wrong company. Publishers are deciding whether to block Google's AI or license to it, Reddit spent part of its July 30 earnings call on exactly that question, and every move at that table gets covered like a summit. Meanwhile the biggest machine reader of the web belongs to Meta. By one count it reads more than every other AI crawler combined, it sends almost nothing back, and nobody seems to be discussing whether Zuckrawlers should be blocked.
Meta's two crawlers made more than half of the AI traffic DataDome saw
DataDome, a bot-defense vendor, reports 17.7 billion AI agent requests across its network in the second quarter of 2026, up 45% from the first. The growth did not come from Google or OpenAI. Meta-ExternalAgent grew 74% quarter over quarter, Meta-WebIndexer grew 163%, and together Meta's two agents now carry the majority of the AI agent traffic DataDome is seeing, with close to nothing returned in referrals. There is something worth saying out loud before you read the rest of the article: DataDome sells bot protection, so its telemetry arrives with an interest attached, treat the exact numbers as one network's view. The direction is harder to dismiss.
HOW AI IS CHANGING THE INTERNET: THE GOOD, THE BAD, AND THE POINTLESS
Nobody asked us if we wanted the internet changed. So every week I send one email about one thing AI did to it, with the receipts, and what to do about it. Sometimes the honest answer is "not yet."
From someone who pays attention to this for a living.
By one count, Facebook's AI robots read websites more than ChatGPT's, and send almost no one back
Google's crawler came with a traffic deal, Meta's crawlers never did
I think this gap exists because of what Google and Meta have been historically. Google was the one driving traffic. For twenty years the deal was clear: Google reads your website, Google sends you visitors. When Google's AI summaries started keeping those visitors, publishers experienced it as a broken promise, which is why the response looks like a renegotiation, with blocking, licensing, and lawsuits as the levers. Blocking is the lever a website pulls itself, and what courts have said about blocking AI bots is thin so far: no case holds that ignoring a robots.txt disallow creates liability on its own.
Meta has always been about keeping the user inside, something Google is trying to evolve to now. Nobody expected a visit from Meta's crawlers, so when Meta's crawlers became the heaviest readers of the open web, there was no promise to break. It simply does not register as a loss. And yet: the company that perfected the keep-the-user model is now the biggest consumer of everyone else's work, and the company being renegotiated with is busy copying that model.
Once the traffic goes, only the biggest publishers have anything to negotiate
Publishers have nothing to negotiate for if the platforms are trying to keep the user for themselves. The traffic that made the old deal work is the thing being phased out, so the bargaining chip shrinks to getting paid for the content that trains and feeds the AI. That is a losing position for almost everyone, unless you are already huge. Meta does write those checks, but only at the very top: in March 2026 it signed a licensing deal with News Corp worth up to $50 million a year, covering both Meta AI answers and model training, alongside similar arrangements with CNN, Fox News, USA Today, and a handful of others. That table seats a few dozen of the biggest brands in media while Meta's crawlers read everyone. Reddit can put a licensing deal and a public maybe-we-walk on an earnings call. A store, a blog, a trade publication cannot do this in 2026.
Which leaves one honest way for publishers to win: "fine, we can do this without you," and then actually creating that reality. A stronger direct connection with the audience, owned channels the platforms cannot dilute, maybe something that looks like a step back toward the decentralized web and away from the platform-centered one. None of that is quick, and none of it runs through the meeting rooms where the Google negotiation is happening.
Meta's crawlers cost a website almost nothing, and they still take work without paying for it
Meta's crawlers have sent websites almost no traffic and never promised any, so their reading billions of pages takes nothing you ever had, and costs you almost nothing. But this is Meta building its business on the back of people who are doing real work, without compensating them. That is wrong, and it stays wrong whether or not any single website can feel the cost.
And what DataDome reported is only the first behavior of this visitor class. The same machines that read your website today are the ones that will pay to read it or buy from it tomorrow, and the access terms the web sets by brand recognition now are the terms it will be living with then.
So what does someone with an ordinary website actually do about Meta? My honest answer is "nothing today, but keep an eye on it". Keeping an eye on it means knowing who actually reads your website rather than which AI company is in the headlines, because the two are splitting. The negotiation everyone can see is with Google. The reader almost no one seems to be watching is Meta's. Your log files know the difference, even if the headlines do not. So pay attention to them, not the headlines.