197: Is the Internet Dead? The Rise of Bot-to-Bot Web & the Decline of Human Content with Anne Berlin
Slobodan "Sani" Manic
Website Optimisation Consultant, No Hacks Founder & Keynote Speaker
CXL-certified conversion specialist and WordPress Core Contributor helping companies optimise websites for both humans and AI agents.
One exposed endpoint. That's all it took to generate a transatlantic flight's worth of carbon emissions. Anne Berlin dropped that number during our conversation about the Dead Internet Theory, and it stuck with me. We're not just talking about bots creating content anymore. We're talking about bots consuming it, crawling pages that no human will ever read, burning energy on digital infrastructure that serves no one.
Anne estimates 60% of web traffic is non-human. Only 40% appears to come from actual people. Many pages exist purely as programmatic SEO artifacts, published without anyone intending for a human to find them. The web has shifted from Geocities-era personal expression to AI-generated slop at industrial scale. Budget cuts and digital burnout have left codebases neglected, crawl bloat unchecked, and real content buried under algorithmic noise.
The fix isn't simple, but Anne sees a path forward. Cross-functional collaboration between SEO, DevOps, infrastructure, and security teams can reduce crawl waste and prioritize meaningful content. The alternative is a web that exists primarily for machines to talk to other machines. That's not the internet anyone signed up for.
KEY TAKEAWAYS
- Audit crawl logs regularly to identify bot traffic patterns and endpoints consuming disproportionate server resources
- Cross-train SEO teams with DevOps and security to address crawl waste holistically rather than in silos
- Calculate the environmental cost of neglected code and crawl bloat to build the business case for infrastructure cleanup
- Evaluate programmatic SEO pages for actual human engagement before scaling content production
- Prioritize human-centric content strategies over algorithmic optimization that primarily serves bot consumption
SHOW NOTES
The Web Feels Off for a Reason
Something changed. The internet that once felt like a collection of human voices now reads like a content factory output. Anne Berlin puts numbers to that feeling: 60% of web traffic comes from bots, leaving just 40% that appears human. This isn't speculation from someone watching trends from the outside. Anne's decade of experience analyzing crawl logs and server statistics at enterprise scale reveals a web where machines have become the primary audience.
From Geocities to AI Slop
The early web had personality. Geocities pages with neon backgrounds and autoplay music represented actual humans expressing themselves, however awkwardly. That era feels impossibly distant now.
Today's web runs on programmatic SEO. Pages get published by the thousands without any human intending for another human to read them. These pages exist to satisfy algorithms, to capture long-tail search queries, to fill databases. The content isn't written for people. Anne describes this as the shift from human expression to AI-generated slop at scale.
Budget cuts and digital burnout have accelerated the decay. Teams maintain codebases they inherited rather than built, patching problems without understanding root causes. The result is bloated infrastructure that serves bots more efficiently than humans.
The Carbon Cost Nobody Talks About
One exposed endpoint generated carbon emissions equivalent to a transatlantic flight. That single data point reframes the entire conversation about web efficiency.
Crawl bloat isn't just a technical annoyance or a budget line item. Neglected code and inefficient infrastructure have real environmental costs that compound across millions of websites. Every unnecessary crawl request, every unoptimized endpoint, every abandoned page that still gets indexed burns energy without delivering value to anyone. But who tracks this? Most organizations treat server costs as fixed overhead rather than symptoms of deeper inefficiency.
The Cross-Functional Fix
Anne advocates for breaking down silos between SEO, DevOps, infrastructure, and security teams. Crawl waste isn't purely an SEO problem. Security teams need to know about exposed endpoints. DevOps needs visibility into which pages actually serve users versus which exist as algorithmic artifacts. Infrastructure teams need context about why certain resources get hammered by bot traffic.
This cross-training approach treats the website as a unified system rather than departmental territory. When SEO understands server constraints and DevOps understands search behavior, solutions emerge that neither team would find alone.
Building a Human-Led Internet
Is it too late to save the web? Anne doesn't think so, but the window is narrowing. The alternative to action is a web that exists primarily for machines to communicate with other machines, humans reduced to occasional visitors in their own digital spaces.
Reclaiming the human web requires intentional choices. Prioritizing meaningful content over algorithmic optimization. Measuring success by human engagement rather than crawl efficiency. Treating environmental impact as a genuine constraint rather than an externality. The crusade Anne describes isn't nostalgic. Building a human-first internet means acknowledging that the current trajectory leads somewhere nobody actually wants to go.
QUESTIONS ANSWERED
What is the Dead Internet Theory?
Dead Internet Theory suggests that bots, not humans, now dominate the web as both content creators and consumers. According to Anne Berlin's analysis, approximately 60% of web traffic comes from bots, with only 40% appearing human. The theory describes a web increasingly filled with AI-generated content and automated interactions rather than genuine human expression.
What percentage of web traffic is bots?
Anne Berlin estimates that 60% of web traffic is bot-based, leaving only 40% that appears to come from actual human users. This data comes from analysis of crawl logs, server statistics, and digital infrastructure across enterprise websites. The ratio reflects both legitimate crawlers and various automated systems consuming web content.
How does crawl bloat affect website carbon emissions?
Crawl bloat and web inefficiencies have significant environmental costs. Anne Berlin cites one case where a single exposed endpoint generated carbon emissions equivalent to a transatlantic flight. Neglected code, unnecessary crawl requests, and unoptimized infrastructure burn energy without delivering value, compounding across millions of websites into measurable environmental impact.
What is programmatic SEO and why does it create unread pages?
Programmatic SEO involves generating web pages at scale, often thousands at a time, to capture long-tail search queries. Anne Berlin notes that many of these pages are published without any human ever intending for another human to read them. These pages exist primarily to satisfy algorithms rather than serve actual user needs, contributing to web content bloat.
How can SEO teams reduce crawl waste?
Anne Berlin advocates for cross-functional collaboration between SEO, DevOps, infrastructure, and security teams to reduce crawl waste. This approach treats websites as unified systems rather than departmental territory. When SEO teams understand server constraints and DevOps understands search behavior, organizations can identify exposed endpoints, optimize crawl budget efficiency, and prioritize resources for content that actually serves human users.
What does human-first web mean for content strategy?
Human-first web strategy prioritizes meaningful content over algorithmic optimization, measuring success by genuine human engagement rather than crawl efficiency. Anne Berlin advocates for treating environmental impact as a real constraint and evaluating programmatic SEO pages for actual human value before scaling production. The approach acknowledges that current bot-dominated trends lead to a web that serves machines rather than people.
RELATED ARTICLES
You Can't See How AI Ranks You, So Build What It Can Read
Web strategy in the AI era means optimizing for systems you are not allowed to look inside. The UK's June 17 order forcing Google to explain how it ranks, AI Overviews included, is the first real crack in that opacity. The durable move does not wait for the box to open: build a website a machine can read, parse, and verify.
How to Measure AI Search Visibility
AI search visibility is not one number, and the metric most tools sell you, citations and mentions, is decoupled from the outcome you actually want, being recommended. Here is what to measure instead, why a single visibility score is noise, and how six practitioners and the 2026 data say to read it.
Google's Open Knowledge Format Could Work For Websites, Too
Google published the Open Knowledge Format this week, a way to store knowledge as a folder of linked markdown files. It was built for internal company data, but it could give a website something better than a flat markdown copy of its pages, a graph of how its ideas connect. I tried it on the No Hacks website to show what that takes.
ENJOYING THIS EPISODE?
Practical strategies for making your website work for AI agents and the humans using it. Read by SEOs, developers, and AI researchers. Exclusive tools, free for subscribers.