222: AI VISIBILITY IS A VANITY METRIC WITH WIL REYNOLDS

The Traffic Correlation Has Broken

Seer Interactive's marketing team went from four people to zero over 18 months. Wil Reynolds found himself sending company tweets while watching organic traffic crater by 80%. Then something unexpected happened: the pipeline went up. Social traffic converted at five times the rate of organic search. Branded search increased. Direct traffic climbed. The leading indicator everyone trusted for two decades stopped leading anywhere useful.

The bifurcation appeared in the data around the same time ChatGPT gained traction. Traffic and pipeline had tracked together for years, then suddenly split in opposite directions. Wil overlaid the metrics and realized the correlation that built careers in SEO had fundamentally broken.

Why AI Visibility Numbers Lie

ChatGPT doubled the length of its answers in November 2024. Every AI visibility tracking tool showed improvements. But did cash registers ring more often? The same number of humans saw longer answers containing more brand mentions. Visibility scores climbed while actual exposure remained flat.

Google's AI Overviews compound the measurement problem. Clicking a brand link in an AI Overview triggers a Google search for that brand rather than visiting the company website. Google's traffic numbers stay healthy because every click generates a new search query. The 17 brand links Wil tested for time tracking software sent 15 of them back to Google search results.

This creates perfect conditions for selling vanity metrics. Sophisticated buyers learned decades ago that rankings meant nothing without traffic, and traffic meant nothing without conversions. The same lesson applies to AI visibility, but the pressure on marketing budgets has increased dramatically. When search represents 30% of spend and craters overnight, executives don't want to hear that everyone is still learning.

Real Humans Use Brands in Prompts

Seer's team watched actual people interact with LLMs rather than relying on scraped data and proxy metrics. The finding that 44% of users include brand names in their prompts changes the optimization target entirely. People aren't asking "what's the best project management software" as often as marketers assume. They're asking "how does Asana compare to Monday" or "is Notion better than Confluence for documentation."

Head-to-head brand comparisons represent the prompts worth tracking. Generic category queries generate visibility scores but rarely generate customers.

The Training Data Reality Check

Gemini 3.1 launched with training data from January 2025. Every optimization effort since then has zero provable impact on answers generated from training data. Marketers cannot call Sundar Pichai and request model updates because a client is upset. This 16-month experimental lag means organizations are running optimization programs without any feedback mechanism.

The three types of AI search require different approaches: search-led systems use web indexes plus AI, answer-led hybrids employ tool usage and web access, and fully generative systems rely entirely on training data. Most marketers treat all AI answers identically.

Phone Numbers and Fraud Exposure

LLMs hallucinate phone numbers for businesses. The wrong number appearing in an AI answer creates immediate fraud risk. Financial services companies targeting elderly customers face particularly dangerous exposure when scammers answer calls intended for legitimate advisors. This security vulnerability receives far less attention than visibility metrics.

Trust Compounds, Hacks Mortgage Your Brand

Michelin built a restaurant guide in 1900 that still drives foot traffic and pricing power 125 years later. RAMP launched 52 AI-generated restaurant pages in a single day. Both involve food content. One represents rotisserie chicken prepared by a chef. The other resembles a chicken nugget. Only one builds lasting trust.

The framework Wil proposes moves through three stages: be seen, be believed, be chosen. Visibility gets brands into the room. But when prospects search for your team members and find no shared content, no conference appearances, no engaged newsletter readers, belief collapses. Trust transfers from people who demonstrate expertise publicly. Listicles don't create belief. The humans behind the brand do.