How AI Is Reshaping the Web—and What It Means for Content Creators

In early 2023, Matthew Prince—the CEO of Cloudflare, a company powering roughly 20% of the internet—began getting urgent calls from media executives. But the danger they were warning about wasn’t a cyberattack or a hostile nation-state. It was something much more transformative: artificial intelligence.

These executives had seen what was coming before most—AI was not just disrupting workflows; it was threatening the core structure of how traffic flows across the internet. As users increasingly turned to AI chatbots like ChatGPT instead of traditional search engines, the age-old practice of clicking through to websites began to decline sharply. Instead of links, users now get answers.

And that seemingly small shift is sending shockwaves across the digital ecosystem.

AI Chatbots Are Stealing the Show—And the Clicks

Since its launch in late 2022, OpenAI’s ChatGPT has become a global sensation. With an estimated 800 million users and the top spot in the iPhone App Store, it’s revolutionized how people look for information online. According to Apple, Safari’s traditional web searches saw their first-ever decline earlier this year, largely due to users switching to AI-based queries.

The dominance of OpenAI has also pushed rivals into action. Google, which still holds over 90% of the search market in the U.S., has added AI features to Search, including AI-generated summaries called “overviews” and a full chatbot-like “AI mode” launched in May 2024. Google now encourages users to “let Google do the Googling for you.”

But as Google takes on the job of summarizing content, fewer users are clicking through to actual websites.

The Data Is Clear: Website Traffic Is Falling

According to Similarweb, human-generated search traffic dropped by 15% globally over the past year. Some categories—like hobbyist forums—remain stable. But sites historically relied on to answer user queries are seeing double-digit losses:

• Education and science sites: -10%
• Reference platforms: -15%
• Health sites: -31%

This matters because fewer visitors mean fewer ad impressions, subscriptions, and engagement opportunities.

Neil Vogel, CEO of Dotdash Meredith (publisher of People, Food & Wine, and others), explains the crisis bluntly: “Google broke the deal.” His sites once received 60% of traffic from Google; now it’s barely above 35%. And it’s not just the volume of traffic that’s changing—Google’s AI overviews often answer user queries directly, with no need to click through. The result? A staggering 69% of news-related searches now end without any click at all.

Stack Overflow, Wikipedia, and the Death of the Open Web?

Platforms powered by user contributions, like Stack Overflow and Wikipedia, are seeing a sharp drop in engagement. Prashanth Chandrasekar, CEO of Stack Overflow, says AI is “choking off traffic” to his site. With fewer readers, there are fewer contributors. Wikipedia has warned that AI-generated answers are blocking users from reaching and participating in its ecosystem.

This threatens the very idea of the open web. If people no longer visit sites, how can those sites survive—let alone grow?

“Wooing and Suing”: Content Creators Fight Back

In response, content publishers are pursuing two strategies: negotiating licensing deals with AI companies and launching lawsuits when negotiations fail.

• News Corp (owner of WSJ and NY Post) inked a deal with OpenAI but is simultaneously suing Perplexity AI.
• The New York Times partnered with Amazon while suing OpenAI.
• Reddit licensed its content to Google for $60 million annually—yet saw its stock plunge due to weak traffic growth.

These deals vary wildly in value and coverage. And they’re not always enough to offset the losses.

Legal Setbacks and Political Pressure

Many AI companies claim that training their models on public content qualifies as “fair use.” And so far, U.S. courts are siding with them. In two high-profile rulings in California, judges dismissed copyright claims against Meta and Anthropic.

Adding to the challenge, political support is swinging in favor of Silicon Valley. President Donald Trump has backed the tech industry’s need to innovate rapidly, even removing a top copyright official who opposed AI’s unrestricted access to copyrighted data.

New Business Models: Making AI Pay to Crawl

While large publishers can sue or negotiate, smaller sites can’t. Most of the internet’s content comes from mid-sized or independent platforms, which individually lack the clout to demand compensation.

But new infrastructure is emerging to level the playing field:

1. Cloudflare’s AI Access Controls

Cloudflare now asks new users whether they want to allow AI bots to crawl their content—and under what terms. It’s also testing a “pay-as-you-crawl” system where bots must pay a fee to access site content.

CEO Matthew Prince envisions “a world where humans get content for free, and bots pay a tonne for it.”

2. Tollbit: A Bot Paywall

Tollbit offers a flexible way for sites to charge AI bots based on content value. For example, fresh news articles can cost more to crawl than archived content. In Q1 2025 alone, Tollbit processed 15 million such microtransactions for over 2,000 publishers—including the Associated Press and Newsweek.

CEO Toshit Panigrahi believes this could lead to more original content, as AI firms prioritize unique sources over generic web pages.

3. ProRata’s Ad-Revenue Sharing Model

Led by Bill Gross (inventor of pay-per-click advertising), ProRata proposes a fairer AI search ecosystem. Its engine, Gist.ai, redistributes ad revenue to sites based on how much of their content was used in the AI’s answer.

Its goal isn’t just to compete with Google—it’s to model an equitable future for the web.

Beyond Search: Publishers Rethink Distribution

Realizing that search traffic may never return to previous levels, content creators are pivoting:

• Stack Overflow now emphasizes its enterprise subscription product, Stack Internal.
• News outlets are focusing on apps, newsletters, events, and paywalls to build direct relationships with readers.
• Audio and video formats—which are harder for AI to summarize—are gaining traction.
• YouTube, still hard for AI to replicate, is now the top destination for referrals from AI-based search tools.

Is This the End of the Open Web?

Not necessarily. Despite the challenges, some experts are cautiously optimistic. Robby Stein, a director at Google, says the web is in an “expansionary moment.” According to Google’s own bots, the number of websites online has grown by 45% in the past two years.

AI may also create opportunities for richer, more personalized browsing experiences. By scanning more sources than any human could, answer engines may ultimately deliver more diverse and comprehensive answers.

Still, the future remains uncertain. If creators can’t get compensated fairly, the open internet risks becoming an AI-powered echo chamber—efficient but hollow.

Conclusion:

The web has weathered threats before—from social media takeovers to mobile app dominance—but AI may be its biggest disruption yet. As users flock to AI search tools, traditional content models are crumbling. If the internet is to remain vibrant, diverse, and democratic, AI companies must share the value they extract.

As Bill Gross puts it, “If we want the web—and democracy—to survive, AI search must support the creators who make it possible.”