In 2014 Facebook tried to be the open web's ad network. It mostly failed. Google AdSense is now in quiet retreat under Vidhya Srinivasan's restructured ads org. AI engines from OpenAI, Anthropic, Google, Meta, and Perplexity are scraping publisher content for training data and retrieval. Three moments, one structural question — who pays the open web? The answer is being negotiated in 2026 across licensing deals, federal court filings, and the Cloudflare bot-management dashboard.
The 2014 Facebook bet
Facebook Audience Network launched April 2014 under David Jakubowski and the ads team David Fischer ran at the time. The pitch was clean: take Facebook's identity graph and targeting data, apply it to ad inventory on third-party apps and sites, beat Google's open-web ad network on yield. For a few years, FAN did real volume. Mobile app developers including Spotify, Pinterest, Tinder, and a long tail of mid-market mobile publishers liked the CPMs. DSP integrations with The Trade Desk, MediaMath, and AppNexus (later Xandr) were strong.
Apple's App Tracking Transparency rollout in iOS 14.5 in April 2021 — driven by Craig Federighi and the Apple privacy team — killed most of the model. Without IDFA, FAN's targeting advantage collapsed on iOS, costing Meta what Mark Zuckerberg said on a 2022 earnings call would be roughly $10B in revenue that year. The product still exists in attenuated form. It is no longer a strategic pillar. The lesson from FAN was clear in retrospect: an audience network is only as durable as the identity layer that underwrites the targeting.
Why audience networks lost
Three structural reasons. Identity loss — ATT, third-party cookie deprecation pressure, GDPR enforcement under data protection authorities including France's CNIL and Ireland's DPC. Walled-garden gravity — Meta, TikTok, and YouTube own the engagement, so the off-platform inventory monetizes worse than the on-platform inventory. Brand safety drift — advertisers got burned by adjacency too many times. The 2017 YouTube advertiser boycott led by Procter & Gamble and Unilever after the Times of London investigation, the 2020 Stop Hate For Profit Facebook boycott organized by the ADL and Free Press, the recurring Breitbart and InfoWars adjacency crises that hit any open-web programmatic buyer who didn't run aggressive exclusion lists. The trust cost compounded.
Google's AdSense retreat
Google AdSense has been the open web's default monetization layer for two decades since Susan Wojcicki and the original Google Ads team built it out of the Applied Semantics acquisition. The 2024 announcement that AdSense for Domains and certain AdSense placement formats underwent significant program changes — including the move from per-impression to revenue share recalibration — was the loudest signal yet that Google's interest in the open web is narrowing to YouTube, Search, and the Google Display Network's premium tier.
Independent publishers absorbed the changes one update at a time. The cumulative effect is that fewer mid-tier publishers can run on AdSense alone. The replacements — Mediavine under Eric Hochberger, Raptive (the rebranded AdThrive), Ezoic, and the SSPs Magnite under Michael Barrett, PubMatic under Rajeev Goel, OpenX under John Gentry, TripleLift under Eric Berry, and Sovrn — fill the gap for now. None of them have the scale to replace Google as the open web's economic backbone. The publisher consolidation cycle accelerated: Vice Media filed Chapter 11 in May 2023. BuzzFeed News shut down April 2023. The Messenger collapsed in eight months and 250 jobs gone by January 2024. Sports Illustrated fired most of its staff in early 2024 following the AI-generated content scandal exposed by Futurism. Pitchfork was merged into GQ in January 2024. Jezebel was sold by G/O Media and revived by Paste in late 2023.
The AI crawler economics
AI engines need training data and retrieval data. Both come from the open web. OpenAI, Anthropic, Google, Meta, Mistral, Cohere, and Perplexity have each been engaged in some combination of licensing deals, scraping, and litigation since 2023. The publisher position is straightforward: the value created by AI engines is partly derived from publisher content, and the compensation has not matched the extraction.
Three categories of arrangement have emerged. Licensing deals — OpenAI signed with News Corp (a deal valued in trade press reporting at $250M over five years), Axel Springer (reportedly $5M to $7M per year), the Associated Press (the first major deal, May 2023), the Financial Times under Roula Khalaf in April 2024, Dotdash Meredith, Vox Media, The Atlantic under Nicholas Thompson, Le Monde, Prisa Media, and Hearst Magazines. Litigation — The New York Times v. OpenAI and Microsoft filed December 2023, now in active discovery; multiple author class actions led by Sarah Silverman, Ta-Nehisi Coates, and Jonathan Franzen; the Center for Investigative Reporting's case; the Daily News and Mother Jones suits. Scraping standoffs — most of the rest. Anthropic's content licensing posture has been more conservative than OpenAI's. Perplexity has faced the most public criticism for citation and content-handling practices, including a heated dispute with Forbes' Randall Lane in mid-2024 and a separate Wired investigation into republished content.
Per-query micropayments — proposed by writers and publishers, not implemented at scale. Bulk licensing deals — implemented for top-tier publishers, not for mid-tier or independent. Crawl-access pricing — Cloudflare under Matthew Prince launched its Pay Per Crawl product in July 2025 and is the most credible scalable model; the company's bot-management dashboard now identifies more than two dozen AI crawlers and allows per-bot allow, block, and price actions. The llms.txt convention proposed by Jeremy Howard at Answer.AI and the ai.txt proposals from Spawning AI address access, not compensation. The Creative Commons CC Signals project, launched 2024, attempts to formalize machine-readable content-use signals. The compensation layer is unsolved.
What publishers should do
Audit which AI crawlers are accessing the site and at what depth — through server logs or Cloudflare's bot-management product. Decide which to allow and which to block — the calculation is different for general news (different again for Reuters than for the New York Post), for trade publications, for reference content, and for vertical media. Pursue direct licensing where the publisher has structural negotiating leverage, which now means primary reporting, proprietary data, named authority, and category dominance. Diversify monetization beyond AdSense and direct programmatic into newsletter sponsorships through Substack-adjacent channels, paid memberships, events, and licensed data products. The publishers that survive the next reset are the ones that produce content the AI engines need and cannot replace from a hundred other sources.
The open web's economic layer has been rewritten three times in twelve years. The fourth rewrite is in progress. The publishers building leverage now are the ones who will set the terms.
Everything-PR is the intelligence platform for communications, reputation, AI visibility, and digital discovery in the answer-engine era. Thirty-plus publications. Publishing since 2009. Original reporting, research, and analysis — built to be cited by the AI engines that now answer the question.
