An AI Overview told searchers that a bakery near Kitchener was closing on July 31. It wasn’t. An automated buying agent invented a CPM because it had no real pricing data. These are not edge cases anymore: hallucination has become a live operational risk for marketing, search, commerce and programmatic teams.
Why generative AI guesses confidently
Large language models write one token at a time. Each token is chosen because it is statistically probable, not because a fact-checker approved it. When a fact appears constantly in training data, the most likely continuation is usually right. When it appears rarely or not at all, the model still produces an answer that looks just as fluent.
Research from OpenAI and Georgia Tech, published on September 4, 2025, framed this as a binary classification problem. If a model cannot reliably separate valid statements from invalid ones, the probability engine that drives text generation will still produce errors. The paper argued that the base hallucination rate should be at least the share of facts that appear exactly once in training data. Asked for a researcher’s own dissertation title, three chatbots gave different confident answers, and none was correct.
Evaluation makes it worse. Most benchmarks score right or wrong and do not reward “I don’t know,” so a model that guesses can outscore one that abstains. Retrieval layers reduce errors but do not eliminate them. In one Munich case, a court found AI Overview claims did not appear in the cited pages.
Three marketing exposures to watch
- Brands as subjects: AI can invent lawsuits, ages, phone numbers or store closures. A Minnesota solar contractor sued Google after a false claim about a nonexistent state attorney general lawsuit; one nonprofit customer cancelled $174,044.12 in projects.
- Marketers as users: NP Digital survey data shows 47.1% of 565 marketers encounter AI inaccuracies several times a week, and 36.5% said hallucinated content had already been published.
- Commercial pipelines: Product metadata, prices and ads can pass through models. Gracenote reported an ungrounded model fabricated every measured field for 19.5% of 2,600 streaming titles.
A practical guardrail stack
The goal is not to ban generative AI. It is to use it where speed matters and add verification where truth matters.
First, ground any output that touches prices, product details, legal or factual claims in source documents. If an answer cannot cite a source, treat it as a draft. Second, make “I don’t know” acceptable in internal workflows; otherwise people and models will both be rewarded for guessing. Third, add a human review step for anything customer-facing, especially SEO content, chat responses and product descriptions. Fourth, monitor AI-generated search results and brand mentions, because corrections often start with a small business denying a fake closure.
For programmatic and agentic buying, watch provenance. IAB Tech Lab’s Agentic Advertising Management Protocols version 2.3 added a pricing provenance field on July 30, 2026 after identifying CPM hallucination in buying agents. The field tracks where a price came from at each negotiation step—a pattern marketing teams should apply to AI-assisted content and data decisions.
The legal and reputational cost is already visible. Courts in the United States, Canada and Germany have started assigning responsibility, and advertising standards bodies are writing hallucination into specifications. As of September 2026, no major model provider claims to have eliminated the behaviour.
Source: PPC Land
