A not-yet-peer-reviewed preprint posted to arXiv on September 14 puts a caution sign on one of the biggest assumptions in AI search optimization: that ranking higher in the results an AI agent sees guarantees more citations.
Researchers Sriram Selvam and Anneswa Ghosh tested a GPT-5.4 search agent that uses Exa as its search provider. They saved 129 question-and-search transcripts, selected pairs of pages that supported the same fact, and replayed conversations with source order flipped or text formatted differently.
The raw gap looks big, but the swap test tells another story
On the surface, position looked powerful. In saved transcripts, pages in the first search result position were cited 85.1% of the time, compared with 42.8% for pages in the fifth position. That’s a 42.3-point gap.
But those averages combine two variables: where the search provider placed a page and how relevant that page actually was. When researchers swapped the order of matched pages, the effect shrank dramatically.
- Moving the same page higher within a pair increased citation likelihood by 7.9 percentage points, but the result was not statistically significant after multiple-test corrections.
- A follow-up set of 56 order-only swaps estimated the effect at 0.0 points, with a 95% confidence interval from -5.4 to +5.4.
- Reruns of the same inputs changed citation decisions about 15% of the time, meaning roughly one in seven wins or losses could be chance.
The authors estimate about 45% of variation in a single run’s effect comes from model randomness. That’s a strong reason not to label a page “cited” or “not cited” from one AI answer.
Structure may concentrate credit more than win new citations
The preprint also compared plain paragraphs with AI-rewritten structured versions using headings, lists or tables. Structured rewrites received an average of 0.50 more citation markers per answer. The total citation count did not rise; the extra credit appeared to move toward the structured page.
However, the planned main test—whether structure changed the chance of being cited at all—found only a 4.5-point lift, with an interval from -1.4 to +10.4. The paper notes the study could detect effects of about 8.5 points or more, so the evidence is not conclusive.
The authors describe the finding as “an attribution-sensitivity warning, not an optimization tactic.”
What this means for your AI visibility workflow
The study can’t tell you that reformatting live pages will boost AI citations because it only tested rewritten text already retrieved—before crawling, ranking and retrieval are even involved. It also used one model and one search provider.
Still, the pattern aligns with earlier practical warnings. SparkToro reported in January that ChatGPT and Google’s AI Overviews produced the same brand list less than 1% of the time on repeated prompts. Ahrefs found in May that pages cited by AI were about three times more likely to include JSON-LD schema, but adding schema did not clearly increase citations.
For marketing teams, the move is to rerun citation checks several times, separate position from relevance in any vendor report, and treat format changes as a distribution lever rather than a guaranteed new-citation play.
Source: Search Engine Journal
