Most AI visibility audits run in one language: English. A new experiment suggests that’s a blind spot. Same model, same question, different language can mean a different brand comes out on top.
What the test found
Search Engine Journal collected 1,650 responses on September 7, 2026. The core test ran six buying scenarios in English, Arabic, Russian, Simplified Chinese, and Persian, with Claude’s web search off and 40 repeated queries per cell. Comparison runs added ChatGPT, Gemini, and live web search.
For a London-to-Tenerife flight, Claude recommended British Airways 52.5% of the time in Persian and easyJet only 12.5%. In Russian, it flipped: easyJet won 72.5% of recommendations and British Airways didn’t appear once. That’s a 60-point swing driven only by language.
Hotel recommendations were just as volatile. Bahía del Duque led in 43.6% of Chinese responses but only 7.7% of Arabic ones. Hotel Botánico took nearly a third of English recommendations and vanished in Russian.
Web search changes the answer again
When the model could search the live web, seven of ten language-category combinations produced a different winner. JOIA by Iberostar, which barely appeared in memory-only answers, became the leading hotel pick in three languages, including 60% of English responses. The catch: 42.7% of web-search responses were truncated, particularly in Persian and Arabic.
Anthropic’s earlier research found Claude’s values shift by language, leaning toward warmth in Hindi and Arabic and rigor in English and Russian. This new experiment suggests that shift also reaches brand mentions.
Why this matters
Multilingual audiences are not an edge case. More than half of Los Angeles County residents speak another language at home, including roughly 74,000 Persian speakers. If your AI visibility checks stop at English, you’re likely missing how buyers in key markets hear about your brand.
What to do now
The findings are suggestive, not definitive, but the practical fix is clear: run your own repeated tests. Here’s a starting framework:
- List the purchase questions customers ask before choosing in your category.
- Run them in every target-market language across at least three models.
- Repeat the tests over time and compare web search on versus off.
Don’t treat a single snapshot as truth. The experiment found brand recommendations are fluid, and live web access changed winners in most cases, which means your content and PR efforts may influence what AI recommends in each language.
Source: Search Engine Journal
