Google has quietly reopened a small door for SEOs who want to keep some page elements away from Googlebot. John Mueller confirmed that a legally required age-verification popup can be blocked with robots.txt, even though real visitors see it.
The question
An SEO asked on Reddit about a hypothetical site that loads an age-verification popup from a separate domain through an API. They wanted to know if adding a disallow to that external domain’s robots.txt would stop Google from rendering the popup.
Mueller’s answer
Mueller said the approach “would generally work.” He then laid out two ways to prevent Googlebot from fetching the resource:
- Block the external domain. If the popup is embedded via iframe or script from a domain you can influence, that domain’s robots.txt can disallow crawling. Googlebot will not fetch the content and cannot render it.
- Block the JavaScript file. If your own site uses a specific script such as load-verification-popup.js, you can disallow that file in your robots.txt so it never loads during rendering.
Mueller added one important condition: your pages still need to load properly when the resource is not fetched. For an age gate, he indicated that is usually not a problem.
Why this matters
For years the standard advice has been to let Googlebot crawl everything a user sees. Early SEOs blocked CSS and JavaScript to save crawl budget. After Google began rendering pages, sites that hid critical resources lost rankings. This answer draws a clearer line: it is not about hiding content you want to rank for; it is about non-essential overlays.
Use it carefully
This is not a license to serve Googlebot a clean page while humans wade through popups that hide the actual content. That is still cloaking. Mueller’s guidance is specific to an interruptive, legally required overlay that does not affect the page’s core content.
The useful frame is this: if a resource is noise, not the ranking signal, blocking it can simplify rendering. If it is the signal, let Googlebot see it. For D2C and performance teams, age gates, cookie banners, ad-loading scripts and retargeting embeds are candidates for a rendering audit. Block them at the domain or file level, then confirm in Search Console that key content still renders.
Source: Search Engine Journal
