Google’s deprecation of the AJAX crawling scheme in 2015 shifted the fundamental expectation for JavaScript-heavy sites from “provide Google a pre-rendered snapshot in place of your real content” to “produce a DOM that Googlebot itself can render and understand directly.” Before that shift, site owners were expected to detect a special crawler request and serve a separate, pre-rendered HTML snapshot in place of the JavaScript-driven experience a real user got. After it, the expectation became that Googlebot would render the actual page using a headless version of Chrome, execute its JavaScript, and index whatever that render produced, no special snapshot required. The SPAs that still run into trouble today are the ones that never fully adjusted to that second model: sites using purely client-side routing with nothing rendered on first load, and sites that gate meaningful content behind interactions Googlebot doesn’t simulate.
What the old scheme required, and why it existed
The original AJAX crawling scheme was Google’s proposed workaround for a real limitation of the time: crawlers generally could not execute JavaScript, so a page that built its content dynamically in the browser was, from a crawler’s point of view, mostly empty. Google’s scheme asked developers to use a specific URL convention, hash-fragment URLs prefixed with #! (the “hashbang” pattern), and to make those URLs resolve, for crawlers, to a special escaped-fragment version at a URL containing ?_escaped_fragment_=. When Googlebot encountered a hashbang URL, it would request the escaped-fragment equivalent instead, and the server was expected to respond with a fully rendered HTML snapshot of what that JavaScript state would have produced for a real user, generated ahead of time or on the fly via a headless browser or a separate rendering service running server-side.
This was a genuinely useful stopgap for its era, but it had obvious downsides even at the time. It required maintaining an entirely separate rendering path solely for crawlers, distinct from what real users experienced, which is inherently fragile since the two versions can drift out of sync. It also depended on a specific URL structure (the hashbang) that had its own downsides for usability and wasn’t necessary for the JavaScript frameworks of the time to function.
The shift after deprecation
Google announced the deprecation of the AJAX crawling scheme in 2015, explaining that Googlebot had become capable of rendering pages more like a modern browser does, including executing JavaScript, which meant the escaped-fragment workaround was no longer necessary as a general recommendation. This is the crux of the shift: instead of asking developers to produce two separate versions of a page (a real one and a crawler-specific pre-rendered one), Google’s expectation became that a site should produce a single implementation that, when its JavaScript executes, results in a DOM containing the actual content, and that Googlebot would run that JavaScript itself as part of a rendering step and index the result.
This changed what “crawlable” means in practice. Under the old model, crawlability was a question of whether you’d built and correctly served the escaped-fragment alternative. Under the new model, crawlability became a question of whether the page’s client-side code, when executed in a Chromium-based renderer without any special crawler-specific accommodation, actually produces the intended content in the DOM within the resources and time Google’s rendering pipeline allocates to it. Google’s current JavaScript SEO guidance reflects this directly, describing rendering as a distinct processing step that follows initial crawling, and recommending patterns like server-side rendering, static generation, or hydration approaches specifically because they reduce reliance on client-side execution being the only path to visible content, rather than because a special crawler-only snapshot is expected.
What still fails to meet the expectation today
Even though the underlying capability shifted years ago, plenty of SPAs built since then still run into indexing problems that are, structurally, a modern variant of the same old gap: content that exists only after client-side execution succeeds under conditions the site’s build doesn’t guarantee for a crawler.
Purely client-side-routed applications with no server-rendered initial content are the most common failure. A framework that ships an essentially empty HTML shell, with all routing, data fetching, and content rendering happening entirely in the browser after JavaScript loads, places the entire burden of “does this page have content” on the rendering step succeeding cleanly. If that JavaScript bundle is large, depends on multiple sequential API calls, or has any execution error under Googlebot’s environment that doesn’t show up in a developer’s local browser, the page can render as empty or partial from an indexing standpoint, even though Google is technically capable of running the JavaScript. This is why server-side rendering, static site generation, or hydration-based frameworks are recommended: they guarantee that meaningful content exists in the initial HTML response independent of whether client-side JavaScript execution goes perfectly, which removes an entire category of risk rather than just reducing it.
Content gated behind interactions Googlebot does not perform is the other persistent failure category. Googlebot does not click buttons, does not fill in and submit forms, does not simulate hovering to reveal a dropdown, and does not otherwise interact with a page the way a human visitor would to reveal additional content. It’s worth being precise here: Googlebot renders the page and executes JavaScript that runs automatically as part of page load, but it does not perform arbitrary user gestures to trigger additional application states. A single-page application that requires a user to click “Load More” to fetch and display the next set of results, or that hides primary content behind a tab that only renders its content on a click event, will have that content invisible to indexing regardless of how well the initial render otherwise works. This applies just as much to content hidden in accordions that lazy-render only on expand, or multi-step flows where later steps’ content only exists in the DOM after a prior step’s interaction completes.
A related, subtler version of this problem shows up in SPAs that technically render the right content eventually, but only after conditions Googlebot’s rendering environment may not satisfy the same way a real browser session does: content dependent on persistent state from a previous visit (cookies, local storage set by an earlier session), content that only loads after a delay tied to a real user’s dwell time, or content whose fetch depends on viewport-related triggers implemented in ways incompatible with how Googlebot’s rendering environment behaves.
A hypothetical illustration
Imagine a hypothetical single-page application, “Example Dashboard,” where the product catalog only renders after a user clicks a “Show all products” button, and the initial HTML shell served to any visitor, including Googlebot, contains no product content at all. Hypothetically, even though Google is fully capable of executing that app’s JavaScript, the catalog would still fail to get indexed, not because of any leftover AJAX-crawling-scheme limitation, but because Googlebot never performs the click needed to trigger the content’s appearance in the DOM, which is the modern version of the same underlying gap described above.
The practical takeaway
The historical arc matters less for its own sake than for what it implies about diagnosing a modern SPA indexing problem: the question is never “does Googlebot run JavaScript,” since it clearly does and has for years, the question is always “does this specific page’s JavaScript, executed automatically without any user interaction, reliably produce the actual content in the DOM.” Any SPA where the honest answer involves “only if the user clicks something first” or “only if every one of several sequential API calls succeeds within whatever time and resource budget Google’s renderer allocates” is still operating, in effect, under the old expectations, even though the old escaped-fragment workaround it might have once used is long gone and no longer functions as a fallback.
