SUPERCHARGE YOUR ONLINE VISIBILITY! CONTACT US AND LET’S ACHIEVE EXCELLENCE TOGETHER!
Google Search has always exposed more information to a machine than a normal searcher necessarily notices. A result page is visually simple, but underneath the interface sits a structured collection of titles, snippets, result types, metadata, links and destination addresses. For years, that underlying structure created an important assumption for SEO software and search-data companies: once a system could reliably retrieve the Google results page, it could generally identify where each result pointed without having to ask Google a second question.
The google.com/goto rollout changes that assumption.
In the affected Search results, the clickable destination is no longer necessarily present as a directly readable publisher URL in the result’s href. Instead, Google can place an intermediary URL such as:
https://www.google.com/goto?url=CAES…
The browser follows that Google URL, receives a redirect, and ultimately reaches the publisher’s page. To the human searcher, the difference is almost invisible. To a system collecting Search data at scale, it introduces another stage into the data path.
Google confirmed the rollout publicly on 26 August. Its explanation was deliberately broad: a spokesperson said Google has a history of deploying technical measures against evolving forms of abuse and regularly takes steps to protect its services and users. Google did not publicly specify a complete technical description of the /goto mechanism or identify every category of automated activity it is intended to address.
That distinction is important. The existence of the redirect is documented. Its practical effect on automated collection can be measured. But claims about Google’s exact internal motivation, cryptographic construction or long-term design should not be presented as established facts unless the underlying evidence supports them.
The technical question is therefore more interesting than “How do I decode this URL?”
The real question is:
What happens when a search result stops being a directly addressable data object and becomes a Google-mediated resolution event?
That is the boundary this first part examines.
What Google Changed With /goto
From Direct Destination URLs to Google-Controlled Intermediaries
The traditional model of a Google organic result was conceptually straightforward:
Google Search
↓
Result
↓
https://publisher.example/page
A scraper parsing the HTML could extract the anchor’s href, normalise it, and associate that URL with the result’s position.
The new model can instead look like this:
Google Search
↓
Result
↓
https://www.google.com/goto?url=CAES…
↓
HTTP redirect
↓
https://publisher.example/page
The publisher page has not disappeared. The user can still reach it by clicking the result. What has changed is where the publisher URL exists in the machine-observable chain.
Previously, the destination was available at the first extraction point.
Now, for an affected result, the first extraction point can provide only a Google-controlled intermediary.
Independent testing published in September found that Google Search can return /goto links instead of direct destination links, while the final publisher URL remains accessible after resolving the redirect. The same research also found that the implementation is not necessarily identical across every browser and environment, which is an important warning against treating /goto as a perfectly uniform replacement for all Google Search links.
This produces an important conceptual distinction:
A Google Search result and the destination URL of that result are no longer necessarily the same data object.
The result has its own identity inside Google’s Search system. The publisher URL exists at the end of a resolution process. A measurement system has to connect those two identities correctly.
That sounds like a small implementation detail.
At scale, it is not.
Anatomy of a /goto Search Result
A simplified affected result can be represented as:
<a href=”https://www.google.com/goto?url=CAES…”>
<h3>Example Result</h3>
</a>
The critical part is the href.
The visible search result may still display information that allows a human to recognise the destination domain. But the machine extracting the anchor does not necessarily receive:
https://example.com/article
It receives:
https://www.google.com/goto?url=CAES…
The url query parameter is therefore misleading if interpreted according to the conventions of ordinary URL parameters. It is not simply a query-string representation of the destination.
This is why a conventional URL extraction pipeline can appear to work while quietly returning the wrong object.
A parser might successfully report:
{
“position”: 1,
“link”: “https://www.google.com/goto?url=CAES…”
}
From a pure HTML-parsing perspective, that output is valid.
From an SEO-data perspective, it is incomplete.
The expected object was:
{
“position”: 1,
“link”: “https://example.com/article”
}
The difference between those two outputs is the entire problem.
The Redirect Chain and HTTP Behaviour
The intermediary is not merely decorative.
Testing reported by multiple technical observers shows the /goto endpoint can issue an HTTP redirect to the final destination. One independent analysis documented the chain as:
google.com/search
↓
google.com/goto?url=…
↓
302 Found
↓
publisher URL
The important HTTP detail is the 302 response. The final destination can therefore be obtained from the redirect response rather than by decoding the token itself.
This creates two separate operations:
- Extract the Google intermediary URL
- Resolve the intermediary to discover the destination
A traditional parser performed essentially the first operation and stopped.
A /goto-aware collector has to perform both.
That distinction also explains why the change can be operationally significant even if the token is never reverse-engineered.
The token does not have to be cracked.
Google itself will tell the requesting client where the token resolves by returning the redirect.
From an engineering standpoint, this is a crucial observation because it moves the problem away from cryptanalysis.
The practical question becomes one of request orchestration, resolution, attribution and scale.
What Google Has Officially Confirmed
The strongest public statement currently available from Google is relatively narrow.
On 26 August, Google confirmed that the /goto mechanism is being rolled out and characterised it as part of its technical measures against evolving forms of abuse. The company did not provide a public specification of the token format, its generation process, its cryptographic properties or the exact classes of automated systems affected.
That leaves several layers of the story that must be kept separate.
Confirmed:
- Google Search is using /goto redirects for affected result links.
- The links can redirect users to the intended publisher destination.
- Google considers the rollout part of its broader technical measures.
- The mechanism changes what appears in the Search result link itself.
Observed independently:
- The url= parameter can contain an opaque CAES… value.
- Base64URL decoding does not simply produce the publisher URL.
- The token can be resolved through Google’s redirect endpoint.
- Behaviour can vary between browsers, sessions and Search environments.
Not publicly established:
- The exact token-generation algorithm.
- Whether the payload should technically be described as encrypted.
- Whether every byte has a documented semantic meaning.
- Whether the present format is permanent.
- Whether the same implementation will eventually cover every Search surface.
This distinction matters because the /goto story has already attracted explanations that go further than the available evidence.
Reverse-engineering becomes useful precisely where speculation stops.
Inside the CAES Token
Why the Token Looks Like Base64
The first thing a technically minded observer notices is the CAES prefix.
The value is URL-safe enough to resemble a Base64URL-encoded object. That naturally leads to the first hypothesis:
Perhaps Google has simply Base64-encoded the destination URL.
If that were true, the problem would be trivial.
Take:
CAES…
Decode Base64URL.
Recover:
Done.
That is not what current testing shows.
Independent reverse-engineering work published in September found that the CAES… value can indeed be Base64URL-decoded, but the output is binary rather than a readable destination URL. Further structural analysis indicates a small protobuf-like envelope followed by protected binary data.
That is the first major break from the traditional URL model.
The string is encoded, but decoding the outer representation does not reveal the destination.
What Happens When the Payload Is Base64URL-Decoded
Consider the conceptual transformation:
CAESVAHrOzAV…
↓
Base64URL decode
↓
binary bytes
The result is not:
https://publisher.example/article
Instead, researchers have observed a structured binary representation.
One September examination reported that a sample token contained a small protobuf-like envelope, with an integer field and a binary payload. Another test of hundreds of tokens found a consistent basic envelope structure and binary material that did not expose the destination as plaintext.
The important technical conclusion is not that the payload has definitively been identified as one particular cryptographic construction.
It is this:
The outer Base64URL layer is not the final encoding layer.
There is additional structure underneath it.
That means a scraper cannot solve the problem by applying a generic decoder.
The Binary and Protobuf-Like Structure
The presence of protobuf-like fields is particularly interesting because it suggests that the token is not merely a random string generated independently of Google’s internal data model.
However, “protobuf-like” should not be casually converted into “Google is using protobuf with field X meaning Y”.
A binary format can resemble protobuf because of its field encoding conventions without giving an observer enough information to reconstruct the full underlying schema.
A responsible reverse-engineering description therefore looks like this:
URL-safe representation
↓
binary envelope
↓
structured fields
↓
opaque/protected payload
rather than:
Base64
↓
Protobuf
↓
encrypted URL
The latter asserts more than the evidence currently demonstrates.
This distinction becomes particularly important when discussing whether the token is “encrypted”.
What the CAES Prefix Tells Us
The prefix itself is evidence of structure, not an explanation of the structure.
It tells us that the token has a repeatable outer representation.
It does not, by itself, tell us:
- what key was used;
- what cipher was used;
- whether encryption is involved;
- whether the destination URL is contained directly;
- whether the token refers to an internal Google object;
- whether part of the token represents request context;
- or whether the token can be reconstructed independently.
This is a recurring problem in reverse-engineering web systems.
A recognisable encoding is not the same thing as a documented protocol.
The right question is therefore not:
“What does CAES stand for?”
The better question is:
“What invariant structure survives decoding, and what information remains inaccessible?”
That produces a more useful engineering conclusion.
Why Decoding the Token Does Not Reveal the Destination
Suppose a collector receives:
https://www.google.com/goto?url=CAES…
It can:
- Parse the URL.
- Extract the url parameter.
- Base64URL-decode the parameter.
- Inspect the resulting bytes.
- Identify repeated structural characteristics.
None of those operations necessarily provides the final publisher URL.
The destination appears to be recoverable through resolution, not straightforward local decoding.
That is the key distinction.
A local parser asks:
“What does this string contain?”
The resolver asks:
“Where does Google send this string?”
Those are different questions.
Is the /goto Token Encrypted?
Encoding, Serialisation and Encryption Are Different Problems
The terminology needs to be precise.
Encoding changes representation.
For example:
text → Base64URL
The original information remains recoverable without a secret key.
Serialisation converts structured data into a machine-readable byte representation.
For example:
fields → binary message
Encryption transforms data using a cryptographic algorithm and secret material so that recovering the plaintext requires the relevant cryptographic capability.
An opaque result after Base64 decoding does not automatically prove encryption.
It could be:
- serialised structured data;
- compressed data;
- encrypted data;
- signed data;
- an internal reference;
- a combination of several mechanisms;
- or something else entirely.
Current public reverse-engineering evidence demonstrates opacity and structured binary data, but does not provide enough information to confidently publish a complete cryptographic specification.
Testing the Plain-URL Hypothesis
The simplest hypothesis is:
token = Base64URL(destination)
Observed behaviour contradicts that hypothesis.
If the destination were simply encoded in that way, decoding should expose a recognisable URL string.
It does not.
That makes ordinary Base64 extraction an insufficient solution.
Testing the Deterministic-Hash Hypothesis
The next possibility is that Google could derive the token deterministically from the destination.
Conceptually:
destination URL
↓
deterministic transformation
↓
same token every time
If that were the model, the same destination should tend to produce the same token across independent Search results.
Current observations do not support such a simple model.
Independent testing found that the same destination can receive different tokens in different SERPs, while repeated instances of the same destination within a particular SERP can show token reuse.
That is a significant clue.
It suggests that the token should not be treated as a universal identifier for a URL.
A system should not assume:
token A = page X forever
or even:
hash(page X) = token A
The token appears to have a stronger relationship with the Search-result context than a simple permanent URL encoding would imply.
Same Destination, Different Token
Imagine the same publisher page appearing for two different queries:
Query A
↓
example.com/page
↓
CAES-Token-A
and:
Query B
↓
example.com/page
↓
CAES-Token-B
If the tokens differ, then the token cannot safely be treated as the page’s canonical identifier.
This matters for caching.
A naïve system might try:
token → destination
as its primary persistent mapping.
A more robust system should store:
query
SERP context
position
token
resolved destination
timestamp
environment
and use the resolved destination as the durable entity wherever possible.
Same SERP, Repeated Destination and Token Reuse
The inverse observation is also important.
If the same destination appears more than once within the same Search context, token reuse can occur according to current testing.
That suggests the token is not simply random noise.
But neither does it prove exactly what Google is encoding.
Several architectural explanations remain possible:
- result-level identity;
- destination-level identity within a request;
- an internal reference;
- contextual metadata;
- signed or protected state;
- or another Google-specific representation.
The available evidence supports describing the token as opaque and context-sensitive, rather than assigning it a precise internal meaning that has not been publicly documented.
What Can Actually Be Established From the Evidence
The strongest current conclusion is surprisingly practical:
There is no demonstrated need to decode the token locally in order to obtain the destination URL.
The HTTP redirect already provides a resolution mechanism.
That means engineering effort spent trying to reconstruct Google’s internal token format may be academically interesting but unnecessary for a production collector.
The more important engineering problem is making the resolution step:
- reliable;
- low-latency;
- correctly attributed;
- observable;
- cache-aware;
- and resistant to changes in Google’s implementation.
That is where the /goto wall starts to become economically important.
/goto Changes the Problem From Parsing to Resolution
The Old SERP Extraction Model
The conventional extraction pipeline looked roughly like:
Request Google
↓
Receive SERP HTML
↓
Identify result
↓
Read href
↓
Normalise URL
↓
Store result
The destination was embedded directly in the result object.
A result could therefore be represented as:
{
“position”: 4,
“title”: “Example Page”,
“url”: “https://example.com/page”
}
The collector’s primary challenge was finding and correctly parsing the relevant HTML.
The New Resolution Model
An affected /goto result changes the pipeline:
Request Google
↓
Receive SERP HTML
↓
Identify result
↓
Read href
↓
Detect /goto
↓
Resolve Google redirect
↓
Read destination
↓
Normalise URL
↓
Store result
The additional stage is not theoretical.
Independent testing has observed the /goto request returning the destination through an HTTP redirect, while Search-data providers have reported having to adapt their pipelines to resolve the intermediary and continue returning direct destination URLs. SerpApi, for example, reported on 5 September that it had deployed a fix and was again returning direct destination URLs through its Search API.
This is a useful distinction when interpreting reports that a SERP API has “fixed” the problem.
The underlying Google behaviour may still exist.
The API has simply inserted a resolution layer between Google’s output and the API’s customer-facing schema.
Why Following the Redirect Is an Additional Data-Collection Operation
Consider a SERP with ten affected results.
Under the old model:
1 SERP request
↓
10 destination URLs
Under the new model:
1 SERP request
↓
10 intermediary URLs
↓
10 resolution operations
↓
10 destination URLs
The exact implementation can optimise this substantially. For example, duplicate intermediary URLs can potentially be resolved once, and a resolver can issue requests concurrently.
But the fundamental change remains:
The destination is no longer available entirely from the initial SERP document.
That creates additional work even if the work is small per result.
At low volume, this may be negligible.
At large scale, it becomes infrastructure.
Why Local Token Decoding Is Not the Same as Destination Recovery
This is perhaps the most important technical distinction in Part 1.
A developer might successfully decode:
CAES…
into binary.
They might even reverse-engineer parts of the binary structure.
That does not mean they have recovered:
https://example.com/page
The successful chain is:
Token
↓
Google resolution endpoint
↓
HTTP Location
↓
Destination
not necessarily:
Token
↓
local decoder
↓
Destination
The first is currently demonstrable.
The second has not been demonstrated as a general-purpose local solution.
For production systems, that distinction should drive architecture.
Reverse-Engineering Methodology
The /goto mechanism is especially useful as a case study in how modern Search reverse-engineering should be performed.
The objective should not be to collect one interesting token and speculate about its internals.
The objective should be to establish repeatable behavioural invariants.
Capturing Raw SERP HTML
Start with the raw document.
For every test, retain:
- complete HTML;
- timestamp;
- query;
- language;
- country;
- device class;
- logged-in state;
- browser;
- IP environment;
- result position;
- result type.
The raw document matters because post-processed API output can hide the behaviour being investigated.
A SERP provider may already resolve /goto, making the API response appear unchanged even though the underlying Google HTML has changed.
That is exactly what happened with some commercial Search-data systems during the rollout. SerpApi reported that the redirect format briefly affected certain URL fields before its engineering team adapted the extraction process.
Capturing Browser Network Traffic
The next layer is network observation.
Instead of asking only:
“What href did the DOM contain?”
capture:
Which URL did the browser request?
What status code did it receive?
What was in Location?
How many hops occurred?
How long did resolution take?
This turns an apparently opaque URL into a measurable transaction.
A browser session can therefore provide two distinct datasets:
DOM dataset
result → /goto?url=…
Network dataset
/goto?url=…
→ 302
→ https://publisher.example/page
Both are necessary if the objective is to understand the full Search-result path.
Building a Token Corpus
One token proves almost nothing.
A useful research corpus should contain hundreds or thousands of observations.
Each observation can include:
| Field | Purpose |
| Query | Search context |
| Position | SERP location |
| Result type | Organic/rich/other |
| Displayed domain | Human-visible identity |
| /goto token | Raw intermediary |
| Resolved URL | Final destination |
| Timestamp | Temporal comparison |
| Browser | Environment comparison |
| Session state | Behaviour comparison |
| Location | Geographic comparison |
| IP class | Network comparison |
Once this corpus exists, several hypotheses can be tested rather than guessed.
Differential Testing Across Queries and Sessions
The strongest experiments deliberately change one variable at a time.
For example:
Same query
Same location
Same browser
Different session
Then:
Same query
Different location
Then:
Same destination
Different query
Then:
Same query
Different browser
The purpose is to determine which characteristics of the token are stable.
A token that changes when the query changes behaves differently from a token that changes only when the destination changes.
Likewise, a token that changes across browsers may contain or depend upon context that a browser-independent identifier would not.
Comparing Browsers, IPs, Locations and User Agents
Current testing indicates that /goto is not necessarily exposed identically to every browser and environment.
A September study reported /goto results in Chrome and Edge, direct destination links in Brave and LibreWolf for comparable searches, and another Google redirect format in Safari. The same research also observed /goto in both manual and automated browser sessions.
This is exactly why broad statements such as:
“Google has replaced every Search URL with /goto.”
should be treated cautiously.
The more technically defensible formulation is:
Google is deploying /goto redirects across affected Search result environments, with observed behaviour varying by browser, session, location and result context.
The distinction is not semantic nitpicking.
It determines how a production data collector should detect the condition.
Measuring Redirect Consistency
For every intermediary URL, capture:
HTTP status
Location
response time
headers
success/failure
Then ask:
- Does the same token resolve repeatedly?
- Does it resolve from another connection?
- Does it resolve after a delay?
- Does it require cookies?
- Does it require the same User-Agent?
- Does it behave differently under HEAD and GET?
- Does the destination remain stable?
Some September testing found tokens could remain resolvable across different connections and across subsequent days, although the maximum lifetime of a token has not been established publicly.
That makes one operational rule particularly sensible:
Do not build a long-term identity system around Google’s token. Store the resolved destination.
What /goto Does and Does Not Reveal About Google’s Architecture
The technical evidence tells us something important about the boundary between Search presentation and Search infrastructure.
Confirmed Behaviour
The following is now well established:
- Google Search can place google.com/goto?url=… links in Search results.
- The url parameter can contain an opaque CAES… token.
- The token is not simply the destination URL in ordinary readable form.
- The /goto endpoint can redirect the requester to the final destination.
- Google has publicly confirmed the rollout as a technical measure against evolving abuse.
Strong Technical Observations
Independent testing further indicates:
- Base64URL decoding produces structured binary data rather than the destination URL.
- The binary representation contains a repeatable envelope.
- The same destination can be represented by different tokens in different contexts.
- Browser and environment differences can affect whether /goto appears.
- The implementation is therefore better understood as a dynamic Search-result mechanism than as a static URL transformation.
Reasonable Inferences
From those observations, several architectural conclusions are reasonable.
First, Google has introduced an additional indirection layer between the Search result and the publisher destination.
Second, the intermediary can be resolved without knowing the internal token-generation algorithm.
Third, the token should not be treated as a durable substitute for the publisher URL.
Fourth, the cost of extracting Search data can increase even if the visible Search experience barely changes.
None of these conclusions requires knowing exactly how the CAES payload is generated.
Unresolved Questions
Several important questions remain open.
What exactly is represented by the binary payload?
Is the protected data encrypted, signed, obfuscated or generated through another mechanism?
How much contextual information does Google encode?
What determines when /goto is served instead of another link representation?
What is the token’s maximum lifetime?
Will the format remain stable?
Will the same mechanism expand to additional Search surfaces?
These are not merely academic questions.
A change in any of them could alter how a SERP collector handles the redirect.
For that reason, robust systems should detect /goto based on observed behaviour rather than assume that today’s token format will remain unchanged.
The First Major Consequence: A Search Result Now Has Two Identities
The most useful way to conceptualise the entire change is to separate result identity from destination identity.
The Google Result Identity
Google knows a result as part of a Search context.
Conceptually:
Query
+
Location
+
Language
+
Device
+
Search features
+
Position
+
Result
The result exists inside that context.
The visible title, snippet and destination are representations of that result.
The Publisher Destination Identity
The destination is something different:
https://example.com/article
It is a web resource that exists independently of the Search result.
Historically, the two identities were easy to connect because the publisher URL appeared directly in the Search result’s link.
The /goto layer separates them:
Google result
↓
opaque Google reference
↓
redirect resolution
↓
publisher destination
This creates a new relationship that data collectors must preserve.
Why the Separation Matters for Search Data
Suppose a rank tracker records:
Keyword: enterprise SEO
Position: 4
URL: https://www.google.com/goto?url=CAES…
Technically, the tracker has captured a URL.
But it has not captured the information that an SEO user actually needs.
The useful relationship is:
Keyword
↓
Position 4
↓
Publisher
↓
https://example.com/enterprise-seo
The intermediary is an implementation detail.
The destination is the analytical entity.
That distinction becomes even more important when a system performs:
- keyword-to-page mapping;
- ranking history;
- landing-page analysis;
- cannibalisation detection;
- competitor analysis;
- content-gap research;
- SERP clustering;
- AI citation monitoring.
A system that loses the destination URL has not merely lost a convenient field.
It has lost part of the relationship between Search visibility and the actual web document receiving that visibility.
This is why /goto should not primarily be understood as a URL-format change.
It is a data-model change.
Why /goto Changes SERP Scraping Economics
Part 1 established the critical architectural change: an affected Google Search result can now expose a Google-controlled intermediary instead of the publisher’s destination URL. The CAES… token does not need to be cracked for the system to work, because the /goto endpoint can resolve the token through a redirect.
That observation leads to the more consequential question.
What happens when this additional resolution step is multiplied across millions of searches, thousands of keywords, multiple locations, devices, languages and SERP features?
The answer is not simply “scraping becomes harder”.
The more precise answer is that the cost structure of Search-data acquisition changes.
A scraper that previously extracted a destination URL while parsing the SERP now has to perform another operation to establish the same fact. That operation consumes some combination of network capacity, latency, proxy resources, concurrency, retries and engineering complexity.
And /goto did not arrive in isolation.
It follows a sequence of changes that had already made large-scale Google SERP collection more expensive. In September previous year, Google’s handling of the &num=100 parameter changed, disrupting workflows that had traditionally retrieved up to 100 results in one request. Industry testing and subsequent reporting showed that tools had to move towards pagination or alternative collection methods. Ahrefs documented that the change reduced its ability to assess results beyond the first page before it developed alternative approaches.
Seen together, the changes are more significant than either mechanism alone.
The problem is no longer simply:
“Can a program download Google’s results?”
It becomes:
“How much infrastructure must a program operate to obtain a sufficiently complete, reliable and attributable representation of Google’s results?”
That is the economics of the /goto wall.
The Real Barrier Is Not the Opaque Token, It Is the Additional Work
The temptation is to focus on the CAES string because it is the most visible technical novelty.
But from the perspective of a production SERP platform, the token itself may be the least important part.
If the destination can be recovered by following a redirect, the cryptographic or serialisation details are largely irrelevant to the end result.
What matters is that one piece of information now requires an additional interaction with Google’s infrastructure.
Request Multiplication
Consider a conventional organic SERP containing ten results.
Under the older extraction model:
1 Google Search request
↓
SERP HTML
↓
10 destination URLs
The destination URLs were part of the document being parsed.
The number of HTTP interactions required to establish those ten destinations could therefore remain close to one Search request, excluding ancillary resources and the infrastructure required to obtain the SERP itself.
With /goto, the conceptual flow becomes:
1 Google Search request
↓
SERP HTML
↓
10 Google /goto URLs
↓
10 resolution operations
↓
10 destination URLs
This does not necessarily mean that every implementation must literally issue ten independent requests in every circumstance. A sophisticated collector can deduplicate, parallelise, cache and optimise the resolution layer.
But the information path has changed.
The destination is no longer fully represented in the original SERP response.
That means the collector has acquired a new unit of work.
The difference becomes particularly important at scale.
Suppose a system monitors 100,000 keyword queries per day and resolves ten affected result links per query.
That creates a theoretical upper bound of:
100,000 SERPs
× 10 results
=
1,000,000 resolution events
The exact number in a real system would be lower or higher depending on:
- duplicate URLs;
- result types;
- cached resolutions;
- failed requests;
- pagination;
- geographic variants;
- mobile versus desktop;
- query depth;
- and how the provider structures its collection pipeline.
The point is not that every provider will incur exactly one million additional requests.
The point is that the unit of cost has moved from “SERP acquired” towards “SERP plus destination resolution”.
That is a fundamentally different scaling model.
Latency Multiplication
Request count is only half the problem.
Latency is the other half.
If a collector resolves ten links sequentially, the theoretical chain becomes:
SERP request
↓
Result 1 resolution
↓
Result 2 resolution
↓
Result 3 resolution
↓
…
↓
Result 10 resolution
Even relatively small per-resolution delays can accumulate.
A production system therefore has a strong incentive to resolve links concurrently:
┌─ Result 1
├─ Result 2
SERP ─ Resolver ─┼─ Result 3
├─ Result 4
├─ …
└─ Result 10
But concurrency introduces its own constraints.
The collector must now manage:
- connection pools;
- concurrent request limits;
- timeouts;
- retries;
- backoff;
- duplicate resolution;
- cancellation;
- partial failure;
- and resource exhaustion.
A system that previously spent most of its effort obtaining the SERP may now need a second subsystem dedicated to resolving the objects contained within that SERP.
This is a recurring pattern in distributed systems.
Moving information behind an additional network boundary rarely makes the information disappear. It makes retrieval stateful and operationally observable.
Proxy and IP Economics
The next constraint is network identity.
A large-scale SERP system may already use a pool of IP addresses or sessions to obtain geographically and behaviourally representative Search results.
Adding a second request layer creates another question:
Which network identity should perform the /goto resolution?
There are several possible models.
Model 1: Same session
SERP request
↓
same session
↓
/goto resolution
This preserves contextual continuity.
Model 2: Same IP, independent connection
SERP request
↓
IP A
/goto resolution
↓
IP A
This may preserve network continuity while allowing connection-level optimisation.
Model 3: Resolver pool
SERP acquisition
↓
Resolver queue
↓
IP pool
↓
Destination
This can increase throughput but introduces another layer of attribution and consistency questions.
A system that collects Search results from one geographic IP and resolves them through an unrelated network may create difficult edge cases if Google’s behaviour varies by location, cookies, session or request context.
The point is not that Google necessarily rejects a resolution performed from another IP.
The point is that a new request creates a new place where network policy can matter.
Concurrency and Connection Management
At modest volume, an extra HTTP request is trivial.
At millions of results, connection management becomes an engineering problem.
Suppose a resolver needs to process 100,000 destination resolutions per minute.
A naïve design might launch an unbounded number of requests.
That creates obvious problems:
- socket exhaustion;
- connection churn;
- excessive simultaneous traffic;
- queue instability;
- retry amplification;
- proxy exhaustion.
A better architecture imposes controlled concurrency.
Conceptually:
SERP Queue
↓
URL Extraction
↓
Resolution Queue
↓
┌──────────────────────┐
│ Resolver Workers │
│ │
│ Worker 1 │
│ Worker 2 │
│ Worker 3 │
│ … │
│ Worker N │
└──────────────────────┘
↓
Destination Store
The number of workers becomes a tunable infrastructure parameter.
Too few:
latency rises.
Too many:
resource pressure rises.
This is precisely why /goto turns an HTML-parsing problem into a systems-engineering problem.
Retry and Failure Costs
A direct href either exists in the document or it does not.
A network resolution can fail in many ways.
For example:
/goto
↓
timeout
or:
/goto
↓
HTTP error
or:
/goto
↓
unexpected response
or:
/goto
↓
redirect chain cannot be completed
The collector must decide what each failure means.
Is the result:
- missing?
- temporarily unresolved?
- permanently unavailable?
- blocked?
- malformed?
- a valid result with an unknown destination?
This distinction is critical.
If the system silently converts:
resolution failed
into:
URL does not exist
it has transformed an infrastructure failure into false SEO data.
That is considerably more dangerous than simply returning an error.
Why Position-100 Tracking Becomes a Different Engineering Problem
The significance of /goto becomes clearer when combined with the previous year &num=100 change.
The previous year Baseline Shift
For years, adding:
&num=100
to a Google Search URL provided a convenient way to request a large result set in one Search response.
Around September previous year, Google began disabling or no longer reliably honouring that parameter. Industry reports documented the disruption to rank trackers and SERP providers, while Ahrefs described the change as affecting its ability to retrieve results beyond Google’s first page until alternative approaches were developed.
The economic distinction is straightforward.
Previously:
1 request
→ roughly 100 results
After the change, systems seeking equivalent depth could need multiple paginated requests.
The industry’s own reporting described this as potentially turning one request into approximately ten requests for the same nominal top-100 depth.
Now introduce /goto.
The conceptual pipeline becomes:
Query
↓
Page 1
↓
10 results
↓
10 URL resolutions
Page 2
↓
10 results
↓
10 URL resolutions
Page 3
↓
10 results
↓
10 URL resolutions
…
If a system wants positions 1–100, the data-acquisition problem is no longer simply “retrieve 100 results”.
It becomes:
Retrieve multiple SERP pages
+
extract affected intermediary links
+
resolve those links
+
preserve position
+
validate destination
That is a very different cost model from the old one-request/top-100 workflow.
This is why /goto should be analysed as part of a sequence of access-cost changes, rather than as an isolated redirect feature.
/goto in the Context of Google’s Broader Search Access Restrictions
The strongest analysis should resist the temptation to treat every Google change as a single coordinated anti-scraping programme.
Google has not publicly provided such a complete architecture.
But the technical effect of multiple changes can still be analysed together.
JavaScript and Browser Execution
Modern Google Search is not simply a static HTML page.
Search can contain:
- dynamic components;
- interactive result modules;
- AI-generated features;
- location-dependent content;
- personalised elements;
- client-side behaviour.
This means a collector seeking parity with a normal browser may increasingly need to reproduce more of the browser environment.
That increases the distance between:
HTTP request
and:
human-equivalent Search session
Bot Detection and Abuse Prevention
Automated access can also encounter:
- rate controls;
- CAPTCHA or challenge behaviour;
- IP reputation effects;
- session differences;
- browser-specific differences;
- traffic-pattern detection.
The previous year num=100 episode itself produced reports of inconsistent behaviour across browsers and sessions, including CAPTCHA-related observations from rank-tracking practitioners. Those observations were industry reports rather than a detailed Google technical specification, so they should be treated as evidence of operational variability rather than proof of one particular detection algorithm.
Search Pagination and Result-Depth Changes
The num=100 change was significant because it altered how much Search data could be obtained per Search request.
Ahrefs subsequently reported that it was gradually restoring top-100 tracking through alternative collection methods, illustrating that the industry did not simply lose the ability to collect deeper rankings. Instead, the collection architecture became more complicated.
That distinction is important.
Google did not make positions 11–100 conceptually nonexistent.
It changed the mechanism through which third parties could retrieve them efficiently.
Dynamic SERP Rendering
The Search result page itself is also becoming more heterogeneous.
A single query can contain combinations of:
- organic results;
- People Also Ask;
- video results;
- local results;
- recipes;
- AI Overviews;
- knowledge features;
- images;
- shopping modules;
- news;
- other specialised result blocks.
A modern collector therefore cannot safely assume:
10 links = 10 organic results
It must identify the semantic result type.
That becomes especially important when /goto appears in non-traditional result surfaces.
SerpApi’s own release history shows that its engineers were resolving /goto behaviour across several Google APIs and result types, including AI Overview, questions and answers, recipes, video key moments, local results and videos.
The significance is architectural:
The intermediary-link problem is not necessarily confined to the ten classic blue links.
Redirect-Based URL Obfuscation
This is the /goto layer.
It does not necessarily stop the result from being displayed.
It does not necessarily stop the destination from being reached.
It changes how the destination is exposed to the machine.
That makes it different from a conventional access block.
The result still exists.
The click still works.
The destination still resolves.
The extraction pathway has simply acquired an additional boundary.
Why These Barriers Compound
Each individual barrier can be manageable.
The difficulty comes from composition.
A simplified modern collection pipeline can look like:
Query generation
↓
Search request
↓
Bot/access validation
↓
Browser/session state
↓
SERP retrieval
↓
Pagination
↓
Result classification
↓
/goto detection
↓
Destination resolution
↓
URL normalisation
↓
Data validation
↓
Storage
A failure at any layer can contaminate the final dataset.
That is why modern SERP collection increasingly resembles distributed data infrastructure rather than a simple scraper.
What /goto Breaks
HTML-Only SERP Parsers
The most obvious failure occurs when a parser assumes:
<a href=”https://publisher.com/page”>
will always contain the final URL.
An affected result may instead contain:
<a href=”https://www.google.com/goto?url=CAES…”>
The parser has not necessarily failed technically.
It has correctly extracted the href.
Its data model has become outdated.
Regex-Based URL Extraction
A regex can successfully identify:
https://www.google.com/goto?url=CAES…
But that does not answer the question:
What page does this result represent?
This is the difference between syntactic extraction and semantic extraction.
The first is easy.
The second now requires resolution.
Rank Trackers
Rank tracking has several dependencies beyond position.
A serious ranking record often needs:
keyword
country
city
device
search engine
position
SERP feature
domain
landing URL
title
result type
If the URL field becomes:
google.com/goto?url=…
the position itself may still be correct.
But downstream workflows can break.
For example:
Keyword
↓
Position 4
↓
Landing URL
↓
Page classification
If the landing URL cannot be established, page-level analysis fails even though the position remains known.
SerpApi explicitly noted that its /goto issue did not change visible rankings, titles, snippets or other parsed result data, but temporarily made the link field less useful for workflows requiring direct destination URLs.
That distinction should not be overlooked.
/goto is not itself a ranking change.
It is a data-representation change.
SERP APIs
The impact on SERP APIs is more subtle.
A good API abstracts away the underlying Search implementation.
If Google changes:
direct URL
to:
/goto
the API provider can add a resolver and continue returning:
{
“link”: “https://example.com/page”
}
From the customer’s perspective, nothing may appear to change.
But the provider has absorbed the complexity.
SerpApi reported exactly this sequence: the rollout initially affected some returned URL fields, engineering work was performed, and by 5 September this year the company reported that direct destination URLs had been restored across affected Google Search result types.
That means a “working SERP API” should not be interpreted as evidence that /goto has disappeared.
It can simply mean the intermediary layer has been successfully abstracted away.
Competitive Intelligence Systems
Competitive intelligence platforms often need more than:
domain = example.com
They may need:
domain
subdomain
directory
landing page
content type
ranking keyword
position
SERP feature
An unresolved /goto link can therefore degrade page-level competitive intelligence while leaving domain-level visibility apparently intact.
That creates a dangerous asymmetry:
the report can look healthy at the domain level while becoming incomplete at the page level.
SEO Crawlers That Depend on SERP Discovery
A conventional crawler that starts with a known URL does not care about /goto.
If it already knows:
https://example.com/page
it can crawl the page directly.
But a system whose workflow is:
Google SERP
↓
discover URLs
↓
crawl discovered URLs
does care.
The intermediary therefore sits precisely at the discovery boundary.
That is why /goto affects some SEO workflows much more than others.
Historical SERP Datasets
Historical data creates another problem.
Suppose a dataset contains:
January this year:
position 5
URL = example.com/page
and a later collection contains:
September this year:
position 5
URL = google.com/goto?url=CAES…
A naïve data pipeline may interpret that as a URL change.
It is not.
It is a collection-method change.
This is one reason provenance becomes increasingly important in Search datasets.
What /goto Does Not Break
A technically serious analysis also needs to define the boundaries of the problem.
Search Console
Google Search Console is fundamentally different from third-party SERP scraping.
Search Console is a Google-controlled reporting system that exposes Search performance data through Google’s own infrastructure.
A /goto intermediary in a browser-visible Search link does not imply that Search Console suddenly needs to parse that link to calculate a site’s performance data.
The two systems have different data paths.
This is why it would be incorrect to say:
“/goto breaks Search Console.”
It does not follow from the architecture.
Direct Website Crawling
If a crawler already has:
https://example.com/article
Google’s intermediary is irrelevant.
The crawler can request the publisher directly, subject to the publisher’s own access controls and normal web-crawling considerations.
Therefore:
/goto is primarily a Search-result discovery and resolution issue, not a general web-crawling mechanism.
First-Party Analytics
Analytics systems operating on the publisher’s website continue to receive the information available through their own measurement architecture.
The Google intermediary does not automatically erase:
- page views;
- server requests;
- campaign parameters;
- internal navigation;
- conversion events.
Its effect is primarily on systems trying to infer destination information by extracting Google’s Search result links.
Known-URL Monitoring
If the system already knows the target URL, it can monitor it directly.
For example:
Known:
example.com/pricing
Need:
check status
check canonical
check content
check rankings
No /goto resolution is required for the page-monitoring component.
Domain-Level SERP Measurement
A collector may still be able to establish:
example.com appears in position 4
even if exact URL resolution fails.
This distinction is extremely important.
A system can have:
high confidence in domain presence
while having:
low confidence in exact landing-page identity.
Those should not be collapsed into a single binary success/failure state.
The Exact-URL Attribution Problem
This is where /goto becomes more consequential for SEO than it first appears.
Domain Visibility Versus Landing-Page Visibility
Consider:
Query: enterprise SEO
Position: 3
Domain: example.com
That may be enough for a basic share-of-voice report.
But an advanced SEO platform may need:
URL: example.com/enterprise-seo
because the URL determines:
- page intent;
- content type;
- internal linking;
- conversion path;
- cannibalisation;
- template;
- topical cluster.
A domain-level answer cannot replace a URL-level answer.
Keyword-to-URL Mapping
SEO systems often construct a graph:
Keyword
↓
Ranking URL
↓
Topic
↓
Traffic
↓
Conversion
If the ranking URL becomes unknown, the graph becomes incomplete.
This can affect:
- keyword clustering;
- page-level reporting;
- content refresh recommendations;
- cannibalisation analysis;
- internal-link recommendations.
Cannibalisation Detection
Cannibalisation systems need to know whether:
keyword A
maps to:
/page-a
or:
/page-b
If both pages alternate in Search, that pattern can be analytically important.
A domain-only record cannot reliably identify it.
Therefore, URL resolution is not cosmetic.
It affects the semantic interpretation of the ranking dataset.
Page-Level Share of Voice
Suppose:
example.com
owns 20 rankings.
A domain-level system can count all 20.
But suppose those 20 rankings are distributed across:
- 2 product pages;
- 8 category pages;
- 6 editorial pages;
- 4 location pages.
That is a very different SEO picture.
Exact URL attribution enables the second level of analysis.
SERP Intent Classification
The landing page can also reveal intent.
For example:
/search-term
might resolve to:
/product/
rather than:
/blog/
That distinction can inform how a SERP behaves.
If URL resolution fails, some classification systems must rely on weaker signals such as:
- title;
- snippet;
- visible breadcrumb;
- displayed domain;
- structured result metadata.
Those can be useful, but they are not equivalent to knowing the actual destination.
Why a Missing URL Can Create a False SEO Signal
Imagine an SEO platform reports:
Keyword:
“enterprise seo agency”
Position:
4
Ranking URL:
Unknown
An analyst may interpret that as a data-quality issue.
Correct.
But if the platform instead drops the result entirely, the problem becomes more serious.
The dataset now says:
No result detected
rather than:
Result detected, destination unresolved
Those are radically different states.
The first implies absence.
The second implies incomplete observation.
A robust /goto-aware platform must preserve that distinction.
The New Failure Modes in SEO Measurement
Apparent Ranking Loss Caused by Resolution Failure
A ranking collector can identify a result but fail to resolve its destination.
If its downstream data model requires a valid URL before storing the ranking, the ranking may disappear from the final report.
That can produce:
Google:
result exists
Rank tracker:
result missing
The discrepancy is methodological, not necessarily algorithmic.
Missing Landing Pages
A platform may still report:
Domain: example.com
Position: 7
but fail to report:
Landing page: /service/enterprise-seo
This is less severe than losing the ranking entirely, but it still corrupts page-level analysis.
Incorrect URL Attribution
A more dangerous failure occurs when the system assigns the wrong destination.
For example:
Keyword A
Position 4
URL = page B
when the actual destination is page A.
That can create false cannibalisation signals and incorrect content recommendations.
In measurement systems, incorrect data can be worse than missing data.
Distorted Share of Voice
If URL resolution failures are non-random, share-of-voice calculations can become biased.
For example, suppose /goto resolution fails disproportionately for one category of result.
The resulting dataset may undercount that category.
That is not a simple 5% missing-data problem.
It is a sampling problem.
SERP Feature Undercounting
The same issue can occur outside conventional organic results.
SerpApi’s September release notes show that /goto handling had to be addressed in several specialised result types, including local results and videos, while its weekly changelog described fixes for AI Overview, questions and answers, recipes and video key moments.
This means an implementation that checks only:
organic_result.link
may be incomplete.
A /goto-aware architecture needs to consider result-type-specific URL fields.
Historical Data Discontinuity
Any major Search collection change introduces a possible break in longitudinal datasets.
A rank-tracking graph might show:
Aug this year
100-result coverage
Sep this year
partial result coverage
If that transition is not recorded, analysts may interpret the difference as a genuine change in Search visibility.
The same problem already appeared during the num=100 disruption, when rank-tracking providers had to adapt their methods for deeper results. Ahrefs documented the disruption and later reported progressive restoration of top-100 tracking.
The lesson is broader:
Every Search collection-method change needs a measurement-version boundary.
How to Detect /goto-Driven Measurement Errors
Detecting Google Intermediary URLs
The simplest diagnostic is to inspect raw link fields for:
google.com/goto
Do not stop there.
Also inspect:
- encoded variants;
- redirects;
- result-type-specific fields;
- API-normalised URLs.
A provider may already resolve the intermediary before returning its output.
Monitoring Resolution Success Rates
Track:
resolved / total affected links
over time.
For example:
Resolution success:
99.8%
↓
97.1%
↓
91.4%
A sudden change should trigger investigation.
Comparing Domain and URL-Level Visibility
Maintain separate metrics:
Domain identification success
Exact URL identification success
If domain identification remains stable while URL identification falls, the problem is likely in destination resolution rather than Search-result detection.
Identifying Sudden Resolver Latency
Track:
- median resolution time;
- p95;
- p99;
- timeout rate;
- retry rate.
A resolver that technically succeeds but takes dramatically longer can still make the collection system operationally unstable.
Comparing Third-Party Data With First-Party Search Data
Search Console and rank trackers answer different questions, so they should not be expected to match exactly.
But large unexplained changes in:
- impressions;
- ranking coverage;
- landing-page attribution;
- keyword counts
can provide useful diagnostic signals when they coincide with a collection-method change.
The key is to avoid treating correlation as proof.
Establishing a Measurement Breakpoint
A serious Search dataset should retain something like:
collector_version
google_link_format
resolution_strategy
timestamp
Then an analyst can distinguish:
SEO performance change
from:
measurement-system change
That distinction is becoming increasingly important.
The SerpApi Litigation and the Legal Context
The /goto rollout also occurred against a background of litigation involving Google and SerpApi.
That context is relevant, but it must be handled carefully.
The existence of the lawsuit does not establish why Google introduced /goto.
What Google Alleged
Google sued SerpApi over claims involving the collection and use of information from Google services.
In July this year, the U.S. District Court for the Northern District of California dismissed Google’s DMCA claims, while giving Google an opportunity to amend portions of the case involving copyrighted material. Search Engine Land reported that the court permanently dismissed claims based on Search results that did not contain copyrighted content and allowed Google to revise claims concerning Search results containing copyrighted content.
What SerpApi Argued
SerpApi disputed Google’s claims and sought dismissal.
After Google filed an amended complaint in August this year, SerpApi described the amended case as narrower, saying that the surviving theories focused on licensed content, principally material appearing in Knowledge Panels, rather than the broader Shopping and Maps theories present in the original complaint. That is SerpApi’s description of the amended pleading, not an independent judicial finding.
What the Court Actually Decided
The July order matters because it established that Google’s original legal theories did not all survive the motion-to-dismiss stage.
But it is equally important not to overread the ruling.
A dismissal of particular claims is not a judicial finding that:
- Search scraping is universally lawful;
- all Search data is freely reusable;
- Google’s technical protections are unlawful;
- or /goto was introduced because of the litigation.
Those are separate questions.
The Significance of the July this year Order
The timing is nevertheless notable.
The sequence was:
July this year
Court dismisses key Google claims
August this year
Google files amended complaint
August 26, this year
Google publicly confirms /goto rollout
The chronology is factual.
The causal explanation is not established by that chronology.
That distinction is essential.
A report can accurately say the two developments occurred in close temporal proximity without claiming that one caused the other.
The /goto Timeline Versus the Litigation Timeline
Some industry commentators have explicitly connected the redirect rollout with Google’s broader dispute over automated access to Search.
That interpretation should be attributed to those commentators.
Google’s public statement did not cite the SerpApi litigation. It described the mechanism generally as part of technical measures against evolving forms of abuse.
Therefore, the strongest formulation is:
The /goto rollout occurred during an active legal and technical dispute over automated access to Google Search, but the public record does not establish that the litigation caused the rollout.
That is both more defensible and more interesting than reducing the story to a simple legal narrative.
Timeline of Google’s Search Access Barriers
The Pre-/goto Architecture
For much of the modern SEO-tool era, the basic model was straightforward:
Google Search
↓
SERP
↓
direct destination URLs
Search-data providers still had to deal with:
- IP reputation;
- request limits;
- browser differences;
- JavaScript;
- pagination;
- result variability.
But destination extraction itself was comparatively direct.
The previous year Search Access Changes
The September previous year &num=100 change was a major event for SERP measurement.
Industry reports documented that the parameter stopped reliably returning 100 results, disrupting rank trackers and forcing providers to reconsider how they collected deeper rankings.
Ahrefs subsequently described a sequence of engineering adaptations, including gradual restoration of top-100 tracking.
This matters because it established a precedent:
Google could materially alter the economics of SERP collection without changing the underlying rankings.
The Emergence of /goto
Public observations of /goto appeared before Google’s August confirmation.
SerpApi’s public roadmap contains an issue dating to June 30, this year documenting /goto links in Google AI Overview results, showing that the behaviour had been observed in specialised Search contexts before the broader August rollout.
By late August, multiple industry observers were reporting broader Search-result use.
The this year Rollout
On 26 August this year, Google confirmed that it was deploying the mechanism.
Its public statement was intentionally non-specific:
Google said it has a history of deploying technical measures against evolving forms of abuse and regularly takes steps to protect its services and users.
That statement is significant precisely because it does not describe the implementation as a ranking change.
The September Adaptation Phase
The first days of September became an engineering race between Google-side changes and Search-data providers.
SerpApi reported on 30 August that it was actively resolving /goto URLs as the implementation evolved.
By 5 September, it reported that it had deployed fixes and was again returning direct destination URLs.
Its release notes subsequently documented additional /goto-related fixes for Google Search, local results, videos and other APIs.
This sequence demonstrates something important:
The problem is not necessarily solved once a provider implements one resolver.
If Google changes where or how intermediary links appear, the provider must continue adapting.
/goto Versus Google’s Older Redirect Mechanisms
Google has used intermediary URLs and click-tracking mechanisms before.
So what makes /goto different?
google.com/url
Older Google Search interfaces and specialised environments have historically used redirect wrappers in which the destination could often be extracted from an explicit parameter.
Conceptually:
google.com/url?q=https://example.com
The important property was that the destination remained comparatively accessible from the wrapper itself.
google.com/aclk
Advertising links use Google-controlled click-tracking infrastructure for measurement and attribution.
The purpose and context are different from organic Search result links.
The broader point is that Google has long had mechanisms for placing itself between a user click and a publisher.
That means /goto is not historically unprecedented as a concept.
Tracking and Attribution Redirects
A redirect can serve many purposes:
- click measurement;
- attribution;
- security;
- policy enforcement;
- link management;
- abuse prevention;
- experimentation.
The existence of an intermediary therefore does not by itself establish its purpose.
What makes /goto notable is the combination of:
- its appearance in Search result links;
- its opaque token;
- its displacement of the readable destination in the result markup;
- and its implications for automated SERP data extraction.
What Makes /goto Technically Different
The key difference is not simply:
Google redirects users
Google has done that before.
The important change is:
Google Search result
↓
opaque Google reference
↓
Google-controlled resolution
↓
publisher destination
The destination is no longer a simple property of the result’s initial HTML representation.
That changes the machine-readable data model.
Engineering a /goto-Aware SERP Pipeline
A production system should treat destination resolution as a separate subsystem.
Detection Layer
The first layer identifies whether a result contains:
google.com/goto
or another Google intermediary format.
The detector should not assume that every result uses /goto.
It should support:
direct
goto
legacy_redirect
unknown
This allows Google to change link formats without requiring a complete rewrite of the extraction pipeline.
Resolution Layer
The next stage resolves affected URLs.
Conceptually:
SERP
↓
Result parser
↓
URL classifier
├── direct ────────────→ destination
├── /goto ─────────────→ resolver
└── unknown ───────────→ fallback
The resolver should return more than a string.
It should return structured state:
{
“status”: “resolved”,
“destination”: “https://example.com/page”,
“http_status”: 302,
“latency_ms”: 184
}
That makes the system observable.
Resolver Queues
A queue separates Search acquisition from destination resolution.
That allows:
SERP acquisition
↓
resolution queue
↓
workers
↓
destination store
If resolution slows down, the system can measure queue depth rather than silently stalling the entire Search collector.
Concurrency Control
Resolution should be bounded.
A resolver can dynamically adjust concurrency based on:
- success rate;
- latency;
- queue size;
- available connections;
- provider limits.
This is standard distributed-systems engineering, but it becomes newly relevant to Search extraction because Google has inserted another network operation into the data path.
Deduplication
If the same intermediary URL occurs multiple times within a collection batch, resolving it repeatedly may be wasteful.
A cache can temporarily map:
goto_token
→
resolved_destination
But that mapping should not automatically be treated as permanent.
As Part 1 established, current evidence does not justify treating the token as a durable URL identifier.
Redirect Caching
A sensible system can maintain short-lived resolution caches.
For example:
Cache key:
exact intermediary URL
Value:
destination + timestamp + status
The TTL should be conservative unless the token’s lifetime and semantics are known.
The goal is not to reverse-engineer Google’s token system.
The goal is to avoid unnecessary repeated resolution while preserving correctness.
Failure-State Modelling
This is one of the most important architectural requirements.
Do not store only:
url = null
Instead use states such as:
direct
resolved
unresolved
timeout
blocked
invalid
unknown
Then downstream analytics can distinguish:
No destination exists
from:
Destination exists but collection failed
That distinction protects the integrity of the dataset.
Preserving SERP Position During Resolution
The resolver must never lose the relationship between:
query
position
result
token
destination
A poor implementation can produce a destination list:
URL A
URL B
URL C
without knowing which URL corresponded to which result position after asynchronous resolution.
A robust implementation preserves the result identity:
{
“query”: “enterprise seo”,
“position”: 4,
“intermediary_url”: “https://www.google.com/goto?url=…”,
“destination_url”: “https://example.com/enterprise-seo”
}
The resolver is asynchronous.
The ranking relationship is not.
Raw-Evidence Retention
The final structured URL should not replace the original evidence.
Store:
raw href
resolved URL
HTTP status
timestamp
collector version
That provides an audit trail.
If Google changes /goto tomorrow, engineers can inspect historical raw data instead of trying to reconstruct what the Search page looked like from a normalised database.
Why the Naïve Workarounds Fail
“Just Decode the Base64”
This addresses the wrong layer.
Base64URL decoding can expose the underlying binary representation, but current public research does not show a practical local transformation from the CAES… token directly to the final URL.
The redirect itself remains the more practical resolution mechanism.
“Just Use Regex”
Regex can identify the intermediary.
It cannot infer the destination if the destination is not present in the string.
The problem is no longer pattern matching.
It is network resolution.
“Just Follow Every Link”
This is closer to the real solution, but it ignores scale.
Following every link introduces:
- additional requests;
- latency;
- concurrency requirements;
- failure handling;
- caching;
- attribution problems.
It solves the logical problem while creating an infrastructure problem.
“Just Use HEAD Requests”
A common optimisation instinct is to replace:
GET
with:
HEAD
But redirect behaviour and server handling cannot simply be assumed to be identical across methods and environments.
A production resolver needs empirical validation rather than assuming that every intermediary will provide an equivalent Location response to every HTTP method.
“Just Rotate User Agents”
User-Agent rotation does not remove the fundamental need to resolve an intermediary URL.
It may change what Search returns, but it does not eliminate the second stage.
It therefore addresses a different problem.
“Just Buy More Proxies”
More IP addresses can increase capacity.
They cannot fix:
- incorrect URL attribution;
- poor resolver architecture;
- token parsing assumptions;
- missing result states;
- historical-data corruption.
Infrastructure scale is not a substitute for a correct data model.
“Just Use Search Console”
Search Console is valuable, but it answers a different question.
It can provide first-party Search performance information for a site.
It does not replace a third-party system that needs to answer:
For this exact query,
in this exact location,
on this exact device,
what appeared at position 7,
and which URL did that result point to?
First-party Search data and independent SERP observation are complementary, not interchangeable.
Part 3: /goto, AI Search and the Future of Search Observability
The first two parts established two separate facts that become much more significant when combined.
First, Google’s /goto mechanism can place an opaque intermediary between a Search result and the publisher destination. Independent testing shows that the CAES… payload can be structurally decoded but does not practically expose the destination URL through ordinary local decoding. The destination can instead be recovered through Google’s redirect behaviour.
Second, the resulting engineering problem is not limited to URL parsing. A Search-data system must now acquire the SERP, identify the intermediary, resolve it, preserve the relationship between the result and destination, handle failures and maintain enough provenance to know whether a change in its dataset represents an actual Search change or a collection change.
That matters even more as Search increasingly becomes an input into systems that do not simply report rankings.
AI retrieval systems, answer engines, citation monitors, competitive-intelligence platforms and agentic browsing systems all depend on one fundamental relationship:
Query
↓
Search result
↓
Source
↓
Source URL
↓
Source content
If the source URL becomes a separate resolution event, then URL resolution becomes part of retrieval infrastructure.
That is the larger significance of /goto.
It is not simply a new Google URL format.
It is a change in the observability layer between Search and the systems that consume Search.
/goto and AI Search Systems
SERP Scraping Versus Web Crawling
One of the easiest mistakes in discussions about /goto is to treat all automated access to the web as one activity.
It is not.
A conventional crawler can begin with:
https://example.com/article
and request that page directly.
A SERP collector begins with:
and asks:
Which pages did Google return?
In what order?
For which Search environment?
The two workflows have fundamentally different starting points.
A simplified crawler looks like:
Known URL
↓
HTTP request
↓
Page
↓
Links
↓
More pages
A SERP-driven retrieval system looks like:
Query
↓
Search engine
↓
Candidate sources
↓
Destination URLs
↓
Source retrieval
/goto primarily inserts friction into the second architecture.
That distinction is crucial.
It would be incorrect to conclude that Google has somehow placed the entire open web behind /goto. It has not.
A publisher’s URL remains an ordinary web address.
What has changed is how easily an automated system can extract that address from an affected Google Search result.
Search Retrieval Versus Destination Retrieval
An AI system performing web retrieval often has at least two logically separate stages.
Stage 1: Candidate discovery
Query
↓
Search
↓
Candidate results
Stage 2: Evidence acquisition
Candidate result
↓
Destination URL
↓
Web request
↓
Page content
The /goto mechanism sits between those stages.
Before:
Candidate result
↓
URL already visible
↓
Fetch page
After:
Candidate result
↓
Google intermediary
↓
Resolve
↓
Fetch page
This adds a network-dependent step before the retrieval system can even decide whether it wants to fetch the page.
For an occasional human-style agent, that extra step may be trivial.
For a system evaluating millions of Search results, it becomes a measurable component of retrieval cost.
Where /goto Sits in an AI Retrieval Pipeline
Consider an AI research system answering:
What are the leading approaches to enterprise search optimisation?
A conventional retrieval pipeline might look like:
User question
↓
Query generation
↓
Google Search
↓
Top results
↓
URLs
↓
Page retrieval
↓
Content extraction
↓
Chunking
↓
Ranking
↓
With affected /goto results:
User question
↓
Query generation
↓
Google Search
↓
Top results
↓
Google /goto links
↓
Destination resolution
↓
URLs
↓
Page retrieval
↓
Chunking
↓
Ranking
↓
LLM synthesis
The additional stage has an interesting property.
It does not necessarily change the information available to the final language model.
It changes the cost and reliability of obtaining that information.
That distinction is central to understanding the effect on AI systems.
Search APIs Versus Browser-Based Agents
Not every AI system interacts with Google in the same way.
There are at least three broad architectures.
Search API architecture
AI system
↓
Search API
↓
structured results
↓
source URLs
The API provider absorbs the complexity of Google’s implementation.
SerpApi, for example, reported that after the /goto rollout it adapted its infrastructure so that its customer-facing APIs continued returning direct destination URLs. It also reported fixing affected result types including AI Overview and several specialised Search result structures.
For an AI company using such an API, /goto may therefore be largely invisible.
The infrastructure provider has effectively become the resolution layer.
Browser-agent architecture
A browser agent can operate more like a human:
Open Search
↓
Observe result
↓
Click
↓
Follow redirect
↓
Read page
The intermediary may therefore be handled naturally by the browser.
But the cost profile is different.
Browser automation is significantly more expensive than extracting URLs from static HTML.
The /goto change therefore matters differently depending on whether the AI system is already paying the cost of a browser environment.
Hybrid retrieval architecture
The third model combines Search APIs with direct retrieval:
Search API
↓
Candidate URLs
↓
Direct HTTP fetch
↓
Content
This model depends heavily on the API provider correctly resolving /goto.
That makes the provider’s extraction quality part of the AI system’s retrieval quality.
Agentic Browsing and Redirect Resolution
An agent that literally clicks a result does not need to understand the CAES token.
It only needs a browser capable of following the link.
This is an important reason not to exaggerate the impact of /goto on AI.
The mechanism does not automatically prevent an agent from navigating to the source.
The more significant effect is on systems that want to inspect, classify or manipulate Search-result URLs before clicking them.
For example, an agent may want to:
- reject certain domains;
- classify source types;
- compare URLs;
- deduplicate results;
- prioritise canonical pages;
- inspect URL paths;
- estimate source relevance;
- build a source graph.
Those tasks benefit from knowing the destination before navigation.
An opaque intermediary makes that pre-navigation analysis more expensive.
This creates a subtle distinction:
Click-based agents can often tolerate indirection.
Data-centric retrieval systems care much more about indirection.
/goto and AI Citation Monitoring
Citation Discovery
AI visibility platforms increasingly ask questions such as:
Which sources were cited?
How often?
For which prompts?
At what position?
Which pages were referenced?
The answer is fundamentally URL-oriented.
Consider an AI answer containing:
Source: example.com
A serious monitoring platform may want to determine whether the cited source was:
example.com/
or:
example.com/resources/enterprise-seo
or:
example.com/blog/search-visibility
The domain alone is insufficient.
The page is the actual evidence source.
Source URL Identification
This is where /goto becomes relevant.
If Google Search is used as one of the discovery layers for an AI citation-monitoring system, the system may encounter:
google.com/goto?url=CAES…
instead of:
example.com/article
The platform must resolve the intermediary before it can store the source identity.
This is not fundamentally different from the problem faced by a rank tracker.
But the downstream consequences can be larger.
A rank tracker may only need to report:
Position 5
An AI visibility platform may need to build a source-level evidence graph.
For example:
Prompt
↓
AI answer
↓
Citation
↓
Publisher
↓
URL
↓
Canonical
↓
Content
↓
Entity/topic
An unresolved URL interrupts that graph.
Canonical URL Resolution
Even after resolving /goto, the job may not be finished.
The destination could itself redirect:
Google /goto
↓
https://example.com/old-page
↓
301
↓
https://example.com/new-page
Or:
Google /goto
↓
tracking URL
↓
canonical page
Therefore, an advanced retrieval system should distinguish at least four layers:
1. Search-result href
2. Google intermediary
3. HTTP destination
4. Canonical document URL
That creates a much richer model:
SERP Result
↓
Google transport URL
↓
Resolved URL
↓
Final URL
↓
Canonical URL
Collapsing those into a single url field destroys useful information.
Citation-Level Attribution
This becomes particularly important when measuring AI visibility.
Suppose a system reports:
Brand:
Example Company
Citation:
example.com
That is a weak measurement.
A more useful record is:
Prompt:
enterprise SEO tools
Answer:
…
Citation:
https://example.com/resources/enterprise-seo-tools
Citation type:
direct source
Discovery path:
Google Search
Resolution:
Google /goto → destination
Canonical:
https://example.com/resources/enterprise-seo-tools
That dataset can support longitudinal analysis.
The first cannot.
Why Domain-Level Visibility Is Not Enough
A domain may have thousands of pages.
If an AI system cites:
example.com
that does not tell an SEO team which content earned the citation.
The actionable question is:
Which document became evidence?
That is a URL-level question.
Google’s intermediary therefore has disproportionate relevance to systems measuring content-level AI visibility.
The New Architecture of Search Data Acquisition
The larger shift can now be expressed more clearly.
From HTML Parsing to Behavioural Resolution
The old SERP extraction mindset was:
Download
↓
Parse
↓
Extract
The newer model is:
Acquire
↓
Observe
↓
Classify
↓
Resolve
↓
Validate
↓
Normalise
↓
Store
This is a fundamentally more sophisticated data pipeline.
The distinction resembles the difference between parsing a document and interacting with a service.
A static document contains its data.
A service can require an additional transaction to reveal the data.
/goto moves part of Search-result information towards the second model.
Search Data as an Acquisition Pipeline
A mature Search-data platform should therefore be thought of as an acquisition system rather than a scraper.
A useful abstraction is:
Google Search
│
▼
Search Collector
│
▼
Result Classifier
│
┌─────────────┴─────────────┐
▼ ▼
Direct URL /goto
│ │
│ ▼
│ URL Resolver
│ │
└─────────────┬─────────────┘
▼
URL Normaliser
│
▼
Result Store
│
▼
Analytics Layer
Each component can fail independently.
That means each component also needs observability.
Provenance Becomes a First-Class Data Attribute
Traditional ranking databases often store something like:
keyword
position
url
date
That is increasingly insufficient.
A modern Search observation should be closer to:
keyword
position
result_type
raw_href
transport_url
resolved_url
canonical_url
collection_timestamp
location
language
device
collector_version
resolution_status
Why?
Because two identical URLs can be obtained through different collection pathways.
And two apparently different URLs can represent the same canonical document.
Without provenance, future analysts cannot distinguish:
Google changed
from:
collector changed
from:
provider changed
from:
publisher changed
That is a major shift in how Search datasets need to be designed.
Measurement Confidence
A mature platform should also stop treating every result as equally certain.
Consider:
Result detected:
YES
Domain:
example.com
Exact destination:
UNRESOLVED
That should not become:
Result:
MISSING
Nor should it become:
URL:
example.com/page
The correct representation is uncertainty.
For example:
Domain confidence: High
Position confidence: High
Exact URL confidence: Low
Canonical confidence: Unknown
This allows downstream analytics to remain honest.
Collector Versioning
Suppose a ranking report changes significantly on 28 August this year.
Without collector metadata, an analyst may investigate:
- Google’s ranking algorithm;
- competitor activity;
- content changes;
- backlinks;
- technical SEO.
But the real cause could be a Search collection change.
A robust system should therefore store:
collector_version = this year.08.28
link_resolution = goto-aware
The exact implementation can vary, but the principle is universal:
Search measurements need methodology versioning.
Search-Environment Versioning
Search results are not generated in a vacuum.
A result can depend on:
- location;
- language;
- device;
- browser;
- session;
- personalisation;
- Search features;
- time.
A /goto investigation makes that even more obvious because independent tests have shown that the link representation can vary across environments. A September this year test found direct and /goto link behaviour differing by browser and session, while other research observed variations across Search result types.
Therefore:
“Google Search”
is not a sufficiently precise description of a measurement environment.
A reproducible observation should identify the environment in which it was collected.
How SEO Platforms Need to Adapt
Separating Ranking Collection From URL Resolution
This is perhaps the most important architectural recommendation for Search-data providers.
Do not build:
parse_result()
→ url
→ save()
as one indivisible function.
Separate:
collect_result()
from:
resolve_destination()
and:
normalise_destination()
That makes the system resilient to changes in Google’s link architecture.
Building Resolver Abstractions
The resolver should not be hard-coded around one exact token structure.
Instead:
Transport URL
↓
Resolver interface
↓
Resolved destination
The implementation can then change without changing the rest of the analytics system.
This matters because Google’s current /goto representation should not be assumed to be permanent.
The September this year industry response already demonstrates how quickly implementations have had to evolve. SerpApi described the rollout as changing across Search features and continued shipping fixes for affected result types.
Preserving Raw SERP Evidence
Normalised data is convenient.
Raw evidence is what allows future debugging.
A platform should retain, where permitted and operationally appropriate:
raw href
result metadata
intermediary URL
HTTP response
resolved destination
timestamp
That allows engineers to reconstruct the event.
Without the raw link, a later investigation may only reveal:
URL = example.com/page
which tells nothing about whether Google originally exposed that URL directly or through /goto.
Handling Partial Resolution
A resolver should be allowed to return partial results.
For example:
{
“position”: 7,
“domain”: “example.com”,
“destination_status”: “unresolved”
}
This is superior to dropping the result.
Partial information can be analytically useful if its limitations are explicit.
Confidence-Aware Reporting
A platform can expose separate quality metrics:
SERP detection rate
Result classification rate
Destination resolution rate
Canonical resolution rate
This creates a data-quality dashboard for the Search measurement system itself.
That is a significant conceptual upgrade.
The SEO platform is no longer merely reporting Google’s Search data.
It is reporting how confidently it was able to observe Google’s Search data.
Re-Baselining Historical Data
When collection architecture changes, historical comparisons need to be annotated.
For example:
Before Aug this year:
direct-link extraction
Aug–Sep this year:
mixed direct + /goto
After resolver deployment:
resolved destination extraction
If a provider fails to record this transition, users may interpret measurement changes as ranking changes.
This is especially important for long-term enterprise SEO reporting.
Avoiding Silent Data Corruption
The worst possible implementation is one that continues producing numbers while silently changing what those numbers mean.
For example:
Before:
URL = publisher destination
After:
URL = Google intermediary
If the schema remains:
url
downstream systems may never notice.
The result is not an obvious outage.
It is semantic corruption.
That is much harder to detect.
A Technical Taxonomy of Google’s Search Access Barriers
The /goto mechanism becomes easier to understand when placed into a broader taxonomy.
| Layer | Mechanism | What changes |
| Rendering | Dynamic/client-side Search | How results become observable |
| Access | Anti-automation controls | Who can reliably retrieve results |
| Depth | Pagination/result-count changes | How much data each request returns |
| Representation | /goto | How destination URLs are exposed |
| Resolution | Redirect chain | How destinations are discovered |
| Rate | Request controls | How quickly data can be collected |
| Semantics | Dynamic SERP features | How results must be classified |
The important point is that these layers operate differently.
A rendering barrier increases client complexity.
A pagination change increases request volume.
A /goto mechanism increases destination-resolution work.
A rate-control mechanism limits throughput.
A dynamic SERP feature increases classification complexity.
Together, they produce a system in which Search observation itself becomes an engineering discipline.
What We Know, What We Suspect and What Remains Unknown
The most responsible conclusion about /goto is not that every popular explanation is correct.
It is that enough evidence now exists to separate fact from inference.
Confirmed by Google
Google confirmed on 26 August this year that it was rolling out the /goto redirect mechanism as part of its technical measures against evolving forms of abuse. The company did not publicly identify the precise abuse categories or provide a full technical specification of the token.
That is the official position.
It should remain the baseline.
Demonstrated Through Technical Testing
Independent testing has established several useful observations.
Affected Search results can expose:
google.com/goto?url=CAES…
instead of the direct publisher URL.
The token can be Base64URL-decoded into binary, but the final destination is not practically recoverable through ordinary local decoding. Research has identified a protobuf-like envelope and protected binary payload.
Other testing has found that the representation is not uniform across every browser or environment.
Third-Party Observations
Search-data providers have documented real-world impact.
SerpApi reported that the rollout temporarily affected direct URL fields and required engineering changes. It later reported restoring direct destination URLs and continuing to monitor evolving variations.
Its changelog also records fixes for /goto handling across multiple result types, including AI Overview, questions and answers, recipes, video key moments, Lens and other Search interfaces.
These reports demonstrate that the mechanism is operationally significant for systems consuming Google Search at scale.
Plausible Architectural Interpretations
It is reasonable to conclude that /goto makes automated extraction of destination URLs more complicated because the destination is no longer necessarily present directly in the initial result link.
It is also reasonable to conclude that the mechanism increases the engineering work required for systems that need exact destination URLs.
It is less defensible to assert the precise internal purpose of every part of the token.
Claims That Should Not Yet Be Treated as Facts
Several questions remain unresolved:
- the complete token-generation algorithm;
- the exact cryptographic construction, if any;
- the semantic meaning of every payload field;
- the complete rollout percentage globally;
- the exact lifetime of tokens;
- the permanent scope of /goto;
- whether all Search surfaces will ultimately use the same mechanism;
- whether Google’s implementation will remain stable.
A September this year report described the rollout as broad, while other testing continues to find variation by browser and environment. That means claims of a single universal implementation should be treated cautiously.
A Forensic Checklist for Investigating /goto
A serious investigation should begin with evidence rather than assumptions.
Capture
Capture:
- raw SERP HTML;
- DOM representation;
- result position;
- result type;
- raw href;
- visible destination text;
- network requests;
- redirect status;
- final destination.
Compare
Run controlled comparisons across:
- browser;
- User-Agent;
- session;
- IP;
- location;
- language;
- device;
- signed-in state;
- query.
Do not change all variables simultaneously.
Otherwise, a difference cannot be attributed confidently.
Measure
Record:
/go-to detection rate
resolution success rate
resolution latency
timeout rate
redirect status
destination stability
These measurements can reveal whether a problem is:
- Search acquisition;
- URL resolution;
- network reliability;
- or downstream parsing.
Validate
For every resolved destination, where relevant, validate:
- final URL;
- canonical URL;
- redirect chain;
- domain;
- path;
- result position;
- result type.
This prevents the resolver from becoming another source of silent data corruption.
Preserve
Retain enough evidence to reconstruct the observation later.
At minimum:
timestamp
query
position
raw href
resolved URL
collector version
resolution status
environment
That turns a scraper into a measurable data-acquisition system.
The Broader Shift From SERP Scraping to Search Observability
This is where the /goto story ultimately becomes bigger than /goto.
For years, third-party SEO platforms operated under a relatively simple conceptual division.
Google determined the rankings.
Third-party systems observed them.
The observation layer was not necessarily trivial, but the basic premise was clear.
A Search result could be retrieved, parsed and converted into a structured record.
The increasing sophistication of Google’s Search interfaces changes that relationship.
Google Controls Ranking, But Third Parties Traditionally Observed It
The third-party measurement ecosystem developed around independent observation.
Rank trackers effectively said:
We queried Google under defined conditions.
Here is what appeared.
That observation model depends on the ability to reproduce and inspect Search.
When Google changes the representation of the Search result, the observer must adapt.
That does not mean independent observation becomes impossible.
It means the observer must increasingly operate infrastructure specifically designed for that purpose.
The Growing Cost of Independent SERP Observation
The sequence is now easier to see:
Static HTML extraction
↓
Browser-aware extraction
↓
Anti-bot adaptation
↓
Pagination adaptation
↓
Dynamic result classification
↓
URL resolution
↓
Provenance management
Each additional layer increases the cost of maintaining an independent observation system.
The important economic effect is therefore not necessarily that a competitor disappears.
It is that the barrier to building and maintaining a high-quality independent Search dataset rises.
Search Results as a Controlled Data Boundary
The /goto mechanism is particularly interesting because it changes a boundary without changing the visible result.
A human sees:
Result
↓
Website
A data system can see:
Result
↓
Google-controlled object
↓
Google-controlled resolution
↓
Website
The publisher page is still public.
The Search result is still visible.
But the machine-readable relationship between them is mediated.
That is a different kind of control.
It is not access denial.
It is access mediation.
The Rise of Search-Data Infrastructure
This is likely to be the lasting lesson.
A sophisticated Search-data platform increasingly needs:
- collectors;
- browser environments;
- network infrastructure;
- parsers;
- result classifiers;
- redirect resolvers;
- URL normalisers;
- canonicalisation engines;
- caches;
- quality monitoring;
- provenance systems;
- anomaly detection.
The old mental model was:
scraper
The newer model is:
Search data infrastructure
That is a significant shift.
The Implications for SEO and AI Visibility Measurement
Ranking Reports
Ranking reports remain fundamentally possible.
The important change is that the quality of the report depends increasingly on the quality of the collection architecture.
A platform should therefore distinguish:
ranking detected
from:
destination resolved
and:
canonical resolved
Share of Search
Domain-level visibility can survive even when URL-level resolution temporarily fails.
But platforms should disclose the distinction.
A report might say:
Domain visibility:
confirmed
Exact landing-page visibility:
partial
rather than presenting both as equally certain.
Landing-Page Attribution
This is one of the areas most exposed to /goto.
A ranking without a destination URL can tell an SEO team:
You appeared.
It cannot reliably tell them:
This specific page appeared.
That difference matters for:
- content strategy;
- internal linking;
- cannibalisation;
- conversion analysis;
- page-level optimisation.
AI Search Visibility
The same principle applies to AI visibility.
A brand-level mention is one measurement.
A citation to a specific page is another.
An enterprise may want to know:
Which page was cited?
For which prompt?
In which AI system?
In which context?
Was the page canonical?
Was the citation direct or indirect?
That requires source identity.
Citation Tracking
Citation tracking is therefore moving towards a provenance problem.
The system should know:
Prompt
↓
AI response
↓
Citation
↓
Source URL
↓
Discovery path
↓
Resolution path
↓
Canonical URL
/goto introduces one more node into that graph when Google Search is part of the discovery process.
Competitive Intelligence
Competitive intelligence platforms also need page-level identity.
Knowing:
Competitor X ranks
is useful.
Knowing:
Competitor X’s /enterprise-seo/ page ranks
is much more actionable.
The difference between those two levels of information is exactly where destination resolution matters.
Longitudinal Search Research
Researchers tracking Search over months or years need to know whether the observation methodology changed.
A dataset should therefore treat collection methodology as part of the dataset itself.
For example:
Observation
=
Search result
+
Environment
+
Collection method
+
Resolution method
+
Timestamp
This is the only reliable way to interpret long-term changes when the Search interface itself evolves.
The Strategic Question: Who Controls Search Observability?
The deepest implication of /goto is not technical.
It is architectural.
Google Controls Ranking
Google controls:
- indexing;
- ranking;
- Search presentation;
- many Search features;
- Search result delivery.
That has always been understood.
Third Parties Traditionally Controlled Measurement
SEO platforms historically built independent measurement systems around Google’s output.
Their value came partly from converting Search observations into:
- historical rankings;
- competitive intelligence;
- visibility metrics;
- keyword databases;
- market analysis.
The /goto mechanism does not eliminate that capability.
But it introduces another point at which Google’s Search implementation controls what the observer can see directly.
Search Becomes Increasingly API-Like Without Being a Public API
This produces an unusual situation.
Google Search is a public interface.
But it is not a public, stable, general-purpose SERP data API.
Third-party systems therefore build observation layers around a continuously changing interface.
When Google changes the interface, those systems must adapt.
/goto is another example of that dynamic.
The Search result can remain perfectly usable for its intended human experience while becoming substantially different for a machine attempting to treat it as structured data.
Measurement Becomes Infrastructure
That means SEO measurement increasingly resembles observability engineering.
An observability platform does not merely collect data.
It asks:
Can I trust this observation?
How was it collected?
What changed?
What failed?
How complete is it?
That is exactly the direction advanced Search measurement is moving towards.
What /goto Means for the Future of Search Data
It would be premature to conclude that Google’s /goto rollout represents the final architecture of Search.
The implementation is still evolving.
The public evidence already shows variation across:
- browsers;
- Search features;
- result types;
- API providers;
- sessions.
SerpApi’s August and September updates demonstrate that its engineering team has had to adapt repeatedly as different Search result types began exposing /goto links.
That makes one conclusion particularly important:
The right response is not to build a permanent parser for today’s token.
It is to build a system capable of detecting tomorrow’s change.
Don’t Hard-Code Google’s Current Representation
A fragile implementation assumes:
if href contains /goto:
decode CAES
A resilient implementation assumes:
if destination is not directly available:
invoke resolution strategy
The second design survives changes in Google’s representation.
Build for Behaviour, Not Format
The permanent invariant is not:
CAES…
The more useful invariant is:
Google result
→
intermediary
→
destination
Even if Google changes:
/goto?url=…
to another mechanism, a behaviour-oriented collector can adapt.
Treat Google as a Changing Data Source
This does not require assuming malicious intent.
It is simply good engineering.
A data source can change:
- schema;
- transport;
- response format;
- pagination;
- rate limits;
- authentication;
- result structure.
Google Search is no different.
The appropriate architecture therefore includes:
- monitoring;
- regression tests;
- sample SERP fixtures;
- format detection;
- resolution tests;
- alerting.
Build Observability Into the Collector
A collector should know when its own accuracy changes.
For example:
Yesterday:
98.9% exact URL resolution
Today:
87.3%
That should trigger an alert.
Likewise:
Organic results:
99% resolved
AI Overview:
76% resolved
That should trigger a result-type-specific investigation.
The collector becomes an observable system in its own right.
The /goto Investigation Checklist for Enterprise SEO Teams
If an organisation depends on third-party Search data, the following questions should now be part of vendor evaluation.
Question 1: Does the platform return the final destination URL?
Not merely the Google intermediary.
Question 2: Does the platform preserve the raw result link?
This matters for auditability.
Question 3: How does the provider resolve /goto?
The exact implementation may be proprietary, but the methodology should be understood at a high level.
Question 4: What happens when resolution fails?
Does the platform:
- drop the result;
- mark it unresolved;
- retry;
- return the intermediary;
- or report an error?
Question 5: Does the provider distinguish URL confidence?
This becomes increasingly valuable.
Question 6: Does the provider support specialised Search features?
The public evidence shows that /goto handling has affected more than conventional organic results. SerpApi has documented work involving AI Overview, questions and answers, recipes, video key moments, Lens, local results and other Search surfaces.
Question 7: Does historical data preserve methodology?
Without that, long-term comparisons become harder to trust.
Question 8: Can the platform detect collection changes?
A mature provider should have monitoring around its own data quality.
Final Analysis: The SERP Has Become a Controlled Data Boundary
The /goto story is easy to describe incorrectly.
It is tempting to say:
Google has blocked scraping.
That is too broad.
It is equally tempting to say:
Nothing has changed because the links still work.
That is also too simplistic.
The technically meaningful change sits between those two statements.
Google has changed how the relationship between a Search result and its destination is exposed to machines.
The resulting architecture can be represented as:
OLD MODEL
Google Search
↓
SERP Result
↓
Direct Destination URL
↓
Publisher
versus:
/GOTO MODEL
Google Search
↓
SERP Result
↓
Google /goto intermediary
↓
Opaque token
↓
Google resolution
↓
Destination URL
↓
Publisher
The publisher has not moved.
The ranking has not inherently changed.
The canonical URL has not inherently changed.
The page itself has not disappeared.
The observation boundary has moved.
That is the important finding.
The first consequence is technical.
A parser must become a resolver.
The second is economic.
A Search result may require additional network work to become a complete structured record.
The third is analytical.
A ranking observation must preserve the relationship between position and destination through an additional layer.
The fourth is methodological.
Search datasets increasingly need provenance, confidence and collection-version metadata.
The fifth is strategic.
AI retrieval systems and visibility platforms that depend on Search as a source-discovery mechanism must treat URL resolution as part of their retrieval infrastructure.
And the sixth is architectural.
The modern SERP is no longer best understood as a static document containing all the information a downstream system needs.
It is increasingly a dynamic interface whose observable data is produced through a combination of document structure, browser state, network behaviour, redirects and Search-specific infrastructure.
That is why the CAES token is not actually the central mystery.
The token is merely the visible artefact.
The deeper change is this:
Search result
↓
was once sufficient
↓
to identify destination
Now:
Search result
↓
may require resolution
↓
to identify destination
That one additional arrow has consequences far beyond URL parsing.
It changes the economics of rank tracking.
It changes how SERP APIs must be engineered.
It changes the provenance requirements of Search datasets.
It changes how page-level competitive intelligence should validate its observations.
It introduces another failure point into AI retrieval pipelines.
And it makes the distinction between observing Search and accessing Search increasingly important.
The future of Search measurement will therefore not be determined solely by whether engineers can find another way to decode Google’s latest token.
A more durable question is whether the industry can build observation systems that remain reliable when the underlying Search interface is continuously changing.
That means moving away from:
“Scrape the HTML.”
towards:
“Observe the Search system.”
The distinction is fundamental.
A scraper expects a format.
An observation system expects change.
And Google’s /goto wall is a clear signal that advanced Search data infrastructure now needs to be designed for the latter.
The final model
The evolution can be reduced to one progression:
Direct URL
↓
Google intermediary
↓
Opaque representation
↓
Resolution event
↓
Destination
↓
Provenance
↓
Measurement
The important new component is not the opaque token.
It is the resolution event.
Once Google makes destination identity dependent on an additional interaction, that interaction becomes part of the data model, the cost model and the measurement model.
That is why the /goto redirect wall matters.
It does not make Google Search unreadable.
It makes the path from Search result to machine-readable source identity more controlled, more operationally expensive and more dependent on infrastructure.
And for an industry whose products increasingly depend on knowing exactly what Google showed, where it showed it, and which page received that visibility, that is a much bigger change than a new URL format.
It is a change in the economics of Search observability.
