URL canonicalization is the process search engines use to pick one representative URL, the canonical, from a group of URLs that show the same or very similar content. Every other URL in the group is a non-canonical URL: it may still be crawled, but it isn't the one shown in search results.
Most sites create these duplicates without meaning to. The same page loads with and without a trailing slash, with and without www, with a tracking parameter on the end. Each small variation is a separate URL to a crawler, and if your redirects, canonical tags, sitemap and internal links don't agree on which one is the real one, the search engine decides for you.
This guide covers the URL variations that cause duplicates, how Google actually deduplicates them (including what Google shared at Search Central Live Deep Dive Europe 2026), and how to keep your URL signals consistent.
URL variations that search engines treat as different pages
A URL is an exact string. Change one character and you have a different URL, even when the server returns the same page. These are the variations that create duplicates most often:
| Variation | Example of two URLs for one page | Preferred fix |
|---|---|---|
| Protocol | http://example.com/shoes/ and https://example.com/shoes/ | 301 redirect HTTP to HTTPS |
| Subdomain | https://www.example.com/shoes/ and https://example.com/shoes/ | Pick one host, 301 redirect the other |
| Trailing slash | /shoes and /shoes/ | Pick one convention, 301 redirect the other |
| Letter case | /Shoes/ and /shoes/ | Use lowercase paths, 301 redirect mixed case |
| Default document | /shoes/ and /shoes/index.html | 301 redirect the file name to the directory URL |
| Tracking parameters | /shoes/ and /shoes/?utm_source=newsletter | Self-referencing canonical on the clean URL |
| Sort and filter parameters | /shoes/ and /shoes/?sort=price | Canonical to the unfiltered URL, or block crawling of low-value combinations |
| Parameter order | ?color=red&size=9 and ?size=9&color=red | Generate parameters in one fixed order |
| Session IDs | /shoes/ and /shoes/?sessionid=123 | Keep session state in cookies, not the URL |
| Separate mobile or AMP URLs | example.com/shoes/ and m.example.com/shoes/ | rel=canonical from the alternate to the main URL |
Two things that look like variations but aren't: the root URL with and without a slash (example.com and example.com/ are the same URL), and fragments (/shoes/#reviews), which search engines generally ignore.
I cover two of these in more depth elsewhere: trailing slashes in URLs and how to write URL slugs.
How Google deduplicates URLs
At Search Central Live Deep Dive Europe 2026, Google walked through what deduplication involves. It broke the process into three parts:
- Building clusters of URLs that seem to be the same page.
- Localization: working out which pages in a cluster are language or country versions.
- Selecting a representative URL for each cluster. This is the canonical.
Your canonical tag doesn't act on its own. It is one input into clustering, and then one input into choosing the representative.
Step 1: Building clusters
Google listed the inputs for clustering as redirects, content, rel=canonical, "etc." Each one behaves differently.
Redirects. A redirect puts the source and the destination into the same cluster. It's the clearest way to say two URLs are one page.
Content. Pages can be clustered because of what's on them, with no redirect or tag involved. Google described three levels:
- Exact-match content, detected with content checksums.
- Near-match content, for example pages where the boilerplate is localized but the main content isn't.
- Structurally similar content, where the URL pattern itself suggests duplication.
The third one is the least obvious. Google's example: if /buy/fax, /buy/typewriter and /office-equipment/ all turn out to be duplicates, does it even need to crawl /buy/laptop? The same goes for location pages: if /barcelona/services, /madrid/services and /valencia/services are the same, what should it assume about /zurich/services?
Google called this a misleading site structure and asked for clear signals. The practical reading: if most URLs in a pattern are duplicates, a new URL in that pattern may be treated as one before it's judged on its own content. Templated location and category pages need genuinely different main content.
Soft errors create "soft duplicates"
Google also flagged soft errors and what it called "angry CDN noises". When a bot-protection layer serves the same "Are you a robot?" page on every URL with a 200 status, those URLs all have identical content, so they get clustered as duplicates of each other.
The advice from the session: don't block by default, use proper HTTP status codes, and check the canonicals Search Console shows you. As the slide put it, AI agents won't tell you about these errors.
rel=canonical. Google uses link rel=canonical annotations to cluster "sometimes", because its systems want to be sure the annotations are good, and many are bad. The example on screen was a canonical tag that still contained its template placeholder:
<link rel="canonical" href="{{ CANONICAL_TARGET }}">So a canonical tag is a hint that has to earn trust. A tag that points to a redirect, a 404, the homepage, or an unrendered template variable is more likely to be ignored.
That’s also why keeping your internal linking clean is important. I’ve seen writers copying the URL directly from SERPs with tracking parameters in the content - if your internal links are pointing to the different variations of the same page, it will dilute the canonical signal you are sending to Google.
Even Google may ultimately figure out the canonical version, it may take a few weeks or months to do so.
Step 2: Localization
When the main content is the same, pages may be clustered even if they are meant for different markets. Google gave three warnings:
- Boilerplate-only localization. Translating the header, footer and navigation while leaving the main content untouched doesn't make a page unique.
- Same language, other country. An en-GB and an en-US page with identical text are duplicates by content.
- "Clever" geo-redirecting, which Google said is often not so clever. If every visitor is redirected by IP, the crawler only ever sees one version. As Google mainly crawl from US, your Japanese page may never get indexed.
Google said it tries to use hreflang alternates here. Note the word "tries": hreflang helps Google serve the right version from a cluster, but it won't keep near-identical pages indexed separately. See my hreflang best practices for the setup.
Step 3: Picking the representative URL
Once a cluster exists, Google has to pick one URL to represent it. According to the session, candidates compete on weighted criteria, and the weights are trained by machine learning. The slide carried a warning: the rel=canonical is not always the chosen canonical.
Google grouped its thinking about that choice into three areas:
- Hijacking, benevolent or not, across pages or sites. The system has to guard against one URL taking over another's place in a cluster.
- User experience. Whether the page can load, meta refreshes, security and so on.
- Site owner signals. Redirects, rel=canonical, sitemaps and so on.
Key takeaway
No single signal decides the canonical. Your redirects, canonical tags and sitemap are weighed together, alongside things you don't control. The more they agree, the more likely Google picks the URL you want.
Bing works on the same principle: rel=canonical is treated as a hint, not a directive, and conflicting signals weaken it.
Why URL consistency matters
Indexing. Only the canonical URL is shown in results. If Google picks a different URL from the one you expected, your reporting, your hreflang and your structured data can all end up attached to the wrong page.
Crawling. Every duplicate URL a crawler fetches is a request not spent on a new or updated page. On large sites with parameters, that adds up quickly.
Duplicate content. There's no penalty for technical duplicates, but they do cause real problems for SEO when search engines can't tell which version to rank.
Authority consolidation. Links pointing at different variations of a page are signals for the same cluster. Clear canonicalization lets them count towards one URL instead of leaving the search engine to work it out.
Canonical tag vs 301 redirect
Both tell search engines which URL you prefer. The difference is what happens to the user. Use a redirect when nobody needs the duplicate URL to load, and a canonical tag when it has to stay accessible.
| 301 redirect | rel=canonical | |
|---|---|---|
| What users see | They are sent to the preferred URL | They stay on the duplicate URL |
| Strength | Strong signal | Hint, checked against other signals |
| Use it for | HTTP to HTTPS, www, trailing slash, letter case, moved pages | Tracking and filter parameters, syndicated copies, print versions |
| Works across domains | Yes | Yes |
| Common mistake | Chains and loops | Pointing at a URL that redirects, errors or is noindexed |
Add a self-referencing canonical to every indexable page, including the preferred version. It covers the variations you can't redirect, such as tracking parameters added by other sites. Always use absolute URLs:
<link rel="canonical" href="https://www.example.com/shoes/">How to keep your URLs consistent
Google's list of site owner signals was redirects, rel=canonical and sitemaps. So our job as an SEO is to make sure that they all say the same thing.
- Choose one URL format. Decide on HTTPS, www or non-www, trailing slash or not, and lowercase. Write it down so developers and content editors use the same rule.
- Redirect every other format in one hop. A request for http://example.com/Shoes should go straight to https://www.example.com/shoes/, not through three redirects that each fix one thing.
- Link internally to the canonical URL only. Navigation, body links, breadcrumbs and pagination should never point at a URL that redirects or canonicalizes elsewhere.
- List only canonical URLs in your sitemap. A sitemap that includes redirected or duplicate URLs contradicts your other signals. Here's how to find and fix non-canonical URLs in a sitemap.
- Match everything else to it. hreflang annotations, Open Graph URLs and structured data should use the same canonical URL.
- Return honest status codes. Missing pages should return 404 or 410, and blocked or rate-limited requests a 4xx or 5xx, not a 200 with an error message. Otherwise those pages become the soft duplicates described above.
Where you can, enforce the convention in the platform so it isn't left to memory. This site runs on Next.js with the trailingSlash option turned on, so every route has one slash-terminated form and the other redirects to it.
Common canonical issues and how to fix them
| Issue | What's happening | Fix |
|---|---|---|
| Canonical points to a redirect or error | The tag names a URL that isn't the final, working page | Point it at the final 200 URL |
| Canonical contains a placeholder, relative path or wrong domain | A template or staging value shipped to production | Output an absolute production URL and test the rendered HTML |
| Every page canonicalizes to the homepage | A global template setting | Make the tag self-referencing per page |
| Signals conflict | The canonical says A, but the sitemap and internal links say B | Align the redirects, tag, sitemap and links on one URL |
| Canonical URL has no incoming internal links | The site links to a variation instead | Update internal links to the canonical |
| "Duplicate without user-selected canonical" | Google clustered pages you didn't mark | Add a canonical tag or redirect, or differentiate the content |
| "Duplicate, Google chose different canonical than user" | Your tag was outweighed by other signals | Check which signals disagree with it, and whether the content really differs |
| "Alternate page with proper canonical tag" | Working as intended | Nothing, unless the page was supposed to be indexed |
How to audit URL consistency
- URL Inspection in Search Console. It shows the user-declared canonical next to the Google-selected canonical. When they differ, you have conflicting signals or content Google sees as duplicate.
- Page indexing report. Review the duplicate statuses in bulk, and look for URL patterns in the examples.
- Bing Webmaster Tools. Its URL Inspection tool shows how Bing indexed a URL, which is worth checking separately because Bing may choose differently.
- A site crawl. Tools such as Screaming Frog report canonicalized URLs, non-indexable canonicals, redirect chains and internal links to non-canonical URLs.
- Your sitemap. Pull the URL list with my sitemap extractor and check that every entry returns 200 and matches its own canonical tag.
- A manual variation test. Request a page with the wrong protocol, host, slash and letter case, and confirm each one reaches the canonical URL in a single redirect:
curl -sIL http://example.com/Shoes | grep -iE "^(HTTP|location)"FAQs about URL canonicalization
What is a canonical URL example?
If https://www.example.com/shoes/ and https://www.example.com/shoes/?utm_source=newsletter show the same page, the first is the canonical URL and the second is a non-canonical variation. The second should carry a rel=canonical tag pointing to the first.
What is a non-canonical URL?
A non-canonical URL is any URL in a group of duplicates that wasn't chosen as the representative. It can still be crawled, but it isn't the version shown in search results, and its signals are attributed to the canonical URL.
Are URLs case sensitive?
The protocol and domain are not case sensitive, but the path and query string are. /Shoes/ and /shoes/ are different URLs to a search engine, even if your server returns the same page for both. Use lowercase paths and redirect other variations.
Does Google always follow my canonical tag?
No. Google treats rel=canonical as one signal among several. At Search Central Live Deep Dive Europe 2026, Google said the rel=canonical is not always the chosen canonical, and that its systems check whether the annotations look reliable before using them.
Should every page have a canonical tag?
Every indexable page should have a self-referencing canonical tag. It protects against variations you can't control, such as tracking parameters added by other sites.
Should canonical URLs be relative or absolute?
Absolute. A relative path can resolve to the wrong protocol or host, for example on a staging domain, and a small typo can turn it into an invalid URL.
Can I use a canonical tag instead of a 301 redirect?
Only when the duplicate URL has to stay accessible to users. If nobody needs the duplicate to load, a 301 redirect is the stronger and cleaner signal.
