A cached 301 can break your site after a redesign

A site goes live with a new structure. The owner opens it, clicks a link, and the browser says the page is not working. Every check from the server side looks perfect: the right status codes, the right headers, the right pages. Open the same address in a private window and it works. The problem is not on the server at all. It is a permanent redirect the browser wrote down months ago and never questioned again.

What actually happens

A 301 means the move is permanent, and browsers take that literally. When a response has no explicit cache headers, the browser is allowed to guess how long to keep it, and for a 301 both Chrome and Safari keep it on disk. The next time the address comes up, the browser does not ask the server. It goes straight to the target it remembers.

That is harmless until the direction changes. The pattern I see most often after a rebuild looks like this:

1. The old site answered / with 301 -> /index.html, because the index file was generated that way.
2. Visitors collected that redirect in their cache.
3. The new site is clean, so / is the real page and /index.html is a leftover. The obvious tidy up is 301 /index.html -> /.
4. A returning visitor asks for /, the cache answers /index.html, the server answers /, the cache answers /index.html, and the loop runs until the browser gives up.

New visitors never see it. Crawlers usually never see it, because they do not carry a year of browsing history. The people most likely to hit it are the owner, the team and the regular readers, which is the worst possible sample.

How to see it

The first instinct is to run curl, and curl will tell you everything is fine:

curl -sI https://example.com/ | head -3
# HTTP/2 200
# content-type: text/html; charset=utf-8
# cache-control: no-cache

Curl starts with an empty cache every time, so it can never reproduce the bug. Use the browser instead, with the network panel open and "disable cache" switched off. A cached redirect shows up as a request served (from disk cache) with a 301 status and no network time.

If you want a reproduction you can run over and over, script it. Serve the old behaviour, visit it once, switch the server to the new behaviour and click through:

// one server, two behaviours, one browser profile
if (mode === 'old' && url === '/') return redirect(301, '/index.html');
if (mode === 'new' && url === '/index.html') return redirect(301, '/');

With the old rule visited first, the new rule produces the loop in a normal window every time. That is the test that tells you whether your fix works, and it takes a minute to write.

The fix

Stop reversing the redirect. Serve the old URL as a real page and let the canonical tag carry the search engine side:

# /index.html answers with the page itself, no redirect, no loop
location / {
  try_files $uri $uri/ =404;
}

<link rel="canonical" href="https://example.com/">

Search engines are happy with this. A canonical tag is the intended way to say "these two addresses are the same page, index this one", and it does not require the browser to follow anything.

Then clear the stale entry. Clear-Site-Data: "cache" tells the browser to drop the HTTP cache for the origin, cached redirects included. Send it only on the URL that needs it, so a normal visit never pays for it:

map $request_uri $clear_cache {
  "~^/index\.html"  '"cache"';
  default           "";
}

server {
  add_header Clear-Site-Data $clear_cache;
}

An empty value means nginx does not send the header at all, so / stays untouched while /index.html cleans up after itself once.

Finally, tidy the address bar. The visitor arrived on the old URL through a redirect they cannot see, and there is no reason to leave it on screen:

if (location.pathname.endsWith('/index.html')) {
  history.replaceState(null, '', location.pathname.slice(0, -10) + location.search + location.hash);
}

How to check it worked

Two commands and one click. The old URL answers with the page and the cleanup header, the clean URL answers without it:

curl -sI https://example.com/index.html | grep -i "HTTP/\|clear-site-data"
# HTTP/2 200
# clear-site-data: "cache"

curl -sI https://example.com/ | grep -ci clear-site-data
# 0

Then repeat the scripted reproduction. The first click goes through the old URL and lands on the page, and the second click goes straight to the clean URL because the stale redirect is gone.

What to watch out for

• Clear-Site-Data only works on a secure origin, and only the values you list. Sending "cache" on every page would throw away fonts and images on every visit, which is a performance bug of your own making.
• Firefox support has lagged behind Chrome on this header. Treat it as a cleanup that helps most people, not as the thing that makes the page correct. Serving 200 instead of a reverse redirect is what makes it correct.
• Redirect chains are the same class of problem. If /a goes to /b and /b later goes to /c, a returning visitor can be sent through a path you never tested. Test with a profile that has history, not a fresh one.
• If the old redirect went to a path you no longer serve at all, you cannot rely on the browser asking you about it. Keep a real response at that address for at least one release cycle.

The general lesson is smaller than the bug: a permanent redirect is a promise, and browsers keep promises longer than we remember making them. Before you flip the direction of an old redirect, assume some readers are still carrying the old one, and give that address something valid to answer with. When you are not certain a move is permanent, a 302 costs nothing and keeps the decision reversible. The same habit of checking with a browser that has history, not a clean profile, catches a surprising number of "works for me" bugs, and it is the same discipline as measuring before optimising rather than guessing.