AdvancedLesson 25 of 307 min read

Site Migrations and Redesigns Without Losing Visibility

Plan a site move: build a URL mapping, review automatic matches by hand, validate redirects for chains and loops and test them against a live server.

In this lesson you will

  • Build a URL mapping
  • Review automatic matches
  • Validate chains and loops
  • Follow a migration checklist

About 20 minutes

The redesign that must not lose the old pages

Fatima’s bakery site is getting a new design and, with it, a new structure. Old blog addresses that include the year and month will become short ones like /recipes/sourdough-starter/. Her agency says, ‘We will sort out redirects after launch.’ Fatima knows from lesson 14 what that can mean: broken links, lost visitors and weeks of recovery. A site migration is any big change to addresses, domain, platform or structure. The risk is not the redesign. It is losing the connections that search engines and other sites have built to your old pages.

What Google recommends

Google’s guidance on moving a site with URL changes comes down to a few habits. Prepare the new site first. Create a URL mapping, a list that pairs each old address with its closest new equivalent. Redirect old addresses permanently (301 or 308) to the new ones, one hop each. Update your own links, sitemaps and canonical tags to the new addresses. Then monitor. Keep the redirects in place for as long as possible, which Google puts at generally at least a year. If you are changing domain, there is also the Change of Address tool in Search Console, which Google says is for moving between domains, not for switching from http to https, and its help page mentions keeping redirects at least 180 days. Google also advises doing one kind of change at a time where you can, such as moving first and redesigning afterwards. Check the current guide before you start.

Building the map

With hundreds of pages, nobody writes a redirect map by hand. A program can make a first draft by comparing the last part of each address, called the slug. The program below does that for nine old addresses and eight new ones. It matches exact slugs, tries approximate matches with Python’s standard text-matching tool and flags anything that is not exact for a person to review. Then it validates the final map for chains, loops and missing targets, deliberately breaks one rule to prove the check works, and finally starts a test server and asks it for every old address. The output is real, and we repeated the test on the live server with curl.

import difflib, http.client, threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

old = ["/blog/2023/05/sourdough-starter/", "/blog/2022/11/bread-baking-tips/", "/blog/2021/03/best-flours/",
       "/shop/rye-loaf/", "/shop/seeded-loaf/", "/about-us/", "/contact-us/",
       "/blog/2020/01/old-news/", "/shop/sourdough-loaf/"]
new = ["/recipes/sourdough-starter/", "/recipes/baking-tips-for-bread/", "/guides/best-flours/",
       "/bakery/rye-loaf/", "/bakery/seeded-loaf/", "/about/", "/contact/", "/bakery/sourdough/"]

def slug(u): return u.strip("/").split("/")[-1]
new_by_slug = {slug(n): n for n in new}

draft = {}
for o in old:
    s = slug(o)
    if s in new_by_slug:
        draft[o] = (new_by_slug[s], "exact", 1.0)
    else:
        m = difflib.get_close_matches(s, list(new_by_slug), n=1, cutoff=0.6)
        score = round(difflib.SequenceMatcher(None, s, m[0]).ratio(), 2) if m else 0
        draft[o] = (new_by_slug[m[0]] if m else None, "fuzzy" if m else "none", score)

for o, (t, how, score) in draft.items():
    print(f"{slug(o)}: exact" if how == "exact" else f"{slug(o)}: {how} {score}, REVIEW")

# a person reviews the list: the matcher missed one obvious pair, and old-news has no equivalent (410)
manual = {"/blog/2022/11/bread-baking-tips/": "/recipes/baking-tips-for-bread/"}
final = {o: t for o, (t, how, score) in draft.items() if t}
final.update(manual)

def validate(rules, targets):
    issues = []
    for o, t in rules.items():
        if t not in targets: issues.append(f"{slug(o)}: target is not a live page")
        if t in rules: issues.append(f"{slug(o)}: chain, target redirects again")
        if rules.get(t) == o: issues.append(f"{slug(o)}: loop")
    return sorted(set(issues))

print("Redirect rules:", len(final), "| problems:", len(validate(final, set(new))))
broken = dict(final)
broken["/bakery/rye-loaf/"] = "/shop/rye-loaf/"          # a new page pointed back at an old one
print("With one careless rule:", validate({**broken}, set(new) | {"/shop/rye-loaf/"}))

PAGES = {u: (200, None) for u in new}
PAGES.update({o: (301, t) for o, t in final.items()})
PAGES["/blog/2020/01/old-news/"] = (410, None)
class Site(BaseHTTPRequestHandler):
    def do_GET(self):
        code, to = PAGES.get(self.path, (404, None))
        self.send_response(code)
        if to: self.send_header("Location", to)
        self.end_headers()
    def log_message(self, *a): pass
server = ThreadingHTTPServer(("127.0.0.1", 0), Site)
port = server.server_address[1]
threading.Thread(target=server.serve_forever, daemon=True).start()

def follow(path):
    codes = []
    for _ in range(5):
        c = http.client.HTTPConnection("127.0.0.1", port); c.request("GET", path)
        r = c.getresponse(); codes.append(r.status); to = r.getheader("Location"); c.close()
        if r.status == 301 and to: path = to
        else: break
    return codes

results = {o: follow(o) for o in old}
one_hop = sum(1 for c in results.values() if c == [301, 200])
print(f"Tested {len(old)} old URLs: {one_hop} redirect once to a 200, "
      f"{sum(1 for c in results.values() if c == [410])} return 410")
server.shutdown()
sourdough-starter: exact
bread-baking-tips: none 0, REVIEW
best-flours: exact
rye-loaf: exact
seeded-loaf: exact
about-us: fuzzy 0.77, REVIEW
contact-us: fuzzy 0.82, REVIEW
old-news: none 0, REVIEW
sourdough-loaf: fuzzy 0.78, REVIEW
Redirect rules: 8 | problems: 0
With one careless rule: ['rye-loaf: chain, target redirects again', 'rye-loaf: loop']
Tested 9 old URLs: 8 redirect once to a 200, 1 return 410

The draft found 4 exact matches, 3 approximate ones and 2 with no match. Notice what the approximate matcher got right and wrong. about-us to about (score 0.77) and contact-us to contact (0.82) are correct, and so is sourdough-loaf to sourdough (0.78), but a person should still confirm each one. Meanwhile the matcher completely missed bread-baking-tips, which obviously belongs with baking-tips-for-bread. The words are the same in a different order. A human spotted that and added it as a manual rule. The page old-news has no equivalent, so we planned to return a 410, which means gone on purpose.

The final map has 8 rules and passes validation. To show the check really works, the program then adds one careless rule that points a new page back at an old one. The validator catches it immediately, reporting a chain and a loop. Finally the test: of the nine old addresses, 8 redirect once to a working page and 1 returns 410.

A migration checklist

  1. Before: crawl the old site and save the full list of addresses, titles and statuses. Add pages that have traffic or links, using Search Console data.
  2. Plan: build the redirect map, review every approximate match by hand, and decide which pages truly have no equivalent.
  3. Test: run the map against a test version of the new site before launch. Check for chains, loops and 404s.
  4. Launch: switch the redirects on, submit the new sitemap and check that robots.txt does not block the new site.
  5. After: crawl again, watch Search Console for errors and compare traffic by page group. Expect some movement while search engines process the change, and keep the redirects.
Everything to the home page

Redirecting every old page to the new home page. It tells search engines the old page has been replaced by something unrelated, and visitors land in the wrong place. Map each page to its closest equivalent.

Draft by machine, decide by person

This lesson’s matcher is a teaching tool. Real migrations often have thousands of pages and use rules based on patterns, plus spreadsheets reviewed by a person. The principle is the same: automate the first draft and review the result.

Choose ten pages of a site you know and invent a new structure for them. Write the old and new addresses side by side. Mark which pairs you are sure about, which need a decision, and which have no equivalent. Check that no new address is also an old one.

Practice for this lesson

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

What is a URL mapping?

  1. A picture of the site
  2. A sitemap of images
  3. A list pairing each old address with its new one
  4. A list of keywords

It is the plan that drives the redirects, and it is made before launch.

How long does Google suggest keeping redirects after a site move?

  1. As long as possible, generally at least a year
  2. One week
  3. Until the next update
  4. Never keep them

Google says generally at least one year, and longer for users if you can.

What did the matcher miss in the demo?

  1. About matching home
  2. bread-baking-tips matching baking-tips-for-bread
  3. Contact matching blog
  4. Rye matching seeded

The words were the same in a different order, so a person had to add that match.

What does the validator report for a rule pointing back at an old page?

  1. Nothing
  2. A 410
  3. A faster page
  4. A chain and a loop

Pointing a new page back at an old one creates a loop, which the check catches.

Where should a page with no equivalent go?

  1. Always the home page
  2. A 410 or 404, not the home page
  3. A random page
  4. A 302 forever

An honest 410 or 404 is better than misleading redirects to unrelated pages.

Summary

  • A migration is safe when every old address has a planned destination.
  • Use permanent one-hop redirects and keep them for at least a year.
  • Automate a draft map but review approximate matches by hand.
  • Test redirects before launch and monitor afterwards.

Next lesson

Lesson 26 looks at content quality, algorithm updates and recovery.

Questions

Will I lose rankings in a migration?

Some movement is normal while search engines process the change, and careful planning keeps it small. Poor redirects, blocked pages or changed content can cause lasting loss.

Is changing from http to https a migration?

It is a site move with URL changes, so follow the same care with redirects. Google says to use its normal move guidance for it rather than the Change of Address tool.

When should I change design and structure?

If you can, change one thing at a time, for example move first and redesign later. It makes problems much easier to trace.

Progress is saved in this browser.