Skip to main content

Tutorial

How to create a sitemap.xml (and make sure it actually works)

A practical setup guide for marketers and developers — from generating the file to listing it in robots.txt and confirming Google can read it.

What is a sitemap?

A sitemap is an XML file that lists the public URLs on your site. Search engines use it to discover pages faster. Tools like ClimbPast also read robots.txt and sitemap.xml to know which pages to scan for CTAs and forms.

A working sitemap returns XML at a stable URL (usually /sitemap.xml) with a 200 status — not an HTML 404 page.

Step 1 — List it in robots.txt

At the root of your site, publish a robots.txt file that includes your sitemap URL:

User-agent: *
Allow: /

Sitemap: https://www.example.com/sitemap.xml

Use the same host your site redirects to (www vs apex). If robots.txt says one host but your canonical site uses another, crawlers may fetch the wrong sitemap.

Step 2 — Generate sitemap.xml

Next.js (App Router)

Add app/sitemap.ts that exports a sitemap function. Next.js serves it at /sitemap.xml automatically.

WordPress

Core WordPress exposes /wp-sitemap.xml. Point robots.txt to that URL, or use an SEO plugin (Yoast, Rank Math) to serve /sitemap_index.xml.

Static / manual XML

Minimum valid structure:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/</loc>
  </url>
  <url>
    <loc>https://www.example.com/pricing</loc>
  </url>
</urlset>

Step 3 — Verify it works

  • Open /sitemap.xml — response should be XML, not an HTML 404 page.
  • HTTP status should be 200.
  • Every <loc> should use your canonical host.
  • Re-open ClimbPast Tracking Health — the sitemap warning should clear once discovery succeeds.

Step 4 — Submit to Google Search Console

After the file is live, open Search Console → Sitemaps, enter sitemap.xml, and submit. Status should show Success with a discovered URL count.

Full Search Console setup guide

FAQ

Do I need a sitemap for a small website?

Sites with fewer than a few dozen well-linked pages can sometimes get by without one, but a sitemap is still cheap insurance. It helps Google and tools like ClimbPast discover pages that are not in your main navigation — especially new landing pages, docs, and blog posts.

Why does my sitemap URL return a 404 HTML page?

The server is serving your normal site template instead of an XML file. Common causes: the sitemap route was never created, a catch-all page router is swallowing /sitemap.xml, or robots.txt points to the wrong host (www vs apex). Fix the route or file, then confirm the URL returns XML with a 200 status.

Should my sitemap use www or non-www URLs?

Match your canonical host. If your site redirects to https://www.example.com, every <loc> in the sitemap should use that same host. Mixed hosts confuse crawlers and break tools that read robots.txt and sitemap together.

How often should I update my sitemap?

Update it whenever you add or remove important public pages. Frameworks like Next.js can generate sitemaps automatically on deploy. You do not need to resubmit to Search Console after every small change — Google re-fetches sitemaps on its own schedule.