What Is a Sitemap (And Why Every Website Needs One)?

You built a website. You have 20 pages. You tell Google about it through Search Console. Google indexes 6 of them. The other 14 are missing from search results. Why? Because Google did not know they existed. It found the homepage, followed 5 links, and stopped. Your other 14 pages are orphans — no link from the homepage, no link from anywhere Google looked.

This is what a sitemap fixes. A sitemap is a file that says "here is every page on my site, please index all of them". Search engines use it as a discovery mechanism: instead of crawling links to find pages, they read the sitemap and index everything in it. For a small site with internal linking, a sitemap is not strictly necessary. For a large site, a site with deep pages, or a site with pages that are not linked from anywhere, a sitemap is essential.

The format

A sitemap is an XML file. The minimum valid sitemap looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
  </url>
  <url>
    <loc>https://example.com/about/</loc>
  </url>
</urlset>

Each <url> element is a page. The <loc> is the URL. Three optional fields: <lastmod> (last modified date, ISO 8601 format), <changefreq> (how often the page changes, e.g. daily, weekly, monthly), <priority> (a number from 0.0 to 1.0, the relative priority of this page vs other pages on your site). Most search engines ignore <changefreq> and <priority>; they pay attention to <lastmod>.

The file lives at the root of your site, usually at https://example.com/sitemap.xml. The path is conventional but not required; the path is whatever you tell the search engine, and whatever you list in your robots.txt.

The sitemap index (for large sites)

Google limits a single sitemap to 50,000 URLs or 50 MB, whichever comes first. If you have more than 50,000 URLs, you split the sitemap into multiple files and create a sitemap index that lists them. The format:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemap-1.xml</loc>
    <lastmod>2026-08-21</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemap-2.xml</loc>
    <lastmod>2026-08-20</lastmod>
  </sitemap>
</sitemapindex>

You submit the sitemap index to the search engine, and the search engine fetches each child sitemap. The split is usually by section (blog posts, products, authors) or by date (last week, this week, last month). The Uttir Sitemap Generator will produce a sitemap index automatically if your URL list is large enough.

How to tell search engines about the sitemap

Three ways, in order of importance:

1. Submit it in Google Search Console and Bing Webmaster Tools. Both have a "Sitemaps" section where you paste the URL of your sitemap. This is the most reliable way; it tells the search engine explicitly.

2. List it in your robots.txt. Add Sitemap: https://example.com/sitemap.xml as a line at the end of your robots.txt. Search engines that respect robots.txt will pick this up automatically. The Uttir Robots.txt Generator adds this line by default.

3. Include a <link rel="sitemap"> in your HTML head. This is rarely used; most search engines do not follow it. Skip it.

For a new site, do all three: submit in Search Console, submit in Bing Webmaster, list in robots.txt. Once indexed, the search engines will re-fetch the sitemap periodically to discover new pages.

What goes IN the sitemap (and what does not)

Include: pages you want indexed. Blog posts, product pages, about pages, contact pages, documentation pages, landing pages. Pages that exist and are meant to be public.

Exclude: pages you do not want indexed. Admin pages, login pages, internal search results, 404 pages, staging environments, duplicate URLs, URLs with query parameters that produce the same content, URLs blocked by robots.txt (including them in the sitemap sends mixed signals).

Use the canonical URL. If /about and /about/ and /about/index.html all serve the same content, the sitemap should list only one (the canonical URL). Listing all three tells the search engine that the same content lives at three URLs, which dilutes the ranking signal.

The seven things that go wrong

1. URLs that return 404. The page used to exist but was deleted. Remove it from the sitemap, or restore the page.
2. URLs that redirect. The page moved. Update the sitemap to the new URL.
3. Non-canonical URLs. /about and /about/ both in the sitemap. Pick one.
4. Relative URLs. /about instead of https://example.com/about. The sitemap requires absolute URLs.
5. Invalid XML. A typo, an unescaped character, a wrong closing tag. Validate the structure with any XML validator (or run it through xmllint on the command line).
6. Encoding issues. URLs with non-ASCII characters (e.g. /café) must be percent-encoded. The URL Encoder does this.
7. Date format errors. <lastmod> must be in W3C datetime format (2026-08-21 or 2026-08-21T12:00:00+00:00). Other formats (US 08/21/2026, European 21-08-2026) are invalid and the search engine will ignore the value.

How to generate a sitemap without uploading your site

Most sitemap generators work by crawling your live website. They fetch each page, follow the links, and build a sitemap of what they find. The catch: if you generate a sitemap of a public site, the generator is fetching your pages from a server. The server logs the requests. If you care about not being seen, the right answer is to generate the sitemap from a local copy of your site, not the live version.

The Uttir Sitemap Generator lets you either paste a list of URLs (if you know them) or run a local crawl of a site you have downloaded. The output is a valid sitemap.xml file that you can upload to your site root and submit to the search engines.

When the sitemap is not enough

A sitemap tells the search engine about your pages. It does not tell the search engine to rank them. For ranking, you need:

• Content that matches the search query.
• Internal links that point to the page with relevant anchor text.
• External links (backlinks) from other sites.
• A page that loads fast and is mobile-friendly.

The sitemap gets your pages into the index. Everything else is what makes them rank. A site with a perfect sitemap and no content will still rank below a site with mediocre content and no sitemap.