What is a sitemap and why do you need one?

A sitemap, or an XML sitemap, connects together all of the important pages and information on your website. They’re used by search engines to navigate through all the pages on your site when they start a crawl.

How to create a sitemap

Step 1:

Firstly, it’s important to take a look at the current structure of your website. Start with the homepage, where does this link to? Follow your structure like a map, and if you run into any problems then you can alter your site before you start your sitemap. Remember, the further a page is from your homepage, the harder it is for a search engine to crawl and index them. So make sure you can reach each page in about three clicks.

Step 2:

Now it’s time to outline your sitemap. This should look almost like a family tree, but instead of spouses and children, it would be category and product pages. Here you should be able to create a page hierarchy, at the top of the sitemap should be the top level pages, trickling all the way down to the lower level pages.

Step 3:

Make sure that your sitemap follows a logical order. Consider the user journey, is this how a customer would navigate through your site? Thinking like this is key as it’ll help when a search engine crawls your website.

Step 4:

When you’re happy with your sitemap plan, you can start finalising it. You’ll need to create an XML file and code all of the URLs you wish to add. This will have to be coded using HTML.

If you aren’t well versed in how to do this, there are multiple tools available online to guide you through the process.

Step 5:

This step is optional, but you may want to use a sitemap validator to check that all of the code is correct. It can be important to do this as you don’t want to give search engines the wrong info and impact their craws and indexing.

Step six:

Lastly, when you’re happy with your coded sitemap, it’s time to add it both to your site and onto robots.txt.

A HTML sitemap may look like this:

Or an XML sitemap which you'll add to your robots.txt may look like this:

As you can see, all of our URLs are coded and laid out logically. Not everyone will link to their sitemap in their robots.txt file, but as we’ll explain later, this is important.

Now you’ve successfully created a sitemap!

Why it’s important to reference your sitemap in robots.txt

Linking your sitemap within your robots.txt is massively beneficial. As a robots.txt file is the first thing that will be crawled by a search engine, having your sitemap ready and waiting ensures a quick crawl and an efficient indexing of your site. So, after you’ve added your sitemap onto your website, go ahead and add it to your robots.txt too.

Why you need a sitemap

A sitemap isn’t essential, but it can be very beneficial to have one. So, why is it important to have a sitemap?

Well, a sitemap can tell a search engine a number of things, including:

  • Which pages to crawl more often: This is key if you want certain pages on your site to be indexed and visible on search engines. This way users can access them.
  • The importance of various pages: Once a search engine crawls a page on your site, it’ll decide the level of importance. It does so by assessing the content, whether it’s relevant to a user, and also the internal and external links. The higher the importance, the more likely a user is to see this page in their search results.
  • The relationship between the pages on your site: Adding internal links to connect the pages on your site is essential, and it’s also essential to define these relationships within your sitemap. If a search engine can crawl all of your pages, flicking between them easily — you’re onto a winner.

How to check a website’s sitemap and robots.txt

If a website has a sitemap, you can easily access it by using the following text at the end of a URL:

/sitemap or /sitemap.xml

And it would look something like this https://www.fasthosts.co.uk/sitemap for example.

The same goes for accessing a site’s robots.txt, and this is the string to add:

/robots.txt

For example, this would be the URL for our robots.txt: https://www.fasthosts.co.uk/robots.txt

Interested in what else we have to say? Take a look at our blog for more interesting information. Or maybe you’re in need of a website? Our easy-to-use website builder is your best bet.

If you’re wondering what other products we have to offer, have a chat with our sales team to discuss more.