Questions tagged [sitemap]

A sitemap is an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. The sitemap protocol format is defined at http://www.sitemaps.org/protocol.html

Sitemap can be either a document in any form used as a planning tool for web design, or a web page that lists the pages on a web site, typically organized in hierarchical fashion. This helps visitors and search engine bots find pages on the site.

While some developers argue that site index is a more appropriately used term to relay page function, web visitors are used to seeing each term and generally associate both as one and the same. However, a site index is often used to mean an A-Z index that provides access to particular content, while a site map provides a general top-down view of the overall site contents.

XML is a document structure and encoding standard used, amongst many other things, as the standard for webcrawlers to find and parse sitemaps. There is an example of an XML sitemap below (missing link to site). The instructions to the sitemap are given to the crawler bot by a Robots Text file, an example of this is also given below. Site maps can improve search engine optimization of a site by making sure that all the pages can be found. This is especially important if a site uses a dynamic access to content such as Adobe Flash or JavaScript menus that do not include HTML links.

They also act as a navigation aid by providing an overview of a site's content at a single glance.

Source: https://en.wikipedia.org/wiki/Site_map

1915 questions
201
votes
3 answers

Can a relative sitemap url be used in a robots.txt?

In robots.txt can I write the following relative URL for the sitemap file? sitemap: /sitemap.ashx Or do I have to use the complete (absolute) URL for the sitemap file, like: sitemap: http://subdomain.domain.com/sitemap.ashx Why I wonder: I own a…
Easyrider
  • 2,939
  • 4
  • 19
  • 28
133
votes
5 answers

What Content-Type value should I send for my XML sitemap?

I thought I should send "text/xml", but then I read that I should send "application/xml". Does it matter? Can someone explain the difference?
Kyle
  • 20,061
  • 36
  • 105
  • 192
60
votes
6 answers

Google sitemap files for Rails projects

Is there an easy way to create a sitemaps file for Rails projects? Especially for dynamic sites (such as Stack Overflow for example) there should be a way to dynamically create a sitemaps file. What is the way to go in Ruby and/or Rails? What would…
z3cko
  • 3,034
  • 8
  • 38
  • 59
56
votes
5 answers

How to generate sitemap on a highly dynamic website?

Should a highly dynamic website that is constantly generating new pages use a sitemap? If so, how does a site like stackoverflow.com go about regenerating a sitemap? It seems like it would be a drain on precious server resources if it was constantly…
Andrew Garrison
  • 6,670
  • 10
  • 46
  • 76
43
votes
5 answers

How to find sitemap.xml path on websites?

How can I find sitemap.xml file of websites? e.g. Going to stackoverflow/sitemap.xml gets me a 404. In stackoverflow/robots.txt is written the following: "this technically isn't valid, since for some godforsaken reason sitemap paths must be…
jacktrades
  • 6,766
  • 13
  • 50
  • 80
41
votes
1 answer

ASP.NET Site Maps

Does anyone have experience creating SQL-based ASP.NET site-map providers? I have the default XML file web.sitemap working properly with my Menu and SiteMapPath controls, but I'll need a way for the users of my site to create and modify pages…
Zack Peterson
  • 53,106
  • 76
  • 203
  • 279
40
votes
5 answers

Multiple Sitemap: entries in robots.txt?

I have been searching around using Google but I can't find an answer to this question. A robots.txt file can contain the following line: Sitemap: http://www.mysite.com/sitemapindex.xml but is it possible to specify multiple sitemap index files in…
user306942
  • 685
  • 1
  • 7
  • 6
38
votes
6 answers

HTML: How to get my subpages listed on a google search

When you go to Google and perform a search, it will return either one of two type of results: just the title of your webpage, or the title of your web-page plus, lists subpages it found on that web site Here is an example of option #2: …
Teddy
  • 381
  • 1
  • 3
  • 3
37
votes
7 answers

Limitation for google sitemap.xml file size

can any one please let me know the memory limitation(MB) of google sitemap.xml file size.
Chakrapani
  • 548
  • 3
  • 6
  • 8
30
votes
4 answers

Pinging google sitemap after every new article submission?

As you may know, by using the following url, you can 'ping' google to update its record of your sitemap: https://www.google.com/ping?sitemap=URLOFSITEMAP.xml I made a script to do this programmatically after every article submission (I am making a…
hjk
27
votes
5 answers

how to create google sitemap for mvc site?

I was wondering if anyone has done this yet or has any examples on how to create a Google Sitemap for an MVC website. Any help or example would be appreciated. Im talking about this: https://www.google.com/webmasters/tools/docs/en/protocol.html
deadbuddha
25
votes
6 answers

How to properly format last modified (lastmod) time for xml sitemaps

I am creating an app that will automatically update sitemap.xml each time new content is added to, or updated on the site. According to Google's best practices the tag should be formatted as…
petebolduc
  • 1,108
  • 1
  • 11
  • 20
24
votes
3 answers

Is there a maximum size for sitemap files?

I was wondering: Is there a maximum size for sitemap files? I'm trying to build a shopping site and I don't know if it's going to support constant size increasing dynamicaly. I don't even know if it is the correct way to put the products to be find…
msmafra
  • 1,654
  • 3
  • 20
  • 34
24
votes
1 answer

django sitemap http://example.com

Sorry for my bad english but i'm not english! I try to generate a sitemap.xml with my django project. In my project i don't have models (my database is empty), i have just static url (like 'home' or 'About' ). I succeded to generate a sitemap.xml…
salt
  • 678
  • 7
  • 23
23
votes
10 answers

Google Sitemap Date Format

I need date format for sitemaps in php. How can i do that ? Is this output right ? 2012-08-02EEST18:01:18+03:00
user1329212
1
2 3
99 100