9

Here is the situation, i have a website that can be accessed from multiple domains, lets say www.domain1.com, www.domain2.net, www.domain3.com. the domains access the exact same code base, but depending on the domain, different CSS, graphics, etc are loaded.

everything works fine, but now my question is how do i deal with the sitemap.xml? i wrote the sitemap.xml for the default domain (www.domain1.com), but what about when the site is accessed from the other domains? the content of the sitemap.xml will contain the wrong domain.

i read that i can add multiple sitemap files to robots.txt, so does that mean that i can for example create sitemap-domain2.net.xml and sitemap-domain3.com.xml (containing the links with the matching domains) and simply add them to robots.txt?

somehow i have doubts that this would work thus i turn to you experts to shed some light on the subject :)

thanks

John Conde
  • 207,509
  • 96
  • 428
  • 469
Julien
  • 1,613
  • 15
  • 32

5 Answers5

3

You should use server-side code to send the correct sitemap based on the domain name for requests to /sitemap.xml

SLaks
  • 800,742
  • 167
  • 1,811
  • 1,896
  • 1
    how would i go about that approach? specifying some rewrite rule in .htaccess that will serve instead some PHP file that will return the correct XML depending on the domain? heck, i like the sound of that, any idea about the mod_rewrite part? :D – Julien Jun 21 '11 at 14:37
3

Apache rewrite rules for /robots.txt requests

If you're using Apache as a webserver, you can create a directory called robots and put a robots.txt for each website you run on that VHOST by using Rewrite Rules in your .htaccess file like this:

# URL Rewrite solution for robots.txt for multidomains on single docroot
RewriteCond %{REQUEST_FILENAME} !-d # not an existing dir
RewriteCond %{REQUEST_FILENAME} !-f # not an existing file
RewriteCond robots/%{HTTP_HOST}.txt -f # and the specific robots file exists
RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [L]

NginX mapping for /robots.txt requests

When using NginX as a webserver (while taking yourdomain1.tld and yourdomain2.tld as example domains), you can achieve the same goal as post above with the following conditional variable (place this outside your server directive):

map $host $robots_file {
    default /robots/default.txt;
    yourdomain1.tld /robots/yourdomain1.tld.txt;
    yourdomain2.tld /robots/yourdomain2.tld.txt;
}

This way you can use this variable in a try_files statement inside your server directive:

location = /robots.txt {
    try_files /robots/$robots_file =404;
}

Content of /robots/*.txt

After setting up the aliases to the domain-specific robots.txt-files, add the sitemap to each of the robots files (e.g.: /robots/yourdomain1.tld.txt) using this syntax at the bottom of the file:

# Sitemap for this specific domain
Sitemap: https://yourdomain1.tld/sitemaps/yourdomain1.tld.xml

Do this for all domains you have, and you'll be set!

Henry van Megen
  • 1,979
  • 2
  • 19
  • 30
1

You have to make sure URLs in each XML sitemap match within domain/subdomain. But, if you really want, you can host all sitemaps on one domain look using "Sitemaps & Cross Submits"

Tom
  • 3,457
  • 8
  • 64
  • 119
0

The easiest method that I have found to achieve that is to use an XML sitemap generator to create a sitemap for each domain name. Place both the /sitemap.xml in the root directory of your domains or sub-domains. Go to Google Search and create separate properties for each domain name. Submit an appropriate sitemap to each domain in the Search Console. The submission will say show success.

TMT
  • 23
  • 5
0

I'm not an expert with this but I have a similar situation

for my situation is that I have one domain but with 3 sub-domain

so what happen is that each of the sub-domain contain the sitemap.xml

but since my case was different directory for each of the sub-domain

but I'm pretty sure that the sitemap.xml can be specify for which of each domain.

Ali
  • 8,267
  • 18
  • 66
  • 101