Stack Overflow
Stack Overflow

Questions tagged [noindex]

The noindex value of an HTML robots meta tag requests that automated Internet bots avoid indexing a web page.

The noindex value of an HTML robots meta tag requests that automated Internet bots avoid indexing a web page.

Example:

<html>
<head>
 <meta name="robots" content="noindex" />
 <title>Don't index this page</title>
</head>
111 questions
21
votes
4 answers

tag for Google

I would like to tell Google not to index certain parts of the page. In Yandex (russian SE) there's a very useful tag called . How can it be done with Google?
teslasimus
  • 1,128
  • 5
  • 13
  • 22
11
votes
3 answers

Precedence of X-Robots-Tag header vs robots meta tag

I've placed the following Header in my vhost config: Header set X-Robots-Tag "noindex, nofollow" The goal here is to just disable search engines from indexing my testing environment. The site is Wordpress and there is a plugin installed to manage…
Jared Eitnier
  • 6,482
  • 11
  • 62
  • 118
10
votes
1 answer

"rel=nofollow noopener" - Possible to have both at the same time?

I've got a pdf that I'd like to protect and don't want search engines to index it. Currently, my link is as follows: View PDF Would I be able to add nofollow tag to the rel…
Joe Bloggs
  • 1,154
  • 1
  • 15
  • 43
9
votes
9 answers

SEO - noindex, nofollow and canonical tag

I need some explanation about my question. Example on my header already added Should I add canonical tag again to my header?
wow
  • 6,825
  • 17
  • 50
  • 63
6
votes
4 answers

X-Robots noindex specific page in .htaccess

Can I 'noindex, follow' a specific page using x robots in .htaccess? I've found some instructions for noindexing types of files, but I can't find instruction to noindex a single page, and what I have tried so far hasn't worked. This is the page I'm…
moobot
  • 457
  • 1
  • 4
  • 19
4
votes
1 answer

How to set the X-Robots-Tag HTTP header via .htaccess file based on URL query string

Is it possible to apply HTTP header directives based on the URL's query string using an apache .htaccess? For example, based on this resource http://code.google.com/web/controlcrawlindex/docs/robots_meta_tag.html under the section titled "Practical…
John Erck
  • 9,102
  • 5
  • 55
  • 71
4
votes
1 answer

How can I check if page has noindex?

My intention is to fetch the link with PHP and maybe with Simple PHP DOM parser (or something similar) parse the content and look for H1-H6 tags. But prior to that I would need to find out if the page is being indexed at all. Other than parsing the…
Ivan Topić
  • 2,524
  • 5
  • 21
  • 42
4
votes
2 answers

Setting noindex on Amazon S3 objects

We have some publicly shared S3 files that we want to make sure won't be indexed by Google. I can't seem to find any documentation on how to do this. Is there a way to set a "noindex" x-robots-tag response header on individual S3 objects? (We're…
Yarin
  • 144,097
  • 139
  • 361
  • 489
3
votes
3 answers

Does a 'user'-url based website create issues with Google / Search Engines?

I currently maintain the backend php code for a website which allows our sales representatives to sell products and services. If a sales representative is active, he/she is given a "custom" website URL which essentially tags any activity on that…
JM4
  • 6,356
  • 18
  • 74
  • 121
3
votes
3 answers

Google, do not index YET

In the effort of building a live site on its actual live hosting platform is there a way to tell google to not YET index the website? I found the following: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710 But would that tell…
Scott Rowley
  • 478
  • 1
  • 7
  • 24
3
votes
1 answer

robots.txt: Disallow bots to access a given "url depth"

I have links with this structure: http://www.example.com/tags/blah http://www.example.com/tags/blubb http://www.example.com/tags/blah/blubb (for all items that match BOTH tags) I want google & co to spider all links that have ONE tag in the URL,…
BlaM
  • 26,721
  • 31
  • 89
  • 104
3
votes
1 answer

X Robots Tag noindex specific page

I have a Privacy Policy page on my website www.domain/privacy-policy/ which I would like to noindex with the X Robots Tag. I have tried the following code but it does not match # BEGIN WordPress RewriteEngine On RewriteBase…
bondimedical
  • 191
  • 11
3
votes
1 answer

How do you prevent a development site from being indexed?

At the moment my development site is being indexed through SERPs. I wish to prevent this from happening. I thought that you would be able to include a tag: that wmould stop it from being followed and indexed. Does this need to be on every ASP.NET…
a bit stuck
  • 23
  • 5
2
votes
1 answer

How to set noindex for Google only?

These days Google has this tag I can use to tell it not to index a page My question is do I need both the generic index and the Google noindex tags and does the order matter? Suppose I want everyone else…
Patrioticcow
  • 23,784
  • 68
  • 202
  • 327
2
votes
2 answers

Noindex in a robots.txt

I've always stopped google from indexing my website using a robots.txt file. Recently i've read an article from a google employee where he stated you should do this using meta tags. Does this mean Robots.txt won't work? Since i'm working with a CMS…
Kevin
  • 81
  • 8
1
2 3 4 5 6 7 8