How to Remove Your Site from Google Index
- 06/06/2020 8:01 PM
You can use a robots.txt file to request that search engines removeyour site and prevent robots from crawling it in the future. (It'simportant to note that if a robot discovers your site by other means -for example, by following a link to your URL from another site - yourcontent may still appear in our index and our search results. Toentirely prevent a page from being added to the Google index even ifother sites link to it, use a noindex meta tag.)
To prevent robots from crawling your site, place the following robots.txt file in your server root:
User-agent: *
Disallow: /
To remove your site from Google only and prevent just Googlebot fromcrawling your site in the future, place the following robots.txt filein your server root:
User-agent: Googlebot
Disallow: /
Each port must have its own robots.txt file. In particular, if youserve content via both http and https, you'll need a separaterobots.txt file for each of these protocols. For example, to allowGooglebot to index all http pages but no https pages, you'd use therobots.txt files below.
For your http protocol (http://yourserver.com/robots.txt):
User-agent: *
Allow: /
For the https protocol (https://yourserver.com/robots.txt):
User-agent: *
Disallow: /
