Announcement
Collapse
No announcement yet.
How do I stop Google crawling Zoom Search results
Collapse
X
-
David, once again, thank you for your reply. I could set the robots.txt file to disallow all user agents, I suppose. I understand that this won't necessarily resolve the problem entirely, but at least it will hopefully reduce it to a significant degree. I will give it a go. Thank you.
-
Information about Google's implementation of the robots.txt file is here.
https://developers.google.com/search...g/robots/intro
Items in the robots.txt file are just suggestions. Other 3rd party spiders might ignore them.
Example file would be,
User-agent: Googlebot
Disallow: /search.php
There are also other human and bot actors that will be indexing your pages. These include white and black hat hackers doing penetration testing, companies that gather information for resale (e.g. to advertisers, or to your competitors), people scraping the content of your site, online services that check for updates to pages, etc, etc...
Leave a comment:
-
PS I can add "noindex" to the search_template page, but what exactly would I need to put in the robots.txt file to stop Google indexing the results? And, having read another post on the subject I see that spammers are putting junk into the search box. Are they doing this in the hope that the pages will be indexed or just to cause nuisance by using up bandwidth? Thanks.
Leave a comment:
-
How do I stop Google crawling Zoom Search results
Hello. How can I prevent Google from crawling the Zoom search results? I have discovered that it has been crawling many 1000s of these. For example:
https://www.libraryireland.com/searc...=0&zoom_sort=0
Many thanks for any help.Tags: None
Leave a comment: