Mod_security

[myob9a]

In indexing one of my sites I have recently encountered the error message "Could not download file: xxxx [Forbiden].

In discussions with my hoster the issue of MOD_SECURITY has been raised. Does ZoomSearch have some reaction to MOD_SECURITY settings? If so, what is required to enable safe operation of Zoom?

[Ray]

If you are using Spider Mode, then Zoom is indexing your website as a "client", in other words, as if it was a user with a web browser, looking at each page of your website.

This means pages which you have restricted access to for the end-user will also be restricted for Zoom to index. I presume if you go to that URL in the error message, from your browser, you should find a similar behaviour.

A Forbidden response from the server could be because the folder does not have permission to be listed. Or it could be that you have setup a user login and require HTTP authentication, or it might mean you have setup the MOD_SECURITY firewall incorrectly and are incorrectly blocking valid requests.

If you are running MOD_SECURITY on your server, and you think it might be blocking requests, then it should be trival to check the MOD_SECURITY logs to determine if and why requests were blocked.

If it does require authentication, see this FAQ:
Q. How do I index protected parts of my website requiring user authentication?