Error - Check that the URL exists and satisfies the conditio

[joleneb]

I am using spider mode to index.

I receive the following error window:
Check that the URL exists and satisfies the conditions in the configuration window.

The status log (verbose mode) shows that it goes to download the file to be indexed and then says, "Could not download file" (filename) "/(forbidden)"

I am indexing an online web site.

This URL is a subdomain of a URL that I have already indexed.

i.e. /library/

I want to index this subdomain, as well as 6 others.

/library/folder1/
/library/folder2/
etc.


Could I be experiencing errors because I've already indexed the parent directory? I turned off the "duplicate page detection" option.

[Ray]

No, the "Forbidden" status is a message returned from the web server. It would seem that you are specifying a start URL which is not actually accessible. Try entering this same URL into a browser and you should see a similar error.

Note that spider mode works by crawling actual web pages, and following the links on each page, in order to find other pages. You generally do not ask it to index the contents of a folder and all subfolders (this is how Offline mode works) - although it is possible if you enable directory listings on your web server. Typically however, you will need to give it pages which link to the files that you wish to index.

If you give us the URL to your site, we can provide more specific details to help.