Originally posted by timhodge
View Post
Q. I am indexing with spider mode but it is not finding all the pages on my web site
Originally posted by timhodge
View Post
For other users (or other sites in the future), where you do not have directory listing enabled, you may add the URL to each file you wish to index as individual start points. This would essentially work like an "opposite skip list" and allow you to specify files that have to be indexed even when the spider fails to find links to them.
Leave a comment: