Added more starting URL's after clicking MORE in spider mode. Only the URL listed in "Start Spider From This URL" is being indexed. The other URL's are being ignored.
Announcement
Collapse
No announcement yet.
Spider Mode Only indexing starting URL
Collapse
X
-
Originally posted by webtrade View PostAdded more starting URL's after clicking MORE in spider mode. Only the URL listed in "Start Spider From This URL" is being indexed. The other URL's are being ignored.
This will tell you why Zoom may have skipped the start point. Some common reasons include the URL being invalid, or that the additional start point is a URL which has already been indexed by the first start point.
Originally posted by webtrade View PostAlso I don't see how choosing to search "ALL CATEGORIES" in a select menu is going to display results from files in different categories on a website or websites on different domains as it is implied.
Comment
-
The pages are being indexed for all starting points now. There was a variable in Skip Options that should not have been there causing one starting point to not be indexed. Both questions above have been answered.
This results in another problem. The starting points entered in the form after clicking the MORE button act as a second level to the "Start Spider From This URL" url. Setting the "Spider Options" for these second level starting points to "follow links only" will only index the page and not follow any links.
Comment
-
The start points are indexed in the order shown on the "Additional start points" window (which you see by clicking on the More button).
If your first start point has already indexed a page which is to be indexed as a second start point later on in the list, then the later start point will be skipped over, and the spider option for that start point will not apply.
If you wish to enforce a page to index with a certain spider option setting when it is actually part of two start points with different spider options, then you will need to specify them in an order which will allow this to happen.
For example, if you switch the order around, by specifying the first start point (aka the one that you can specify on the main "Start spider from this URL" box) to be the URL which you wish to set the "Follow links only" option, and have the second start point set to your original start point (which will supposedly include the first start point's page but which will now be ignored because it has already been indexed), you would be able to achieve what you describe.
Comment
Comment