PassMark Logo
Home » Forum

Announcement

Collapse
No announcement yet.

Spider Mode Only indexing starting URL

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spider Mode Only indexing starting URL

    Added more starting URL's after clicking MORE in spider mode. Only the URL listed in "Start Spider From This URL" is being indexed. The other URL's are being ignored.

  • #2
    Also I don't see how choosing to search "ALL CATEGORIES" in a select menu is going to display results from files in different categories on a website or websites on different domains as it is implied.

    Comment


    • #3
      Originally posted by webtrade View Post
      Added more starting URL's after clicking MORE in spider mode. Only the URL listed in "Start Spider From This URL" is being indexed. The other URL's are being ignored.
      If this is the case, there should have been an error message as to why the additional start points were not indexed. Check the index log. If you have Error and Warning messages disabled, re-enable them on the "Index Log" tab of the Configuration window. For the most information, including "Skipped" messages, you can turn on "Verbose mode".

      This will tell you why Zoom may have skipped the start point. Some common reasons include the URL being invalid, or that the additional start point is a URL which has already been indexed by the first start point.

      Originally posted by webtrade View Post
      Also I don't see how choosing to search "ALL CATEGORIES" in a select menu is going to display results from files in different categories on a website or websites on different domains as it is implied.
      Not sure what you mean here. If you have indexed multiple domains within one set of index files (usually using multiple start points), then the default search behaviour will search across all the domains/pages you have indexed. If you have categories setup so that you can filter results by certain domain names, then you would be able to perform searches within a specific domain by selecting one of the categories, or search across all your indexed domains by selecting the "All" category.
      --Ray
      Wrensoft Web Software
      Sydney, Australia
      Zoom Search Engine

      Comment


      • #4
        The pages are being indexed for all starting points now. There was a variable in Skip Options that should not have been there causing one starting point to not be indexed. Both questions above have been answered.

        This results in another problem. The starting points entered in the form after clicking the MORE button act as a second level to the "Start Spider From This URL" url. Setting the "Spider Options" for these second level starting points to "follow links only" will only index the page and not follow any links.

        Comment


        • #5
          The start points are indexed in the order shown on the "Additional start points" window (which you see by clicking on the More button).

          If your first start point has already indexed a page which is to be indexed as a second start point later on in the list, then the later start point will be skipped over, and the spider option for that start point will not apply.

          If you wish to enforce a page to index with a certain spider option setting when it is actually part of two start points with different spider options, then you will need to specify them in an order which will allow this to happen.

          For example, if you switch the order around, by specifying the first start point (aka the one that you can specify on the main "Start spider from this URL" box) to be the URL which you wish to set the "Follow links only" option, and have the second start point set to your original start point (which will supposedly include the first start point's page but which will now be ignored because it has already been indexed), you would be able to achieve what you describe.
          --Ray
          Wrensoft Web Software
          Sydney, Australia
          Zoom Search Engine

          Comment

          Working...
          X