PassMark Logo
Home » Forum

Announcement

Collapse
No announcement yet.

Advise on incremental Index large site

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Advise on incremental Index large site

    How should I go about an incremental indexing of a site with 100,000+
    Pages. Do you think it would be best to just have a scheduled reindex for the whole section or have a scheduled incremental .

    The index will grow about 1000 pages per 3 days or 1 indexing.
    chopped it down to 4 words per page to be indexed.

    OFFLINE <- Server 2008 4gb . Medium to mild traffic CGI.


    Thank you

  • #2
    If you have a way of managing a list of the new pages that need to be added to the index, then you could certainly call Zoom to add that incrementally to the existing index. Assuming also that these pages are completely new, and not changes to existing pages.

    If you have a Content Management System (CMS) of some sort and you have control over this, you could automate this process (for example, it could call it at the end of every 3 days) and call Zoom with a text file containing the list of files to increment. This is detailed in the command-line options section here.
    --Ray
    Wrensoft Web Software
    Sydney, Australia
    Zoom Search Engine

    Comment


    • #3
      Hum

      Well, I think I'll be ok re-indexing every 3 days or so. I have it automated so I should be all right. I have the max index set to 35 Charters Par page.
      Zoom should zoom through in a few bits.

      As for CMS. Still wonder what that stands for. LOL

      I will not be using such a thing. I have attempted to implement no less the 5 CMS brands. NONE of them worked out. It seems 1/2 a million pages and 1.3 million images won't site well with some Free CMS.....


      Thank you
      Regards

      Comment


      • #4
        I just mean that you can programmatically call Zoom to add pages, if there is any sort of scripted backend to add articles/pages to your website. That's all I mean by CMS (technically, any such scripted backend would count as a "CMS"), I wasn't referring to any particular CMS product that is out on the market.

        If the pages are all manually added, then you can probably ignore that idea.
        --Ray
        Wrensoft Web Software
        Sydney, Australia
        Zoom Search Engine

        Comment

        Working...
        X