"Error: Corrupted or invalid index files" when zoom_query parameter is empty

[jookyg]

Hi there. Love Zoom Search. My company has successfully rolled it out in two different client sites now, with more in the works. Each time the implementation gets more complex, and Zoom handles the complexities with ease.

I have a question...

One of our search pages requires us to return all results for a given category and then post-process them using custom business logic. We use the CGI implementation and parse the XML results using .NET code.

When the zoom_query parameter is empty, the list of <item></item> nodes in the XML ends prematurely with the error "Corrupted or invalid index files."

<zoom:error>Error: Corrupted or invalid index files. Please re-index and try again.</zoom:error>

Here is a live example:

http://schiffhardin.com.prod.tenrec.com/ZoomSearch/search.cgi?zoom_query=&zoom_cat=0&zoom_xml=1&zoom_ and=1&zoom_per_page=5000&SERVER=1863

(The extra SERVER parameter in the URL above directs the request to a particular server in our cluster).

If I use the double-asterisk wildcard (**) in the query, this error does not occur, and everything works as expected. It's simple enough to ensure the wildcard is always there, but I'm concerned there might be something else going on.

My question is this: Is this expected behavior, or is there possibly some other sort of corruption in my index files that gets exposed only when zoom_query is empty?

Thanks!

-Greg

[Ray]

I can't reproduce that error here, using a fresh index created with V7 build 1023 -- with an empty search query, and a custom meta field specified.

So it does seem possible that there's something wrong with your index. Note that I can also see the error when XML mode is disabled (it's the first line if you view source).

The error is generally related to the "zoom_pagedata.zdat" file. Or the "settings.zdat" file contains information that does not match the former. Either way, it would be worth confirming that your full set of index files are up-to-date (all files listed at the end of indexing as "Required Files"). If you do not update the "settings.zdat" file for example (or you modify it), you would then be mixing index files from different sessions.

[jookyg]

Ok, that all makes sense. So the double-star wildcard shouldn't be necessary, if the files are in a good state?

Also, based on this information, I assume that the zoom_pagedata.zdat and settings.zdat files get appended and not completely regenerated when a full reindex is performed (using the scheduler, for example). Is that correct? Otherwise, I would expect that the error should no longer exist, since we perform a full reindex nightly.

Thanks

[jookyg]

Ray,

I deleted and completely rebuilt the index files, and the corruption error message went away. I can now perform searches without the wildcard and get all the results I'm expecting, but I plan to leave the wildcard in place, in case it occurs again. I believe it started when I had two indexing processes collide--our server launches the indexer via command line whenever a page in the site is saved. We've implemented an indexing queue to prevent these collisions.

Follow-up question: is it possible that the indexer process collision could have corrupted the config file? There was an instance where some of the settings "disappeared." It was strange because I think they were default settings (Indexing Options).

Thanks!

[Ray]

If the command line parameters you are using invokes incremental indexing, then yes, it could potentially corrupt the configuration (since incremental indexing may require updating the config file). Otherwise, I can't imagine how it could with normal (full) indexing.

Having said that, yes, do avoid running two simultaneous indexing processes with the same configuration file (which means it will attempt to write to the same index files, etc.). We haven't come up with an effective way to prevent this just yet, but may need to do so in the future.