v6.0 50% slower than v5.1???

[Liebel]

Greetings,
i just upgrade to v.6 and recognized immediately that searches took 2x as long as the good old 5.1
(1,2-2sec compared to the old version (350-700ms)
..First i thought it was the settings and "pushed" "optimization button to ""fast search" but putting both search script on the same machine on the same side...it seems pretty obvious...(is there something i did wrong? (i like these sort of questions
What i did...same configfile (v.5.1 ->)

See the comparison here
http://harvester.fzk.de
(the search-windows for human/mouse/rat/zebrafish/arabidposis have the new v6.0 engine (1,2-2sec for any search term)
the last one (drosophila uses the old one (v.51.) ~300ms)


Search for "protein" finds "all pages in each individual subset.

the number of pages in the indexes

human: 73907 - searchtime 2,3 sec
mouse: 55290 - searchtime 1,6 sec
rat: 39888 - searchtime 1,3 sec
zebrafish. 42963 - searchtime 1,9 sec
arabidopsis 36654 - searchtime 1,0
drosophila 29527 - searchtime ~ 0,3 sec


donīt get me wrong..we have been using zoom for the last 2 years and love it...just wondering if there is a new "feature that slows us down .-)

Tech: details:
server: 2x quad (2,0Ghz), RAID 2x 10.0000 Rpm , 300GB, windows 2003 server, apache

Greetings from Karlsruhe Germany

Urban

[David]

If you have turned on new V6 features like stemming then the search might be slower. The V6 index files are also slightly larger, to have the space to cater for very large data sets, so the disk access times are likely to slightly longer. The new templating options probably also take up a few dozen milliseconds to complete.

We also haven't put much effort into code profiling V6 as yet. So there is probably tweaks that can be done over time to lift the performance.

Further we added new algorithms in V6 to significantly speed up exact phase search in medium to large documents. But I think you are only indexing the page titles (and not page content) and only doing single word searches. So you aren't seeing this improvement in your scenario.

But you also aren't doing an apples for apples comparision. Your V6 first data set is more than twice as large as your V5 data set.

[Liebel]

v6->v5.1

thansk for the fast answer,
i reindexed the "arabidopsis" data set with the v5.1

http://harvester.fzk.de

before (see above) search for protein was
~ 1,0 sec...with 5.1 it is now back to 0,36 sec..
(same config file ...that much i can say)...

...i will do this for the other datasets too...
if there is anything i can do to give you more testing/help/info..please let me know...

p.s. i do index the entire HTML file (not only the title as you speculated above)

greetings and thanks again

Urban

[Ray]

As mentioned before, the examples in the original post is quite misleading because you are using different data sets between different versions.

Nonetheless, if you can send us your config file, we can do some tests on this side. I presume you are indexing with Spider Mode and the content is online so we can index the same data.

As mentioned, some of the new features such as Stemming, may have had an affect. I noticed that your website feature very uncommon words (such as "papillomavirus", or HBeAG-binding" and "Granulocyte"), which would not benefit much from stemming and certainly demonstrates a very unusual scenario. You should perhaps at least do your comparisons with Stemming disabled in V6 (on the "Languages" panel of the "Configure" tab).

If you can send us your ZCFG configuration file, we can look further into it.