Search result sorting by score - bug?

[Rene]

Dear Wrensoft Team,

it seems as if the search result sorting function is not working correctly on our V6 installation.
We have used Zoom Search for years, but are just now starting to really optimize the search results.

For this reason I have turned on the "Scores" option in the Results layout Tab.

What I can see now is that the results are not always sorted by the actual score.

For one search phrase for example I would get:
15
160
106
54
44
20
....

the rest of the list is ok in this case, but the first result is in the wrong place.

In another case it looks like this:
979
808
122
94
72
286
286
272
245
241
217
191
171
171
137
71
54
178
......
Theres more wrong in this example, but the list would be too long.

So what could be the reason for this behaviour? And how would I fix this?

I have deleted and reindexed lots of times, as well as changed weightings and used ZOOMBOOST Metatags before I turned on the Score to check why nothing changes.

Whatever I do changes the Scores, but not the sorting it seems.
It is not sorted by date either, all results show todays date.

Thanks for your help!
Rene

[David]

Are these single word searches or multi-word / phrase searches?

For multi-word searches there are two sorting factors. The number of words matched and the overall score. So pages that contain all the search words always get sorted first.

[Rene]

Are these single word searches or multi-word / phrase searches?

For multi-word searches there are two sorting factors. The number of words matched and the overall score. So pages that contain all the search words always get sorted first.

These searches are single word searches.
Right now were are only testing with single word searches.

[Ray]

Are you sure you are using V6? We discovered a bug in the V7 beta where "sorting by relevance" will not work when date sorting is available (basically it will always sort by date, either ascendingly or descendingly).

If you have previously installed V7, then you may still have the V7 scripts sitting around as well. Make sure to uninstall V7 completely before re-installing V6 if you are trying to rollback. You can also double check the scripts in:
C:\ProgramData\Wrensoft\Zoom Search Engine Indexer\scripts\

Let us know if you still have the problem. Also, please tell us which script platform you are using, and the exact version and build of the indexer. If possible, zip up your output files and e-mail them to us.

If you are using V7, then let us know. We're about to put out a new beta release with the bug fixed.

[Rene]

Let us know if you still have the problem. Also, please tell us which script platform you are using, and the exact version and build of the indexer. If possible, zip up your output files and e-mail them to us.

Ray,

we are using Version 6.0, Build 1027 at the moment and we never had V7 installed. The search script is the ASP version running on a IIS 7 webserver. I will email the output files to info@wrensoft.com.

Thank you!
Rene

[Ray]

We've had a look at the files and verified that this is not a bug.

The reason things look odd here is that you have disabled the "Terms matched" information for the search results. When you turn this back on, you will see that the results which are ranked higher, will have a greater number of terms matched.

Note that the sorting by relevance is determined based on a combination of the "Terms matched" value and the "Score". So sometimes, the following is normal behaviour:

Result #1
Terms matched: 4 - Score: 116

Result #2
Terms matched: 4 - Score: 9

Result #3
Terms matched: 3 - Score: 54

Result #4
Terms matched: 2 - Score: 43


This is doubly unexpected in your search configuration because you have enabled "Substring match for all searches" (under "Configure"->"Languages"). This is not really recommended for any languages besides Asian languages which have no space characters.

In the case of English or German, this feature allows a query of the word "test" to match "Praxistest", "getestet", "spatestens", "Testbetrieb", etc. Which is not very helpful, in most cases. This is why there were so many terms matched even when searching with a one word query.

We would suggest disabling this option. But if stemming is essential, there is German stemming available for the CGI version.

[Rene]

Thanks for your efforts Ray.

We do want to use substring search for all searches though.

I have found that the score does not seem to overrule the terms matched in any of our tests.
If I have a page with 3 terms matches and a score of 71, it will show before a page with 2 terms matched and a score of 2492.

I am not entirely sure how the score is calculated but it seems that most of the time pages with a higher score have more terms matched anyway.
I think due to the fact that we use a "strong adjustment" for "word position" and weighting boosts for title and headline there are many pages that have less terms matched but a higher score that we really want to show further up on the results.

I have now made changes to the sorting function in the ASP search script so that it will sort by score only, which improved the results for us on all our test-searches.

In addition to that we have a requirement to really boost some specific pages a lot, so I gave the ZOOMPAGEBOOST Metatags even more of a "boost" in the search script.
Now the +5 boosted pages actually come up on top (or on second/third) place all the time.

Thanks again!
Rene