Problem with exact phrase search - incomplete results

[Ruud Hendrickx]

Whenever I try to search for an exact phrase on my website, the result page is not complete. It's a bit hard to explain, but just try it.

http://vrttaal.net

Search for my name between double quotation marks: "ruud hendrickx"

The result page always ends with:

Pagina's: 1 <a href="search.cgi?zoom_query=%22ruud+

Any idea?

Regards,

Ruud

[David]

Yes, I agree this is strange. The page is being cut short with the final few lines of HTML not being outputted.

I think your server is killing the CGI script. Probably due to it hitting a CPU or memory limit that has been set too low on the server. We see this problem a lot with the slower PHP script. But it can happen with the CGI as well if the limits are really low.

It will happen for exact phrase searches, and not others, becuase the exact phrase search takes slightly more CPU time than a normal word search.

See the link above for futher details.

[Ruud Hendrickx]

I contacted Serverpoint.com, but they say it's probably a script problem.

Maybe, you could try some searches and see if there's a pattern:

http://taalmail.net/cgi-bin/search.cgi

- "vindt jij" should find 0 results (vindt 78 rslts, jij 51 rslts)
- "ruud hendrickx" should find 12 results (ruud 42 rslts, hendrickx 26 rslts)
- "hendrickx ruud" should find 1 result
- "los angeles" should find 16 results (los 97 rslts, angeles 16 rslts)

Another interesting experiment: try searching for
- hoogachtend "ruud hendrickx"
Then put a space in front of hoogachtend and search again.

Looking forward to your reply.

Ruud

[David]

Without wanting to offend. Your hosting company are either lazy or stupid. Or both. Or maybe they aren't being paid enough to care.

Every week we see the same thing with different hosting companies. They don't investigate anything, they just blame someone else (it's quicker, costs less, and they don't need to employ competent staff if they do this).

On the page talking about this server side problem there is the test script. It is in PHP, but it should still prove the point, and show the problem is definitely on the server.

[Ruud Hendrickx]

My host company, Serverpoint, is willing to solve the problem. They invite you to have a look at

http://taalmail.net/cgi-bin/diag.php

and see what parameters need to be changed.

Thanks again for you reply!

Ruud

[Ruud Hendrickx]

Just downloaded the new release of Zoom. Guess what: everything is working fine now. Must have had something to do with my website using UTF-8.

[David]

It is unlikely that it is related to UTF-8.

Much more likely is that your hosting company fixed the configuration problem around the same time.

[Ray]

The UTF-8 bug that was fixed in the latest build does not cause the CGI to stop running prematurely. It will always execute completely, the bug only affects the number of search results returned. So I agree that the problem was much more likely to be a server limit.

[Ruud Hendrickx]

zoom 5.1.1011 vs zoom 5.1.1012

If you're interested in doing some more experimenting.

I installed the latest and the previous version of Zoom Search on a new domain at Webmasters.com, with a memory limit of 128 MB (http://www.taalmail.info/diag.php). Test.php stops at "allocated memory for 1845000 integers"). Now try this:

latest version:
http://www.taalmail.info/search.cgi?zoom_query="ruud%20hendrickx"

previous version:
http://www.taalmail.info/zoomoud/search.cgi?zoom_query="ruud%20hendrickx"

Doesn't that make you suspect that there was a bug in the previous version?

Ruud

[Ray]

I installed the latest and the previous version of Zoom Search on a new domain at Webmasters.com, with a memory limit of 128 MB (http://www.taalmail.info/diag.php). Test.php stops at "allocated memory for 1845000 integers").

Note that the memory limit reported by php_info() only applies to PHP. While the PHP script engine itself runs as a CGI, it does not guarantee that the same memory limit is available to other CGI's on the server.

It also does not rule out the possibility of a limit on CPU usage, which could still be in place.

Now try this:

latest version:
http://www.taalmail.info/search.cgi?zoom_query="ruud%20hendrickx"

previous version:
http://www.taalmail.info/zoomoud/search.cgi?zoom_query="ruud%20hendrickx"

Doesn't that make you suspect that there was a bug in the previous version?

Couple of things to note:

1.) The "previous version" you have hosted up there is actually using index files from the latest build (Version 5.1 build 1012). This is reported in the settings file, and also visible when you "View Source" the search page. Mixing index files from different sessions with different CGI builds can lead to unexpected behaviour, so we can not accept this as being indicative of what you are describing.

2.) We tried a different exact phrase query on the problematic CGI:

http://www.taalmail.info/zoomoud/search.cgi?zoom_query="zes%20miljoen"

This caused the CGI to stop running much earlier than any handling of the search query or index files in the code (which does not happen until after the search form is displayed, or at the very least, after the template file is read in and displayed up to the point of the <!--ZOOMSEARCH--> marker). There were no changes in the code in this early part of the CGI.

I can not be sure what exactly is happening on your server, but one possibility is that there are different CGI limits for different folders, or specific files. It would be interesting to see if the problem occurs when you copy ALL of the files for your "latest version" to the folder for your "previous version".

If you want us to look into it further, e-mail us and provide us with temporary FTP access to your server. We can then take a closer look to confirm if this is the case.

[Ruud Hendrickx]

more experimenting

Hi Ray,

Some more experimenting, because I still can't get most of the versions of search.cgi to work. Luckily, there is one search.cgi that works fine.

I set up four versions on two different servers (Taalmail.info is an Apache server, Taalmail.net is a Zeus server):

http://www.taalmail.info/zoom511/search.cgi (dated 7 DEC 2007)
http://www.taalmail.info/zoom512/search.cgi (dated 27 FEB 200
http://www.taalmail.info/zoom512a/search.cgi (dated 22 FEB 200
http://www.taalmail.info/zoom513/search.cgi (dated 5 MAR 200

http://www.taalmail.net/zoom511/search.cgi (dated 7 DEC 2007)
http://www.taalmail.net/zoom512/search.cgi (dated 27 FEB 200
http://www.taalmail.net/zoom512a/search.cgi (dated 22 FEB 200
http://www.taalmail.net/zoom513/search.cgi (dated 5 MAR 200

Try searching "ruud hendrickx" or "zes miljoen". Only version 512a will work flawlessly on both servers. On Taalmail.info, you'll get a customized error page.

The search.cgi in 512a was installed when I first downloaded zoom 5_1_1012. Now another version of search.cgi is included in the same release. That search.cgi is in 512.

The error log on taalmail.info has:

[Tue Mar 18 08:18:39 2008] [error] [client 83.101.2.227] Premature end of script headers: /usr/local/4admin/apache/vhosts/taalmail.info/httpdocs/zoom511/search.cgi
[Tue Mar 18 08:18:45 2008] [error] [client 83.101.2.227] Premature end of script headers: /usr/local/4admin/apache/vhosts/taalmail.info/httpdocs/zoom512/search.cgi
[Tue Mar 18 08:18:49 2008] [error] [client 83.101.2.227] Premature end of script headers: /usr/local/4admin/apache/vhosts/taalmail.info/httpdocs/zoom513/search.cgi

Maybe, you can find out what the difference is between 512a and the rest?

Regards,

Ruud

[Ray]

Ruud - As I mentioned in my previous post, I think we'll really need temporary FTP access to investigate further. We would need to test it on your server directly. Contact us via e-mail to arrange for this.

[Ray]

After investigating the issue on Ruud's server directly, we were able to determine the real cause of the problem, which was something much more obscure than originally anticipated. The nature of the problem makes it behave inconsistently, and that is why it looked like many other different possible issues.

The real problem is due to an obscure difference between versions of Linux. We have previously always compiled on an older version of Linux to ensure the widest compatibility possible with the vast variations of Linux servers out there. Unfortunately, there seems to be a bug in the older Linux C run-time library that is only exhibiting itself when it is running on some newer servers.

The build that Ruud found to have behaved differently was one that we have built in a newer Linux environment. This would indicate that the bug must have been fixed in newer versions of Linux. Unfortunately, builds that were made with the new compiler/environment raised compatibility issues with our other users hosting on older servers. So we had to switch back to our previous environment.

The good news is that we have found a way to avoid the bug in Linux, and the new build will be made using our original (and more widely compatible) development environment, while at the same time, it should fix the issues Ruud was experiencing on his server.

You can download the latest build (5.1 build 1014) from here:
http://www.wrensoft.com/zoom/whatsnew.html

[Ruud Hendrickx]

Success!

Yes, it's working! Thanks a million, you guys down-under.

Ruud