High CPU Usage

[gravity101]

Software: WinServer 2003, V5.1 Enterprise
Hardware:: Quad Core Zeon, 8gb
Usage: , 480,000 Pages, 15 Categories, CGI version (wrapped for asp.net)

Hi, In our final release testing, we came across CPU usage statistics of zoom search. Averaging 85% usage for 25 concurrent users.

Stats are as follows:

1 user, 0.85sec (3% cpu)
10 user 2.43sec (36% cpu
25 user 5.46 secs (85% cpu)

Any recommendations? Is this normal for our setup? How is version 6 on handling concurrency?

Thanks

Graham

[David]

My comments would be,

• Having 25 people performing a search in the same second would be extremely rare for all but the most popular sites on the net. Our moderately popular sites would only get to maybe 10 searches per minute at most.
• If you really have 25 people performing searches every second on your site, then you need to look at load balancing or a dedicated machine for the search function.
• The numbers you reported seem to be linear scaling of load vs CPU load. Which would seem normal.
• You could probably at least double the performance of the CGI by removing the .NET wrapping. Our CGI benchmarks show more than double the performance on much much slower hardware. In most cases there is no real need for wrapping, you can call the CGI direct.
• I don't know what else is running on your machine, but having enough RAM for effective caching of the index files can also dramatically improve performance. (Note that this is probably not an issue on your machine). Failing this a solid state disk can also help.
  
• We haven't done benchmarking on the V6 CGI as yet. But the performance should be similar to V5 for most searches. Maybe slightly better in some extreme cases.
  
• In V6 there is a native .NET control available. But we aren't expecting .NET to significantly outperform the CGI. They share a lot of the same code in fact.
  

[gravity101]

Hi

Thanks for your input.

No your right, 25 concurrent requests (not a second, thats not the same thing) is way exceeding our current demand, but in capacity planning we have to test what its limits are and how that effects the server right?

As the search feature is very prominant on our site, we currently get 3/s with our limited DB search service, im expecting that to at least double when users see a site search ability..

You mention caching of the index files, As we just 'call' the cgi i dont think we have control over caching, does the cgi script cache it for us? Or would it be accessing the file i/o each time?

Thanks

Graham

[David]

The O/S will cache the files automatically if you have free RAM. In the past we recommended using RAM disks or solid state drives. As this will increase performance dramatically compare to a slow hard drive. But we did some testing a few weeks ago and found that having ample free RAM had a similar effect as putting everything on a RAM disk (after the initial few searches forced the files into cache).

It would be good if you could redo the test calling the search.cgi directly rather than your wrapping ASPX code and post the results.

Besides load sharing, there are exotic / more expensive solutions to further improve performance including,
- MasterNode distributed search
- Using FastCGI instead of CGI (this would require some additional coding, and can be harder to install)
- Study all the data in your index and look at documents can be removed without effecting the user experience too much.
- Look at all options you are using in Zoom and work out what is the minimum set of features you can live with. Some of them impact on performance. For example, turning off context descriptions and exact phrase can give a significant gain in performance.