What's the maximum simultaneous search capacity for enterprise cgi?

[Jerren]

Hi,

Can anyone tel me how many simultaneous requests the enterprise Zoom CGI interface can safely handle?

Are there any guidelines/formulas for sizing? For example say you have X number of users using the search engine and doing Y searches per minute you will need Z servers running zoom to handle the load?

Thanks!

[David]

There are many factors that effect the number of searches that can be performed in a given time. These include,

1. The hardware in the server. Disk speed, and CPU speed are critical
2. If there is enough RAM in the machine to cache the index files
3. How much load is already on the server, and if the server is shared with other web sites
4. How large the index files are
5. What type of searches are performed. (Wildcard and exact phrase searches need more resources)
6. What options are selected in the Zoom configuration window. e.g. logging, what content was indexed, categories, the optimization slider position, etc..
7. The Internet bandwidth available

You can however find the results of some benchmark testing we did here. Even on the old Athlon XP 3200+ the CGI got search times of 0.08 seconds for a 60,000 page site. There is some web server overhead that is not included in this number (e.g sending the results out over the Ethernet connection) but on new hardware it would be reasonable to expect 200+ searches per minute. For a smaller site, with a new server, you might get to 1000+ searches per minute.

But for a good estimate you really need to test with your own data and own hardware.

[Jerren]

Thank's I was using that information as a baseline, but I guess a better question would be what happens if you feed Zoom more requests than it can handle in a small period of time?

[David]

It slows down.

Then your web server might complain about about too many connections. If it doesn't and you keep going, then the hard disk would start thrashing, available RAM would decrease as more instances of the CGI are started by the server and eventually something would break due to lack of RAM, server timeouts, or lack of server resources. What breaks first would depend on your system.

[Ray]

Just wanted to add that the same would happen with any other CGI running on the system. Generally, your web server should have limits setup to prevent things degrading to this level and would simply start denying requests with the "too many connections" or "server busy" type of messages.

[Jerren]

Hmm ok I'll set up some lab testing but the current minimum spec for the web server that will be hosting this is dual Xenon Dual core 2.4Ghz or better with minimum of 4 gigs of ram. System drive mirrored, Application drive on raid 5.

The application in question is a knowledge base for a helpdesk, which means there may be peak times where we would see "bursts" of activity followed by moderate periods of low usage. Of course we are most concerned with the "bursts".

I'm conservatively estimating that we should be able to handle a Maximum 500 simultaneous searches per server based on the above configuration. (Servers clustered behind a f5 BigIP 6400 Enterprise balancer/application accelerator.) With the assumption that a) the servers should have the capacity to handle that level of load from time to time, and b) there will be sufficient recovery time between bursts.

I guess I'm just looking for a sanity check here, we will be monitoring the application post go live and will be adding additional servers as needed as we add more clients to the system.

[David]

While a search in running (for ~0.1 sec) it might use 16MB - 64MB+ of RAM depending on the size of your index and what is searched for. You claim you are going to get 500 simultaneous searches, then this implies (in theory) that you will need ~16GB of RAM.

But in reality the server can't do 500 things at the same time and requests will be somewhat serialized. So it might take 30 sec to process the 500 requests, which means only 16 searches actually happen at the same time (512MB RAM).

Depending on your server, there is also the possibility of switching from CGI to FastCGI. This would involve some work on our side and on your server. But we think it would at least double performance and significantly reduce RAM usage when multiple instances are running. Search times should get down into the 0.01 - 0.05 sec range.

In short you need to do some testing with your hardware and index data.

But please post the results.

[Jerren]

That's interesting I didn't know you supported FastCGI out of the box, can it be run multi threaded or just single threaded under FastCGI? There's an ISAPI filter out there to implement the FASTCGI interface easily under IIS and I'm looking at that now, but I would be interested if there is a FAQ available on the subject from WrenSoft that addresses FASTCGI in general...

[David]

I didn't know you supported FastCGI out of the box

We don't.
We have looked into what is required as we thought that eventually someone would ask about it. It would involve some work on our side to make it work (i.e a bunch of exta code and testing). It was one of the things we were considering for the next major release. But very very few people know what FastCGI is and not many shared hosting companies support it. So demand is low. In fact no one has asked for it. I only mentioned it as an option if the normal CGI is not adequate. We could come to some arrangement to fast track the development.

[Jerren]

We don't.
We have looked into what is required as we thought that eventually someone would ask about it. It would involve some work on our side to make it work (i.e a bunch of exta code and testing). It was one of the things we were considering for the next major release. But very very few people know what FastCGI is and not many shared hosting companies support it. So demand is low. In fact no one has asked for it. I only mentioned it as an option if the normal CGI is not adequate. We could come to some arrangement to fast track the development.

LOL, well I have a test platform now if you decide you want to pursue it further. After some additional analysis I think we will be ok as is, but as we move forward we may want to work with you more on this item to reduce hardware dependence for larger sites.