Hangs when using zoom ftp

[adman4054]

I'm using the 5.0 professional version for a newspaper site. The ftp hangs when uploading a very large file and freezes the server (I'm running the program from my own server to a commercial host). I read one post that said that it is probably a "timed out" issue on the host and I think thats what the problem is. Same as the other post I can use an ftp program and can upload the files just fine. I tried the "pasv" setting and that produced the same results.
There is a group of 6000 html files that are old stories that do not change. Is there anyway that I could still include the 6000 files in the search results, but not index them everytime Zoom indexes the site? This is just an idea to get around the time out issue, but I'm open to any other suggestions.

Thanks

[David]

We are not aware of any problems with the FTP function in V5 of the software. (There were a few issues in V4, but we think we fixed all of them in V5).

You can avoid (re)indexing the files from your web sire by using the incremental indexing option, or by indexing in offline mode (if you just have static HTML files). But this would seem to be completely unrelated to the FTP issue, because the overall size of the index files will not change by using incremental or offline indexing.

If you control the server and it is a windows server, then you could install the indexer on the server and avoid FTP entirely.

If you want us to investigate the FTP issue, then we would need a copy of your index files and FTP login details.

When you say it "freezes the server", do you really mean that the server totally locks up and that you need to power it off and on to restart it?

How far does it get into the FTP process before stopping? Can you post the log? Was this a one off error or does it work from time to time. Do you know what server side FTP software the remote host is running?

[adman4054]

I'm assuming its an ftp problem after reading an almost identical post on this forum, but maybe not.
Its a web hosting package from Network solutions, so installing the program on their server is not an option.
It does lock up my server and I have to power it down and restart before I can continue.
Just checked and I didnt have the log file checked.
The ftp stops at the "wordmap" file. According to the zoom program its gets to 100% of the file size and then just locks up. This is the 5th file of 7 that I upload to the server.
Finally, it hasnt worked at all and I have noa idea what ftp server software Network Solutions is running.

[csiemers]

As strange as this might sound, minimize Zoom Search when it starts indexing your site.

I had problems when Zoom would reach the point to FTP the files, and would freeze.

I started to minimize Zoom to my task bar, and have yet to have it freeze on me.

When I do bring it to the forefront to process, about 75% of the time it freezes.

It's worth a try at least.

[David]

The size of the window should make no difference. If you have FTP problems, but you have a stable internet connection, then the first thing to try is using PASV mode.

In no case should the Zoom software lock up. But due to some timeouts it can take a minute or more before it errors out as a result of a protocol error.

If your entire machine locks up, (all applications freeze & the mouse stops working, etc..) and you need to power off the machine, then it is really unlikely this is due to our software. This is much more likely to be a hardware fault or hardware device driver bug.

If you continue to have FTP problems, we can try and reproduce the problem from here. But we will need, a zipped up copy of your index files, your FTP login details and system details about your client machine.

[David]

Also, if your connection is not stable, you should make sure you select the "upload and tmp filenames, then rename" option. This can help prevent half a file being left on the server in the case of a disconnect.

[David]

With FTP details supplied by adman4054, we attempted some uploads and downloads from the server in question.

Small transfers work OK. But larger transfers don't seem to work. Here is the log from the 3rd party Ipswitch FTP tool,


==================
150 Data connection accepted from 150.101.113.240:2899; transfer starting for zoom_titles.zdat (168397 bytes).
226 Transfer ok
transferred 168397 bytes in 2.688 seconds, 489.526 Kbps ( 61.191 Kbps), transfer succeeded.
TYPE I
200 Type set to I.
PASV
227 Entering Passive Mode (205,145,137,195,19,140)
connecting data channel to 205.145.137.195:19,140(5004)
data channel connected to 205.145.137.195:19,140(5004)
RETR zoom_pagetext.zdat
150 Data connection accepted from 150.101.113.240:2900; transfer starting for zoom_pagetext.zdat (17911926 bytes).
Error reading response from server.
transferred 17911926 bytes in 319.281 seconds, 438.287 Kbps ( 54.786 Kbps), transfer failed.
It appears that the connection is dead. Attempting reconnect...
Connecting to 205.145.137.195:21
Connected to 205.145.137.195:21 in 0.000000 seconds, Waiting for Server Response
220 stuffsites2
Host type (1): Automatic Detect
==================

Some advanced FTP tools will attempt to do an auto-reconnect if the connection is lost, but Zoom doesn't do this at the moment. So I think this is probably the problem.

[Hex Angel]

I had problems when Zoom would reach the point to FTP the files, and would freeze.

Yep. I have the same problem, though less than 75% freeze-up.

Running windows XP Media Center edition.

I have another issue specific to Media Center I'll report in another thread.

p.

[Ray]

We've done some further testing with adman4054's FTP server and have concluded that the problem is on the server side.

We tested file transfers to and from the FTP server with several different third-party FTP applications (including Ipswitch WS_FTP and FTP Voyager) and the problem exhibited itself in all of them. The server would always disconnect during large transfers (eg. of 18MB or more) - ignoring anti-timeout commands such as NOOP; requiring the FTP clients to reconnect, and resume their transfers. Note that this might not be obvious if your client is automatically reconnecting and resuming and if you are not looking at the transfer log window.

Zoom's built-in FTP functionality does not currently support automatic reconnects, and it can not resume file transfers for interrupted uploads (note also that resume is not actually supported on many FTP servers). However, in our tests, Zoom never crashed when it encountered this problem with the FTP server - it only stopped uploading and displayed an error message stating that the server connection has been reset. This is expected and correct behaviour.

It may be worth contacting your host about this, and pointing out how all FTP clients seem to be disconnecting during transfers for large files. I suspect that the problem is timeout related, and that it is because the server is ignoring NOOP messages. This may be due to a firewall on the side of your web host which is configured to filter out some or all ICMP messages. When this is incorrectly setup on the server, this can be the cause of much problems, including what we are seeing here. More information can be found on the web on such common ISP mistakes:
http://www.freelabs.com/~whitis/isp_mistakes.html
http://www.znep.com/~marcs/mtu/

Hope that helps.