Unable to rename tmp files

[mkrob]

We are currently running V6 asp.net and doing a full nightly index. Occassionally (2-3 times per week) some of the .tmp files cannont be renamed causing the search engine to throw various errors. The log files end with the following message:

Unable to rename file from zoom_recommended.zdat.tmp to zoom_recommended.zdat

Sometimes it's one file; sometimes mutliple files. When I try to manually delete the zdat file and rename the tmp file it tells me the file is locked. The only solution seems to be restarting IIS which then releases the file. Obviously this is not a great solution.

I have upgraded both the search and the asp control. Other thoughts?

[Ray]

Are you scheduling the indexer to run automatically? Is it possible that the Indexer is still running from another session and holding the files opened? Or they might even have crashed.

If you are scheduling the Indexer, make sure to enable saving of the index log file to disk ("Configure"->"Index log"), so you can check afterwards what happened.

Can you also tell us the version of Windows you are running the Indexer on, and if you are running it on the web server itself. If not, then also the version of Windows that the web server is running. I presume the Indexer is either running on the server, or it is on a machine locally networked to the server and writing the index files across the network?

[mkrob]

The indexer is currently scheduled to run every morning at 3 am. It runs from our development server and uploads the files to our production server. Both of the servers are running Windows 2003. Everything runs fine and the tmp files are upload without any problem. The problem comes when it tries to rename the .tmp files to .zdat files. At that point I get "Unable to rename file from zoom_recommended.zdat.tmp to zoom_recommended.zdat" (file name varies) in the log file and some of the zdat files are from the current index and some (the ones that it wasn't able to rename) are from the older index. This doesn't happen every day, but seems to be about once or twice a week making it impossible to have this as a completely automated process.

[David]

How many searches per minute are being run on your site, and what is the average duration of a search? I am just wondering if it is the search script holding the files open. But for this to happen you'd need a lot of load on the site. If you don't have this data you might want to turn on the logging function.

[mkrob]

We don't have that many searches. The problem occurred again this morning so I checked the logs and the indexer finished running (with the "Unable to rename" error) at 3:36 this morning. The only search that was done on the site around that time was at 3:41 a.m. - the search before that was at 2:12 a.m.

[Ray]

Can you run SysInternals Handle to determine what is holding the files open next time this occurs. This is a free Microsoft utility provided for download. This would help us eliminate the possibility that a third party application (e.g. a security product or antivirus) is holding the files open.

Can you also provide us with some examples of your log file, namely the relevant tail end of the log which shows the renaming and the error messages. This will give us some idea of the variation of files that fail to be renamed.

Can you check the Event Log for any indications of crashes of the ASP.NET Server Control. Note that crash messages from Windows Server operating systems can be silent and not evident until you check the event log or you next restart the machine.

[mkrob]

I didn't find anything in the Event log the last time this happened but will check it again if/when it happens again. I'll also run SysInternals Handle next time and let you know.

Here's the end of the last failed log file:

13|04/06/11 03:36:18|Created handle
05|04/06/11 03:36:18|C:\Zoom-ala5\settings.zdat uploaded
13|04/06/11 03:36:18|Handle Closed
13|04/06/11 03:36:18|Created handle
05|04/06/11 03:36:18|C:\Zoom-ala5\zoom_dictionary.zdat uploaded
13|04/06/11 03:36:18|Handle Closed
13|04/06/11 03:36:18|Created handle
05|04/06/11 03:36:19|C:\Zoom-ala5\zoom_wordmap.zdat uploaded
13|04/06/11 03:36:19|Handle Closed
13|04/06/11 03:36:19|Created handle
05|04/06/11 03:36:20|C:\Zoom-ala5\zoom_pagetext.zdat uploaded
13|04/06/11 03:36:20|Handle Closed
13|04/06/11 03:36:20|Created handle
05|04/06/11 03:36:20|C:\Zoom-ala5\zoom_pagedata.zdat uploaded
13|04/06/11 03:36:20|Handle Closed
13|04/06/11 03:36:20|Created handle
05|04/06/11 03:36:20|C:\Zoom-ala5\zoom_pageinfo.zdat uploaded
13|04/06/11 03:36:20|Handle Closed
13|04/06/11 03:36:20|Created handle
05|04/06/11 03:36:20|C:\Zoom-ala5\zoom_recommended.zdat uploaded
13|04/06/11 03:36:20|Handle Closed
13|04/06/11 03:36:20|Created handle
05|04/06/11 03:36:20|C:\Zoom-ala5\zoom_spelling.zdat uploaded
13|04/06/11 03:36:20|Handle Closed
05|04/06/11 03:36:20|Renaming files from .tmp filenames ...
08|04/06/11 03:36:21|FTP upload error:
Unable to rename file from zoom_pagetext.zdat.tmp to zoom_pagetext.zdat
08|04/06/11 03:36:21|FTP upload error:
Unable to rename file from zoom_pagetext.zdat.tmp to zoom_pagetext.zdat
This may be due to a temporary problem with your connection or
your FTP server. Please check your settings, and try again by clicking
"Upload" from the FTP window (accessible from the "Tools" menu).

[mkrob]

OK, it happened again twice this last week. The first time there were no searches during the time that the index was running. The second time there were. I ran SysInternals Handle after each time and all the files were being held by wpw3.exe.

When I check the Event Log I get a series of unhandled exception warnings - corresponding to searches, not the index. The warnings are all as follows:

Process information:
Process ID: 2328
Process name: w3wp.exe
Account name: NT AUTHORITY\NETWORK SERVICE

Exception information:
Exception type: AccessViolationException
Exception message: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

Request information:
Request URL: http://alanet.org/zoomsearch/search.aspx?zoom_query=juris&zoom_cat=1&zoom_per_p age=10&zoom_xml=0&zoom_and=0&zoom_sort=1
Request path: /zoomsearch/search.aspx
User host address: 66.249.71.112
User:
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\NETWORK SERVICE

Thread information:
Thread ID: 7
Thread account name: NT AUTHORITY\NETWORK SERVICE
Is impersonating: False
Stack trace: at calloc(UInt32 , UInt32 )
at zoomSearch.PrintHighlightDescription(zoomSearch* , SByte* line, Int32 bufferIndex)
at zoomSearch.search(zoomSearch* , zoomSearchSetup* zsSetup)
at ZoomASPX.ASPXSearch.Render(HtmlTextWriter output)
at System.Web.UI.Control.RenderControlInternal(HtmlTe xtWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
at System.Web.UI.Control.RenderChildrenInternal(HtmlT extWriter writer, ICollection children)
at System.Web.UI.Control.RenderChildren(HtmlTextWrite r writer)
at System.Web.UI.Page.Render(HtmlTextWriter writer)
at System.Web.UI.Control.RenderControlInternal(HtmlTe xtWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

[Ray]

We've had another user report this. We suspect the problem occurs when a search is submitted at the same time as when the index files are updated. While the indexer should take very little time to rename the files over, the problem is that IIS' application pool (depending on configuration) can keep the files opened for longer than expected and the indexer could be prevented from updating certain files.

We don't have that many searches. The problem occurred again this morning so I checked the logs and the indexer finished running (with the "Unable to rename" error) at 3:36 this morning. The only search that was done on the site around that time was at 3:41 a.m. - the search before that was at 2:12 a.m.

It's possible a search was submitted around 3:36 but it could not be logged because the process failed. The searchwords.txt logging only happens at the end of a successful query. If you can check the HTTP requests logs from IIS, this might be able to confirm this. Of course, there's also the possibility you are experiencing a different problem than what we're suspecting.

Nonetheless, we will have to implement a locking mechanism to prevent this from happening. This will be in place for the next public build (V6.0 build 1026), which should hopefully be available sometime next week.

In the meantime, you should be able to avoid the issue if you schedule indexing to happen during less busy periods on your server or less regularly (e.g. once a week as opposed to daily) if your content does not change so rapidly.

[mkrob]

Good news on the upcoming fix. I really want to do an index nightly but don't want to the frustration of dealing with this first thing every morning.

I'll keep an eye out for the new build. Thanks.

[mkrob]

Any news on the new build? The product page still lists build 1025. I'll be going out of town in a couple of weeks and would like to get this resolved as soon as possible. Thanks.

[sguptaca]

I am also facing this problem i.e. change from tmp file to zdat files

I am also facing this problem.

I have one suggestion that, since my server is running parallel plesk panel, it may be better to schedule through plesk panel schedule.

I have tried to copy and past the path from zoom scheduler to plesk but could not.

Kindly help us to solve the query as how to schedule the same through plesk panel.

[David]

Initially can you follow the suggestions made above and collect a log file.

Can you also provide details of what software version you are using, on what hardware, how often this problem happens, and what scripting option you are using.

We have fixed one issue in the upcoming 1026 build that might result in files being locked. Especially if you are using the .NET option with spelling suggestions turned on. So you could also try turning off spelling suggestions.

[sguptaca]

I am using windows 2003 enterprises version

and

Zoom Search Engine V6 Enterprise

I have purchase it as on May 21, 2011

I am facing problem frequently

[sguptaca]

Further, I have enabled spelling suggestions but now disabled.

i am using cgi/win 32 option

visit: http://www.taxmanagementindia.com