v. 8.1004 indexing log error: missing .tmp file for Zoom Office 2007 plugin

[rkg82]

Noticed a new "Could not open" error in log that we never saw before, appearing periodically in the midst of other Office 2007 plugin errors.
The error does prevent completion of indexing. Nor does it the appear to compromise search operations and results.
Note: Some paths in the following log excerpt have been altered.

09:40:21 - Start indexing (offline mode) at Wed Jun 26 09:40:21 2019
09:41:13 - [ERROR] Error: Could not open file: C:\inetpub\wwwroot\test\zoomindex\ServerTWT\zoom_7 1.plugin.out.tmp (Error: The system cannot find the file specified. )
...
09:42:05 - [ERROR] [Office 2007 plugin error] Could not recognise OOXML file format (C:\Users\richardg\Documents\Personal - DO NOT BACKUP\Qualcomm_Hosted_Sharepoint\ext\MyFolder\MyW ordDoc1.docx)
...
21:42:32 - [ERROR] [Office 2007 plugin error] Could not open OOXML file (error reading from: C:\Users\richardg\svn\qitc-doc\4.x\hw\rtl\obs\verif\MyPlan1.xlsm) (C:\Users\richardg\svn\qitc-doc\4.x\hw\rtl\obs\verif\MyPlan1.xlsm)
21:42:32 - [ERROR] Error: Could not open file: C:\inetpub\wwwroot\test\zoomindex\ServerVERIF\zoom _807.plugin.out.tmp (Error: The system cannot find the file specified. )
21:42:33 - [ERROR] [Office 2007 plugin error] Could not open OOXML file (error reading from: C:\Users\richardg\svn\qitc-doc\4.x\hw\rtl\obs\verif\MyPlan2.xlsm) (C:\Users\richardg\svn\qitc-doc\4.x\hw\rtl\obs\verif\MyPlan2.xlsm)

[rkg82]

Sorry, typo: the error does NOT prevent completion of indexing.

[David]

Can you Email us your entire log. So we can see what settings are being used, etc..

Do the error keep happening on the same documents all the time, or is it random?

[rkg82]

Have done some quick testing with a subset of the original offline configuration, building back up to the full config. progressively, but have not been able to reproduce the issue so far.

I believe we see this issue intermittently only when we have two or three instances of Zoom running simultaneously, typically in full admin mode, but not always. Unfortunately not much time at the moment for testing under these conditions, but will send a complete log, which I'll need to obfuscate a bit first, if and when I can reproduce the problem. I just noticed in the log excerpt I posted the unexpected reference to the "zoom 7.1 plugin". We no longer have v. 7 installed.

I'll have another opportunity tonight or tomorrow morning to run the full config in real conditions, with 3 instances of Zoom running respectively one spider config and two offline configs. I have only seen the issue, I believe, with the offline configs. All three configs index a certain number of the same files, as well as config.-specific files.

[David]

In theory you can run multiple instances of Zoom at the same time on the same machine with the same document set. But it is an area that doesn't get nearly as much testing as running a single instance.

[Ray]

The important thing is to make sure the simultaneous instances do not write to the same folder (Output Directory), or (as I think is happening in the case above), it's trying to read from a folder where another instances is writing to.

It seems to me that you have an instance of Zoom writing to the "ServerTWT" folder, while another instance is writing to "ServerVERIF" folder, and here in this 3rd instance, you are trying to index (Read) from those output folders. This scenario will certainly cause the problem above.

You should be able to avoid this somewhat, by either specifying the skip folders, or at the very least, disallow ".tmp" files from indexing.

[rkg82]

Thanks for the suggestions.

All our output folders are indeed distinct for each Zoom instance running. These output directories do reside in the same parent folder on our test machine, so that we can test the result before deploying the output files to our production server.

I'll check my configurations again to see if there is indeed an incorrect path that might be lead to an indexing attempt in an output folder, but to me that is highly unlikely.

Given that my two offline profiles do index a certain number of the same files stored locally in target folder X, and that the profiles are both configured to use the max. number of threads, I presume there is a relatively high chance that they sometimes will be reading the same file in target folder X at the same time. Does Zoom search by design write these .tmp files, reported by the log, in the distinct output directories when certain types of target file is being read in another directory?

I will exclude .tmp files from indexing, but again, I don't think any of my declared indexing target paths are pointing to my output folders. If the .tmp files were being written in the indexing target folders, which does not appear to be the case according to the log, and if my two instances where trying to simultaneously read the same file in a target folder that suddenly had a .tmp file written to it by one of the one Zoom instances, I'd expect a potential read/write conflict.

Presumably, if I alter the ordering of the indexing target paths in my two offline configurations so that the common target paths are located at opposite ends of the path list in the res^pective config. files, and if I limit the threads to just one, the likelihood of a simultaneous read/write attempt would diminish considerably. This assumes that Zoom search indexes the target paths in the order they are listed in the config file. I assume, however, that when multiple threads are being used, execution order might not be exactly the same every time, which might explain why the issue does not appear systematically.

[rkg82]

For what it's worth, I just noticed in the list of new v. 8 features that there is a RAM drive used to store temporary files. Is there a possible relationship to the issue at hand? I know our machines have agents monitoring read/write activity at times, which sometimes slows things down on the local machine.

[Ray]

Does Zoom search by design write these .tmp files, reported by the log, in the distinct output directories when certain types of target file is being read in another directory?

Yes, Zoom only writes the .tmp files in the Output Directory, which should be distinct between your configurations. So there shouldn't be any problem writing these files. There certainly could be a problem reading from the same files however, if your multiple simultaneous indexing sessions will be attempting to index the same content. It would depend on the file type.
However, the error seen above in the original post should not be caused by this.

Presumably, if I alter the ordering of the indexing target paths in my two offline configurations so that the common target paths are located at opposite ends of the path list in the res^pective config. files, and if I limit the threads to just one, the likelihood of a simultaneous read/write attempt would diminish considerably. This assumes that Zoom search indexes the target paths in the order they are listed in the config file. I assume, however, that when multiple threads are being used, execution order might not be exactly the same every time, which might explain why the issue does not appear systematically.

As you noted, multi-thread would make this difficult to predict, so I wouldn't rely on the "opposite ends of the path list". There is no such thing as a definite order to the path list, whether it is alphabetic, date sorted, etc. is all display purposes.

The reason I guessed that you're actually indexing .tmp files from another session (i.e. your indexing path is overlapping with the OUTPUT path of another instance), is because the file path of the two .tmp files are different. I can't quite imagine how else this might be happening, unless your log quoted above is actually extracted from multiple sessions, writing to different folders.

RAM drive is not enabled by default, and actually not enabled in the current release. We'll be adding a feature to toggle this on/off later. So this isn't the cause.