programatically invoke indexing

[kunalkamboj]

Hi,
I am using the zoom search enginge on my we site.
In here i am creating some static pages in the website dynamically.
I mean that some pages are created according to a requirement.
I am using ASp .Net 2.0 as the platform.
My question is there any what to invoke the indexing process programatically so that the new page that is created is automatically indexed and the required index file are uploaded?.

Please Help.
Thanks and Regards,
Kunal Kamboj.

[David]

See this FAQ.
Q. How do I use the command line options with the Zoom Indexer?

[kunalkamboj]

Problems with programatic implementation

Hi,
I am facing the folowing Problems with programatic implementation.
Say i added an html file to my website named encyclopedia.htm . Now i want to index this file in my web site.
I added the following code for invoking the process of indexing.

System.Diagnostics.Process Process = new System.Diagnostics.Process();
Process.StartInfo.FileName = Server.MapPath("~/ZoomExe/ZoomIndexer.exe");
Process.StartInfo.Arguments = "-o zoom.zcfg -addpage D:\\Inetpub\\wwwroot\\ZoomSearchWebsite\\encyclope dia.htm;http://localhost:4921/ZoomSearchWebsite/";
Process.Start();
Process.WaitForExit();

I am saving the exe and the config file in a folder placed at the root named ZoomExe .
Now the i can see the indexing process going fine the window opens and closes automatically.
But Get the followin error at my search page.

"Content-type: text/html Unable to load settings.zdat file.
The file appears to be invalid and may have been corrupted. "

Currently i am using the free version is it because it that the indexing is not being done properly.

Please help.

Thanks and Regards,
Kunal Kamboj.

[Ray]

The Free Edition does not limit functionality this way. Please see this page for information on how the Free Edition differs from the other editions.

If however, you have already reached the Max Pages scanned limit of 50 pages (or the max words limit) for the Free Edition, then you should receive an error message during indexing (a red line in the index log).

Since you are doing this programmatically, you may be missing the message, and you should have enabled logging to file (see "Index Log" tab of Configuration window), so that all error messages are written to disk. This way you can confirm if the indexing session ran successfully before proceeding to the search page (this is mentioned in the FAQ above).

Please check your index log, and let us know if there is a more informative error message there which may indicate what the problem is.

[kunalkamboj]

Hi,
I created a log file for the following commad in the cmd window.

D:\NewFolder>zoomindexer.exe -o zoom.zcfg -addpage D:\Inetpub\wwwroot\zoomsearchwebsite\Invitations.h tm;http://localhost:4921/ZoomSearchWebsite/

But looking at the index file kind of surprised me ,, i passed the parameter "-addpage" it generated the following log file amd was mentioned there deleteing file.


11/30/07 12:28:36 - Config file loaded: D:\zoom\zoom.zcfg
11/30/07 12:28:38 - Start indexing (offline mode)
11/30/07 12:28:38 - Maximum number of words: 15000
11/30/07 12:28:38 - Maximum number of files: 50
11/30/07 12:28:38 - Will scan files with extensions
11/30/07 12:28:38 - .htm
11/30/07 12:28:38 - .html
11/30/07 12:28:38 - .txt
11/30/07 12:28:38 - .php
11/30/07 12:28:38 - .asp
11/30/07 12:28:38 - .cgi
11/30/07 12:28:38 - .aspx
11/30/07 12:28:38 - .pl
11/30/07 12:28:38 - .php3
11/30/07 12:28:38 - Search root directory: D:\Inetpub\wwwroot\ZoomSearchWebsite
11/30/07 12:28:38 - Web site URL: http://localhost:4921/ZoomSearchWebsite/
11/30/07 12:28:38 - Estimated RAM required during index process: 4396 KB
11/30/07 12:28:38 - Reading old settings from existing index data ...
11/30/07 12:28:38 - Successfully loaded and verified existing index files and settings.
11/30/07 12:28:38 - Deleting pages from existing wordmap ...
11/30/07 12:28:38 - 1 page(s) successfully deleted from existing wordmap
11/30/07 12:28:38 - 1 page(s) successfully deleted from existing pageinfo
11/30/07 12:28:38 - Loading in existing dictionary file ...
11/30/07 12:28:38 - Preparing existing wordmap file for incremental update ...
11/30/07 12:28:38 - Error: Could not open file: http://localhost:4921/ZoomSearchWebsite/Invitations.htm
11/30/07 12:28:38 - Failed to complete incremental index
11/30/07 12:28:38 - Indexing failed
11/30/07 12:28:38 - Cleaning up memory used for index data... please wait.
11/30/07 12:28:38 - Finished cleaning up memory.

also i used the same parameters in the code. but there i was not able to generate any log file with the same exe and config file.It runs the application with 2 red lines and nothing is added in the log file.

System.Diagnostics.Process pProcess = new System.Diagnostics.Process();
pProcess.StartInfo.FileName = Server.MapPath("ZoomIndexer.exe");
pProcess.StartInfo.Arguments = "-o zoom.zcfg -addpage D:\\Inetpub\\wwwroot\\ZoomSearchWebsite\\Invitatio ns.htm; http://localhost:4921/ZoomSearchWebsite/";
pProcess.Start();
pProcess.WaitForExit();

Please help and sorry for bothring you on mail as well as fourm. we will talk here now..
Thanks..
Kunal Kamboj

[kunalkamboj]

Hi,
I created a log file for the following commad in the cmd window.

D:\NewFolder>zoomindexer.exe -o zoom.zcfg -addpage D:\Inetpub\wwwroot\zoomsearchwebsite\Invitations.h tm;http://localhost:4921/ZoomSearchWebsite/

But looking at the index file kind of surprised me ,, i passed the parameter "-addpage" it generated the following log file amd was mentioned there deleteing file.


11/30/07 12:28:36 - Config file loaded: D:\zoom\zoom.zcfg
11/30/07 12:28:38 - Start indexing (offline mode)
11/30/07 12:28:38 - Maximum number of words: 15000
11/30/07 12:28:38 - Maximum number of files: 50
11/30/07 12:28:38 - Will scan files with extensions
11/30/07 12:28:38 - .htm
11/30/07 12:28:38 - .html
11/30/07 12:28:38 - .txt
11/30/07 12:28:38 - .php
11/30/07 12:28:38 - .asp
11/30/07 12:28:38 - .cgi
11/30/07 12:28:38 - .aspx
11/30/07 12:28:38 - .pl
11/30/07 12:28:38 - .php3
11/30/07 12:28:38 - Search root directory: D:\Inetpub\wwwroot\ZoomSearchWebsite
11/30/07 12:28:38 - Web site URL: http://localhost:4921/ZoomSearchWebsite/
11/30/07 12:28:38 - Estimated RAM required during index process: 4396 KB
11/30/07 12:28:38 - Reading old settings from existing index data ...
11/30/07 12:28:38 - Successfully loaded and verified existing index files and settings.
11/30/07 12:28:38 - Deleting pages from existing wordmap ...
11/30/07 12:28:38 - 1 page(s) successfully deleted from existing wordmap
11/30/07 12:28:38 - 1 page(s) successfully deleted from existing pageinfo
11/30/07 12:28:38 - Loading in existing dictionary file ...
11/30/07 12:28:38 - Preparing existing wordmap file for incremental update ...
11/30/07 12:28:38 - Error: Could not open file: http://localhost:4921/ZoomSearchWebsite/Invitations.htm
11/30/07 12:28:38 - Failed to complete incremental index
11/30/07 12:28:38 - Indexing failed
11/30/07 12:28:38 - Cleaning up memory used for index data... please wait.
11/30/07 12:28:38 - Finished cleaning up memory.

also i used the same parameters in the code. but there i was not able to generate any log file with the same exe and config file.It runs the application with 2 red lines and nothing is added in the log file.

System.Diagnostics.Process pProcess = new System.Diagnostics.Process();
pProcess.StartInfo.FileName = Server.MapPath("ZoomIndexer.exe");
pProcess.StartInfo.Arguments = "-o zoom.zcfg -addpage D:\\Inetpub\\wwwroot\\ZoomSearchWebsite\\Invitatio ns.htm; http://localhost:4921/ZoomSearchWebsite/";
pProcess.Start();
pProcess.WaitForExit();

Please help and sorry for bothring you on mail as well as fourm. we will talk here now..
Thanks..
Kunal Kamboj





Hi,
Well creating the log file kind of helped a lot.Thanks for the response.

I have reached to some level of sucess programatically.
the followong is the Log file generated .
Please have a look.
As you can see there is one error.."Could not open language file: English.zlang"



11/30/07 15:24:25 - Config file loaded: D:\Inetpub\wwwroot\ZoomSearchWebsite\zoom.zcfg
11/30/07 15:24:27 - Start indexing (offline mode)
11/30/07 15:24:27 - Maximum number of words: 15000
11/30/07 15:24:27 - Maximum number of files: 50
11/30/07 15:24:27 - Will scan files with extensions
11/30/07 15:24:27 - .htm
11/30/07 15:24:27 - .html
11/30/07 15:24:27 - Search root directory: D:\Inetpub\wwwroot\ZoomSearchWebsite
11/30/07 15:24:27 - Web site URL: http://localhost:4921/ZoomSearchWebsite/
11/30/07 15:24:27 - Estimated RAM required during index process: 4396 KB
11/30/07 15:24:27 - Reading old settings from existing index data ...
11/30/07 15:24:27 - Successfully loaded and verified existing index files and settings.
11/30/07 15:24:27 - Loading in existing dictionary file ...
11/30/07 15:24:27 - Preparing existing wordmap file for incremental update ...
11/30/07 15:24:27 - Indexing D:\Inetpub\wwwroot\ZoomSearchWebsite\Manual.htm
11/30/07 15:24:27 - Writing index data for CGI/Win32 search... (Please wait)
11/30/07 15:24:27 - Created pagedata data file (zoom_pagedata.zdat)
11/30/07 15:24:27 - Created pageinfo data file (zoom_pageinfo.zdat)
11/30/07 15:24:27 - Flushing index data to disk ...
11/30/07 15:24:27 - Merging all flushed wordmap data files ...
11/30/07 15:24:27 - Created wordmap data file (zoom_wordmap.zdat)
11/30/07 15:24:27 - Created dictionary data file (zoom_dictionary.zdat)
11/30/07 15:24:27 - Could not open language file: English.zlang
11/30/07 15:24:27 - Created script settings file (settings.zdat)
11/30/07 15:24:27 - Indexing completed
11/30/07 15:24:27 - INDEX SUMMARY
11/30/07 15:24:27 - Files indexed: 14
11/30/07 15:24:27 - Files skipped: 0
11/30/07 15:24:27 - Files filtered: 0
11/30/07 15:24:27 - Files downloaded: 0
11/30/07 15:24:27 - Unique words found: 194
11/30/07 15:24:27 - Total words found: 194
11/30/07 15:24:27 - Avg. unique words per page: 13.86
11/30/07 15:24:27 - Avg. words per page: 13
11/30/07 15:24:27 - Start index time: 15:24:27 (2007/11/30)
11/30/07 15:24:27 - Elapsed index time: 00:00:00
11/30/07 15:24:27 - Errors: 0
11/30/07 15:24:27 - Total bytes scanned/downloaded: 39327
11/30/07 15:24:27 - File extensions:
11/30/07 15:24:27 - .htm indexed: 1
11/30/07 15:24:27 - .html indexed: 0
11/30/07 15:24:27 - No extensions indexed: 0
11/30/07 15:24:27 - Cleaning up memory used for index data... please wait.
11/30/07 15:24:27 - Finished cleaning up memory.
11/30/07 15:24:27 - Successfully created all required files

Now beacuse of this i am getting the error "Content-type: text/html Unable to load settings.zdat file.
The file appears to be invalid and may have been corrupted. "

What possibly could be the reason for this.
Please help

[Ray]

Based on your earlier post, it seems like you have moved the "ZoomIndexer.exe" executable to a different folder than your original installation folder, and you are running the application from there (your second post referred to a path "~/ZoomExe/ZoomIndexer.exe").

If this is the case, then it is most likely that you have not copied the necessary subfolders across, namely the "lang" folder containing all the ZLANG language files (including "English.zlang"), as well as "scripts", "plugins", "Help", etc. IF you are attempting to copy the ZoomIndexer to a different folder, you need to move all the files and subfolders with you. We don't recommend this generally, and would advise running Zoom from the installation folder (or installing Zoom in the folder you wanted to begin with).

[kunalkamboj]

HI,
thanks for the reply it worked thanks a lot.
Well i have one more query can i Rewrite Links for the search output while indexing the site prgramatically?

I get the folowing error in the log file while doing do.

"You have enabled the Rewrite Links option in your indexing
configuration.

Index files which have been created with the Rewrite Links option
enabled can not be used with incremental indexing."

[Ray]

The Rewrite Link option allows you to change the URL of the files as they are indexed, to something else when the search results are returned. For example, indexing from http://dev.mysite.com/ but returning search results to http://www.mysite.com/

This involves changing the URLs in the index files permenantly, thus losing the ability to perform incremental indexing because we no longer know the exact URL that was originally indexed.

One alternative to using the Rewrite Links feature is to post-process the search results. Since you are already using an ASP.NET wrapper, you could modify the output HTML before displaying it to the user.

[kunalkamboj]

Hi thanks got the point.
now what i am trying is that on a button click in my website i am trying to start the index process for a particular file.
i mean i kept the required file in a paricular folder and then programatically invoke the indexing process.
It works fine when i am debugging the code but when i publish the same code i dont know why but nothing is done....
no index file is generated i am not sure why
please help..

[Ray]

You should look at the index log, as you have before, to get more of an idea as to what happened when you tried to start the indexer from the command-line, using your script.

If the index log was not updated at all, then it most likely means your script is not executing the Indexer correctly. Perhaps you have the executable hosted in a different folder than your debugging setup.

If you are implying that you were testing on your own local desktop/test server when you were "debugging", and that now you are running the same script on a remote server when you "publish", then you will need to look at whether you have the Indexer installed on your server (in the location you're expecting in the script).

[kunalkamboj]

Hi ,
I have the Indexer installed at the remote server,also there is no index log generated and i am setting the Start directory , output directory as the local physical path on the server also i have placed the same directory structure at the server and also kept the same folder containing the ZoomIndexer.exe and the config file , the script and the lang directories are also there...But still am not sure what else could be the isuue.
Please Help...
Please let me know a your contact number so that i could call you actually it is need to be delivered soon...
Please see the code attached:

System.Diagnostics.Process pProcess = new System.Diagnostics.Process();
pProcess.StartInfo.FileName = Server.MapPath("~\\ZoomSearch\\ZoomIndexer");
pProcess.StartInfo.Arguments = "-o zoom.zcfg -addpage " + Server.MapPath("~\\ContentPages\\" + PageName) + ";" + Request.Url.ToString().Substring(0, Request.Url.ToString().IndexOf("Admin")) + "ContentPages";
pProcess.Start();
pProcess.WaitForExit();

I would be mailing you the Config file as well.

Thanks.
Kunal Kamboj

[Ray]

As far as I know, the tilda character ("~") is not supported by the Server.MapPath() function. It also does not do anything in Windows paths (it is just an ordinary character, so you can actually create a directory named "~"). Are you mistaking this for Unix paths, where it is a reserved character to refer to your home directory?

I do not know how this could possibly work on your local machine as you claimed, unless it is a Linux machine running an ASP interpreter, which is a pretty rare scenario, and you should really mention if this is the case. I can imagine alot of different behaviour between an ASP interpreter on Linux and ASP on Windows. And this wouldn't make sense either, since you wouldn't be able to run the ZoomIndexer executable on a Linux server.

Speaking of which, there is also a typo in your path which refers to "ZoomIndexer" as opposed to "ZoomIndexer.exe".

Your config file indicates that logs are supposed to be saved to "D:\indexlog.txt". Does this path exist and does it have appropriate write permissions on your remote server? It would seem unlikely that your server is configured to have write permissions in the root directory.

You can also narrow down the problem by trying to make your script start any other executable as a test, in the same folder. For example, you could make a copy of "Notepad.exe" and place it in the same "ZoomSearch" folder on your remote server, that you are trying to run from. Change your script to execute "notepad.exe", and if it still fails, then the problem should be clear that it is an issue with your paths.

You can find our phone number on our Contact Us page. Please note the time difference as we are in Sydney, Australia.