URL Does not exist

[myob9a]

I am having a problem indexing a website that previously was fine. I recently switched the site to Wordpress but I cannot see how this would create the problem.

In starting the index I receive the following error message:
Check that the URL exists and satisfies the settings in the configuration window.

The site is at http:www/cla.asn.au with some 1600 posts. The log does indicate that there is an Invalid URL or domain name - which there is not!

Thoughts!

The log file reports:

02|09/23/13 12:10:10|Config file loaded: C:\Zoom Search Engine 6.0\cla1.zcfg
10|09/23/13 12:10:17|Start indexing (spider mode) at Mon Sep 23 12:10:17 2013
02|09/23/13 12:10:17|Maximum number of words: 500000
02|09/23/13 12:10:17|Maximum number of files: 20000
02|09/23/13 12:10:17|Will scan files with extensions
02|09/23/13 12:10:17| .htm
02|09/23/13 12:10:17| .html
02|09/23/13 12:10:17| .txt
02|09/23/13 12:10:17| .php
02|09/23/13 12:10:17| .asp
02|09/23/13 12:10:17| .cgi
02|09/23/13 12:10:17| .aspx
02|09/23/13 12:10:17| .pl
02|09/23/13 12:10:17| .php3
02|09/23/13 12:10:17| .pdf
02|09/23/13 12:10:17| .doc
02|09/23/13 12:10:17| .dot
02|09/23/13 12:10:17| .xls
02|09/23/13 12:10:17| .xlt
02|09/23/13 12:10:17| .ppt
02|09/23/13 12:10:17| .pot
02|09/23/13 12:10:17| .pps
02|09/23/13 12:10:17| .wpd
02|09/23/13 12:10:17| .djvu
02|09/23/13 12:10:17| .swf
02|09/23/13 12:10:17| .mp3
02|09/23/13 12:10:17| .dwf
02|09/23/13 12:10:17|Spider from: http://www.cla.asn.au/
02|09/23/13 12:10:17|Web site URL: http://www.cla.asn.au/
02|09/23/13 12:10:17|Estimated RAM required during index process: 389280 KB
02|09/23/13 12:10:17|Initiating HTTP session (thread #1) ...
14|09/23/13 12:10:17|DL Thread #1, got URL (http://www.cla.asn.au/) off queue
04|09/23/13 12:10:17|Downloading file http://www.cla.asn.au/
09|09/23/13 12:10:17|Could not download file: http://www.cla.asn.au/ (Invalid URL or domain name)
02|09/23/13 12:10:17|Initiating HTTP session (thread #5) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread # ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #7) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #4) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #9) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #3) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #2) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #6) ...
02|09/23/13 12:10:17|Initiating HTTP session (thread #10) ...
08|09/23/13 12:10:19|No files found to spider from http://www.cla.asn.au/
07|09/23/13 12:10:19|Indexing failed
14|09/23/13 12:10:19|Waiting for threads to finish ...
02|09/23/13 12:10:19|Cleaning up memory used for index data... please wait.
02|09/23/13 12:10:19|Finished cleaning up memory.
10|09/23/13 12:10:42|Start indexing (spider mode) at Mon Sep 23 12:10:42 2013
02|09/23/13 12:10:42|Maximum number of words: 500000
02|09/23/13 12:10:42|Maximum number of files: 20000
02|09/23/13 12:10:42|Will scan files with extensions
02|09/23/13 12:10:42| .htm
02|09/23/13 12:10:42| .html
02|09/23/13 12:10:42| .txt
02|09/23/13 12:10:42| .php
02|09/23/13 12:10:42| .asp
02|09/23/13 12:10:42| .cgi
02|09/23/13 12:10:42| .aspx
02|09/23/13 12:10:42| .pl
02|09/23/13 12:10:42| .php3
02|09/23/13 12:10:42| .pdf
02|09/23/13 12:10:42| .doc
02|09/23/13 12:10:42| .dot
02|09/23/13 12:10:42| .xls
02|09/23/13 12:10:42| .xlt
02|09/23/13 12:10:42| .ppt
02|09/23/13 12:10:42| .pot
02|09/23/13 12:10:42| .pps
02|09/23/13 12:10:42| .wpd
02|09/23/13 12:10:42| .djvu
02|09/23/13 12:10:42| .swf
02|09/23/13 12:10:42| .mp3
02|09/23/13 12:10:42| .dwf
02|09/23/13 12:10:42|Spider from: http://www.cla.asn.au/
02|09/23/13 12:10:42|Web site URL: http://www.cla.asn.au/
02|09/23/13 12:10:42|Estimated RAM required during index process: 389280 KB
02|09/23/13 12:10:43|Initiating HTTP session (thread #1) ...
14|09/23/13 12:10:43|DL Thread #1, got URL (http://www.cla.asn.au/) off queue
04|09/23/13 12:10:43|Downloading file http://www.cla.asn.au/
09|09/23/13 12:10:43|Could not download file: http://www.cla.asn.au/ (Invalid URL or domain name)
02|09/23/13 12:10:43|Initiating HTTP session (thread #2) ...
02|09/23/13 12:10:43|Initiating HTTP session (thread #6) ...
02|09/23/13 12:10:43|Initiating HTTP session (thread #3) ...
02|09/23/13 12:10:43|Initiating HTTP session (thread #4) ...
02|09/23/13 12:10:43|Initiating HTTP session (thread # ...
02|09/23/13 12:10:43|Initiating HTTP session (thread

[David]

The site seems to index OK from here.

So maybe,

• The site was down for a short period. e.g. bad wireless connection
• You internet connection was down for a short period.
• You have some security software or firewall blocking internet access for the Zoom application

As an experiment, can you try doing the indexing from a different PC.

[myob9a]

The site seems to index OK from here.

So maybe,

• The site was down for a short period. e.g. bad wireless connection
• You internet connection was down for a short period.
• You have some security software or firewall blocking internet access for the Zoom application

As an experiment, can you try doing the indexing from a different PC.

As I also index six other sites from this pc without issue I don't think it can be the firewall nor the PC, hence the call for help. The site has been up and functioning and the issue has been recurring over days and not just as a once off.

Thoughts.

[David]

Can you E-mail me your configuration file (C:\Zoom Search Engine 6.0\cla1.zcfg)

What exact version of Zoom are you using?

Also, as an experiment, can you try doing the indexing from a different PC.

[myob9a]

Thanks for the help. I rebuilt the configuration file from scratch and it is now working.

I initially had problems as it was giving some wildly differing index results (300+ to 900+). After changing down to 5 multiple threads and a 0.2sec delay I am receiving the expected 1900+ indexed files. I am using V6 (build 1029).

[David]

If the delay fixes the problem, then it sounds like your site might be overloaded and starts failing when put under a bit of load. e.g. returning HTTP 503 busy error codes, or just failing to serve pages. Even if you have solved the problem for Zoom, you might have the same problem from time to time with real visitors to the site not being able to access pages when the site is busy.