Can't start index - no files found

[Fork]

I have been indexing the same site for years. I have not changed the settings for Zoom for months, perhaps a year. The computer, OS, etc. are also the same.

The URL is in the format http://mysite.com/index.html

Today I suddenly received an error message window titled "no files found to spider" that says "Check that the URL exists and satisfies the settings in the configuration window."

I can access the URL via Firefox and IE and navigate the site from it -- the pages aren't cached in the browsers. I can also use Filezilla to FTP into the site.

The only change to my system is a new DSL modem/router combo - all computers on my local network, both wired and wireless, can access the Internet without any problems. If browsers and FTP can reach the site, what might be preventing Zoom from connecting to it?

[David]

Zoom uses TCP/IP port 80. The same as a normal browser. So external firewalls (if there was one in the router) should not see any difference between Zoom and a browser.

I would think that the real problem is elsewhere. e.g. something did change on the site around the same time. What is the real URL for the site?

[Fork]

http://www.globalgourmet.com/index.html

I agree that the behaviour doesn't make any logical sense. I can access the site in various programs from other computers on the network and the PC with Zoom on it - just can't do it from within Zoom.

To repeat, it's been working fine in Zoom until today and I haven't made any changes to the settings. I simply launched Zoom and clicked the "Start Indexing" button.

[David]

I just indexed 20 odd pages from your site from here, without any problem. So it probalby isn't the site at fault (unless the site is doing something fancy, like blocking based on IP and user agent).

So it is more likely the problem is on your PC or network. Can you post the log from Zoom showing the failure, it might give a hint.

I assume you tried this several times in case it was just something mundane like your wireless network being down for a minute or so.

[Fork]

Log

06:44:35 - Zoom Search Engine Indexer (Professional Edition)
06:44:35 - Version 6.0 (Build: 1014) on Windows Vista
06:44:35 - Copyright Wrensoft 2000-2008 (http://www.wrensoft.com/)
06:44:35 - Config file loaded: C:\Users\Administrator\Documents\zoom.zcfg
06:44:44 - Start indexing (spider mode) at Tue Oct 06 06:44:44 2009
06:44:44 - Maximum number of words: 200000
06:44:44 - Maximum number of files: 30000
06:44:44 - Will scan files with extensions
06:44:44 - .html
06:44:44 - Spider from: http://www.globalgourmet.com/index.html
06:44:44 - Web site URL: http://www.globalgourmet.com/
06:44:44 - Estimated RAM required during index process: 284776 KB
06:44:44 - Initiating HTTP session (thread #1) ...
06:44:44 - DL Thread #1, got URL (http://www.globalgourmet.com/index.html) off queue
06:44:44 - [DOWNLOAD] Downloading file http://www.globalgourmet.com/index.html
06:44:44 - [WARNING] Could not download file: http://www.globalgourmet.com/index.html (Invalid URL or domain name)
06:44:44 - Initiating HTTP session (thread #2) ...
07:38:19 - [ERROR] No files found to spider from http://www.globalgourmet.com/index.html
07:38:19 - Indexing failed
07:38:19 - Waiting for threads to finish ...
07:38:19 - Cleaning up memory used for index data... please wait.
07:38:19 - Finished cleaning up memory.

I can use manual FTP (like Filezilla) to reach the same url and can access it easily from Firefox and IE. It does not appear to be a normal networking problem. ZOOM was working fine until a few days ago, before I first posted about the problem. I cannot think of any changes made to the computer recently and I know the ZOOM settings have not been changed in months. I am the only person with access to this PC and it is only turned on a few hours a day for accounting and weekly indexing. It has been scanned for viruses etc and come up clean.

[David]

It does look like a firewall issue. Do you have any internet security or similar applications running on your PC? Can you try turning them off as an experiment.

[Fork]

We have turned off UAC and Windows Firewall and Defender. This particular computer is rarely used for Internet access or email - we mostly use Macs.

There are three uses for the computer:
1 - accounting (Quickbooks Pro does access the Internet in order to run)
2 - Zoom indexing
3 - occasional testing of our website in IE to make sure it looks okay in Windows

We use Avast to monitor the machine for viruses, etc.

If it were a firewall issue, why would browsers and FTP programs work and Zoom not?

The only thing I can think of that changed on this computer recently are the normal updates that Microsoft installs automatically every few days.

Unless you have other suggestions my plan is to install ZOOM on one of our Intel Macs that have Bootcamp installed, which will allow the Mac to operate as a PC on startup.

Do I need to de-license ZOOM on the PC first and then re-license it on the other machine?

[David]

If it were a firewall issue, why would browsers and FTP programs work and Zoom not?

Firewalls can block specific programs.
(In addition to doing blanket blocking on ports, or IP addresses)

You might also want to turn off Avast and give it a try.

You can move the license between machines with an uninstall and re-install.

[Ray]

Firewalls can also block all programs except for a specified list. It is likely that by default it is configured to allow Firefox, IE, and other popular browsers to work. But it will block everything else it can't identify, such as Zoom. So just because you haven't asked it to block Zoom specifically, doesn't mean it's not doing it.

There is a similar configuration issue with most anti-virus and internet security programs. Which is why it's worth disabling them temporarily to see if it makes a difference.

[Fork]

Logic, Who Needs Loigic?

At your suggestion, I turned off Avast (though Avast does not includes a firewall - it only scans incoming and outgoing files). No change - I still couldn't get Zoom to index.

Though Windows Firewall had been turned off through all of this I rechecked its preferences. There were no settings (that I found) relating to specific ports or URLs that had been blocked. In any case, I turned off everything I could find that related to "firewall". Still no change.

Remember, everything else that could connect to the Internet worked through all this and continued to work at this point.

I noticed that my Windows Update control panel had five "suggested" updates that I know had been installed last year. Four of them related to the networking card. I assume the automatic Windows Updates may have "uninstalled" them for some reason, perhaps to install something else. I decided to "re-install" these updates.

Now Zoom works normally.

I still don't understand why a downloaded FTP program could access my site through all this but ZOOM couldn't.

Guess that's why I've preferred using Macs for the past twenty years. I use both, but the Windows OS problems are often the most difficult to solve using everyday logic.

[Ray]

Glad to hear you've got to the bottom of it and it's all working fine now.

But just to clarify and for your information:

At your suggestion, I turned off Avast (though Avast does not includes a firewall - it only scans incoming and outgoing files). No change - I still couldn't get Zoom to index.

According to this, Avast has a "Web Shield" module which "is able to monitor and filter all HTTP traffic coming from the Web sites on the Internet. It’s implemented as a HTTP proxy running on your PC.
Connections from your Web browser are redirected to the Web Shield module. Web Shield in turn connects to the requested web server and while downloading the content it scans it for viruses and Trojans. Only the clean data is delivered to the browser, every malware is stopped before it gets saved on your PC." This could certainly affect a program such as Zoom which needs to access HTTP traffic in Spider Mode.

I can't imagine how Windows Update would have automatically uninstalled an update (it can be manually uninstalled from the Control Panel however), unless the computer was rolled back at some point through a System Restore.

[Fork]

I had turned off the Avast Web Shield early on in the troubleshooting process. But Zoom had worked with Avast for months with Web Shield turned on so I didn't really consider that was the problem. With Web Shield off or on, Zoom didn't work beginning last week. FTP and browsers did work with Web Shield on.

Only after I installed the previously installed network card updates did Zoom finally work.

Curiously, my Windows Update log shows these network card updates were installed back in 2008. Windows offered to re-install them this past week as a Recommended Update, after Microsoft installed some other updates. So somehow they got uninstalled and since I'm the only user of this particular computer and do not use it very often, or for Internet tasks other than Zoom indexing, and I never manually uninstalled those network drivers, logic points at some sort of Microsoft manipulation. I can't prove it but the other alternatives are not very compelling (at least to me, a 20+ years computer user).

I do want to thank you for trying to help me solve the problem. Zoom is an excellent product.

[Ray]

It is certainly possible that Microsoft bunged up an update. It would not be the first time. The "logic" that is dominant in the Microsoft world is that new versions are not necessarily better, just more likely to have introduced new bugs. So one is always wary with new Microsoft releases.