Cache Reload Problem

[Liza]

Hi

I am having problems with Zoom's reload cache facilty. I'd like the spider to reload the pages I've uploaded to the Net (i.e. not the cached versions). But when I check the box to enable reload, the spider doesn't index; it gives the error message that the URL does not satisfy the configuration settings. In the log, the message is that Zoom cannot find a file to spider. When I uncheck the same box, the spider works (though it is not indexing all files successfully, which is why I wanted to enable reload in the first place).

My Base and Start URLs are (example) http://www.mywebsite.com/

I've tried changing the Start URL to http://www.mywebsite.com/index.htm but it hasn't made any difference.

Thanks for your help in advance

Almost forgot - I'm indexing around 40 files, all htm or images (1.5 MB ish for testing purposes). My browser is IE, and my hosting server is a Linux.

[David]

Maybe the file you are trying to start the spider at no longer exists (except in your local cache)?

Or maybe you have lost your internet connection, so you can now only access cached files.

Or maybe there was a change in configuration the server, and / or the server is down.

If you still have a problem can you post the real URLs that you are using so that we can test them from here.

[Liza]

Thank for the suggestions. I don't think it's a net connection problem. Nor is the index file missing or mistyped - I checked via my FTP programme just in case. I've been having a think and the only inconsistent issue on my webpages is the encoding language. It was UTF-8 Unicode on the searchpage template by default (if I recall correctly) and the rest of the site is iso-8859-1 (Western Latin). The config was set to Unicode for the template's sake. I'll change the template page and config to match the rest of the site and see if that makes a difference. I'll post the result back here for the benefit of others who may run into the same issue.

[Liza]

Okay, I tried changing the encoding, and it made no difference. I've noticed now that the search form itself is not working properly either. Sometimes I will query the engine with the same word at two different times of the day, and on the first attempt it will find the pages, and sometimes it won't. I've also just learned that my host provider insists that all CGI and Perl scripts must be uploaded in ASCII mode and not binary. Their solution for getting round binary scripts is to amend the first line to read #!/usr/bin/perl , but I assume that would only work if Zoom was a Perl script anyway?

So I suspect that it is the binary vs ASCII that is the problem. Can you suggest any way round it (other than switching hosts )?

The urls for the site btw are
http://jinnaharchive.com
http://www.jinnaharchive.com/searcher/search.cgi

[David]

I think you have multiple problems, or maybe you solved your initial problem and moved on to the 2nd problem?

If you are now trying to do some searches, then I assume you must have solved your earlier problem of not being able to index your site?

Regarding the 1st problem: Indexing
The character set selected and the template file have nothing to do with the ability of the indexer to find your home page. If your home page can not be found you need to be looking at server problems, or Internet connection problems (e.g. firewalls).

I indexed your site from here without caching and it worked fine. I used the start URL, http://jinnaharchive.com/


Regarding the 2nd problem: Searching
From what I can see you only have 13 accessible pages on your entire site. So this barely justifies the use of a search engine (or maybe the site will be much larger in the future?).

You have a bunch of broken links on your site. Like this one, which you should fix
http://jinnaharchive.com/news.htm

Links in your Javascript menus being skipped, so you also need to fix that.

The search script does not change it's behaviour depending on the time of the day. If the behaviour changes from one search to the next, then there is some other factor at play. Like maybe the server is overloaded (just an example, I am not saying this is the cause).

..my host provider insists that all CGI and Perl scripts must be uploaded in ASCII mode and not binary.

Your hosting company don't know what they are talking about. Maybe suggest to them they actually do some reading about CGI's before making such misinformed statements.

Their solution for getting round binary scripts is to amend the first line to read #!/usr/bin/perl

This comment from your hosting company is hopelessly confused and misinformed. Ignore them and move to a hosting company that have qualified support people.

When using FTP to upload files, YOU have control over the binary and ASCII setting. So just upload the CGI and indexing files in binary.

But if you have only 13 pages on your site, it might be easier for you (and your hosting company) to use the PHP option instead of the GGI option. PHP scripts are easier to install than a CGI.

[Liza]

Wow - didn't realise it was as bad as that!

Thank you so much for the detailed help. Nope, I've never sucessfully indexed when reload is enabled. I'll try using jinnaharchive.com minus the www. I can't believe I never thought of it.

To answer your question about the site's size, yes, it will become very large in the future, running into thousands of pages, which is why I opted to run Zoom in cgi mode rather than php. I was aware of the broken links; the site is still in test mode.

And I'll fix the Javascript issue too - thanks very much for that - I had no idea about it.

I think I'll take your advice and look into a new host if they can't help me from here.

Many thanks again. I'll be back to let you know once everything's sorted (whenever that is!)

[David]

...didn't realise it was as bad as that

I didn't intend to make it sound that bad . I think it is all fairly easy to fix.

You need to stick with either using www. or not using www. for all of your links. Mixing them up makes it appear like you have two subdomains.

[Liza]

Hi again,

Well, you were absolutely right - it was easy to fix. I'd tried removing the www and still couldn't get it to work. And since you said it had worked fine at your end, I wondered whether it was actually my installation of Zoom that was the problem rather than the server or anything else. So I uninstalled Zoom, removed the Javascript menus (which to be frank I'd never liked too much anyway) and reinstalled Zoom. And it worked.

The search engine is absolutely fine now, no more issues with cache. It was just my installation of Zoom that was corrupted (not sure when it must have happened). But thank you again for all your help. Your software and tech support are second to none.

All the Best!