Repeating sub folder problem

[sherbert99]

Hi when I scan my site using "Spider Mode" I have a problem.

It scans folders that don't exist. It seems like it duplicates a folder inside its self. For example I have a folder like this.

http://mydomain.com/articles/enviroment/default.asp

It will scan that page but then also starts scanning folders like this.

http://mydomain.com/articles/enviroment/enviroment/enviroment/enviroment/
enviroment/enviroment/enviroment/enviroment/enviromen

Any ideas why this is happing? When I scan it in offline mode it works fine but I'd rather scan using spider mode.

I'm using V6 Build 1008

Thanks

[David]

We have seen this once before, many years ago.

I think it can happen if you site is a dymanic site (i.e. not just static pages in folders) and one of you dymanic page is generating a semi-broken URL.

So there will be on page on your site that incorrectly adds an additional folder name to the URL, thus generating a new URL. When that new URL is indexed, that page that incorrectly adds an another additional folder name, etc.. in a loop.

See also this old post for details of a similar problem and some tips for how to debug it.
http://www.wrensoft.com/forum/showthread.php?p=7276

[sherbert99]

Thanks for the info.

I don't have any dynamic pages, but it does use includes and site paths starting with "/". Such as "/articles/enviroment/default.asp". I have a feeling there might be a bad link like with those which is causing the problem.

I'll start stripping stuff out until I figure what page is causing this.

Thanks

[David]

If you can't find it after a while, post the real URL for your site and we'll have a look.

[Ray]

I don't have any dynamic pages, but it does use includes and site paths starting with "/". Such as "/articles/enviroment/default.asp".

FYI, an ASP page is a dynamic page.

[sherbert99]

I thought he ment dynamic as the page's content being generated from a database using a query such as

"http://mydomain.com/links.asp?page=1"

but you are right I guess it's all the same. I'll start by looking at my asp and CFM pages.

If I find or don't find it I'll let you know.

Thanks

[sherbert99]

Thanks guys, I finally fixed the problem. It took me a while but there were tons of broken links throughout the site on my asp pages. Once I fixed the links it scans now with no problem.

Thanks