wordpress pages not indexing

[boxoffice]

Hi guys
As part of our site in a subdirectory we have a wordpress blog.
But zoom isn't indexing those files.
I have made sure there is nothing in the skip list, and no zoomstop / zoomrestart code.
In 'start options' > 'start spider from this url' I put in the wordpress folder.
But it doesn't index it, it says 'Empty XML sitemap created (no valid URLs found.) Check if your Sitemap Base URL is incorrect.'
If I set a post from wordpress as the base url, it runs the indexer, but finishes with the same message as above in the log.

Example URLs are
http://example.com.au/hubbub-blog/
http://example.com.au/hubbub-blog/film-review-arbitrage
http://example.com.au/hubbub-blog/whats-hot/celebrity-tv-film-news

[Ray]

We'll need to see the actual URLs and what your Wordpress blog is responding with for those URLs, than a hypothetical one.

Also, check the Skipped messages on your Log tab. Some reasons might have been given as to why the URL was not indexed. Perhaps it redirected to somewhere that's outside of the Base URL.

[boxoffice]

Hi Ray

Thanks for you suggestion of checking the skipped messages in my log tab. Zoom was skipping all the files in the wordpress folder, saying they weren't in the base url.

So I checked the wordpress pages on our blog and if I went to 'www.example.com.au/hubbub-blog' the browser would change the url to 'example.com.au/hubbub-blog' (without the www). The base url set in zoom was 'www.example.com.au'.

There are wordpress settings of 'WordPress Address (URL)' and 'Site Address (URL)' which were set to http://example.com.au/hubbub-blog (with no www)
I changed this to
http://www.example.com.au/hubbub-blog (including the www)
and then the indexer worked and it indexed my wordpress pages. Success!

PS
I found that changing the base url in zoom to 'example.com.au' (without www) did not help, in the log for each page from wordpress it said the same error 'External site - does not match base url'.

[Ray]

Good to hear you've worked it out.

Another way to solve that is to change the base URL to two base URLs (separated with a semi-colon ";" character). i.e.

http://example.com.au;http://www.example.com.au

This way, both domains will be considered part of the base URL. You can change the base URL by clicking on "More" and "Edit".

And on a technical note, as to why this isn't done automatically -- the standards of internet domains actually do mean that these could potentially point to completely different sites. It's just that most servers are configured to consider them the same. But there are sites where the "www." domain is not the same as the one without.