Problem indexing PHPbb with SID session IDs

[GregRaiz]

I seem to be hitting a problem where when I index a my site's PHPbb message board many of the topics show up multiple times.

For example:
http://www.raizlabs.com/search/searc...=0&zoom_sort=0

Many of the results seem to link to the same page. Unfortunatly the engine doesn't see these as being the same.

I tried to filter out the &SID= but then I wouldn't get any results.
Any idea how to work around this issue?

[Ray]

Here's a previous thread which discussed a similar problem:
http://wrensoft.com/forum/viewtopic.php?t=347

As suggested, this is because your phpBB is setup to append session ID numbers (&sid=...) to each of the links. This causes the pages to appear to be different to a spider, everytime the session ID changes.

To avoid this, make sure phpBB does not assign a session ID for the spider. In most cases, phpBB will not use session ID if the client is logged in and has the correct cookie. As mentioned in the thread above, you can allow Zoom to share your IE cookies, meaning that you can login to your forums with IE, and when Zoom's spider accesses those pages, they will also be logged in. Make sure however, that you skip the link to "log out", or the spider will log itself out.

Alternatively, disable the use of session IDs in phpBB. Note that SID's are also commonly known to hamper Internet-wide search engine rankings (on google, msn, etc.)

Hope that helps.

[GregRaiz]

Problem fixed

Adding the IE cookies did the trick.
I think that phpbb adds the sessionID only if it first tries to add a cookie and fails. I'm not sure if the software uses it's own cookies or not, as a guest I don't see sessionID's as I browse.

Anyway. Thanks for the help.