I have been able to use the skip function to bypass all the duplicate content and other stuff easily, but I have run into one snag. Part of our site is a wordpress blog http://www.crimethinc.com/blog/ and it makes pages for each month, like
http://www.crimethinc.com/blog/2008/04/
but i want to skip those when indexing since the individual posts are much more useful in search results. However, because a normal post has a url that is exactly the same, but with stuff on the end of it, such as
http://www.crimethinc.com/blog/2008/04/09/from-the-archives-seattle-logistics-zine/
if it put the monthly page on the skip list, it also skips the individual posts from that month, which is bad. Right now I am just manually removing all the monthly pages from the index post-indexing, but I was hoping there could be some automatic way. Thanks!
http://www.crimethinc.com/blog/2008/04/
but i want to skip those when indexing since the individual posts are much more useful in search results. However, because a normal post has a url that is exactly the same, but with stuff on the end of it, such as
http://www.crimethinc.com/blog/2008/04/09/from-the-archives-seattle-logistics-zine/
if it put the monthly page on the skip list, it also skips the individual posts from that month, which is bad. Right now I am just manually removing all the monthly pages from the index post-indexing, but I was hoping there could be some automatic way. Thanks!
Comment