homerlex Posted March 30, 2005 Share Posted March 30, 2005 Is there any way to configure the site so that web crawlers only index the product_info pages? Obviously we want the crawler to visit all the pages but I just want it to index the product page. I thought perhaps there would be a way to do this via robots.txt but I do not see how. I'm a bit of a search engine newbie. :blink: Link to comment Share on other sites More sharing options...
Guest Posted March 30, 2005 Share Posted March 30, 2005 The first question is WHY ON EARTH WOULD YOU WANT TO EXCLUDE YOUR CATEGORY PAGES? If you have the category descriptions contribution installed it could be a valuable page of related resources. Search engines group items of related theme and the more relevant the better placement you'll get. EDIT: wrong examples. My apologies... Bobby Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.