christianbright Posted May 20, 2010 Posted May 20, 2010 The way my site is configured, I do not need product pages to be indexed at all. I only want category pages and the main index to get indexed by Google. What is the most effective way to do this? I know one method could be by adding something to robot.txt, correct? Though my shop has an SEO module installed and my URLs look like this site.com/category/product Google is indexing them as site.com/product. Please help, my site is dropping pages each day...
Jillvpj197 Posted May 20, 2010 Posted May 20, 2010 <br />The way my site is configured, I do not need product pages to be indexed at all. I only want category pages and the main index to get indexed by Google. What is the most effective way to do this? I know one method could be by adding something to robot.txt, correct? Though my shop has an SEO module installed and my URLs look like this site.com/category/product Google is indexing them as site.com/product. Please help, my site is dropping pages each day...<br /><br /><br /><br /> The most effective way is is to set the meta flag noindex on. I wouldn't block them in robots.txt,because then they won't come out. You see, if the pages are already indexed, and you want them out, by blocking them in robots.txt they will not get the new meta tag. So, leave robots.txt as is, and set the noindex tag for each page you want out. Once you've done that, if you are desperate to get them out of google, you can then use the google URL removal tool, which works within 12-24 hours. But first, I would strongly suggest you make sure you REALLY do want them out of google. That makes no sense to me, unless the pages have no content or are duplicate content... The more pages indexed in google, the greater the chance one of your pages will get google traffic...
christianbright Posted May 20, 2010 Author Posted May 20, 2010 Well they are showing up as duplicate content in the webmaster tools. You can't manually add a noindex meta tag to product pages, because they aren't actual pages. It's all stored in the database, so that's what I'm wondering is how to add the noindex tag to product_info.php for example...Which seems to be where it would have to be.
Hotclutch Posted May 20, 2010 Posted May 20, 2010 Well they are showing up as duplicate content in the webmaster tools. You can't manually add a noindex meta tag to product pages, because they aren't actual pages. It's all stored in the database, so that's what I'm wondering is how to add the noindex tag to product_info.php for example...Which seems to be where it would have to be. If you really want to remove all your product pages from Google index, then that is quite easy to achieve. Just add <meta name="robots" content="noindex"> below <head> in the product_info.php If you are using a meta tags addon such as Header Tags SEO, then remove the code and restore the file to original.
christianbright Posted May 20, 2010 Author Posted May 20, 2010 And by default it should still follow links correct? Any possible ramifications? I think it is the wise choice for me because I only want the main page and top categories/subcategories to be indexed. I don't want the products to be indexed at all. I essentially have the same products under every subcategory, and I had already disabled the link to view the product page. So they are simply listed.
Hotclutch Posted May 20, 2010 Posted May 20, 2010 And by default it should still follow links correct? Any possible ramifications? .... Yes, default behaviour of robots is to follow, unless nofollow is specified.
christianbright Posted May 20, 2010 Author Posted May 20, 2010 One last question. Disallow: /*sort=* Disallow: /*manufacturers_id=*products_id=* Having this in robots.txt should help clear up duplicate content as well yes? I know that there is duplicate content from sort, language, currency, and even action for notifications. If one wanted to disable product notifications completely what lines should I look for?
Hotclutch Posted May 20, 2010 Posted May 20, 2010 One last question. Disallow: /*sort=* Disallow: /*manufacturers_id=*products_id=* Having this in robots.txt should help clear up duplicate content as well yes? I know that there is duplicate content from sort, language, currency, and even action for notifications. If one wanted to disable product notifications completely what lines should I look for? It is not recommended to block URLs like that to remedy duplicate content issues. Rather use <link rel="canonical"> in the the head sections of the relevant pages.
christianbright Posted May 21, 2010 Author Posted May 21, 2010 Hmm, while creating an xml sitemap I noticed that my category pages are being indexed like so: http://site.net/catalog/subcategory?sort=1 and catalog/product?action=notify Is there a way to prevent the sort from showing in the URL, also I'm a bit confused now. Is product_info.php controlling the subcategory too? The SEO module I used rewrites URLs so that the category and product display in the store root. For example: http://site.net/catalog/category and if the visitor browses to a subcategory it displays catalog/subcategory. I just want to be certain that by putting noindex in product_info.php that it only stops the products and not categories or subcategories. Thanks!
Hotclutch Posted May 21, 2010 Posted May 21, 2010 Hmm, while creating an xml sitemap I noticed that my category pages are being indexed like so: http://site.net/catalog/subcategory?sort=1 and catalog/product?action=notify Is there a way to prevent the sort from showing in the URL, also I'm a bit confused now. Is product_info.php controlling the subcategory too? The SEO module I used rewrites URLs so that the category and product display in the store root. For example: http://site.net/catalog/category and if the visitor browses to a subcategory it displays catalog/subcategory. I just want to be certain that by putting noindex in product_info.php that it only stops the products and not categories or subcategories. Thanks! Your products are generated from product_info.php ONLY. Your categories from index.php ONLY. This type of URL catalog/product?action=notify is still generated from product_info.php. Putting the noindex tag won't affect how your URLs are displayed at all.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.