ollyno1uk2 Posted February 18, 2008 Posted February 18, 2008 I'm seeing in my webmaster tools in google unreachable urls such as http://www.mysite.co.uk/advanced_search_re...roducts_id=4529 Whilst these are not indexed I would like to tell google not to follow these urls. Is this possible? Perhaps there are other urls this happens to also. Thanks
Guest Posted February 18, 2008 Posted February 18, 2008 i think the best way is in a robot.txt file. some thing like User-agent: Googlebot-Image Disallow: / probably the best place to start would be to google robot.txt and see what comes up
♥FWR Media Posted February 18, 2008 Posted February 18, 2008 There are easy ways to do it but I'd honestly suggest that you don't bother. Googlebot isn't stupid (well it is it's a bot) it recognises such things and as you deduced it doesn't index them .. the best that would happen is that you feel better .. the worst is that you'll block some genuine pages .. leave alone I say. Ultimate SEO Urls 5 PRO - Multi Language Modern, Powerful SEO Urls KissMT Dynamic SEO Meta & Canonical Header Tags KissER Error Handling and Debugging KissIT Image Thumbnailer Security Pro - Querystring protection against hackers ( a KISS contribution ) If you found my post useful please click the "Like This" button to the right. Please only PM me for paid work.
ollyno1uk2 Posted February 18, 2008 Author Posted February 18, 2008 There are easy ways to do it but I'd honestly suggest that you don't bother. Googlebot isn't stupid (well it is it's a bot) it recognises such things and as you deduced it doesn't index them .. the best that would happen is that you feel better .. the worst is that you'll block some genuine pages .. leave alone I say. Hi thanks - the issue is that it is indexing it. My store has 8000 odd pages google is indexing 14500 This is due to advanced_search_results and some kind of filter. This is why I am now worried
♥FWR Media Posted February 18, 2008 Posted February 18, 2008 Hi thanks - the issue is that it is indexing it. My store has 8000 odd pages google is indexing 14500 This is due to advanced_search_results and some kind of filter. This is why I am now worried I replied to a topic a few minutes ago maybe it'll help. http://www.oscommerce.com/forums/index.php?sho...=293373&hl= Ultimate SEO Urls 5 PRO - Multi Language Modern, Powerful SEO Urls KissMT Dynamic SEO Meta & Canonical Header Tags KissER Error Handling and Debugging KissIT Image Thumbnailer Security Pro - Querystring protection against hackers ( a KISS contribution ) If you found my post useful please click the "Like This" button to the right. Please only PM me for paid work.
ollyno1uk2 Posted February 19, 2008 Author Posted February 19, 2008 I replied to a topic a few minutes ago maybe it'll help. http://www.oscommerce.com/forums/index.php?sho...=293373&hl= Thanks I have now been advised to do this: User-agent: * Disallow: /*? in Robots.txt. This will remove all urls with the ? in it. As I use ultimate SEO I believe that there should be no ? in the urls anyhow so hopefully this should sure it
Recommended Posts
Archived
This topic is now archived and is closed to further replies.