knifeman Posted April 11, 2011 Share Posted April 11, 2011 I have had this file blocked for ages. All of a sudden Google sends me an e-mail. Thank you for participating in Google Product Search. It has come to our attention that a robots.txt file is preventing us from crawling some or all of the images on your site. In order for us to access and display the images you provide in your product listings, we'd like you to modify your robots.txt file to allow user-agent 'googlebot' to crawl your site. Failure for Google to access your images may affect the visibility of your items on Google Product Search and Product Ad results. bots should be able to crawl my images, just not the pop up image. We don't want the pop up indexed do we? If customers click on an indexed pop up, they cannot navigate the site and lose interest. Anyone else seen this? Recommendations? Tim Link to comment Share on other sites More sharing options...
germ Posted April 11, 2011 Share Posted April 11, 2011 I disallow it. I don't think it does any good to get indexed by search engines. Like you said, the popup image by itself in search engines seems of no value to me. That's my two cents. Next opinion! :) If I suggest you edit any file(s) make a backup first - I'm not perfect and neither are you. "Given enough impetus a parallelogramatically shaped projectile can egress a circular orifice." - Me - "Headers already sent" - The definitive help "Cannot redeclare ..." - How to find/fix it SSL Implementation Help Like this post? "Like" it again over there > Link to comment Share on other sites More sharing options...
Hotclutch Posted April 11, 2011 Share Posted April 11, 2011 Anyone else seen this? Recommendations? Tim Use meta noindex in the head section of the popup page instead of robots.txt to disallow. Link to comment Share on other sites More sharing options...
knifeman Posted April 11, 2011 Author Share Posted April 11, 2011 Use meta noindex in the head section of the popup page instead of robots.txt to disallow. That sounds like a winner! Thanks. Link to comment Share on other sites More sharing options...
germ Posted April 11, 2011 Share Posted April 11, 2011 Use meta noindex in the head section of the popup page instead of robots.txt to disallow. Never thought about approaching the situation from that angle. I think I like it better, too. :thumbsup: If I suggest you edit any file(s) make a backup first - I'm not perfect and neither are you. "Given enough impetus a parallelogramatically shaped projectile can egress a circular orifice." - Me - "Headers already sent" - The definitive help "Cannot redeclare ..." - How to find/fix it SSL Implementation Help Like this post? "Like" it again over there > Link to comment Share on other sites More sharing options...
Hotclutch Posted April 11, 2011 Share Posted April 11, 2011 Never thought about approaching the situation from that angle. I think I like it better, too. :thumbsup: In general, I always try to allow with robots.txt and block with htacess and meta noindex. :thumbsup: Link to comment Share on other sites More sharing options...
DAVID3733 Posted April 12, 2011 Share Posted April 12, 2011 Hi there I have had this email too but its worded slightly different It has come to our attention that a robots.txt file is preventing us from crawling some or all of the images on your site. In order for us to access and display the images you provide in your product listings, we'd like you to modify your robots.txt file to allow user-agent 'googlebot' to crawl your site. Failure for Google to access your images may affect the visibility of your items on Google Product Search and Product Ad results. To ensure the 'googlebot' is not being blocked, please add the following two lines of text to the end of your robots.txt file: User-agent: googlebot Disallow: For more information on robots.txt files, please visit http://www.robotstxt.org. If you have any questions, please contact your webmaster directly. isnt that telling google it cant scan rather than the image bot or am i reading this wrong, it has definatly some from google to a email address that is not used for other things, in my robots text i have the normal User-agent: googlebot-image Disallow: Unless the google image bot has changed its name or something i am confused David David Link to comment Share on other sites More sharing options...
knifeman Posted April 12, 2011 Author Share Posted April 12, 2011 Hi there I have had this email too but its worded slightly different isnt that telling google it cant scan rather than the image bot or am i reading this wrong, it has definatly some from google to a email address that is not used for other things, in my robots text i have the normal User-agent: googlebot-image Disallow: Unless the google image bot has changed its name or something i am confused David This page explains the code you referenced: http://www.robotstxt.org./robotstxt.html To exclude all robots from the entire server User-agent: * Disallow: / To allow all robots complete access User-agent: * Disallow: Link to comment Share on other sites More sharing options...
krdito Posted April 13, 2011 Share Posted April 13, 2011 I received the same email. Does anyone know where the robots.txt file is located? I have search my File Manager and don't see it anywhere. Link to comment Share on other sites More sharing options...
DAVID3733 Posted April 13, 2011 Share Posted April 13, 2011 Its in your main dir Catalog/robots Regards David Link to comment Share on other sites More sharing options...
germ Posted April 13, 2011 Share Posted April 13, 2011 robots.txt goes in the root of the SITE, not necessarily the catalog. If I suggest you edit any file(s) make a backup first - I'm not perfect and neither are you. "Given enough impetus a parallelogramatically shaped projectile can egress a circular orifice." - Me - "Headers already sent" - The definitive help "Cannot redeclare ..." - How to find/fix it SSL Implementation Help Like this post? "Like" it again over there > Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.