lilmiss2die4 Posted April 9, 2007 Posted April 9, 2007 Can someone check my code for my robots.txt It seems that the googlebot and others are still spending alot of time in places that are not needed. Like in the cookie_usage area and the login area # /robots.txt file for [url="http://just2cutegifts.com/"]http://just2cutegifts.com/[/url] User-agent: Googlebot-Image Disallow: /cgi-bin/ Disallow: /usage/ Disallow: /catalog/ User-agent: * Disallow: /cgi-bin/ Disallow: /usage/ Disallow: /catalog/admin/ Disallow: /catalog/download/ Disallow: /catalog/elmar/ Disallow: /catalog/pub/ Disallow: /catalog/account.php Disallow: /catalog/advanced_search.php Disallow: /catalog/checkout_shipping.php Disallow: /catalog/create_account.php Disallow: /catalog/login.php Disallow: /catalog/password_forgotten.php Disallow: /catalog/popup_image.php Disallow: /catalog/shopping_cart.php
Recommended Posts
Archived
This topic is now archived and is closed to further replies.