Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

robots.txt


lilmiss2die4

Recommended Posts

Can someone check my code for my robots.txt

 

It seems that the googlebot and others are still spending alot of time in places that are not needed. Like in the cookie_usage area and the login area

 

# /robots.txt file for [url="http://just2cutegifts.com/"]http://just2cutegifts.com/[/url] 


User-agent: Googlebot-Image
Disallow: /cgi-bin/
Disallow: /usage/
Disallow: /catalog/

User-agent: *
Disallow: /cgi-bin/
Disallow: /usage/
Disallow: /catalog/admin/
Disallow: /catalog/download/
Disallow: /catalog/elmar/
Disallow: /catalog/pub/
Disallow: /catalog/account.php
Disallow: /catalog/advanced_search.php
Disallow: /catalog/checkout_shipping.php
Disallow: /catalog/create_account.php
Disallow: /catalog/login.php
Disallow: /catalog/password_forgotten.php
Disallow: /catalog/popup_image.php
Disallow: /catalog/shopping_cart.php

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...