Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

Robots.txt


Sincraft

Recommended Posts

I am going to post my robots file in hopes someone here with good SEO experience and osCommerce can tell me if this looks correct or not. I keep logging into my who's online page and keep finding MSNbot crawling around. It basically lives there day and night with a break in the middle of the day, which I guess is a good thing. But I often find it in , account.php or shopping_cart.php etc...so I disallowed a few things, but tonight say that it was in COOKIE_USAGE.PHP...

How does it get into that without a link to it? I can't even figure out how it got there.

 

So anyway, I wanted to know the above question for curiousity reasons and wanted someone to comment on this robot.txt..

 

THis is my ENTIRE robot.txt contents.

User-agent: *
Disallow: /cgi-bin/
Disallow: /usage/
Disallow: /images/
Disallow: /(censoredadmincat)/
Disallow: /tmp/
Disallow: /account.php
Disallow: /shopping_cart.php
Disallow: /checkout_shipping.php
Disallow: /login.php
Disallow: /create_account.php
Disallow: /advanced_search.php
Disallow: /product_review.php
Disallow: /popup_image.php
Disallow: /cookie_usage.php

 

what do you think? Should I also consider putting an * after the .php just incase there are session ids etc as many of my urls are not wrapped with Ultimate SEO URLs (like privacy.php and articles.php&page2_how to buy etc. I noticed it is still looking at a popup image as we speak even though I have it commented out in my robots file. /popup_image.php?pID=43 (for example)

 

S

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...