Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

Has msn gone a little over the top ?


rommany

Recommended Posts

Hello people

 

 

This doesnt look right to me, msn came to my site around 2000 last month in total, but ive just looked at my stats today and msn has been ( MSNBot 726+25 18.80 MB 03 Jul 2005 - 06:42 ) in only 3days why would it have gone mad ? if this goes on i will run out of MB.

 

Many thanks

Link to comment
Share on other sites

  • 2 weeks later...

I also am getting repeated hits on my who's online, and alway's to the products_reviews_write page, and from the same IP addy. I was getting a little concerned. I did a traceroute, and learned it was msnbot.com. So, should I be concerned? I am getting this thing several times a day.

 

Len

LensRc.com

Link to comment
Share on other sites

Leonard,

 

I would consider adding it to your robots page to exclude it. The bot isn't hurting the store per se but it is driving up your traffic.

 

ed

Link to comment
Share on other sites

Hi again

 

With my site the msn bots are getting silly, the rest have visited around 400 times and msn has been 4000 times so i think i will also do what you have recommended above.

 

Thanks

Link to comment
Share on other sites

Leonard,

 

I would consider adding it to your robots page to exclude it.  The bot isn't hurting the store per se but it is driving up your traffic.

 

ed

 

 

How do I do this? I saw references to "robots.txt" when I did a search for msnbot, but I dont seem to have that file........

 

TIA

 

Len

Link to comment
Share on other sites

I'd start by using this mod to make sure all active robots are picked up.

 

Then create / update your robots.txt with every single page in the root directory which you don't want to appear on search engines. A guide to robots.txt is available here.

 

Jon.

Link to comment
Share on other sites

Here's a copy of the robots.txt which I use:

 

User-agent: *
Disallow: /account.php
Disallow: /account_edit.php
Disallow: /account_history.php
Disallow: /account_history_info.php
Disallow: /account_newsletters.php
Disallow: /account_notifications.php
Disallow: /account_password.php
Disallow: /address_book.php
Disallow: /address_book_process.php
Disallow: /advanced_search_result.php
Disallow: /checkout_confirmation.php
Disallow: /checkout_payment.php
Disallow: /checkout_payment_address.php
Disallow: /checkout_process.php
Disallow: /checkout_shipping.php
Disallow: /checkout_shipping_address.php
Disallow: /checkout_success.php
Disallow: /create_account.php
Disallow: /create_account_success.php
Disallow: /default.php
Disallow: /download.php
Disallow: /info_shopping_cart.php
Disallow: /login.php
Disallow: /logoff.php
Disallow: /password_forgotten.php
Disallow: /popup_conditions.php
Disallow: /popup_image.php
Disallow: /popup_search_help.php
Disallow: /product_reviews_info.php
Disallow: /product_reviews_write.php
Disallow: /redirect.php
Disallow: /shopping_cart.php
Disallow: /tell_a_friend.php

 

I think I've removed all the entries specific to my site. There may be some more you need to add to yours dependent upon the mods you've got installed.

 

Jon.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...