Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

Stop Msn.bot indexing Site


gaspower

Recommended Posts

Hello,

 

we have had our site up and running for many years, version prior to MS2.31. Just recently we are having a msn.bot indexing our site and it is absolutely crippling our site. The site almost comes to a complete holt, and we have to wait until the indexing is over. Is there anyway to prevent this from msn.bot? We have other search engines on the site, and no issues.

 

Thanks JR

Link to comment
Share on other sites

I've noticed they seem to be increasing their searches too. But if you stop them altogether, you would prevent your site from being listed by them, which is not good. You can add the following to your robots file and it should help. Change them as you wish, of course.

Crawl-delay: 2
Request-rate: 1/2
Visit-time: 0300-0900

Support Links:

For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc.

All of My Addons

Get the latest versions of my addons

Recommended SEO Addons

Link to comment
Share on other sites

You should have a user agent statement. see http://www.robotstxt.org/ for examples and testing. The permissions don't matter but shouldn't be higher than 644.

Support Links:

For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc.

All of My Addons

Get the latest versions of my addons

Recommended SEO Addons

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...