Jump to content
  • Checkout
  • Login
  • Get in touch


The e-commerce.


Recommended Posts

Robots.txt contains a list of the files and directories that you do not want search engines to spider. For the proper format see robotstxt.org.


Spiders.txt should be in your catalog/includes/ directory (if you are running osC 2.2 MS2 or a recent CVS). This is a list of search engine spiders that you want to avoid giving a SID to. You may need to add new bots to the list when they appear. The format is the name of the bot as it appears in your server logs.




See my profile for a list of my addons and ways to get support.

Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Create New...