♥yesudo Posted March 2, 2004 Share Posted March 2, 2004 i am on ms2: 1) is the sid for spiders already corrected under this release? As i have read somewhere there is a bug in the coding. 2) where can i get a list of bots/spiders to put in my spiders.txt file ? 3) what is the difference between spiders.txt and robots.txt ? THANX. Your online success is Paramount. Link to comment Share on other sites More sharing options...
sam6 Posted March 2, 2004 Share Posted March 2, 2004 a robots.txt is a file that allows you to tell a spider you dont what him in this part of your site a spider.txt in osc is a list of user agents tell osc not to give these user agents oscids . you dont want osids in your links on search engines i guse and it works most of the time right now i am having problems with yahoo but that is another story Link to comment Share on other sites More sharing options...
♥yesudo Posted March 2, 2004 Author Share Posted March 2, 2004 thanks Sam, I was going to ask these questions on your thread but did not want to hijack it. just one more question how do i keep the spiders.txt file upto date? Thanx, Your online success is Paramount. Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.