Xtreme_Stunts7 Posted August 12, 2006 Share Posted August 12, 2006 Hi everybody, I have an osc store which is running since february of this year. i have installed stevel's spiders.txt file and my own robots.txt. however, when i check for the robots that are visiting my site, i only see 8 different ones through my cpanel: Googlebot 413+12 9.82 MB 11 Aug 2006 - 23:20 Unknown robot (identified by 'robot') 159+1 1.40 MB 11 Aug 2006 - 22:15 MSNBot 61+85 1.34 MB 12 Aug 2006 - 00:01 Inktomi Slurp 71+73 1.42 MB 11 Aug 2006 - 00:02 Unknown robot (identified by hit on 'robots.txt') 0+49 58.62 KB 11 Aug 2006 - 06:32 Unknown robot (identified by 'spider') 19+2 211.49 KB 11 Aug 2006 - 00:38 Unknown robot (identified by 'crawl') 10+9 251.50 KB 08 Aug 2006 - 09:14 AskJeeves 1+1 8.81 KB 08 Aug 2006 - 04:50 Could you tell me if i have something setup the wrong way or if it is actually good to have that little robots crawling the site. Thanks a lot Nail Link to comment Share on other sites More sharing options...
abra123cadabra Posted August 12, 2006 Share Posted August 12, 2006 You've got the main ones which is important. There are others that will find you sooner or later and to be honest, my site is crawled by something called baidu which is nice but being listed in a chinese search engine is pretty useless to me as I sell German foodstuffs for delivery in Ireland. abra The First Law of E-Commerce: If the user can't find the product, the user can't buy the product. Feedback and suggestions on my shop welcome. Note: My advice is based on my own experience or on something I read in these forums. No guarantee it'll work for you! Make sure that you always BACKUP the database and the files you are going to change so that you can rollback to a working version if things go wrong. Link to comment Share on other sites More sharing options...
Xtreme_Stunts7 Posted August 12, 2006 Author Share Posted August 12, 2006 Hi Abra, thanks. i was just hoping to get more robots to come and visit my site because i submitted it to many different international search engines to get more traffic. Link to comment Share on other sites More sharing options...
boxtel Posted August 12, 2006 Share Posted August 12, 2006 Hi Abra, thanks. i was just hoping to get more robots to come and visit my site because i submitted it to many different international search engines to get more traffic. yes but more spiders does not mean more traffic, the trick is to keep the valuable spiders coming and to have them rate your pages well, the rest are just an added feature of little value and more often a nusance. Treasurer MFC Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.