scubaworld Posted December 30, 2009 Share Posted December 30, 2009 Hallo, This crawler b3090894.crawl.yahoo.net show in the quest list of whos online. It's the Yahoo slurp. Is there a possibility to let it show as a bot so I can select "show bot" or not. I have installed Whos Online 3.3.6. Thanks Link to comment Share on other sites More sharing options...
emdkay Posted January 11, 2010 Share Posted January 11, 2010 I've had this same issue with Yahoo and MSN for months. I'm using Whos Online 3.6.3. I don't know how/where to update the list of bot IP addresses, but I presume that would be where to start looking. Link to comment Share on other sites More sharing options...
markg-uk Posted January 13, 2010 Share Posted January 13, 2010 Its not done by IP address, its done by user agent. Add the spider's user agent to /includes/spiders.txt and it should then appear as a bot Link to comment Share on other sites More sharing options...
emdkay Posted January 13, 2010 Share Posted January 13, 2010 Its not done by IP address, its done by user agent. Add the spider's user agent to /includes/spiders.txt and it should then appear as a bot What would be the correct entry for yahoo then? They show up as: User agent: Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) I've tried adding: Yahoo! Slurp/3.0 Slurp/3.0 Yahoo! Slurp Yahoo still appears as a visitor in whos online though. However, I updated spiders.txt with this and am waiting to see what happens: http://www.oscommerce.com/community/contributions,2455 Link to comment Share on other sites More sharing options...
emdkay Posted January 13, 2010 Share Posted January 13, 2010 What would be the correct entry for yahoo then? They show up as: User agent: Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) I've tried adding: Yahoo! Slurp/3.0 Slurp/3.0 Yahoo! Slurp Yahoo still appears as a visitor in whos online though. However, I updated spiders.txt with this and am waiting to see what happens: http://www.oscommerce.com/community/contributions,2455 http://www.oscommerce.com/community/contributions,2455 Nice! Yahoo now shows up properly as a bot Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.