scubaworld Posted December 30, 2009 Posted December 30, 2009 Hallo, This crawler b3090894.crawl.yahoo.net show in the quest list of whos online. It's the Yahoo slurp. Is there a possibility to let it show as a bot so I can select "show bot" or not. I have installed Whos Online 3.3.6. Thanks
emdkay Posted January 11, 2010 Posted January 11, 2010 I've had this same issue with Yahoo and MSN for months. I'm using Whos Online 3.6.3. I don't know how/where to update the list of bot IP addresses, but I presume that would be where to start looking.
markg-uk Posted January 13, 2010 Posted January 13, 2010 Its not done by IP address, its done by user agent. Add the spider's user agent to /includes/spiders.txt and it should then appear as a bot
emdkay Posted January 13, 2010 Posted January 13, 2010 Its not done by IP address, its done by user agent. Add the spider's user agent to /includes/spiders.txt and it should then appear as a bot What would be the correct entry for yahoo then? They show up as: User agent: Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) I've tried adding: Yahoo! Slurp/3.0 Slurp/3.0 Yahoo! Slurp Yahoo still appears as a visitor in whos online though. However, I updated spiders.txt with this and am waiting to see what happens: http://www.oscommerce.com/community/contributions,2455
emdkay Posted January 13, 2010 Posted January 13, 2010 What would be the correct entry for yahoo then? They show up as: User agent: Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) I've tried adding: Yahoo! Slurp/3.0 Slurp/3.0 Yahoo! Slurp Yahoo still appears as a visitor in whos online though. However, I updated spiders.txt with this and am waiting to see what happens: http://www.oscommerce.com/community/contributions,2455 http://www.oscommerce.com/community/contributions,2455 Nice! Yahoo now shows up properly as a bot
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.