daz_75 Posted June 26, 2007 Share Posted June 26, 2007 The yahoo spider has gone nuts recently, it's started using 800mb of my bandwidth which seeing as i've only got 5gb adds up to a lot. Anyway of stopping this, or better still telling it to slow down a bit? I know you can do it in google, but is there something similar in yahoo? Cheers Daz Link to comment Share on other sites More sharing options...
germ Posted June 26, 2007 Share Posted June 26, 2007 You could use the "robots.txt" file and "disallow" folders you don't want "crawled". Yahoo! help on the subject: http://help.yahoo.com/help/us/ysearch/slurp/slurp-02.html There are probably other ways as well. :) If I suggest you edit any file(s) make a backup first - I'm not perfect and neither are you. "Given enough impetus a parallelogramatically shaped projectile can egress a circular orifice." - Me - "Headers already sent" - The definitive help "Cannot redeclare ..." - How to find/fix it SSL Implementation Help Like this post? "Like" it again over there > Link to comment Share on other sites More sharing options...
daz_75 Posted June 26, 2007 Author Share Posted June 26, 2007 You could use the "robots.txt" file and "disallow" folders you don't want "crawled". Yahoo! help on the subject: http://help.yahoo.com/help/us/ysearch/slurp/slurp-02.html There are probably other ways as well. :) Already got that in place mate, thanks anyway Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.