Guest Posted July 16, 2003 Share Posted July 16, 2003 Hi all, I've searched this (& other forums) to no avail.. :( Over the past couple of days Inktomi spiders have been constantly indexing my site, even adding products to their cart!! My visitor stats (from a contrib) show they are from a specific IP range: 66.196.64.0 - 66.196.127.255 I've installed SID killer & changed relevent Meta Tags & enabled "search engine friendly URLs, however I think that I'm just a bit too late as the "entry URLs" for these spiders vary across product categorys/items for example: /catalog/product_info.php/products_id/89 /catalog/products_new.php/page/1/action/buy_now/products_id/216 /catalog/default.php/manufacturers_id/10/sort/2a/page/1/action/buy_now/products_id/108 They seem to enter directly on these URLs & go nuts from there. Please help as these bots are really a problem for me. What can I do? Block Inktomi's IPs? How? Please be kind as I'm new at this? thanks, Rob :?: :?: :?: Link to comment Share on other sites More sharing options...
Guest Posted July 16, 2003 Share Posted July 16, 2003 Oh, And they are generating osCid's! thanks, Rob Link to comment Share on other sites More sharing options...
wizardsandwars Posted July 16, 2003 Share Posted July 16, 2003 Look for a contribution called "spider Killer" ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
Guest Posted July 16, 2003 Share Posted July 16, 2003 It's been doing the same to my site for 2 weeks straight now and has consumed about 2 gig of bandwidth but I personally am not worrying about it. Google did the samething and I have about 1900 pages indexed there now so as long as my server can handle it and it doesnt take me over my bandwidth I'm personally going to let it go. Admittedly its a pain to figure out traffic but once its done should but 4 weeks and my site should be idexed there and who ever else Inktomi feeds. Just my 2 cents for what its worth. :) Hi all, I've searched this (& other forums) to no avail.. :( Over the past couple of days Inktomi spiders have been constantly indexing my site, even adding products to their cart!! My visitor stats (from a contrib) show they are from a specific IP range: 66.196.64.0 - 66.196.127.255 I've installed SID killer & changed relevent Meta Tags & enabled "search engine friendly URLs, however I think that I'm just a bit too late as the "entry URLs" for these spiders vary across product categorys/items for example: /catalog/product_info.php/products_id/89 /catalog/products_new.php/page/1/action/buy_now/products_id/216 /catalog/default.php/manufacturers_id/10/sort/2a/page/1/action/buy_now/products_id/108 They seem to enter directly on these URLs & go nuts from there. Please help as these bots are really a problem for me. What can I do? Block Inktomi's IPs? How? Please be kind as I'm new at this? thanks, Rob :?: :?: :?: Link to comment Share on other sites More sharing options...
chfields Posted July 18, 2003 Share Posted July 18, 2003 I have they same problem with inktomi creating sid's. I have 2.2 ms1 and I have spider killer installed and inktomi blocked and the ip blocked and still getting sid's...help.....please Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.