Guest Posted May 27, 2008 Posted May 27, 2008 firstly, i consider them a direct competitor.. so what are they trying to do? secondly, i have my url's rewritten. they aren't looking at that, they're looking at THIS: 00:00:00 Guest ec2-75-101-218-225.compute-1.amazonaws.com 15:33:34 15:33:34 /index.php?manufacturers_id=ManufacturerName last night, I banned their entire range via htaccess: deny from 67.202. today they return with a new ip: 75.101.218.225 anyone else experiencing this? is it possible to 404 the url's they're looking at? and... how many more ip's do they have?
spooks Posted May 27, 2008 Posted May 27, 2008 Are they trying to look at pages they should'nt? Have you tried a IP trap http://addons.oscommerce.com/info/5914 Have you checked if there`s been any hacking? Sam Remember, What you think I ment may not be what I thought I ment when I said it. Contributions: Auto Backup your Database, Easy way Multi Images with Fancy Pop-ups, Easy way Products in columns with multi buy etc etc Disable any Category or Product, Easy way Secure & Improve your account pages et al.
Guest Posted May 27, 2008 Posted May 27, 2008 i don't really want to auto ban anybody as i've read too many horror stories on other boards about mistakes that happen. i'm keeping a close eye on who's online now, though. i've not been hacked, i have logwatch installed on my server, so i get daily email updates about any funny business and ssh logins. and i keep all of the server stuff updated, operating system, cpanel, etc. .. and no, i am not on a shared server. amazonaws.com seems to be some sort of developer site? i'm not really sure, but it looks like they allow customers to "build" things on there.. so i'm suspecting a competitor is probing my site trying to figure out how i do things? pretty lame. it doesn't seem to be identifying itself as a bot.. so i'm wondering if it's possible to ban anything with "amazonaws.com" in the browser agent
♥Vger Posted May 28, 2008 Posted May 28, 2008 amazonaws = amazon web services This is a new service by Amazon (trying to do a bit of a Google). They are renting out space and bandwidth on their servers to developers. Those developers then use the services supplied by Amazon to scan other people's websites - it's not Amazon themselves who are doing it. If your websites' bandwidth is being racked up by amazonaws then you can complain to amazonaws with details of when and what happened and they'll "warn" the person doing it, and then cut them off if they keep doing it (or so they imply). The only good point about this is that the person using their service has to pay them by the hour and by the Gigabyte, so if they rack up your bandwidth they rack up their own bill. The bad point is that they don't charge a lot for this service. As it's not a search engine you can quite safely bar amazonaws ip addresses - but be aware that they have a massive traunch of ip addresses at their disposal (the whole amazon network), so it won't be that easy. But you don't have to ban by ip address and can ban by domain, so you can just simply ban amazonaws.com via a root level .htaccess file - using the same code as for an ip address ban but using the domain name instead e.g. *.amazonaws.com There is no valid reason that I can see why amazonaws.com should be on people's websites at all - so ban away! Vger
Guest Posted May 29, 2008 Posted May 29, 2008 Is it possible that someone there might actually be interested in what you sell? Maybe you sell books and and an employee found it cheaper on your site than on amazon after their discount... Just a thought. Blocking their ip ranges might help for a few minutes, but if they really cared (or were spying), they would spend the 1 minute to proxy to another non-amazon domain server and do their snooping from there.
Guest Posted June 2, 2008 Posted June 2, 2008 The only good point about this is that the person using their service has to pay them by the hour and by the Gigabyte, so if they rack up your bandwidth they rack up their own bill. I'm extremely suspicious of anybody that's willing to pay to scan my site.. that's astounding. But you don't have to ban by ip address and can ban by domain excellent point! i've added this to htaccess: SetEnvIfNoCase User-Agent "amazonaws" bad_bot Order Allow,Deny Allow from all Deny from env=bad_bot in hopes of keeping them away. I haven't spotted one yet since adding this.. but they seem to have popped on at random. Is it possible that someone there might actually be interested in what you sell? if it's a competitor, i really don't want them scanning my site though. :)
Guest Posted June 8, 2008 Posted June 8, 2008 my previous solution did not work. spotted 1 today: zermelo ec2-67-202-41-191.compute-1.amazonaws.com Mozilla/5.0 (compatible; zermelo; +http://www.powerset.com) [ema IP Address: 67.202.41.191 User Agent: Mozilla/5.0 (compatible; zermelo; +http://www.powerset.com) [email:[email protected],[email protected]] i am offically confused. why do the majority of amazon bots spider my sites while ignoring my rewrite rules... yet "paul" obeys them.. nevertheless...anyone feeling the need to buy space via amazon requests an immediate ban in my book. how can i efficiently ban anything from amazon's servers?
♥Vger Posted June 8, 2008 Posted June 8, 2008 Try a simpler approach: <FILES .*> Order Deny, Allow Deny from amazonaws.com Allow From All </FILES> Vger
Guest Posted June 10, 2008 Posted June 10, 2008 thanks i will give this a try. what portion of their bot details does this scan? i currently use something similar to block ips.
♥Vger Posted June 10, 2008 Posted June 10, 2008 My post would completely block all access to a website for amazonaws.com There is no benefit to any webmaster in having that domain scan their website, so I'd recommend that everyone blocks it. Vger
gtremblay Posted July 23, 2008 Posted July 23, 2008 Hi, I receive also many visits from theses, and i don't know why either... Number of Entries: 37 21st July 2008 16:07:26 Multiple visits spread over more than one day Seattle Washington United States ec2-67-202-54-208.compute-1.amazonaws.com (67.202.54.208) Number of Entries: 9 22nd July 2008 19:31:46 Seattle Washington United States ec2-67-202-54-149.compute-1.amazonaws.com (67.202.54.149) and it's seems is does'nt load any page??? Date Time WebPage 21st July 2008 16:07:26 No referring link 21st July 2008 16:09:12 No referring link 21st July 2008 16:10:41 No referring link 21st July 2008 16:13:40 No referring link 21st July 2008 16:14:49 No referring link 21st July 2008 16:17:10 No referring link 21st July 2008 23:36:26 No referring link 22nd July 2008 08:09:17 No referring link 22nd July 2008 08:12:02 No referring link 22nd July 2008 08:14:31 No referring link 22nd July 2008 08:17:23 No referring link 22nd July 2008 08:19:29 No referring link 22nd July 2008 08:21:58 No referring link 22nd July 2008 08:24:42 No referring link 22nd July 2008 08:27:56 No referring link 22nd July 2008 08:31:15 No referring link 22nd July 2008 08:34:29 No referring link 22nd July 2008 08:37:18 No referring link 22nd July 2008 08:40:29 No referring link 22nd July 2008 08:44:06 No referring link 22nd July 2008 08:47:16 No referring link 22nd July 2008 16:51:03 No referring link I also deny access to amazonaws.com in .htaccess ...
Eighteen48 Posted July 30, 2008 Posted July 30, 2008 I would not be so quick to try to ban the Amazon bots, as some of these might be tied into with Alexa and some of their other sites that will help boost your ranks. Or Amazon or any developer making an interface with Amazon web payment systems. You never know it could be a good thing. I know Alexa uses it to thumbnail your site as well as to calculate the speed of your site, ranking and so forth. Powered By osC 2.2RC2a STS 4.5.8 - HTC 2.6.3 - FP 1.5.9 - BCH 1.0.0
Eighteen48 Posted July 30, 2008 Posted July 30, 2008 Try blocking "ia_archiver" in your robots text and see what happens. Powered By osC 2.2RC2a STS 4.5.8 - HTC 2.6.3 - FP 1.5.9 - BCH 1.0.0
♥Vger Posted July 30, 2008 Posted July 30, 2008 I would not be so quick to try to ban the Amazon bots We've banned amazonaws.com at root level on all of our servers. It is not a bot. Amazon are renting out their unused webspace and bandwidth to anyone who will pay to use their "cloud". The Amazon Web Service has, in our experience and the experience of others, been used by hackers to try and find exploits in websites and/or to recursively download complete backups of website files so that they can spoof the website and use it for pfishing. It didn't happen on our servers, but someone that we know of lost 80 Gbs of bandwidth to the activities of one person using amazonaws.com during the course of just one day. It cost them an arm and a leg in extra bandwidth charges. Yes, Amazon does own Alexa but they use a different set of servers - so there's no benefit to you in allowing amazonaws.com to access your website. Don't try to ban them via ip address as there are just too many of them. Ban them by domain name, as in: amazonaws.com *.amazonaws.com Vger
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.