Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

why is AMAZON.COM scanning my site!


Guest

Recommended Posts

firstly, i consider them a direct competitor.. so what are they trying to do?

 

secondly, i have my url's rewritten. they aren't looking at that, they're looking at THIS:

00:00:00 Guest ec2-75-101-218-225.compute-1.amazonaws.com 15:33:34 15:33:34 /index.php?manufacturers_id=ManufacturerName

 

last night, I banned their entire range via htaccess:

deny from 67.202.

 

today they return with a new ip:

75.101.218.225

 

anyone else experiencing this? is it possible to 404 the url's they're looking at?

and... how many more ip's do they have?

Link to comment
Share on other sites

Are they trying to look at pages they should'nt?

 

Have you tried a IP trap

 

http://addons.oscommerce.com/info/5914

 

Have you checked if there`s been any hacking?

Sam

 

Remember, What you think I ment may not be what I thought I ment when I said it.

 

Contributions:

 

Auto Backup your Database, Easy way

 

Multi Images with Fancy Pop-ups, Easy way

 

Products in columns with multi buy etc etc

 

Disable any Category or Product, Easy way

 

Secure & Improve your account pages et al.

Link to comment
Share on other sites

i don't really want to auto ban anybody as i've read too many horror stories on other boards about mistakes that happen. i'm keeping a close eye on who's online now, though.

 

i've not been hacked, i have logwatch installed on my server, so i get daily email updates about any funny business and ssh logins.

and i keep all of the server stuff updated, operating system, cpanel, etc. .. and no, i am not on a shared server.

 

amazonaws.com seems to be some sort of developer site? i'm not really sure, but it looks like they allow customers to "build" things on there.. so i'm suspecting a competitor is probing my site trying to figure out how i do things? pretty lame.

 

it doesn't seem to be identifying itself as a bot.. so i'm wondering if it's possible to ban anything with "amazonaws.com" in the browser agent

Link to comment
Share on other sites

amazonaws = amazon web services

 

This is a new service by Amazon (trying to do a bit of a Google).

 

They are renting out space and bandwidth on their servers to developers. Those developers then use the services supplied by Amazon to scan other people's websites - it's not Amazon themselves who are doing it. If your websites' bandwidth is being racked up by amazonaws then you can complain to amazonaws with details of when and what happened and they'll "warn" the person doing it, and then cut them off if they keep doing it (or so they imply).

 

The only good point about this is that the person using their service has to pay them by the hour and by the Gigabyte, so if they rack up your bandwidth they rack up their own bill. The bad point is that they don't charge a lot for this service.

 

As it's not a search engine you can quite safely bar amazonaws ip addresses - but be aware that they have a massive traunch of ip addresses at their disposal (the whole amazon network), so it won't be that easy.

 

But you don't have to ban by ip address and can ban by domain, so you can just simply ban amazonaws.com via a root level .htaccess file - using the same code as for an ip address ban but using the domain name instead e.g. *.amazonaws.com

 

There is no valid reason that I can see why amazonaws.com should be on people's websites at all - so ban away!

 

Vger

Link to comment
Share on other sites

Is it possible that someone there might actually be interested in what you sell? Maybe you sell books and and an employee found it cheaper on your site than on amazon after their discount... Just a thought.

 

Blocking their ip ranges might help for a few minutes, but if they really cared (or were spying), they would spend the 1 minute to proxy to another non-amazon domain server and do their snooping from there.

Link to comment
Share on other sites

The only good point about this is that the person using their service has to pay them by the hour and by the Gigabyte, so if they rack up your bandwidth they rack up their own bill.

I'm extremely suspicious of anybody that's willing to pay to scan my site.. that's astounding.

 

But you don't have to ban by ip address and can ban by domain

excellent point! i've added this to htaccess:

   SetEnvIfNoCase User-Agent "amazonaws" bad_bot
  Order Allow,Deny
  Allow from all
  Deny from env=bad_bot

in hopes of keeping them away. I haven't spotted one yet since adding this.. but they seem to have popped on at random.

 

Is it possible that someone there might actually be interested in what you sell?

if it's a competitor, i really don't want them scanning my site though. :)

Link to comment
Share on other sites

my previous solution did not work.

 

spotted 1 today:

zermelo

ec2-67-202-41-191.compute-1.amazonaws.com

Mozilla/5.0 (compatible; zermelo; +http://www.powerset.com) [ema

IP Address: 67.202.41.191

User Agent: Mozilla/5.0 (compatible; zermelo; +http://www.powerset.com) [email:[email protected],[email protected]]

 

i am offically confused.

why do the majority of amazon bots spider my sites while ignoring my rewrite rules...

yet "paul" obeys them..

 

nevertheless...anyone feeling the need to buy space via amazon requests an immediate ban in my book. how can i efficiently ban anything from amazon's servers?

Link to comment
Share on other sites

thanks i will give this a try.

what portion of their bot details does this scan? i currently use something similar to block ips.

Link to comment
Share on other sites

My post would completely block all access to a website for amazonaws.com

 

There is no benefit to any webmaster in having that domain scan their website, so I'd recommend that everyone blocks it.

 

Vger

Link to comment
Share on other sites

  • 1 month later...

Hi, I receive also many visits from theses, and i don't know why either...

 

Number of Entries: 37

21st July 2008 16:07:26

Multiple visits spread over more than one day

Seattle Washington United States

ec2-67-202-54-208.compute-1.amazonaws.com (67.202.54.208)

 

 

Number of Entries: 9

22nd July 2008 19:31:46

Seattle Washington United States

ec2-67-202-54-149.compute-1.amazonaws.com (67.202.54.149)

 

and it's seems is does'nt load any page???

Date Time WebPage

21st July 2008 16:07:26 No referring link

21st July 2008 16:09:12 No referring link

21st July 2008 16:10:41 No referring link

21st July 2008 16:13:40 No referring link

21st July 2008 16:14:49 No referring link

21st July 2008 16:17:10 No referring link

21st July 2008 23:36:26 No referring link

22nd July 2008 08:09:17 No referring link

22nd July 2008 08:12:02 No referring link

22nd July 2008 08:14:31 No referring link

22nd July 2008 08:17:23 No referring link

22nd July 2008 08:19:29 No referring link

22nd July 2008 08:21:58 No referring link

22nd July 2008 08:24:42 No referring link

22nd July 2008 08:27:56 No referring link

22nd July 2008 08:31:15 No referring link

22nd July 2008 08:34:29 No referring link

22nd July 2008 08:37:18 No referring link

22nd July 2008 08:40:29 No referring link

22nd July 2008 08:44:06 No referring link

22nd July 2008 08:47:16 No referring link

22nd July 2008 16:51:03 No referring link

 

 

I also deny access to amazonaws.com in .htaccess ...

Link to comment
Share on other sites

I would not be so quick to try to ban the Amazon bots, as some of these might be tied into with Alexa and some of their other sites that will help boost your ranks. Or Amazon or any developer making an interface with Amazon web payment systems. You never know it could be a good thing.

 

I know Alexa uses it to thumbnail your site as well as to calculate the speed of your site, ranking and so forth.

Powered By osC 2.2RC2a STS 4.5.8 - HTC 2.6.3 - FP 1.5.9 - BCH 1.0.0

Link to comment
Share on other sites

I would not be so quick to try to ban the Amazon bots

 

We've banned amazonaws.com at root level on all of our servers. It is not a bot. Amazon are renting out their unused webspace and bandwidth to anyone who will pay to use their "cloud". The Amazon Web Service has, in our experience and the experience of others, been used by hackers to try and find exploits in websites and/or to recursively download complete backups of website files so that they can spoof the website and use it for pfishing.

 

It didn't happen on our servers, but someone that we know of lost 80 Gbs of bandwidth to the activities of one person using amazonaws.com during the course of just one day. It cost them an arm and a leg in extra bandwidth charges.

 

Yes, Amazon does own Alexa but they use a different set of servers - so there's no benefit to you in allowing amazonaws.com to access your website. Don't try to ban them via ip address as there are just too many of them. Ban them by domain name, as in:

 

amazonaws.com

*.amazonaws.com

 

Vger

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...