Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

How do you stop robots using search?


Guest

Recommended Posts

Posted

Hi,

 

I have a contribution installed with logs what "terms" people are searching for on my site. The number one search term on my site is "%%%" and it has been searched for a total of 40839 times, in a matter of a few months. These searches are obviously not done by a human user and they are a plain as they may be causing problems due to the excessive number of times it is being searched for.

 

My question is how do I stop it? I have Disallow: /advanced_search.php in my robots file but that doesn't seem to stop this particular problem, it may be using the quick search box I not sure. Any suggestions welcome.

Posted

I wouldn't be surprised if that "search" is a hack attempt, looking for a vulnerability in certain application search routines. I don't know if osC has any vulnerabilities here -- hackers throw everything at every site, hoping to get lucky. I can't imagine any legitimate search for '%%%'. Anyway, no hacker is going to pay any attention to the robots.txt file, except possibly to see what kind of interesting things are on your site (hint: don't use it to hide sensitive data -- use password controlled access, etc.). What to do about it? If it seems to be a common enough attack, I suppose you might trap it in either .htaccess (if it shows up as a URL Query String) or in the search routine, and ignore it or blow it off in some manner.

Posted

I wouldn't be surprised if that "search" is a hack attempt, looking for a vulnerability in certain application search routines. I don't know if osC has any vulnerabilities here -- hackers throw everything at every site, hoping to get lucky. I can't imagine any legitimate search for '%%%'. Anyway, no hacker is going to pay any attention to the robots.txt file, except possibly to see what kind of interesting things are on your site (hint: don't use it to hide sensitive data -- use password controlled access, etc.). What to do about it? If it seems to be a common enough attack, I suppose you might trap it in either .htaccess (if it shows up as a URL Query String) or in the search routine, and ignore it or blow it off in some manner.

 

Thanks for the response. How do I go about "trapping" it? I would really love to get rid of it.

Posted

Robots do not enter search terms into the search box. If you have links on your pages, like for example those that are created by the Search Tag Cloud contribution, they would look something like this:

 

http://www.mysite.com/advanced_search_results.php?keywords=%%%

 

That's the only way a spider can come to know about the search term. Entering that URL into your robots.txt will prevent them from accessing that URL.

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...