Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

bots and cookie_usage.php


tictric

Recommended Posts

I realise that googlebot and msnbot in particular are indexing cookie_usage.php like mad. At least 20 out of 100 hits call that page.

 

And it's useless. All the more as I don't force cookies.

 

Anybody knows of some way to make it as little attractive to them bots as any page with real content?

 

--tictric

Link to comment
Share on other sites

I realise that googlebot and msnbot in particular are indexing cookie_usage.php like mad. At least 20 out of 100 hits call that page.

 

And it's useless. All the more as I don't force cookies.

 

Anybody knows of some way to make it as little attractive to them bots as any page with real content?

 

--tictric

Unfortunately, this is an all too common problem lately and there doesn't seem to be a fix for it. I have had about 7000 hits on the cookie_usage page by google in the last few weeks alone. The robots file is not being followed for this one file for some reason.

 

Jack

Support Links:

For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc.

All of My Addons

Get the latest versions of my addons

Recommended SEO Addons

Link to comment
Share on other sites

I think that what's is happening here is that if you use 'Prevent Spider Sessions' and Google is in your list in spiders.txt, that Google ends up, whenever it tries to create a session, on the Cookie Usage page. That would explain the 'Google Dance' with the Cookie Usage page.

 

Vger

Link to comment
Share on other sites

I think that what's is happening here is that if you use 'Prevent Spider Sessions' and Google is in your list in spiders.txt, that Google ends up, whenever it tries to create a session, on the Cookie Usage page.  That would explain the 'Google Dance' with the Cookie Usage page.

 

That means we need a patch that doesn't lead a bot to cookie_usage.php if it's listed in spiders.txt

 

Anybody? I might sponsor it or raise a fund :) , depending on the effort it takes.

Or is it already being integrated into ms3?

 

--tictric

Link to comment
Share on other sites

No - you need to keep bots off pages that require a session, such as login, reviews, cart, etc. If you use Buy Now, you have to disable this link if there is no session - I posted code for that recently (will look it up when I get the chance.)

 

That the bots are getting to cookie_usage means that Prevent Spider Sessions is working - but you need a good robots.txt (there is a contrib for that) to keep the bots off pages they don't need to visit.

 

Note that msnbot is not in the MS2 spiders.txt. I have a contrib where i maintain an updated one.

Link to comment
Share on other sites

The problem I am having is that even though I have everything set up as it should be, google is still going to cookie_usage. I added an SSL certificate a few months ago and I think that is when the problem started. I have been trying to find a way to stop google from going there but no luck so far. But I am open for any suggestions since I am getting desperate at this point.

 

Jack

Support Links:

For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc.

All of My Addons

Get the latest versions of my addons

Recommended SEO Addons

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...