Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

How to include Yahoo in my robot.txt file


Trentide

Recommended Posts

Hi thank you for looking at my post. Right now I'm blocking Google from indexing certain pages and want to do the same with Yahoo. I've been getting a lot of bots indexing my cookie_usage page. What a pain!

 

Does anyone know how to include Yahoo in my robot.txt file?

 

Thanks so much for your help!~

 

Jason

Link to comment
Share on other sites

In your Admin sessions section it is not set up properly

 

Force Cookie Use False

Check SSL Session ID False

Check User Agent False

Check IP Address False

Prevent Spider Sessions True

Recreate Session False

 

Your robots.txt file blocks pages such as your SSL pages but if the bots are hitting cookie useage page try the Sessions setting above. The bots you are referring to are in includes/spiders.txt file and can be updated from the contributions section.

Link to comment
Share on other sites

Thanks. I did have to change force cookie use to false. Let's see if that works.

 

Also, right now, I have:

 

User-agent: Googlebot-Image

 

at the top of my robot.txt file.

 

Shouldn't I just include a Yahoo user agent in the file to keep Yahoo bots from accessing certain files?

 

Thank you so much,

 

Jason

 

In your Admin sessions section it is not set up properly

 

Force Cookie Use False

Check SSL Session ID False

Check User Agent False

Check IP Address False

Prevent Spider Sessions True

Recreate Session False

 

Your robots.txt file blocks pages such as your SSL pages but if the bots are hitting cookie useage page try the Sessions setting above. The bots you are referring to are in includes/spiders.txt file and can be updated from the contributions section.

Link to comment
Share on other sites

I think now I'm getting a lot of visitors that go to pages with this type of URL"

 

html?filter_id=23&sort=2a&osCsid=40cc55ab0a021100ce0196ecd4ca02e8

 

Is that normal?

 

Jason

 

 

 

OK, thank you. I made the changes. I hope it works!

 

Jason

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...