Jump to content
  • Checkout
  • Login
  • Get in touch

osCommerce

The e-commerce.

speeding things up with a cookie


boxtel

Recommended Posts

I am an active user of the spider list contribution because I do not like to force cookies.

 

currently the logic for that is:

 

if force cookies {

issue a test cookie

read the test cookie (next page load)

if exists {

start session

}

} elseif block spider sessions {

go over the spiders list

if not in list {

start session

}

} else {

start session

}

this means that if you do not force cookies, we go over the list on every page load.

 

However, we know that spiders do not accept cookies.

Since I set my own cookies for resolution and language etc which are always set. I thought, why not test to see if one of those cookies already exists at the client before all this and if it does, I know it cannot be a spider. Then I need not go over that very long list at all. Furthermore, I do not need to append the session id to the url's on the first page load, but that is another topic.

 

So now I use the following code logic:

 

check for my cookie

if force cookies or my cookie exists {

set test cookie

if test cookie exists or my cookie exists {

start session

}

} elseif block spider sessions {

go over the spiders list

if not in list {

start session

}

} else {

start session

}

 

so no more going over the spider list as long as I can read my cookies.

Treasurer MFC

Link to comment
Share on other sites

Thanks for the tip, can you post a code snippet for example?

 

// start the session

 

// check one of my own cookies or the test cookie

if ((isset($_COOKIE['res'])) or (isset($_COOKIE['cookie_test']))) {

$cookies_exist = true;

} else {

$cookies_exist = false;

}

 

// register value for later usage

if (!tep_session_is_registered('cookies_exist')) {

tep_session_register('cookies_exist');

}

 

$session_started = false;

if ((SESSION_FORCE_COOKIE_USE == 'True') or ($cookies_exist)) {

// they accept cookies, start session

if ($cookies_exist) {

tep_session_start();

$session_started = true;

} else {

// no cookie yet, issue a test one and do not start session

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

} elseif (SESSION_BLOCK_SPIDERS == 'True') {

// go over the spider list to see if this is one of them

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

 

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

 

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if ($spider_flag == false) {

tep_session_start();

$session_started = true;

}

} else {

tep_session_start();

$session_started = true;

}

Treasurer MFC

Link to comment
Share on other sites

// start the session

 

// check one of my own cookies or the test cookie

if ((isset($_COOKIE['res'])) or (isset($_COOKIE['cookie_test']))) {

$cookies_exist = true;

} else {

$cookies_exist = false;

}

 

// register value for later usage

if (!tep_session_is_registered('cookies_exist')) {

tep_session_register('cookies_exist');

}

 

$session_started = false;

if ((SESSION_FORCE_COOKIE_USE == 'True') or ($cookies_exist)) {

// they accept cookies, start session

if ($cookies_exist) {

tep_session_start();

$session_started = true;

} else {

// no cookie yet, issue a test one and do not start session

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

} elseif (SESSION_BLOCK_SPIDERS == 'True') {

// go over the spider list to see if this is one of them

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

 

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

 

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if ($spider_flag == false) {

tep_session_start();

$session_started = true;

}

} else {

tep_session_start();

$session_started = true;

}

 

 

 

 

this code may be even better, it also eliminates the BLOCK_SPIDERS setting as noone in their right mind would not choose to prevent spiders from getting sessions.

instead of the "res" cookie you can use any cookie you set yourself or simply only test on the "cookie_test" cookie.

 

// see if we can obtain a previously set cookie

if ((isset($_COOKIE['res'])) or (isset($_COOKIE['cookie_test']))) {

// cookie present so start the session

tep_session_start();

$session_started = true;

} else {

// try to set the test cookie

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

if (SESSION_FORCE_COOKIE_USE != 'true') {

// we do not force cookies and have none so check if spider

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

if (!$spider_flag) {

// no spider so start the session

tep_session_start();

$session_started = true;

}

}

}

Treasurer MFC

Link to comment
Share on other sites

this code may be even better, it also eliminates the BLOCK_SPIDERS setting as noone in their right mind would not choose to prevent spiders from getting sessions.

instead of the "res" cookie you can use any cookie you set yourself or simply only test on the "cookie_test" cookie.

 

// see if we can obtain a previously set cookie

if ((isset($_COOKIE['res'])) or (isset($_COOKIE['cookie_test']))) {

// cookie present so start the session

tep_session_start();

$session_started = true;

} else {

// try to set the test cookie

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

if (SESSION_FORCE_COOKIE_USE != 'true') {

// we do not force cookies and have none so check if spider

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

if (!$spider_flag) {

// no spider so start the session

tep_session_start();

$session_started = true;

}

}

}

Hi guys,

 

Sorry for a silly question :blush:, but where should this code go?

 

Thanks a lot.

Link to comment
Share on other sites

Hi guys,

 

Sorry for a silly question :blush:, but where should this code go?

 

Thanks a lot.

 

in application_top.php.

 

it replaces this code :

 

if (SESSION_FORCE_COOKIE_USE == 'True') {

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

 

if (isset($HTTP_COOKIE_VARS['cookie_test'])) {

tep_session_start();

$session_started = true;

}

} elseif (SESSION_BLOCK_SPIDERS == 'True') {

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

 

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

 

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if ($spider_flag == false) {

tep_session_start();

$session_started = true;

}

} else {

tep_session_start();

$session_started = true;

}

Treasurer MFC

Link to comment
Share on other sites

  • 3 weeks later...
I am an active user of the spider list contribution because I do not like to force cookies.

 

currently the logic for that is:

 

if force cookies {

issue a test cookie

read the test cookie (next page load)

if exists {

start session

}

} elseif block spider sessions {

go over the spiders list

if not in list {

start session

}

} else {

start session

}

this means that if you do not force cookies, we go over the list on every page load.

 

However, we know that spiders do not accept cookies.

Since I set my own cookies for resolution and language etc which are always set. I thought, why not test to see if one of those cookies already exists at the client before all this and if it does, I know it cannot be a spider. Then I need not go over that very long list at all. Furthermore, I do not need to append the session id to the url's on the first page load, but that is another topic.

 

So now I use the following code logic:

 

check for my cookie

if force cookies or my cookie exists {

set test cookie

if test cookie exists or my cookie exists {

start session

}

} elseif block spider sessions {

go over the spiders list

if not in list {

start session

}

} else {

start session

}

 

so no more going over the spider list as long as I can read my cookies.

 

you won't believe this but I actually have a spider which accepts cookies !

 

63.241.61.7

User Agent: converacrawler/0.9d (+http://www.authoritativeweb.com/crawl)

Treasurer MFC

Link to comment
Share on other sites

  • 5 months later...

ok so I implemented pretty much your solution for my store to speed things up, but now for some spiders that accept cookies.....

 

Normally when the code goes through the spiders.txt it will not start the session even when the spider accepts cookies. And therefore in the application_top.php this code:

 

	if ($session_started == false) {
  tep_redirect(tep_href_link(FILENAME_COOKIE_USAGE));
}

 

will take effect when the _GET 'action' switch is on and it will redirect. But because of the cookie check code, this will now fall through the action switch, so these spiders will be able to add items to the cart.

 

What I was thinking to do was the first time to check the spiders.txt and if a spider is found then I can send a different cookie name. Lets say spider_cookie. Now when the cookie is tested in application top it could be:

 

  if( isset($_COOKIE['spider_cookie']) ) {
// dont start session and dont check spiders.txt
 } elseif( !isset($_COOKIE[tep_session_name()]) ) {
// do default osc operation ie check spiders.txt and start session if necessary
 } else {
//  dont check spiders.txt a regular user is present
 }

 

so the buy now buttons will still redirect to the cookies_usage page.

Link to comment
Share on other sites

ok so I implemented pretty much your solution for my store to speed things up, but now for some spiders that accept cookies.....

 

Normally when the code goes through the spiders.txt it will not start the session even when the spider accepts cookies. And therefore in the application_top.php this code:

 

	if ($session_started == false) {
  tep_redirect(tep_href_link(FILENAME_COOKIE_USAGE));
}

 

will take effect when the _GET 'action' switch is on and it will redirect. But because of the cookie check code, this will now fall through the action switch, so these spiders will be able to add items to the cart.

 

What I was thinking to do was the first time to check the spiders.txt and if a spider is found then I can send a different cookie name. Lets say spider_cookie. Now when the cookie is tested in application top it could be:

 

  if( isset($_COOKIE['spider_cookie']) ) {
// dont start session and dont check spiders.txt
 } elseif( !isset($_COOKIE[tep_session_name()]) ) {
// do default osc operation ie check spiders.txt and start session if necessary
 } else {
//  dont check spiders.txt a regular user is present
 }

 

so the buy now buttons will still redirect to the cookies_usage page.

 

That is a very good solution for that, I personally have moved all cart actions into forms (I think) but this should work.

Treasurer MFC

Link to comment
Share on other sites

That is a very good solution for that, I personally have moved all cart actions into forms (I think) but this should work.

 

I now made it to this which follows your design on the spiders cookie.

 

// try to set a test cookie at all times

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

if (isset($_COOKIE['cookie_test'])) {

// test cookie present

$cookies_exist = true;

if (isset($_COOKIE['spider'])) {

// spider with spider cookie - write to error log to keep an eye on it

error_log('spider with cookie: '.$_COOKIE['cookie_test'].'-'.$_COOKIE['spider']."\n".'Agent:'.$user_agent."\n".'ip: '.$browser_ip."\n");

$spider_flag = true;

} else {

// normal user with cookies - start the session

tep_session_start();

$session_started = true;

}

} else {

// evaluate the spiders agent list

require(DIR_WS_RAM . 'spider_check.php');

if (!$spider_flag) {

// not a spider so start the session

tep_session_start();

$session_started = true;

} else {

// identified spider - try to set a spider cookie for future identification

tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

}

Treasurer MFC

Link to comment
Share on other sites

If you can believe it I found this thread because I was looking to solve a problem with my session regeneration code.

 

The problem I was having being on a shared ssl, when I called the regeneration code it will create a different session id (I did change my regen. code to create a brand new session alltogether) but the cookie on the 'NONSSL' server was still set to the old values. As you see in my case a visitor has 2 separate cookies stored if everything is set correctly.

 

1. SSL domain cookie

2. NONSSL domain cookie.

 

So during the transitions between SSL/NONSSL the session id stored in the cookies was different but the session id on the _GET, _POST arrays was the same. Once in the ssl domain I had no way to update the cookie of the nonssl domain. So to solve this problem and another bug of the osc (at least I do not understand its purpose) see this code from the original application_top.php

 

// set the session ID if it exists
  if (isset($HTTP_POST_VARS[tep_session_name()])) {
 tep_session_id($HTTP_POST_VARS[tep_session_name()]);
  } elseif ( ($request_type == 'SSL') && isset($HTTP_GET_VARS[tep_session_name()]) ) {
 tep_session_id($HTTP_GET_VARS[tep_session_name()]);
  }

 

Now this code goes and sets a session id without even checking if a cookie already sent. In other words as I see it on secure pages it maintains the session id with the links regardless if a session cookie passes (It doesn't even know if the session id of the cookie and the session id passed from the links are the same). Not a good thing. So for regular operation this code needs to be changed so if the cookie is set you do not set the session id.

 

Of course with the session regeneration I had to reverse the operation and unset the cookie if it is set and resend a new one with the new session id (in case it was different). Doing that solved my problem at least.

 

Anyways thanks for that, because there other things are speed up in a store when a cookie is present.

Link to comment
Share on other sites

  • 2 weeks later...
I now made it to this which follows your design on the spiders cookie.

 

// try to set a test cookie at all times

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

if (isset($_COOKIE['cookie_test'])) {

// test cookie present

$cookies_exist = true;

if (isset($_COOKIE['spider'])) {

// spider with spider cookie - write to error log to keep an eye on it

error_log('spider with cookie: '.$_COOKIE['cookie_test'].'-'.$_COOKIE['spider']."\n".'Agent:'.$user_agent."\n".'ip: '.$browser_ip."\n");

$spider_flag = true;

} else {

// normal user with cookies - start the session

tep_session_start();

$session_started = true;

}

} else {

// evaluate the spiders agent list

require(DIR_WS_RAM . 'spider_check.php');

if (!$spider_flag) {

// not a spider so start the session

tep_session_start();

$session_started = true;

} else {

// identified spider - try to set a spider cookie for future identification

tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

}

Hi Amanda

 

Does this relace the previous or is it additional?

Link to comment
Share on other sites

Hi Amanda

 

Does this relace the previous or is it additional?

 

 

you would have this:

 

// start the session

$session_started = false;

if (SESSION_FORCE_COOKIE_USE == 'True') {

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

 

if (isset($HTTP_COOKIE_VARS['cookie_test'])) {

tep_session_start();

$session_started = true;

}

} elseif (SESSION_BLOCK_SPIDERS == 'True') {

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

 

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

 

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if ($spider_flag == false) {

tep_session_start();

$session_started = true;

}

} else {

tep_session_start();

$session_started = true;

}

 

 

replace it with this:

 

$spider_flag = false;

if (isset($_COOKIE['cookie_test'])) {

// test cookie present

$cookies_exist = true;

if (isset($_COOKIE['spider'])) {

// spider with spider cookie

$spider_flag = true;

} else {

// normal user with cookies - start the session

tep_session_start();

$session_started = true;

// renew the existing cookie

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

} else {

// no cookie present (yet) - evaluate the spiders agent list

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if (!$spider_flag) {

// not a spider so start the session

tep_session_start();

$session_started = true;

// try to set the cookie for later

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

} else {

// identified spider - try to set a spider cookie for future identification

tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

}

Treasurer MFC

Link to comment
Share on other sites

you would have this:

 

// start the session

$session_started = false;

if (SESSION_FORCE_COOKIE_USE == 'True') {

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

 

if (isset($HTTP_COOKIE_VARS['cookie_test'])) {

tep_session_start();

$session_started = true;

}

} elseif (SESSION_BLOCK_SPIDERS == 'True') {

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

 

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

 

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if ($spider_flag == false) {

tep_session_start();

$session_started = true;

}

} else {

tep_session_start();

$session_started = true;

}

replace it with this:

 

$spider_flag = false;

if (isset($_COOKIE['cookie_test'])) {

// test cookie present

$cookies_exist = true;

if (isset($_COOKIE['spider'])) {

// spider with spider cookie

$spider_flag = true;

} else {

// normal user with cookies - start the session

tep_session_start();

$session_started = true;

// renew the existing cookie

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

} else {

// no cookie present (yet) - evaluate the spiders agent list

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if (!$spider_flag) {

// not a spider so start the session

tep_session_start();

$session_started = true;

// try to set the cookie for later

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

} else {

// identified spider - try to set a spider cookie for future identification

tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

}

Thanks Amanda

You are the best.

Link to comment
Share on other sites

  • 2 weeks later...
you would have this:

 

// start the session

$session_started = false;

if (SESSION_FORCE_COOKIE_USE == 'True') {

tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);

 

if (isset($HTTP_COOKIE_VARS['cookie_test'])) {

tep_session_start();

$session_started = true;

}

} elseif (SESSION_BLOCK_SPIDERS == 'True') {

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

$spider_flag = false;

 

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

 

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if ($spider_flag == false) {

tep_session_start();

$session_started = true;

}

} else {

tep_session_start();

$session_started = true;

}

replace it with this:

 

$spider_flag = false;

if (isset($_COOKIE['cookie_test'])) {

// test cookie present

$cookies_exist = true;

if (isset($_COOKIE['spider'])) {

// spider with spider cookie

$spider_flag = true;

} else {

// normal user with cookies - start the session

tep_session_start();

$session_started = true;

// renew the existing cookie

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

} else {

// no cookie present (yet) - evaluate the spiders agent list

$user_agent = strtolower(getenv('HTTP_USER_AGENT'));

if (tep_not_null($user_agent)) {

$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {

if (tep_not_null($spiders[$i])) {

if (is_integer(strpos($user_agent, trim($spiders[$i])))) {

$spider_flag = true;

break;

}

}

}

}

 

if (!$spider_flag) {

// not a spider so start the session

tep_session_start();

$session_started = true;

// try to set the cookie for later

tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);

} else {

// identified spider - try to set a spider cookie for future identification

tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);

}

}

Hi Amanda

 

I have "Force cookie useage" & "Prevent Spider sessions" set to TRUE & I have the latest "Spiders list" too, so do I just do the above amendment, or is there anything else I need to do? Or do I not need to do this? :blush: This is slightly above my head understanding what this is all about but I am learning. :thumbsup:

 

Thanks

Julie

Link to comment
Share on other sites

Hi Amanda

 

I have "Force cookie useage" & "Prevent Spider sessions" set to TRUE & I have the latest "Spiders list" too, so do I just do the above amendment, or is there anything else I need to do? Or do I not need to do this? :blush: This is slightly above my head understanding what this is all about but I am learning. :thumbsup:

 

Thanks

Julie

 

in a default install, prevent spider sessions is not relevant if you force cookies. The code for it is simply not executed as force cookies means that the client must accept your cookie or no session is started.

As 99.9% of spiders do not accept cookies, you need not check whether it is a spider or not.

 

However, force cookies is a bad idea, certainly for clients with IE which has various security settings which can at any time prevent your cookies from being accepted and as such your users will be referred to a cookie usage page, think of not having a compact cookie privacy policy for instance.

 

Many people I know do not even know what a cookie is and despite the good efforts of sites to explain the concept in simple language to them it sounds complex and thus scary, the fact that you have to explain some technical feature to your customers is just plain bad and people will just go away, even I would as I am not going to mess around with my browser security settings while browsing/shopping around (unless I really really want what I am looking at).

 

so the replacement code as stated above removes the cookie usage dependancy in the spider prevention code. I other words it always tries to determine whether it is a spider unless we can read a cookie from them and assume it is not a spider, a session is started always unless a spider is identified.

Treasurer MFC

Link to comment
Share on other sites

in a default install, prevent spider sessions is not relevant if you force cookies. The code for it is simply not executed as force cookies means that the client must accept your cookie or no session is started.

As 99.9% of spiders do not accept cookies, you need not check whether it is a spider or not.

 

However, force cookies is a bad idea, certainly for clients with IE which has various security settings which can at any time prevent your cookies from being accepted and as such your users will be referred to a cookie usage page, think of not having a compact cookie privacy policy for instance.

 

Many people I know do not even know what a cookie is and despite the good efforts of sites to explain the concept in simple language to them it sounds complex and thus scary, the fact that you have to explain some technical feature to your customers is just plain bad and people will just go away, even I would as I am not going to mess around with my browser security settings while browsing/shopping around (unless I really really want what I am looking at).

 

so the replacement code as stated above removes the cookie usage dependancy in the spider prevention code. I other words it always tries to determine whether it is a spider unless we can read a cookie from them and assume it is not a spider, a session is started always unless a spider is identified.

Thanks Amanda

I am one of those customers who do not understand cookies hence my question. :blush: Still not completely sure, but I think what you are saying for the best option is:

 

Put "Force cookies" to False

Keep "Prevent Spider sessions" to True

& replace my code in application_top.php with this code. :blink:

 

$spider_flag = false;
if (isset($_COOKIE['cookie_test'])) {
// test cookie present
$cookies_exist = true;
if (isset($_COOKIE['spider'])) { 
// spider with spider cookie
$spider_flag = true;
} else {
// normal user with cookies - start the session
tep_session_start();
$session_started = true;
// renew the existing cookie
tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);
}
} else {
// no cookie present (yet) - evaluate the spiders agent list
$user_agent = strtolower(getenv('HTTP_USER_AGENT'));
if (tep_not_null($user_agent)) {
$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');
for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {
if (tep_not_null($spiders[$i])) {
if (is_integer(strpos($user_agent, trim($spiders[$i])))) {
$spider_flag = true;
break;
}
}
}
}

if (!$spider_flag) {
// not a spider so start the session
tep_session_start();
$session_started = true;
// try to set the cookie for later
tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);
} else {
// identified spider - try to set a spider cookie for future identification
tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);
}
}

 

And then all is OK? :D

 

Thanks for helping me.

Julie

Link to comment
Share on other sites

Thanks Amanda

I am one of those customers who do not understand cookies hence my question. :blush: Still not completely sure, but I think what you are saying for the best option is:

 

Put "Force cookies" to False

Keep "Prevent Spider sessions" to True

& replace my code in application_top.php with this code. :blink:

 

$spider_flag = false;
if (isset($_COOKIE['cookie_test'])) {
// test cookie present
$cookies_exist = true;
if (isset($_COOKIE['spider'])) { 
// spider with spider cookie
$spider_flag = true;
} else {
// normal user with cookies - start the session
tep_session_start();
$session_started = true;
// renew the existing cookie
tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);
}
} else {
// no cookie present (yet) - evaluate the spiders agent list
$user_agent = strtolower(getenv('HTTP_USER_AGENT'));
if (tep_not_null($user_agent)) {
$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');
for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {
if (tep_not_null($spiders[$i])) {
if (is_integer(strpos($user_agent, trim($spiders[$i])))) {
$spider_flag = true;
break;
}
}
}
}

if (!$spider_flag) {
// not a spider so start the session
tep_session_start();
$session_started = true;
// try to set the cookie for later
tep_setcookie('cookie_test', 'ThankYou', time()+60*60*24*30, $cookie_path, $cookie_domain);
} else {
// identified spider - try to set a spider cookie for future identification
tep_setcookie('spider', 'gotya', time()+60*60*24*30, $cookie_path, $cookie_domain);
}
}

 

And then all is OK? :D

 

Thanks for helping me.

Julie

 

correct.

normally if you force cookies (which basically means that no session is started unless cookies are accepted so that osc can put the session id in such a cookie) then the prevent spiders sessions is irrelevant as we assume they do not accept cookies anyway and as such for them no session is ever started which is good.

 

But it also means that if a customer has the browser privacy setting a little too high or has a certain proxy setting and as such also does not allow cookies to be set, they will also get no session and then they get the cookie usage page when they try an action like add to cart. But in that case all session based logic also ceases to work like language and currency changes and all other session based variables from the various contributions and there is no warning for that behaviour.

 

So I would never force cookies and always prevent spider sessions.

the problem with the original code is that with this setting osc is going over the spiders list on every page load even if cookies are accepted and by knowing that should know that it cannot be a spider. Needless performance degration.

 

This code change will always set (renew) a cookie and if it can read the cookie knows that it cannot be a spider and does not go over the spider list. It will only use the spider list as normal if no cookies can be read because then we cannot be sure whether it is a customer who refuses cookies or a spider.

Treasurer MFC

Link to comment
Share on other sites

correct.

normally if you force cookies (which basically means that no session is started unless cookies are accepted so that osc can put the session id in such a cookie) then the prevent spiders sessions is irrelevant as we assume they do not accept cookies anyway and as such for them no session is ever started which is good.

 

But it also means that if a customer has the browser privacy setting a little too high or has a certain proxy setting and as such also does not allow cookies to be set, they will also get no session and then they get the cookie usage page when they try an action like add to cart. But in that case all session based logic also ceases to work like language and currency changes and all other session based variables from the various contributions and there is no warning for that behaviour.

 

So I would never force cookies and always prevent spider sessions.

the problem with the original code is that with this setting osc is going over the spiders list on every page load even if cookies are accepted and by knowing that should know that it cannot be a spider. Needless performance degration.

 

This code change will always set (renew) a cookie and if it can read the cookie knows that it cannot be a spider and does not go over the spider list. It will only use the spider list as normal if no cookies can be read because then we cannot be sure whether it is a customer who refuses cookies or a spider.

Thank you for taking the time to explain that to me Amanda. I understand a bit more now, & will make the changes today. :thumbsup:

 

Julie

Link to comment
Share on other sites

  • 2 weeks later...

Believe it or not I was looking for how to make an info box an individual so I could change its colour - I got a bit sidetracked but wow.

 

This code is great - thanks a lot guys an gals :thumbsup:

Link to comment
Share on other sites

Hello everyone I dont want to be a bother but after reading this thread I found myself a little confused on to witch code or part of code to use or not to use. Blow is the code that I have coppied and using I just want to verify that this is infact right or not.

 

 

// start the session
/ check one of my own cookies or the test cookie
if ((isset($_COOKIE['res'])) or (isset($_COOKIE['cookie_test']))) {
$cookies_exist = true;
} else {
$cookies_exist = false;
}

// register value for later usage
if (!tep_session_is_registered('cookies_exist')) {
tep_session_register('cookies_exist');
}

$session_started = false;
if ((SESSION_FORCE_COOKIE_USE == 'True') or ($cookies_exist)) {
// they accept cookies, start session
if ($cookies_exist) {
tep_session_start();
$session_started = true;
} else {
// no cookie yet, issue a test one and do not start session
tep_setcookie('cookie_test', 'please_accept_for_session', time()+60*60*24*30, $cookie_path, $cookie_domain);
}
} elseif (SESSION_BLOCK_SPIDERS == 'True') {
// go over the spider list to see if this is one of them
$user_agent = strtolower(getenv('HTTP_USER_AGENT'));
$spider_flag = false;

if (tep_not_null($user_agent)) {
$spiders = file(DIR_WS_INCLUDES . 'spiders.txt');

for ($i=0, $n=sizeof($spiders); $i<$n; $i++) {
if (tep_not_null($spiders[$i])) {
if (is_integer(strpos($user_agent, trim($spiders[$i])))) {
$spider_flag = true;
break;
}
}
}
}

if ($spider_flag == false) {
tep_session_start();
$session_started = true;
}
} else {
tep_session_start();
$session_started = true;
}

 

Thank you for your help

Bteck

Link to comment
Share on other sites

  • 2 months later...

I've just added this to my site as it seems ideal not to have to force cookies.

 

However, I've run a few spider simulators across my homepage to test it and they all show that the oscid is present in the URL.

 

Are these a reliable test ? I've used a range including seochat.com and webconfs.com

 

My homepage is www.travelbagsize.com/shop/ and my session settings are:

 

Session Directory /tmp

Force Cookie Use False

Check SSL Session ID True

Check User Agent False

Check IP Address False

Prevent Spider Sessions True

Recreate Session True

 

Regards

 

Bob

Link to comment
Share on other sites

I've just added this to my site as it seems ideal not to have to force cookies.

 

However, I've run a few spider simulators across my homepage to test it and they all show that the oscid is present in the URL.

 

Are these a reliable test ? I've used a range including seochat.com and webconfs.com

 

My homepage is www.travelbagsize.com/shop/ and my session settings are:

 

Session Directory /tmp

Force Cookie Use False

Check SSL Session ID True

Check User Agent False

Check IP Address False

Prevent Spider Sessions True

Recreate Session True

 

Regards

 

Bob

 

 

no that is not a reliable test.

 

seochat spider simulator uses useragent

 

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) SEOChat::Bot v1.1

 

does your spiders.txt file cover that one?

Treasurer MFC

Link to comment
Share on other sites

no that is not a reliable test.

 

seochat spider simulator uses useragent

 

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) SEOChat::Bot v1.1

 

does your spiders.txt file cover that one?

 

Hi Amanda,

 

Thanks for that.

 

I've got the latest spiders.txt (2006-09-09) installed and cannot see it in there. Do I just add the line above and run the test again or is there another test you could recommend please.

 

Bob

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...