Google Visiting my site - is it working?

[Guest]

Hi all,

 

I wanted some advice please on the visits I am having to my site of various bots. I've set up the Allprods contribution, my robots text file and as I have to redirect from www.emvi-htpc.com to www.emvi-htpc.com/store I;ve used the numerous suggestions on the various Google threads to use a file to direct humans to the mainpage and bots to the allprods page....but in my stats i have the following info on the length of bot visits, which seem a tad short to me:

 

This is from my cpanel (AWstats):

 

Robots/Spiders visitors (Top 25)  -  Full list  -  Last visit   

3 different robots            Hits      Bandwidth        Last visit

Alexa (IA Archiver)        69        1.91 MB            13 Jan 2004 - 23:30

Inktomi Slurp                  4        165.39 KB          14 Jan 2004 - 08:33

Googlebot (Google)          1        90.18 KB            14 Jan 2004 - 21:13

 

to me this seems a tad short IF it is going to my allprods and then finding all my products?

 

My robots text file is as follow:

 

# robots.txt file

# For domain: http://www.emvi-htpc.com

 

# All robots will spider the domain

#User-agent: *

#Disallow:

User-agent: *

Disallow: /admin/

Disallow: /phpMyAdmin

Disallow: /catalog/address_book_process.php

Disallow: /catalog/account.php

Disallow: /catalog/account_edit.php

Disallow: /catalog/account_edit_process.php

Disallow: /catalog/account_history.php

Disallow: /catalog/account_history_info.php

Disallow: /catalog/address_book.php

Disallow: /catalog/checkout_process.php

Disallow: /catalog/advanced_search.php

Disallow: /catalog/advanced_search_result.php

Disallow: /catalog/checkout_address.php

Disallow: /catalog/checkout_confirmation.php

Disallow: /catalog/checkout_payment.php

Disallow: /catalog/checkout_success.php

Disallow: /catalog/conditions.php

Disallow: /catalog/contact_us.php

Disallow: /catalog/create_account.php

Disallow: /catalog/create_account_process.php

Disallow: /catalog/create_account_success.php

Disallow: /catalog/download.php

Disallow: /catalog/info_shopping_cart.php

Disallow: /catalog/login.php

Disallow: /catalog/logoff.php

Disallow: /catalog/password_forgotten.php

Disallow: /catalog/popup_image.php

Disallow: /catalog/popup_search_help.php

Disallow: /catalog/privacy.php

Disallow: /catalog/products_new.php

Disallow: /catalog/product_notifications.php

Disallow: /catalog/product_reviews.php

Disallow: /catalog/product_reviews_info.php

Disallow: /catalog/product_reviews_write.php

Disallow: /catalog/redirect.php

Disallow: /catalog/reviews.php

Disallow: /catalog/shipping.php

Disallow: /catalog/shopping_cart.php

Disallow: /catalog/specials.php

Disallow: /catalog/tell_a_friend.php

Disallow: /catalog/disclaimer.php

Disallow: /catalog/admin/

Disallow: /catalog/download/

Disallow: /catalog/images/

Disallow: /catalog/includes/

Disallow: /catalog/pub/

 

and my index.php is:

 

<?php

// Quick script to 'redirect' spiders/robots to 'search engine friendly' page.

// By Dan Zambonini [[email protected]], Sep 2000. V 1.0

// Please send alterations/comments to [email protected]

// Modified from someone else's script, can't remember which one...Sorry!

// Major search engines match either $spider_footprint or $spider_ip.

 

$spider_footprint = array( "bot", "rawler", "pider",  "ppie", "rchitext", "aaland", "igout4u", "cho", "ferret", "ulliver", "arvest", "tdig", "rchiver", "eeves", "inkwalker", "ycos", "ercator", "uscatferret", "yweb", "omad", "eternews", "cooter", "lurp", "oila", "oyager", "ebbase", "eblayers", "get");

$spider_ip = array( "204.123.", "204.74.103.", "203.108.10.", "195.4.183.", "195.242.46.", "198.3.97.", "204.62.245.", "193.189.227.", "209.1.12.", "204.162.96.", "204.162.98.", "194.121.108.", "128.182.72.", "207.77.91.", "206.79.171.", "207.77.90.", "208.213.76.", "194.124.202.", "193.114.89.", "193.131.74.", "131.84.1.", "208.219.77.", "206.64.113.", "195.186.1.", "195.3.97.", "194.191.121.", "139.175.250.", "209.73.233.", "194.191.121.", "198.49.220.", "204.62.245.", "198.3.99.", "198.2.101.", "204.192.112.", "206.181.238", "208.215.47.", "171.64.75.", "204.162.98.", "204.162.96.", "204.123.9.52", "204.123.2.44", "204.74.103.39", "204.123.9.53", "204.62.245.", "206.64.113.", "204.138.115.", "94.22.130.", "164.195.64.1", "205.181.75.169", "129.170.24.57", "204.162.96.", "204.162.96.", "204.162.98.", "204.162.96.", "207.77.90.", "207.77.91.", "208.200.146.", "204.123.9.20", "204.138.115.", "209.1.32.", "209.1.12.", "192.216.46.49", "192.216.46.31", "192.216.46.30", "203.9.252.2", "209.185.253.167", "209.185.253.168", "209.185.253.169", "209.185.253.170", "209.185.253.171", "209.185.253.172", "209.185.253.173", "209.185.253.174", "209.185.253.175", "209.185.253.176", "209.185.253.177", "209.185.253.178", "209.185.253.179", "209.185.253.180", "209.185.253.181", "209.185.253.182", "209.185.253.183", "209.185.253.184", "209.185.253.185", "209.185.253.186", "209.185.253.187", "209.185.253.188", "209.185.108.138", "209.185.108.139", "209.185.108.140", "209.185.108.141", "209.185.108.142", "209.185.108.143", "209.185.108.144", "209.185.108.145", "209.185.108.146", "209.185.108.147", "209.185.108.148", "209.185.108.149", "209.185.108.150", "209.185.108.151", "209.185.108.152", "209.185.108.153", "209.185.108.154", "209.185.108.155", "209.185.108.156", "209.185.108.157", "209.185.108.158", "209.185.108.159", "209.185.108.160", "209.185.108.161", "209.185.108.162", "209.185.108.163", "209.185.108.164", "209.185.108.165", "209.185.108.134", "209.185.108.135", "64.209.181.52", "64.208.33.33", "64.209.181.53", "64.68.82.22", "64.68.82.23", "64.68.82.24", "64.68.82.25", "64.68.82.26", "64.68.82.27", "64.68.82.28", "64.68.82.29", "64.68.82.30", "64.68.82.1", "64.68.82.2", "64.68.82.3", "64.68.82.4", "64.68.82.5", "64.68.82.6", "64.68.82.7", "64.68.82.8", "64.68.82.9", "64.68.82.10", "64.68.82.11", "64.68.82.12", "64.68.82.13", "64.68.82.14", "64.68.82.15", "64.68.82.16", "64.68.82.17", "64.68.82.18", "64.68.82.19", "64.68.82.20", "64.68.82.21", "64.68.82.50", "64.68.82.51", "64.68.82.52", "64.68.82.53", "64.68.82.54", "64.68.82.55", "64.68.82.56", "64.68.82.57", "64.68.82.58", "64.68.82.59", "64.68.82.60", "64.68.82.31", "64.68.82.32", "64.68.82.33", "64.68.82.34", "64.68.82.35", "64.68.82.36", "64.68.82.37", "64.68.82.38", "64.68.82.39", "64.68.82.40", "64.68.82.41", "64.68.82.42", "64.68.82.43", "64.68.82.44", "64.68.82.45", "64.68.82.46", "64.68.82.47", "64.68.82.48", "64.68.82.49", "64.68.82.76", "64.68.82.77", "64.68.82.78", "64.68.82.79", "64.68.82.80", "64.68.82.61", "64.68.82.62", "64.68.82.63", "64.68.82.64", "64.68.82.65", "64.68.82.66", "64.68.82.67", "64.68.82.68", "64.68.82.69", "64.68.82.70", "64.68.82.71", "64.68.82.72", "64.68.82.73", "64.68.82.74", "64.68.82.75"); 

 

$agent = getenv('HTTP_USER_AGENT');

$host_ip = getenv('REMOTE_ADDR');

$is_spider = 0;

 

// Is it a spider?

 

$i = 0;

while ($i < (count($spider_footprint)))

{

if (strstr($agent, $spider_footprint[$i]))

{

$is_spider = 1;

break;

}

$i++;

}

 

if (! $is_spider)

{

$i = 0;

while ($i < (count($spider_ip)))

{

if (strstr($host_ip, $spider_ip[$i]))

{

$is_spider = 1;

break;

}

$i++;

}

}

 

// Re-direct to correct page

// Change the files below to your search-engine 'keyword' page and normal index page.

 

if ($is_spider)

{

readfile("http://www.emvi-htpc.com/store/allprods.php");

}

else

{

readfile("http://www.emvi-htpc.com/store/index.php");

}

?>

 

Can anyone please take a look at the above and suggest what I may be doing wrong??

 

I am at a loss here <_<

 

Thank all for your help in advance, i;m sure your help will be useful to plenty of other too..!!

 

TIA

Steve

[♥kymation]

Here's one possibility: "...setting up pages/links with the sole purpose of fooling search engines may result in permanent removal from our index." That's from Google. Search engines don't like sites that deliver different content to them than what their customers will see. I would suggest that you not do this. Link your All Products page from all of your regular pages and the search engines will find it.

 

In this case Google may have just not finished. It's common practice for Google to grab the first page and list all of the links, then come back later to search all of the links that it found. Since Google does monthly updates, you will need to check your logs for a month to determine what is is really searching.

 

Just being spidered doesn't mean your pages will rank well. I see your company name in all the titles, no H1 tags, "Let's See What We Have Here" for Category headings.... You need to put your keywords in all of these places if you want your store to be seen. Search this forum for more tips on search engine optimization; there's plenty of good information available.

 

Regards

Jim

[tammy507]

no H1 tags,

 

How is this done with php? , I know how to do it with html, but php is still foreign to me.

 

"Let's See What We Have Here" for Category headings.... You need to put your keywords in all of these places if you want your store to be seen.

Can you be a little more specific? How to change, which file? How can you put keywords in a php file?

 

I currently have Header_tags contribution installed , but dont have it completely filled out....... When I fill it all the way out, I get the exact same title on every page. Ive searched the forum, even under contribution support, but have yet to fins a thread that gives step by step instructions on how to use the header_tags contribution or how it effects search engines.

 

Thanks

tammy

[Guest]

Whoa! Talk about overkill. :o

 

I'd lose all of the hocus pocus crap and pray that Google hasn't already banned you. That redirect was a really really bad move. :rolleyes:

[tammy507]

If your referring to my sites redirect to the message board forum...... I search hereand on the net trying to find out how to add an url in the info box, with no luck. The redirect I used was the only one I could find to work. I tried typing the exact url, but it didnt work. If anyone has the answer, they arent sharing it.

 

Tammy

[Guest]

No, I'm talking about that spider redirect script you're using. Google will ban you in a heartbeat for that (and if Google bans you, you might as well close up shop).

[♥kymation]

If you are using a redirect that only works on spiders: don't. This is a very bad idea.

 

To answer your previous question about H1 tags, I modified the Dynamic Meta Tags contribution to get what I wanted. If you already have one of these contributions installed, just use it to provide the product name (or whatever you want to use as keywords.) Then just replace the "Let's See What We Have Here" tag with something like this:

<h1 class="pageHeading"><?php echo $the_product_name; ?></h1>

This is assuming that $the_product_name is defined by your particular Meta Tags contribution.

 

Then add something like this to your stylesheet:

H1.pageHeading {
 font-family: Verdana, Arial, sans-serif;
 font-size: 20px;
 font-weight: bold;
 vertical-align: top;
 color: #3c628c;
}

Modify as desired to make that ugly H1 tag look right on your page.

 

Please ask if you have any more questions.

 

Regards

Jim

[wizardsandwars]

If you are using a redirect that only works on spiders: don't. This is a very bad idea.

 

I couldn't agree more. Someone gave you some very very bad advice.

[sam6]

im not to sure about robot,txt i put one on my site and google goes away i get rid of the robot txt and i cant get rid of google and my robot txt was done properly go figure

[user99999999]

Bad Idea I Agree!

 

Instead of creating spider food you have created a mouse trap!