wwwclaes Posted September 9, 2003 Posted September 9, 2003 http://www.nutritionstore.se has been up for 6 months now. From time to time it indexes my default page. A couple of times it has indexed all of my pages. My default page is listed in Google and Altavista for maybe 5 months now, but *none* of the other pages. This is no good for a store such as this one. I have searched the forums but unfortunately there seems to a number of reasons and speculations why this can happen. I believe I have tried most of them... (external links to the store, Burt's SID killer, Meta Tags, AllProds...) Can anyone with more and better experiences with osC + Google maybe take a look at my site? Clue no. 1: MSN Search does index all my pages. Clue no. 2: My Google PageRank is 3/10. This seems bloody %&/# impossible (sorry about the language). It shouldn't really be that hard, I believe I have proper knowledge and it appears most people here haven't gone through these difficulties. 6 months, many workhours... and counting... :-( Thanks in advance for any help, Claes
wizardsandwars Posted September 9, 2003 Posted September 9, 2003 How long ago did you add the unique Meta Tags to each page? Usually after adding unique meta titles to each page, you cna expect to wait up to 3 months before being fully indexed. Honestly, the 3/10 does seem to be pretty low for having 1,497,024 inbound links accross the internet, such as you do. ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 I can't really remember when adding the Meta Tags contribution, but judging from the file timestamps - I'd say beginning of April. Also, I'm guessing most osC users haven't spent much time adding these type of contributions - and they still manage somehow. Either they get properly listed in searchs anyway or they simply don't know or care (which I doubt). All help truly appreciated, I'd really like to solve this Google trouble now once and for all...
wizardsandwars Posted September 9, 2003 Posted September 9, 2003 Actually, if an OSC user does NOT add the Unique Meta titles to their site, then each of the product listings will not be listed in Google. Your current listing is similar in symptoms. Do you have any IP addresses denied in your htaccess file? ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 No IP addresses denied. As mentioned, I have added functions to dynamically generate meta tags for each page. As is always the case... I may have done something wrong, but I hope and think not.
wizardsandwars Posted September 9, 2003 Posted September 9, 2003 No, your meta tags are fine. In fact, I don't see anything wrong with your pages at all. I ran it through a page similarity tester, and it seems to have a very low page similarity, which is good. I ran it through a spider simulator, and it did recognize your various different pages. You're not running an html redirect, and you are in your web root, both good. It seems to me that everythign looks fine. How have you tested that your SID killer is working? ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 First of all, sincere thanks for your efforts... About the SID killer, actually not quite. I did my best to double check that I indeed have coded everything as instructed. However, I failed to find out how to include any of the spider simulators to check that it really works. I believe this is the code that does it, included in html_output.php. (I removed the long list of spiders from the post). // Added by Claes. Add SID killer for spiders. Not the SID Killer modification, another solution from forum. // Add more Spiders as you find them. MAKE SURE THEY ARE LOWER CASE! //$spiders = array("LONG...", "LIST...", "REMOVED..."); // get useragent and force to lowercase just once $useragent = strtolower(getenv("HTTP_USER_AGENT")); foreach($spiders as $Val) { if (!(strpos($Val, $useragent) === false)) { // found a spider, kill the sid/sess // Edit out one of these as necessary depending upon your version of html_output.php // $sess = NULL; $sid = NULL; break; } } // End spider stopper code
wizardsandwars Posted September 9, 2003 Posted September 9, 2003 I think we'd need to see that long list. :D ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 Some more information just arrived... It just so happens, Google is currently indexing my site. (As I mentioned, this has happened a couple of times before, but has never resulted in results in Google search.) I am using User Tracking and this are some samples from Google's visit: 17:29:38: Chromic Fuel (200 kapslar) /product_info.php?products_id=77 17:28:45: Protein bar /product_info.phpcPath=8&products_id=209 17:26:59: Videos /default.php?cPath=12&page=1&sort=1a I guess this is proof enough that Google does not experience SIDs during its visit. At the login page, there is a SID (probably as expected): /login.php?osCsid=747737e517afd80dea496bacad71bc4a Damn, I guess, otherwise I would have known what to fix... Any more ideas? If I have been banned, I would have had other symptoms, don't you think?
wizardsandwars Posted September 9, 2003 Posted September 9, 2003 No, i'm sure you havn't been banned. However, you should nnot see a sid for this url /login.php?osCsid=747737e517afd80dea496bacad71bc4a Also, what is the ip of the bot spitering you? ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 Ok, I'll look into the login SID issue. But that shouldn't affect the indexing of all the other pages. You can never be too sure, but the odds of that fixing my problems must be near to nothing... The IPs are crawl31.googlebot.com and crawl34.googlebot.com.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 Well, since you actually asked for the IPs, I'll submit them as well... 64.68.88.*
wizardsandwars Posted September 9, 2003 Posted September 9, 2003 Yes, that would be the deep parsing google bot. Well, if you see it on your product pages, then I would expect that you would be included in the next index. ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
wwwclaes Posted September 9, 2003 Author Posted September 9, 2003 This deep parsing has happened so many times without results that I have given up hope :-( Anyway, I just discovered that Google gets a little crazy about the login.php and the SID. That *might* be the problem, Google revisiting that page several times during its work may disrupt the entire process? I have tried to fix the problem by adding a robots.txt file in my root dir as described in: http://www.oscommerce.com/forums/viewtopic.php...ight=robots+txt For now, I guess I'll just have to hold my breath for a couple of months more and see what happens... :-)
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.