Google XML Sitemap SEO

[Jack_mcs]

It looks like it can't find one of the sitemap files but you didn't post the diagnostic results as requested so it I can't be sure.

[lovesdonut]

It looks like it can't find one of the sitemap files but you didn't post the diagnostic results as requested so it I can't be sure.

 

I'm sorry Jack, where do I get the diagnostic result?

[Jack_mcs]

I'm sorry Jack, where do I get the diagnostic result?

Enable the option in the settings in admin and try to create the sitemaps.

[lovesdonut]

Enable the option in the settings in admin and try to create the sitemaps.

 

That's what I did. I set the diagnostic option true and then run http://YOUR_DOMAIN/googlesitemap/index.php. The message I posted above is the result. I just changed moo.myname into moo.xxxxx. Is there something else I can do to identify the problem?

[Jack_mcs]

That's what I did. I set the diagnostic option true and then run http://YOUR_DOMAIN/googlesitemap/index.php. The message I posted above is the result. I just changed moo.myname into moo.xxxxx. Is there something else I can do to identify the problem?

Then something is wrong with your installation since the diagnostic should display more data than what you posted.

[Guest]

I'd be interested in knowing how google handles all of the maps - whether they accept all of the links or not. On a site, two links to the same product with the language set can be considered duplicate pages. If they see that as the case in the maps, they will refuse some of the entries. If they accept them, then I would expect the total number of accepted links per language is the same for all languages.

 

Hi Jack,

 

Long time ago.

As how oscommerce works now with languages there are to many pages (when you click on the flags) who leades to the same page (e.g. page-lang1?language=en, page-lang2?language=en etc.). Google looks also to the contents of a page (domain independed) and when there are too many pages with the same contents they all get a low rank, you have to say to them who's is most important with a canonial link. But because these pages are virtual (seo names) you can't apply such canonial link.

Besides the other pages (e.g. productslist) are all pointing to the default language because there is no session when google visits the website (if you switch that off in the admin settings to prevent the session_id parameter in the url's). So all that pages in the different languages also point to the same contents (default language).

So I removed my language parameter in my solution for generating sitemaps for more languages (see before).

In stead of I changed it like that so all pages are unique in name and contents. E.g. languge en : page-ep-, language ge : page-gp- etc. etc. and changed the .htacces file to apply the language parameter, so e.g. page-ep- will change to products_info.php?language=en etc. So every seo name is an unique page by name (no language parameters) and also by contents, there are no 2 product, category or manufacturer pages anymore leading to the same content.

This brings also with it that you have to use different languages in one page for your language flags url's (seo names in different languages in 1 page).

I changed this in header.php and seo_class.php that I build all my names for every language flag after each other by creating the seo_class object every time again (so the constructor will be executed every time again). This works now too.

I got no indexed pages for some languages before, but now I get them slowly for all languages ...

Edited February 20, 2010 by Felix Scheiffers

[Guest]

Hi Jack (and others interested of course),

 

Here is some code I changed to give the idea (so in this example 3 languages nl, en and de) .

 

1) The changed code for generating a sitemap based on your contribution, you may find it some pages back in this forum (or check in my profile to find it quicker), but I removed the language parameters like .html?language=en etc.

 

2) Changed code in .htacces :

 

RewriteRule ^(.*)-p-(.*).html$ product_info.php?language=nl&products_id=$2&%{QUERY_STRING}

RewriteRule ^(.*)-ep-(.*).html$ product_info.php?language=en&products_id=$2&%{QUERY_STRING}

RewriteRule ^(.*)-dp-(.*).html$ product_info.php?language=de&products_id=$2&%{QUERY_STRING}

RewriteRule ^(.*)-c-(.*).html$ index.php?language=nl&cPath=$2&%{QUERY_STRING}

RewriteRule ^(.*)-ec-(.*).html$ index.php?language=en&cPath=$2&%{QUERY_STRING}

RewriteRule ^(.*)-dc-(.*).html$ index.php?language=de&cPath=$2&%{QUERY_STRING}

RewriteRule ^(.*)-m-([0-9]+).html$ index.php?language=nl&manufacturers_id=$2&%{QUERY_STRING}

RewriteRule ^(.*)-em-([0-9]+).html$ index.php?language=en&manufacturers_id=$2&%{QUERY_STRING}

RewriteRule ^(.*)-dm-([0-9]+).html$ index.php?language=de&manufacturers_id=$2&%{QUERY_STRING}

 

3) Changed code in header.php (but could be in your case in /boxes/languages.php), in my case I have the language flags in the header of the page :

 

if (!isset($lng) || (isset($lng) && !is_object($lng))) {

include(DIR_WS_CLASSES . 'language.php');

$lng = new language;

}

 

$languages_string = '';

reset($lng->catalog_languages);

$i = 1;

while (list($key, $value) = each($lng->catalog_languages)) {

if (($PHP_SELF == '/product_info.php') or (($PHP_SELF == '/index.php') and (isset($_GET['cPath']))) or (($PHP_SELF == '/index.php') and (isset($_GET['manufacturers_id'])))) {

$save_languages_id = $languages_id;

if ($i == 1) {

$languages_id = 4;

unset($seo_urls); // initiate new class for language (now)

} elseif ($i == 2) {

$languages_id = 5;

unset($seo_urls); // initiate new class for language (now)

} elseif ($i == 3) {

$languages_id = 6;

unset($seo_urls); // initiate new class for language (now)

}

$languages_string .= ' <a href="' . tep_href_link(basename($PHP_SELF), tep_get_all_get_params(array('language', 'currency')), $request_type) . '">' . tep_image(DIR_WS_LANGUAGES . $value['directory'] . '/images/' . $value['image'], $value['name']) . '</a> ';

$languages_id = $save_languages_id;

unset($seo_urls); // initiate new class for language (next time)

} else {

$languages_string .= ' <a href="' . tep_href_link(basename($PHP_SELF), tep_get_all_get_params(array('language', 'currency')) . 'language=' . $key, $request_type) . '">' . tep_image(DIR_WS_LANGUAGES . $value['directory'] . '/images/' . $value['image'], $value['name']) . '</a> ';

}

$i++;

}

 

3) Changed code in seo_class.php (here only for the products pages, same for manufacturer and category pages):

 

function make_url($page, $string, $anchor_type, $id, $extension = '.html', &$separator){

// Right now there is but one rewrite method since cName was dropped

// In the future there will be additional methods here in the switch

switch ( $this->attributes['SEO_REWRITE_TYPE'] ){

case 'Rewrite':

// FS start

if ($anchor_type == 'products_id') {

if ($this->languages_id == 4) {

return $string . '-p-' . $id . $extension;

} elseif ($this->languages_id == 5) {

return $string . '-ep-' . $id . $extension;

} elseif ($this->languages_id == 6) {

return $string . '-dp-' . $id . $extension;

}

}

if ($anchor_type == 'cPath') {

if ($this->languages_id == 4) {

return $string . '-c-' . $id . $extension;

} elseif ($this->languages_id == 5) {

return $string . '-ec-' . $id . $extension;

} elseif ($this->languages_id == 6) {

return $string . '-dc-' . $id . $extension;

}

}

if ($anchor_type == 'manufacturers_id') {

if ($this->languages_id == 4) {

return $string . '-m-' . $id . $extension;

} elseif ($this->languages_id == 5) {

return $string . '-em-' . $id . $extension;

} elseif ($this->languages_id == 6) {

return $string . '-dm-' . $id . $extension;

}

}

// FS stop

return $string . $this->reg_anchors[$anchor_type] . $id . $extension;

break;

default:

break;

} # end switch

} # end function

 

function get_product_name($pID){

// FS start stop - [(int)$this->languages_id] added (constants per language)

// - . (int)$this->languages_id . '_'

switch(true){

case ($this->attributes['USE_SEO_CACHE_GLOBAL'] == 'true' && defined('PRODUCT_NAME_' . (int)$this->languages_id . '_' . $pID)):

$this->performance['CACHE_QUERY_SAVINGS']++;

$return = constant('PRODUCT_NAME_' . (int)$this->languages_id . '_' . $pID);

$this->cache['PRODUCTS'][$pID][(int)$this->languages_id] = $return;

break;

case ($this->attributes['USE_SEO_CACHE_GLOBAL'] == 'true' && isset($this->cache['PRODUCTS'][$pID][(int)$this->languages_id])):

$this->performance['CACHE_QUERY_SAVINGS']++;

$return = $this->cache['PRODUCTS'][$pID][(int)$this->languages_id];

break;

default:

$this->performance['NUMBER_QUERIES']++;

$sql = "SELECT products_name as pName

FROM ".TABLE_PRODUCTS_DESCRIPTION."

WHERE products_id='".(int)$pID."'

AND language_id='".(int)$this->languages_id."'

LIMIT 1";

$result = $this->DB->FetchArray( $this->DB->Query( $sql ) );

$pName = $this->strip( $result['pName'] );

$this->cache['PRODUCTS'][$pID][(int)$this->languages_id] = $pName;

$this->performance['QUERIES']['PRODUCTS'][] = $sql;

$return = $pName;

break;

} # end switch

return $return;

} # end function

 

function generate_products_cache(){

$this->is_cached($this->cache_file . 'products', $is_cached, $is_expired);

if ( !$is_cached || $is_expired ) {

$sql = "SELECT p.products_id as id, pd.products_name as name

FROM ".TABLE_PRODUCTS." p

LEFT JOIN ".TABLE_PRODUCTS_DESCRIPTION." pd

ON p.products_id=pd.products_id

AND pd.language_id='".(int)$this->languages_id."'

WHERE p.products_status='1'";

$product_query = $this->DB->Query( $sql );

$prod_cache = '';

while ($product = $this->DB->FetchArray($product_query)) {

// FS start stop - [(int)$this->languages_id] added (constants per language)

// - . (int)$this->languages_id . '_'

$define = 'define(\'PRODUCT_NAME_' . (int)$this->languages_id . '_' . $product['id'] . '\', \'' . $this->strip($product['name']) . '\');';

$prod_cache .= $define . "\n";

eval("$define");

}

$this->DB->Free($product_query);

$this->save_cache($this->cache_file . 'products', $prod_cache, 'EVAL', 1 , 1);

unset($prod_cache);

} else {

$this->get_cache($this->cache_file . 'products');

}

} # end function

 

You see only a few changes in seo_class and it works.

[smartin513]

You're welcome.:)

 

Please enable the diagnostic setting and post the results here.

 

I turned on the diagnostics and I am still receiving the same message in my browser and in the error_log. Am I supposed to go somewhere else to retrieve the diagnostic output?

 

 

Warning: require_once(includes/configure.php) [function.require-once]: failed to open stream: No such file or directory in /home/xxx/public_html/googlexml/googlesitemap/index.php on line 47

 

Fatal error: require_once() [function.require]: Failed opening required 'includes/configure.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/xxx/public_html/googlexml/googlesitemap/index.php on line 47

[Jack_mcs]

Warning: require_once(includes/configure.php) [function.require-once]: failed to open stream: No such file or directory in /home/xxx/public_html/googlexml/googlesitemap/index.php on line 47

You said your shop was located in the root but you have the sitemap code installed in a subdirectory named googlexml. You need to install it correctly before it will work.

[smartin513]

You said your shop was located in the root but you have the sitemap code installed in a subdirectory named googlexml. You need to install it correctly before it will work.

 

I made this change and I made the change you suggested earlier in the thread about switch two lines in the configure.php file and it is working now. If I have URL rewriter on should the URLs in the sitemaps be the rewritten ones?

[Jack_mcs]

If I have URL rewriter on should the URLs in the sitemaps be the rewritten ones?

Yes, the url's should match or google may complain. If you are using Ultimate SEO V 2.2d (recommended), it will work automatically. If you are using some other version, it may not work without additional changes or maybe not at all.

[smartin513]

Yes, the url's should match or google may complain. If you are using Ultimate SEO V 2.2d (recommended), it will work automatically. If you are using some other version, it may not work without additional changes or maybe not at all.

I am using Ultimate SEO Urls 5 and I made the overwrite recommended in that support thread from the extras that come with that contrib. All site maps are generating but I am getting the URLs that are not the "SEO friendly" and when clicking on the links it leads to 404s because of this.

[Jack_mcs]

I am using Ultimate SEO Urls 5 and I made the overwrite recommended in that support thread from the extras that come with that contrib. All site maps are generating but I am getting the URLs that are not the "SEO friendly" and when clicking on the links it leads to 404s because of this.

I don't support that contribution. You'll need to ask in its support thread for assistance.

[smartin513]

I don't support that contribution. You'll need to ask in its support thread for assistance.

 

Ok. Again, thanks for all your help.

[longhorn1999]

Thank you for your comments. I apprecate them.

 

As for the exclude option, yes, that is a comman separated list. I meant to make a note of that but forgot. It won't work with pages that are part of a contribution, like info_id=2, though. You can editt the googlesitemap/sitemap.class.php file (or the alternate if you use it) to add your own though. To exclude regular pages, find this line

$this->excludeList

and enter the file names as the others are. To exclude particular pages for Information Pages contribution, find this line

			  $pagesArray[] = array('filename' => $this->base_url . 'information.php?info_id=' . $result['information_id'],

and add this right before it

if ($result['information_id'] != '2')

to exclude page 2. Multiple pages would be

if ($result['information_id'] != '2' || $result['information_id'] != '3')

 

I'll put adding an exclusion page on the list of changes to be made but it will probably be a while so the above should work until then.

 

Jack

 

 

Hi Jack,

 

I'm using this contribution with USU5 successfully. I saw this piece of code you gave a while back to exclude certain pages created in information pages unlimited from showing up in the sitemap. Unfortunately they still appear in my sitemappages.xml. Is there anything you might suggest besides just manually editing the sitemaps each time they're created?

 

Thanks,

 

Nick

[Jack_mcs]

I'm using this contribution with USU5 successfully. I saw this piece of code you gave a while back to exclude certain pages created in information pages unlimited from showing up in the sitemap. Unfortunately they still appear in my sitemappages.xml. Is there anything you might suggest besides just manually editing the sitemaps each time they're created?

Please read my post two above yours.

[motorcity]

I thought I had this all squared away but I continue to get scrambled results some of the time, that makes setting a cron job out of the picture entirely. Perhaps I'm missing something about using gzip commression?

in home/googlesitemap/index.php

	 * Option to compress the files 021610 switched to true
 */
define('GOOGLE_SITEMAP_COMPRESS', 'true');

I had to adjust the server php.ini so memory_limit = 128 and script time limit = 60 seconds.

It does work sometimes but here is an example of it all going to....

	<url>
	<loc>http://www.mysite.4in-7onsk-craviioipe-wrenatc5freq60e7tml</loc>
	<lastmod>2008-02-27</lastmod>
	<changefreq>weekly</changefreq>
	<priority>0.1</priority>
</url>
<url>
	<loc>http://www.mysite.1e1h-indiana8rightlgnsm-smlarnatc5ftylesheet type="text/xsl" hrefstmo7tmod>
8-02-27<ifeeddk386e-un2prio
	<rly</chkly<icaniorindard.1</pri4>0.1</priority>
</url>
<url>
	<loc>http://www.mysite.4in-7onsk-craviioipe-wrenatc5freq60e7tml</loc>
	<lastmod>2008-02-27</lastmod>
	<changefreq>wpeklrioritreq>
peakman.1</pri6>0.1</priority>
</url>
</url>
lehisolmd/mo7tmo8-02-p10m7-conr-ivumoviewnr-n2159l>
	<l953lid35.com/grit-a2www.mysite.1e1h-indiana8rightlgnsm-smlarnatc5ftylesheet type="text/xsl" hrefstmo7tmod>
8-02-27<ifeeddk386e-un2prio
	spany</co8-freq5hrmlar3x/www.mysite.com/driltc5f7tml</lr5oeqcuext/117</url>
<-s-remoddri12mmivgne553e64we7tml<-smlarnatc5ftylesheet type="text/xsl" hrefstmo7tmod>
8-02-27<ifeeddk386e-un2prio
	spany</co8-freq5hrmlar3x/www.mysite99>
haium-nisod>
queivumovin18672mysite.58-remoddri12mmivgne553e64we7tml<-smlarnatc5ftylesheet type="text/xsl" hrefstmo7tmod>
8-02-27<ifeeddk386e-un2prio
	spany</co8-freq5hrmlar3x/www.mysite99>
haium-nischrns.cn27412mysite.59-remoddri12mmivgne553e64we7tml<-smlarnatc5ftylesheet type="text/xsl" hrefstmo7tmod>
8

[Jack_mcs]

I haven't used/tested that option so I don't have an answer for you. But I've installed this contribution on many sites some quite large, and never had a need to use it. I suspect something else is causing the problem.

[motorcity]

I haven't used/tested that option so I don't have an answer for you. But I've installed this contribution on many sites some quite large, and never had a need to use it. I suspect something else is causing the problem.

I have determined that the problem only comes up when rewriting ~ over writing the existing files. Writing to a blank dummy sitemap.xml.gz seems to work fine.

 

So these other sites you've installed on were also over 50,000 and so had sitemapproducts.xml & sitemapproducts1.xml?

Did those sites require changing the max_execution_time and memory_limit, or are those perhaps more evidence that it's something else?

[motorcity]

I haven't used/tested that option so I don't have an answer for you. But I've installed this contribution on many sites some quite large, and never had a need to use it. I suspect something else is causing the problem.

Well that's weird. I guess I hadn't tried it with the compression off since I changed the time & memory. It seems to be working now.

 

Considering your comment & the fact that I couldn't find anything about anyone else using the gzip maybe it's just a bad idea. Or entirely non-functional.

[Stealth1]

It seems no changefreq is defined when that sitemap is created, this is causing google to show errors when you point it to this sitemap. I have manually added the changefreq but hoping this can be fixed.

[Jack_mcs]

That setting is set when the maps are created. If your maps don't have it, then something is wrong with your installation, though I don't have a guess as to what that might be.

[Stealth1]

That setting is set when the maps are created. If your maps don't have it, then something is wrong with your installation, though I don't have a guess as to what that might be.

 

Ah, I have SEO URLs 5 which has an extra for using Google XML Sitemap and it left out the part for the pages sitemap ChangeFreq

[mr_absinthe]

sitemapspecials.xml

I have total of 8 products in category specials, however sitemapspecials.xml shows four identical (all English language) links for each product, 32 links in total. The sitemapcategories.xml and sitemapproducts.xml are OK - one link per category and one link per product.

 

Any ideas please?

Hi Jack,

 

did you have a chance to have a look at the above issue please?

[Jack_mcs]

Hi Jack,

 

did you have a chance to have a look at the above issue please?

No, I haven't.