French letter before every price - Â.

[SafeMrRomeo]

Before any price on my site, there is this symbol. Â.

 

Is it something to do with the database? or something else?

 

 

Ben

[dmnalven]

Either your server's locale needs to be set or adjusted or the choice of currency symbol is incorrect or a poorly supported one.

[SafeMrRomeo]

Either your server's locale needs to be set or adjusted or the choice of currency symbol is incorrect or a poorly supported one.

 

And how do i do this?

 

The prices display as : £135.00

[♥geoffreywalton]

In all the php files in /includes/languages/

 

Or it could be the collation sequence in your db.

[Guest]

This sometimes happens if you change servers, all I normally do is edit the currency in the admin and remove the  in front of the currency symbol and that normally fixes it.

 

Aran

[dickie]

If you edit your currency with the html code for the symbol. E.g. Instead of putting $ put the html code into the symbol box place the code &_#_3_6_; and the same with other currency symbols. This should rectify the problem.

 

REMOVE THE UNDERSCORES, THE FORUM WAS DISPLAYING THE CODE AS THE DOLLAR SYMBOL SO HAD TO PUT THEM IN

[MrPhil]

You had your currency symbol coded as UTF-8 (either in a file or in the database), and are displaying it on a page in Latin-1 (ISO-8859-1) (or some other single byte encoding). Did you change character encodings at some point? Did you edit something on a UTF-8 PC and upload to a Latin-1 server? Are your database, support files, and page display all consistent encoding?

[Franalo]

Hi!! I have the same problem but with the euro (€) sign and I don't know what to do. Can somebody help me??

 

thanks!

[Xpajun]

Francisco,

 

Use either & euro ; (no spaces) or & #128 ; (no spaces) instead of € in admin/location/currencies where it asks for symbol

[RobC1962]

Quick answer is to use actual HTML entities for your symbols. Long answer it to make the physical changes in your database. Similar problems can happen with copyright symbols etc.

[MrPhil]

Use either & euro ; (no spaces) or & #128 ; (no spaces) instead of €

NO, NO, NO, NO! That's wrong! #128 is not the standard UTF-16 assigned codepoint for the Euro (it's x20AC or #8364, if you want to use the numeric form). #128 is the Microsoft "Smart Quotes" Windows-1252 code page only, and will not work unless you change your page encoding to Windows-1252. It's best to use the named entity ("euro"), as most browsers support that now.

[Xpajun]

NO, NO, NO, NO! That's wrong! #128 is not the standard UTF-16 assigned codepoint for the Euro (it's x20AC or #8364, if you want to use the numeric form). #128 is the Microsoft "Smart Quotes" Windows-1252 code page only, and will not work unless you change your page encoding to Windows-1252. It's best to use the named entity ("euro"), as most browsers support that now.

 

 

 

Oops - looks like I need to update my files :blush: :blush:

 

Actually the last time I used HEX - UTF-16 it didn't work

 

Also seeing that the € is UTF-8 and 16 compatible one should not have to use a character code for it?

 

I would suspect that, like a lot of people, the poster has a latin1_swedish_ci collation database in conjunction with a utf-8 coded store that is causing the problem?

[MrPhil]

UTF web pages are normally UTF-8. When you use numeric HTML entities ( nnnn ; ), the nnnn value is the decimal value of the UTF-16 assigned value (code point) for that glyph. So, a Euro is hex 20AC which is 8364 ;. Microsoft's "Smart Quotes" extension to the Latin-1 code page (Windows-1252) sticks the Euro into x80 (128), but that only works if your page is displayed in Windows-1252. That's why cut-and-paste from Word will leave you with funny "?" symbols when you display your page in the standard Latin-1 or UTF-8 -- x80/128 is supposed to be a control character, not a Euro.

[Xpajun]

Phil,

 

I get the diamonded ? on a lot of characters on my store not just € but ø ü any like this but only when the page is outputed/encoded as utf-8

If I change the encoding in my browser to ISO-8859-1 then they show correctly.

 

My database has latin1_swedish_ci collation as installed (or has been converted by my host (but others have said this as well)) by osC 2.3, the standard encoding for pages in 2.3 is utf-8 - this is why I say everyone may have this problem

 

Would these problems arise from customers entering their addresses from their windows machines and these "bad characters" get stored as latin1_swedish_ci which reads wrong when brought back to the utf-8 page

 

I've converted the collation of my local db to utf-8 (doesn't have the option of utf-16) which did nothing to characters already stored but maybe it will for new characters?

 

Just when I thought I had databases sorted 2.3 comes along and gave it a knock :rolleyes:

 

Phil, the question I'm asking of you is: in your opinion should the db be utf-8 as well as the page encoding or do the core team know something that I haven't found out yet? Oh and will this fix the problem? :lol:

[♥Biancoblu]

I too am having lost of problems with charsets. I have a multilanguage site and have endless issues.

If I set my pages to utf-8 the special characters don't display correctly, they only display correctly if I set pages to iso-8859-1.

My DB is utf8-general-ci collation.

 

Recently I've only just realised that contact form, add comments to order, emails from admin don't display special characters at all whether in utf-8 or iso-8859-1. The special characters just get deleted!!

 

I have thought of converting the DB to latin 1 collation, anyone know if this is good or bad?

[♥Biancoblu]

Jules I'm curious, how does your contact us from behave? if you try sending a message with special characters, for example ö é â £ ü, do you see them in the email you receive or do they just get deleted?

[Xpajun]

Isabella,

 

Just had a check through...

 

First my set up:

 

I use Thunderbird for my email client - I do not use web based email client

 

My contact us (in fact my whole store, I think) has charset=ISO_8859-1 not utf-8

 

The contact us page does not store anything in the database

 

 

I've checked emails that include "special" characters and they all show perfectly - it is only in the store itself that the problem arises - I was under the impression that if the database has a utf-8 collation then no matter what encoding the web page used (or the browser chose) the characters would read correctly.

 

2.3 threw me because the database collation was latin1_swedish_ci collation - my thoughts were, at the time, "if that is what the core team picked it must be right" but of course it may be my host, although others have said their database is latin1_swedish_ci collation

 

 

Incidentally if I send a translated email from eBay it really messes up big time on "special" characters - a) they are written online B) I think the email is stored in, and retrieved from, a database in their case

 

So that would bring us back to the database again

 

BTW have you got 2.2 or 2.3?

 

 

Juls

[♥Biancoblu]

Hi Jules

 

I use 2.3. My old database was imported from my old ms2.2 shop.

 

I am not sure what to do about the charset issue as I'm having the problems described earlier.

 

When you you say your contact us has charset=ISO_8859-1, do you mean catalog>contact_us or catalog>includes>languages>english>contact_us? or both?

[Xpajun]

The charset is set in languages/english.php (or your local language) and should carry through for the whole site (sometimes it doesn't - packing slip has a separate one)

[eroszamp]

Strange problem indeed...

[MrPhil]

Phil,

 

I get the diamonded ? on a lot of characters on my store not just € but ø ü any like this but only when the page is outputed/encoded as utf-8

If I change the encoding in my browser to ISO-8859-1 then they show correctly.

 

My database has latin1_swedish_ci collation as installed (or has been converted by my host (but others have said this as well)) by osC 2.3, the standard encoding for pages in 2.3 is utf-8 - this is why I say everyone may have this problem

 

Would these problems arise from customers entering their addresses from their windows machines and these "bad characters" get stored as latin1_swedish_ci which reads wrong when brought back to the utf-8 page

 

I've converted the collation of my local db to utf-8 (doesn't have the option of utf-16) which did nothing to characters already stored but maybe it will for new characters?

 

Just when I thought I had databases sorted 2.3 comes along and gave it a knock :rolleyes:

 

Phil, the question I'm asking of you is: in your opinion should the db be utf-8 as well as the page encoding or do the core team know something that I haven't found out yet? Oh and will this fix the problem? :lol:

Okaaaaaaay, the accented ("national", or non-ASCII) characters are encoded in Latin-1 (ISO-8859-1) in the range xA0 -- xFF (single bytes), but your page display is UTF-8 for whatever reason. Latin-1 accented characters are invalid codes in UTF-8, and will cause the ?-in-diamond glyph to be displayed. The bottom line is that your database, language support files, and page encoding all have to be the same encoding -- typically either Latin-1 or UTF-8, but never a mixture of the two. As far as "incoming" characters go, it's irrelevant what kind of PC someone is using -- it's the page encoding that counts. Now, if this is old customer input, made back when your site was Latin-1, it would have been stored in the database as Latin-1 encoded text. When the database serves that up now, and osC assembles the page, those Latin-1 accented characters are going to cause a problem when the page is displayed in UTF-8.

 

As part of the upgrade process, if 2.3.1 is UTF-8-only, there should have been a step to convert your database from Latin-1 encoding to UTF-8. Actually, it should be two things: changing the encoding/collation definition for all text fields, and converting existing data. I'm presuming that you either missed that second step or it wasn't run successfully. See if you can find it in the upgrade (or instructions how). Failing that, if you have little or no new UTF-8 data in the database (which thinks it's UTF-8, but the data is still Latin-1), there are two things you can do. One would be to stay at Latin-1 and find where the UTF-8 encoding is being set (probably in english.php) and change it back to ISO-8859-1. You would also change your database back to Latin-1, but don't convert existing data. I can't promise that this won't cause some sort of problem in other parts of osC 2.3.1. Also, any new data in the database, with accented (non-ASCII) characters, is going to be UTF-8 encoded and will display as two or three strange characters each. They will have to be manually changed in phpMyAdmin. The second alternative is to make sure that your language support files are UTF-8 (don't forget things like the Pound Sterling sign), generally not much work with English. Then you convert your MySQL database encoding from Latin-1 to UTF-8. Not only do you declare all text fields to be UTF-8, but you also actually convert any non-ASCII text to its UTF-8 equivalent in the process. You may also be asked whether you want to convert HTML entities to UTF-8 characters -- you can do that if you want, leave it for later, or not do it -- it doesn't matter. Again, any recently added UTF-8 accented characters will be mistranslated and need to be cleaned up manually.

 

When all is said and done, your database, its text data content, your language files, and your page encoding should all be speaking the same character set (encoding). In the long run, I would recommend UTF-8 over Latin-1, simply because you'll never encounter a bona fide accented or non-Latin character that can't be represented (so long as you're not selling Spi\:nal Tap merchandise!). Even if you plan to sell only in English speaking areas, there are enough "furriners" with accented characters in their names or whatnot that you can make happy by not forcing them to mangle their names. Normally, I would not recommend changing over from one to another, simply because of the work involved, but the issue has been forced on you because you have a hybrid system now that's only going to become messier and messier the longer you let it go.

[Xpajun]

Phil, thank you for taking the time to write that - I believe I've now found the "real" problem...

 

When I installed 2.3 I never read the documentation (more on that in a bit) and just created a database and installed - the same as I always have (right or wrong :rolleyes: )

 

However the database I created had my server's default collation which is latin1_swedish_ci collation and so the problem began...

 

Back to the Documentation for 2.3 - I've now taken the opportunity to read it (before posting this) - well not all but certainly the new installation process - Nowhere in "New Installation", "Requirements", "Pre-installation Steps" or "Installation Procedure" is there any mention of setting the collation of the database to utf-8 on creation - also as far as I can see, nothing is mentioned about the database collation in the upgrade guide either.

 

Changing the database:

 

I've found out that changing the database from latin1_swedish_ci collation to utf-8-unicode-ci doesent do anything to tables and columns already created, each column of each table that has latin1_swedish_ci collation has to changed to utf-8-unicode-ci as does each table - I have 57 tables >_< - others may have more

 

What I have done (locally only - not on my store yet) is exported the existing database opened it in my text editor then searched for all:

 

CHARSET=latin1

 

and replaced with:

CHARSET=utf8 COLLATE=utf8_unicode_ci

 

I then deleted the database, created a new database with a utf8_unicode_ci and then imported the modified data

 

So far every thing seems to have gone ok

 

BUT

 

If anyone intends to do this work on a copy of your backup!!!

 

My next step is to manually change the special characters I have in the database - Oh what fun :'(

[burt]

latin1_swedish_ci is fine - the swedish language has many extra characters that english does not have.

 

utf8_general_ci adds even more.

 

Conclusion;

 

DB: latin1_swedish_ci

Languages; iso-8859-1

 

All should be well.

[Xpajun]

latin1_swedish_ci is fine - the swedish language has many extra characters that english does not have.

 

utf8_general_ci adds even more.

 

Conclusion;

 

DB: latin1_swedish_ci

Languages; iso-8859-1

 

All should be well.

 

 

Yes it works ok like that, Gary, until you get cyrillic character sets...

 

utf-8 all around is the best way to go - you just need to remember to set the database before you install :rolleyes:

[♥Biancoblu]

Phil,

 

I get the diamonded ? on a lot of characters on my store not just € but ø ü any like this but only when the page is outputed/encoded as utf-8

If I change the encoding in my browser to ISO-8859-1 then they show correctly.

 

 

 

I can only comment on my own site, but I found that when using special characters (ü ø é, etc), if my page is encoded in iso-5589-1, I can either write it as "& u u m l ;" or just "ü".

If the encoding is in utf-8, I have to use "& u u m l ;" or I'll get the question mark in black diamond.

When I import my DB which contains special characters, I have to import it as latin1_swedish_ci regardless of collation which can be utf-8. If I import it as utf-8, special characters get deleted.

 

As for bowsers, it should detect the encoding of the page you are viewing and show it to you correctly, so you're not supposed to change the browsers encoding which has already been determined by the browser. For example, if I view yahoo.ru, I see it correctly, and my browser tells me that the page is in utf-8, but if I change the browser's encoding to say iso-5589-1, the cyrillic characters don't show correctly. If I view my site, I see it correctly and the browser tells me that the page is in iso-5589-1, if I change the browser to utf-8 I see the black diamonds + question marks.

 

My conclusion, and i only speak for myself, is that a. I keep my pages at iso-8859-1, and all my characters show correctly either written as "& u u m l ;" or just "ü"

b. I change my pages to utf-8 but in this case I rewrite all my texts replacing special characters like "ü" with "& u u m l ;".

 

As for my previous problem when some characters were stripped altogether, turns out this was due to too much security which was stripping special characters, nothing to do with my encoding. Thanks burt for helping me fix that. :)