Russian language is displaying question mark

[jdbanerjee]

I really don't know what happened but suddenly my website is showing question mark in both frontend and backend in oscommerce. Everything was working fine but suddenly it stopped working perhaps one of my colleague did something. Format of my db structure is in latin1_swedish_ci table name: products_description. i dont know if before it was in utf8 format or not. Can anyone tell me how to solve it if i change the structure to utf8 format will it work back to normal? Even if i am adding anything still it is displaying question mark instead of russian language

[MrPhil]

Were you running Russian successfully and then it failed, or is this a new attempt? What osC version (you should be at 2.3.4)? You mentioned UTF-8... did someone try a conversion from some single byte encoding (Latin-1, etc.) to UTF-8? Are the pages marked as being UTF-8, but the database is still Latin-1? That would give you ?-in-black-diamond glyphs instead of Cyrillic characters on most browsers. Is that what you're seeing? Or are they plain "?", which indicates that the database itself is confused about what the data is?

 

If you're lucky, someone marked the pages to be displayed in UTF-8, but the database is still in Latin-1 or something else supporting Cyrillic (Latin-9? CP-1251?). It would simply be a matter of backing up the database, and then going into phpMyAdmin and telling it to convert each table to UTF-8. Note that if the database was marked ("encoded") as Latin-1, but contained Cyrillic data, it will not convert correctly (it will think the Cyrillic characters are accented Western European characters, and convert to their UTF-8 equivalents).

 

If the database says "Latin-1" (which used to be the default for osC, and still is for MySQL), what you might have to do is export the database to a .sql file, edit the backup to show that the encoding is actually CP-1251 or whatever (Cyrillic), import the .sql to rebuild the database with the correct encoding (also, but incorrectly, called "collation"), and then do the conversion to UTF-8. Do this on a test copy of the database until you've got it figured out.

[jdbanerjee]

My full database is in latin1_swedish_ci and i just converted table name "product_description" into utf8 format but still it is showing question mark(?). I also readded text in russian language and save it again still it is showing the same question mark(?)

 

It display each russian character into question mark eg пакет into ????

 

My page meta is in utf-8

[MrPhil]

After doing the conversion to UTF-8, does the correct text show up when you browse the table's data in phpMyAdmin? If it does, and the page encoding (meta) is UTF-8, it should show correctly on the page. Be sure to check with your browser's View > Character Set (or similar), to make sure the browser is being told it's UTF-8 and isn't being overridden by the server. I still need to know: is it ?-in-black-diamond or just regular "?"? They mean two different things.

[♥bruyndoncx]

i had mixed results when testing, found that the browser detecting/interpreting the language set was inconsistent.

 

I force UTF8 as the first entry in my application_top.php and made sure that all my php files are saved into UTF8 nobom.

  header('Content-Type: text/html; charset=utf-8');

also check your includes/functions/database that it is set to UTF8

 

and I also noticed that my notepad++ recent, but not latest version had an ansi bug, that everything went back to ansi mode, instead of maintaining utf8 no bom.

 

I have a rather rudimentary cleanup script (little php skills required to understand to tweak it) to convert php files to utf8 and/or remove bom.

The encoding file is from a github project. I'va attached it if it would be of any help

_fix_bom.php

Encoding.php

[jdbanerjee]

After changing it to utf8 in database it is still showing it in the ??? format. plain question mark

[jdbanerjee]

I guess i found a solution why it was happening. In table name  products_description it has field name-> products_name which was in latin_swedish so when i changed it to utf8_general_ci and save the data in russian language it saved properly in russian format and it is showing the russian data correctly and just now i got one problem which is all the saved russian data in the database is showing in ???(Question mark) so i only want to know is there any way to recover the question mark data into original text?