Greek characters problem In Box Reviews

[panos13]

I make a site 2.3.4bs UTF-8 in Greek language. The database and all the pages of site is working perfect with Greek language .

 

Then i noticed that the module box reviews is appear a number of greek letters as symbols.

Can anyone help me?
 

I fiddle three days and nights searching and reading and I can not find α solution.

Thanks. (Sorry for my english.)

[MrPhil]

Is the Reviews box accepting text in UTF-8 Greek? Ask your browser what character encoding is being used on that page (View > Character Set, or similar). If it's not in UTF-8 for some reason, that's your problem. Are you comfortable looking in the database to see the review text field definition (character set/collation should be UTF-8) and examine some of the text to see if it's UTF-8 Greek text, or is already messed up at that point? Finally, on the page that displays the reviews, is it UTF-8 encoding or something else?

 

If you're seeing lots of ?-in-black-diamond symbols, it's likely that your data was input in Latin-x (single byte encoding) and you're trying to display it in UTF-8. The problem is likely with the original input code. On the other hand, if you're seeing several odd accented Latin alphabet characters per expected Greek character, it's likely that the text was input OK, but is being displayed in Latin-x or another single byte encoding.

 

Is the Reviews you're using part of the base 2.3.4, or is it an add-on? What add-on number? It's possible that an older add-on may have its character encoding hard-coded to Latin-1, rather than allowing other encodings such as UTF-8.

[panos13]

@@MrPhil

 

Thanks for your answer.

 

This is a site 2.3.4 bootstap with this addon.

 

The database and the pages they haven't problem with Greekw characters. The shows all normaly. The problem is with boxes of reviews and testimorial.
Quote the link of my site www.ekdoseis-stemma.gr to see. Βecause you will appear the English page in bar settings and selecting the Greek flag.
 

Click on reviews (Κριτικές) and you will see that there are no symbols. But the Box Reviews (Κριτικές) have symbols.

 

Sorry for my english.

[BrockleyJohn]

Aha - found your thread now! Try approving my review and see if the funny characters come out in English too

[MrPhil]

Did you type these reviews and testimonials directly in to your store, or did you cut and paste from a PC program such as Word? The UTF-8 invalid characters (?-in-diamond) appear to be at the beginning and/or end of a line. If it's at the beginning, it's a Latin-1 I+^, while at the end it's I+: (umlaut or diaseris). Those (xCE and xCF) don't seem to correspond to any MS "Smart Quotes", so it's not clear where they come from. Perhaps some word processor markup that got copy-pasted in?

 

There's also a possibility that there is code to trim a longer line down to a "teaser", but it's either adding some strange quotes or brackets on its own, or it's assuming single byte encoding and cutting off UTF-8 characters in the middle. However, the second explanation would not explain why it always seems to be those two accented I's, and why it also happens at the beginning of the quote. Anyway, if this is a "teaser" function add-on, you might ask the author to confirm that it works with UTF-8 and if it adds any brackets or quotation marks, that they are UTF-8 legal.

 

PHP changed its default handling of encodings in string functions around 5.5 or 5.6 (I think) -- what PHP are you running with?

[auzStar]

Hi guys,

 

Panos originally posted this problem in the "Customer Testimonials" support thread since he assumed it was an issue with this add-on (he hadn't created any reviews yet). I instructed him to create reviews to see if he has the same issue which he did. The testimonials add-on uses the same built-in osCommerce functions as the reviews system, so it's not a specific issue related to the testimonial add-on, but more a general osC issue.

 

He's not using a reviews add-on, it's just the one that's included with osC.

 

It seems it could be an issue with how long words get broken into smaller parts inside the reviews and testimonials infoboxes.

 

Example:

in catalog\includes\modules\boxes\bm_reviews.php there is this line of code which does the slicing:

        $rand_review_text = tep_break_string(tep_output_string_protected($rand_review['reviews_text']), 15, '-<br />');

The testimonial add-on does the same:

        $testimonial = tep_break_string(tep_output_string_protected($random_testimonial['text']), 15, '-<br />');

It's possibly something to do with how the strings are manipulated inside these functions that seems to break the output when used with certain languages.

 

When I get a chance I'm going to try removing tep_break_string to see if long words will wrap OK in the responsive boxes, or if someone wants to try? :)

 

[♥Tsimi]

@@auzStar

 

Hi Dominic

 

I am not really familiar with encoding stuff inside the php code and more less familiar with the reviews and testimonials modules but would something like this not work?

substr(strip_tags($rand_review['reviews_text']),0,20)

substr(strip_tags($random_testimonial['text']),0,20)

and what about using something like this?

utf8_decode

would this have any effect on the encoding?

[♥Tsimi]

Just out of curiosity i quickly added 2 new reviews to my local testshop.

One in German with the Umlauts and one in Greek. Both show just fine... :rolleyes:

No clue what went wrong with Panos's version but I can't reproduce that error.

[tgely]

Hi all,

 

there is no good solution for tep_break_string in all lenght or in all languages in utf8 collation. Hyphenization is language dependent but we can not find good rule libraries for all language. Its horrible.

So I dont suggest to break words..

[panos13]

Hi all

 

@@auzStar thank you and all others.

 

The Greek language has some special features but it is very nice.

A change of 15 to 30 had a job to me.

Now there is no problem unless something occurred later but this is the beauty of the Greek language.
Furthermore we have learned to overcome our problems with " race".

 

Thanks.

Sorry for my english.

[auzStar]

Hi all

 

@@auzStar thank you and all others.

 

The Greek language has some special features but it is very nice.

 

A change of 15 to 30 had a job to me.

 

Now there is no problem unless something occurred later but this is the beauty of the Greek language.

Furthermore we have learned to overcome our problems with " race".

 

Thanks.

Sorry for my english.

 

@@panos13

 

While that maybe a quick solution, the problem still exists, since longer words will re-create the problem.

I copied your Greek text from your website and was able to replicate the issue with a test review on my test site, with 15 as the breakpoint. I changed it to 30 like you did and all looked good till I joined a couple of the words (by removing the spaces) to make a longer word. Again the text appeared corrupted in the reviews box.

 

No joy in removing tep_break_string(). Long words did not wrap in the responsive boxes without any assistance. Go figure. :-

 

There is some CSS styling available though, which I tested and seems to work OK, if you'd like to try:

1. word-wrap: break-word; - deprecated

2. overflow-wrap: break-word; - is the replacement for word-wrap but not fully supported yet

 

We can use both for future proofing.

 

 

Reviews

 

1. in catalog\includes\modules\boxes\bm_reviews.php

find at around line 50:

        $rand_review_text = tep_break_string(tep_output_string_protected($rand_review['reviews_text']), 15, '-<br />');

 replace with:

        $rand_review_text = tep_output_string_protected($rand_review['reviews_text']);

2. in catalog\includes\modules\boxes\templates\reviews.php

find at around line 3:

  <div class="panel-body"><?php echo $reviews_box_contents; ?></div>

replace with:

  <div class="panel-body" style="word-wrap: break-word; overflow-wrap: break-word;"><?php echo $reviews_box_contents; ?></div>

Testimonials

 

1. in catalog\includes\modules\boxes\bm_testimonials.php

find at around lines 96 and 128:

	        $testimonial = tep_break_string(tep_output_string_protected($random_testimonial['text']), 15, '-<br />');

replace both lines with:

	        $testimonial = tep_output_string_protected($random_testimonial['text']);

2. in catalog\includes\modules\boxes\templates\testimonials.php

find at around line 3:

  <div class="panel-body"><?php echo $testimonials_box_contents; ?></div>

replace with:

  <div class="panel-body" style="word-wrap: break-word; overflow-wrap: break-word;"><?php echo $testimonials_box_contents; ?></div>

cheers

[MrPhil]

tep_break_string() uses strlen() and substr() calls, which were not UTF-8 (multibyte character) aware for most versions of PHP, and may still not be (mb_* versions can be used). The conversion to UTF-8 awareness may have overlooked this function (among others). It appears that the function was cutting words at 15 (or 30) bytes rather than characters, leaving incomplete (invalid) characters at the beginning and end of most cuts.

 

Changing the maximum word length from 15 to 30 bytes will not help for any word longer than about 10 characters (assuming 3 bytes per character for Greek alphabet). Any function using strlen() and substr() really needs to be fixed to be UTF-8 friendly.