Bizarre special characters problem

[candycanes]

I have a variety of websites, copied off each other. The most recent one was copied from UK to Portuguese.

 

I have found that despite everything being set to UTF-8 (in database.php, lang/portuguese.php) that when I for example go to add a new address as a customer would, if there is more than one special character in the input text field, it crops off the second special char and everything after it:

 

'Rua Álvaro Castel' would save fine in the db but 'Rua Álvaro Urbanização' would be saved as 'Rua Álvaro Urbaniza'

 

This is infuriating, and I have checked using my admin and putting lots of special chars in product descriptions and saving works just fine. I compared php files between admin and non-admin, they match.

 

I don't understand at any rate why it lets me save one special char but not more than one! Any ideas?

[MrPhil]

Have you confirmed that the individual database (text) fields are themselves UTF-8? Maybe they're still Latin-1 and MySQL is getting tangled up with UTF-8 multibyte strings going in as Latin-1 and trying to come back out as UTF-8? Now, why the first accented character would be OK but the second isn't, is a puzzler. These characters aren't being cut and pasted from Word or Outlook (MS Smart Quotes for some accented letters), but directly typed in? What happens if you have two Á letters instead of Á and ç? In other words, is this behavior sensitive to the specific letters? This is only with customer-entered text, and not anywhere else? I would suspect the customer information table field encoding -- take a close look at it in phpMyAdmin.

[candycanes]

Thanks for the reply.

 

In phpMyAdmin, it says it's UTF8 and collated to utf8_generic_ci. I believe when the site and db were copied, the product data was translated and imported, but the customer and address tables were truncated, so all data in them should be fresh. I have copied the special chars from the webpage and pasted in, so it is from the rendered HTML. It is not character centric either, and furthermore, if I edit the address from within the web admin site, it works fine. It is only using the shop front end from my account > address book update that fails.

 

If I use admin and update, then view my addresses in the shop front end, I can see all my special chars just fine. It is not a display issue, just a saving from the shop form. But I can't understand why. Each page is set:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

set names is UTF-8 setlocale is pt_PT.UTF-8

 

And it's driving me insane. :)