[Addon] Twitter Typeahead Autocomplete Search for osC 2.3.4 (BS and nonBS)

[auzStar]

@@Zoltan

@@presentemusical

 

And as a separate test, please try the below change (without above workaround):

 

replace the following code in "catalog\ext\typeahead\js\ht-twitter-typeahead.js.php" lines 89 to 140:

    var products = new Bloodhound({
      datumTokenizer: function(datum) {
        return Bloodhound.tokenizers.whitespace(datum.name);
      },
      queryTokenizer: Bloodhound.tokenizers.whitespace,
      remote: {
        url: "ext/modules/header_tags/twitter_typeahead/autocomplete.php?term=%query",
        wildcard : '%query',
        rateLimitBy : rateLimit_By,
        rateLimitWait : rateLimit_Wait,
        transform: function(data) {
          suggestion_length = data.length;

          if (sort_list) {
            var key = $.trim(search_input.val());
            key = key.replace(/["\\()]+/g,'');
            key = key.replace(/ +/g," ");
            var srchTerms = key.split(" ");
            var sort_key = "";

            $.each(srchTerms, function( index, value ){
              if (index > 0) {
                sort_key += " ";
              }
              sort_key += value;

              var startsWithMatcher = new RegExp("^" + sort_key, "i")
                , startsWith = $.grep(data, function(value) {
                    return startsWithMatcher.test(value.name);
                })
                , notStartsWith = $.grep(data, function(value) {
                    return 0 > $.inArray(value, startsWith);
                });
              data = startsWith.concat(notStartsWith);
            });
          }

          return $.map(data, function(product) {
              return {
                  name: product.name,
                  price: product.price,
                  model: product.model,
                  manufacturer: product.manufacturer,
                  image: product.img,
                  link: product.link,
                  pid: product.pid
              };
          });

        }
      }
    });

with:

    var charMap = {
        'a': /[áàâãªä]/gi,
        'A': /[ÁÀÂÃÄ]/gi,
        'I': /[ÍÌÎÏ]/gi,
        'i': /[íìîï]/gi,
        'e': /[éèêë]/gi,
        'E': /[ÉÈÊË]/gi,
        'o': /[óòôõºö]/gi,
        'O': /[ÓÒÔÕÖ]/gi,
        'u': /[úùûü]/gi,
        'U': /[ÚÙÛÜ]/gi,
        'c': /[ç]/gi,
        'C': /[Ç]/gi,
        'n': /[ñ]/gi,
        'N': /[Ñ]/gi,
        '-': /[–]/gi,
        ' ': /[’‘‹›‚]/gi,
        ' ': /[“”«»„]/gi
    };

    var normalize = function (str) {
        $.each(charMap, function (normalized, regex) {
            str = str.replace(regex, normalized);
        });
        return str;
    };

    var queryTokenizer = function (q) {
        var normalized = normalize(q);
        return Bloodhound.tokenizers.whitespace(normalized);
    };

    var products = new Bloodhound({
      datumTokenizer: function(datum) {
        return Bloodhound.tokenizers.whitespace(datum.name);
      },
      queryTokenizer: queryTokenizer,
      remote: {
        url: "ext/modules/header_tags/twitter_typeahead/autocomplete.php?term=%query",
        wildcard : '%query',
        rateLimitBy : rateLimit_By,
        rateLimitWait : rateLimit_Wait,
        transform: function(data) {
          suggestion_length = data.length;

          if (sort_list) {
            var key = $.trim(search_input.val());
            key = key.replace(/["\\()]+/g,'');
            key = key.replace(/ +/g," ");
            var srchTerms = key.split(" ");
            var sort_key = "";

            $.each(srchTerms, function( index, value ){
              if (index > 0) {
                sort_key += " ";
              }
              sort_key += value;

              var startsWithMatcher = new RegExp("^" + sort_key, "i")
                , startsWith = $.grep(data, function(value) {
                    return startsWithMatcher.test(value.name);
                })
                , notStartsWith = $.grep(data, function(value) {
                    return 0 > $.inArray(value, startsWith);
                });
              data = startsWith.concat(notStartsWith);
            });
          }

          return $.map(data, function(product) {
              var name = normalize(product.name);
              return {
                  name: name,
                  price: product.price,
                  model: product.model,
                  manufacturer: product.manufacturer,
                  image: product.img,
                  link: product.link,
                  pid: product.pid
              };
          });

        }
      }
    });

NOTE: new changes only has sample special characters to test with

 

cheers

[♥raiwa]

Hello Dominic @@auzStar,

 

I tried your solution and it finds the results but shows the product name with the replaced characters.

Example: search term " Latón" shows in the results "Laton".

 

The php workaround shows the correct result "Latón".

 

regards

Rainer

Edited February 28, 2016 by raiwa

[auzStar]

OK. Thanks Rainer.

[Zoltan]

Hi Dominic and Rainer,

 

I tried the two solutions with no success. Autocomplete and Search Result remains empty in both cases. Any suggestions ?

 

Kind regards,

Zoltan

[♥raiwa]

Hello Robin@@Zoltan,
 
did you use search terms with special characters which are included in the substitution list?
 
For example german "ß" is not included.

[♥raiwa]

I checked my files and found that I have one modification in advanced_search_result.php:

 

line 233 added

        default:
          $keyword = tep_db_prepare_input($search_keywords[$i]);
          $keyword = utf8_encode($keyword);
          $where_str .= "(pd.products_name like '%" . tep_db_input($keyword) . "%' or p.products_model like '%" . tep_db_input($keyword) . "%' or m.manufacturers_name like '%" . tep_db_input($keyword) . "%'";
          if (isset($HTTP_GET_VARS['search_in_description']) && ($HTTP_GET_VARS['search_in_description'] == '1')) $where_str .= " or pd.products_description like '%" . tep_db_input($keyword) . "%'";
          $where_str .= ')';
          break;

I do not remember why I added that line, but it may have an influence.

 

Oohps, forget, not related.

Tried it to apply in autocomplete.php, no result, sorry.

Edited March 1, 2016 by raiwa

[♥raiwa]

@@Zoltan, @@auzStar,

 

I installed the php solution to my real store under UNIX server (until now I tested under XAMPP and WIndows local store) and searched for german "ß" and surprise, it worked even it is not included in the substitution list.

So I reverted the workaround and tried with the original files and it works with all special characters.

Conclusion: the problem seems to exist only with windows server. Can you confirm this?

 

regards

Rainer

[auzStar]

Rainer you are correct. Nice pickup. I tested on local Windows/IIS where it seemed to be a bit hit and miss, for example:

when searching for this product caneta violão putting in both words works and so does entering a ã and a violão, but just putting in violão didn't work.

 

But on  my host's Linux/Apache server it worked every time.

 

You can test here http://www.auzcommerce.com.au/osc234bs_test/

for these products:

caneta violão

Adesivo Amo música

 

(These tests were also without any changes to the files)

 

cheers

[Zoltan]

Yes, Rainer, I also tried the "ß". It does not work yet.

[auzStar]

@@Zoltan

 

Have you read all the comments above?

Rainer discovered that it works on his host's UNIX server without any file changes, and I have confirmed on my host server.

 

It works here:

http://www.auzcommerce.com.au/osc234bs_test/

 

with these 3 test products:

caneta violão

Adesivo Amo música

another testß ß testßtest

 

(NOTE: without any changes to the files)

Edited March 3, 2016 by auzStar

[Zoltan]

@@auzStar

Yes I read the comments, and tried both file exchange versions above as well as not change anything. It did not succeed in any case. 

 

Regards,

Zoltan

[♥raiwa]

@@Zoltan,

 

1. Do not apply any modification, use the original files.

2. Try it on UNIX server NOT Windows.

[Zoltan]

@@Rainer,

 

Thank you, I can not try it as my provider uses Linux Servers.

 

Kind regards,

Robin

[♥raiwa]

@@Zoltan,,

 

Linux is UNIX

[auzStar]

The test site I provided in an earlier post is Linux. I mentioned it here: http://www.oscommerce.com/forums/topic/408210-addon-twitter-typeahead-autocomplete-search-for-osc-234-bs-and-nonbs/page-3#entry1740122. Seems to work there.

 

I thought you said you read everything :-

 

Have you tried it on your host Linux server like we've asked. We're just trying to help.

[Zoltan]

@auzStar

 

Yes I picked everything up again, but the addon is not working on the Linux server: http://www.holistikum.de. Would you please check it there ? Thanks.

 

Regards,

Robin

[♥raiwa]

@@Zoltan,

 

It looks like you character encoding is not consistant clean utf-8:

Neue Produkte im M�rz

 

rgds

Rainer

[auzStar]

@@Zoltan

 

Rainer is correct (see sample image attached, taken from your site).

 

[♥raiwa]

@@Zoltan,

 

I just remembered that the wrong encoded month is typical for a windows server running with unix date format definition. Are you sure your server is running Linux/Unix?

 

rgds

Rainer

[Zoltan]

@ Raiwa, @auzStar,

 

I just verified with my provider, they assured 100 % it is a Linux.

 

Regards, Robin

[♥raiwa]

@@Zoltan,
 
ok, but:
Neue Produkte im M�rz

 

still doesn't show correct!

Edited March 30, 2016 by raiwa

[Zoltan]

@ Raiwa,

 

I dunno why.

 

Robin

[♥raiwa]

@@Zoltan,

 

You need to check all your character encoding setup.

check these definitions in the main language file: includes/languages/german.php:

@setlocale(LC_ALL, array('de_DE.UTF-8', 'de_DE.UTF8', 'deu_deu'));

// charset for web pages and emails
define('CHARSET', 'utf-8');

Then check with your provider if the default encoding in php.ini is set to another than  utf-8. It should be undefined or set to utf-8.

 

Another but more rare issue could be that you ahve any add-on installed which adds another than utf-8 encoding definition.

 

Check also for any case if all your database tables collation is set to utf8_unicode_ci.

You can check this in:

Admin => Tools => database tables

and convert them there if needed.

 

Your source code looks like you updated from previous versions to 2.3.4.

Make sure you made all update steps correct. Full UTF-8 support was introduced in the 2.3.3.3  to 2.3.3.4 update. Maybe you missed something there.

 

 

rgds

Rainer

[Zoltan]

@ Raiwa,

 

thanks, I will follow your suggestions but this does not solve the search issue described above.

 

Regards,

Robin

[♥raiwa]

It is related.