CODEWITHSUNDEEP
utf-8character-encodingdojox.grid.datagridxpages-extlibnon-english
Panu Haaramo
Panu Haaramo

Reputation: 2932

Scandinavian characters get messed up when saved from DataGrid

I'm working on a xe:djxDataGrid that loads and saves data via xe:restService. It loads the data from Domino documents and at this point all scandinavian characters like ä and ö look OK.

But if data is modified and saved back to server ä turns to ä and ö turns to ö. I think this is because data is UTF-8 encoded but gets interpreted as ISO-8559-1 at some point.

I'm trying to use UTF-8 everywhere:

Response header of the page has this line: Content-Type:text/html;charset=utf-8

I've also tried changing the charset to ISO-8559-1 everywhere but that does not help. I've tried setting the computeWithForm property of REST control to true to see if that makes it use the form charset but it has no effect.

In Firefox (17.0.1) everything works fine! The problem occurs at least in Chrome and IE9.

Because it's browser specific I think it breaks when Dojo sends data to REST service. But I haven't seen a way to tell specifically to Dojo to use UTF-8.

HTML tag looks like this:

Chrome: <html lang="fi"> Firefox: <html class="dj_gecko dj_contentbox" lang="fi">

djConfig is this: djConfig="locale: 'fi-fi'.

Domino version is 8.5.3FP3 and ExtLib date is 20121217.

Unfortunately I can't force the users to use Firefox only. Any ideas how to fix this?

edit 1

The same problem occurs in ExtLib demo application: xpagesext.nsf/REST_DojoGrid.xsp and xpagesext.nsf/REST_DojoGridJsonRest.xsp.

edit 2

As a workaround I can do this in field Input Translations on form:

@ReplaceSubstring(@ThisValue; "ä":"ö"; "ä":"ö");

This works when I enable computeWithForm in REST control. I need to include all non-english characters likely to be used. Or is there a generic way to convert all?

edit 3

As adviced by @Esailija I examined the HTTP PUT request which saves the data to server. This time I was testing with Opera 12.12 which also has the problem. The content type of the PUT request is:

Content-Type: application/json

With Firefox it is:

Content-Type:application/json; charset=UTF-8

This explains the problem but how to fix it? In my understanding the problem is with dojox.grid.DataGrid (1.6) control which does not set the charset in the PUT request. Firefox seems to set it automatically. Or is it actually in ExtLib DataGrid which fails to set the charset in the Dojo control? I haven't found a way to set charset in DataGrid.

edit 4

Tried changing contentType property in REST control from application/json to application/json; charset=UTF-8. That did not help and the content type of the PUT is still application/json.

thanks,

Upvotes: 0

Views: 1239

Answers (3)

Vyacheslav Danilov
Vyacheslav Danilov

Reputation: 1

It's better to do it so that it works 100%.

 dojo.addOnLoad(function() {
        if (!dojo._xhr) {
            dojo._xhr = dojo.xhr;
        }
    
        dojo.xhr = function(method, args){        
            try {
                if (method=="PUT" || method=="POST") {
                    args["contentType"] += ";charset=UTF-8";                        
                }
            } catch(e){}
            
            return dojo._xhr.apply(dojo, arguments);
        }
    });

Upvotes: 0

Roman
Roman

Reputation: 127

I modified code, its work

dojo.addOnLoad( 
    function() {
        if( !(dojo._xhrPost )) {
            dojo._xhrPost = dojo.xhrPost;
        }

        dojo.xhrPost = function (args) {
            if ( args.headers && ( args.headers["Content-Type"] == "application/json" ) ) {
                args.headers["Content-Type"] = "application/json;charset=UTF-8";
            }
            return dojo._xhrPost(args);
        }
    }
)

Upvotes: 1

Panu Haaramo
Panu Haaramo

Reputation: 2932

I should be able to modify the charset by overriding Dojo xhrPut and modifying HTTP headers.

Here is how Sven Hasselbach is doing it but this needs to be modified so that we do not override the existing headers (because they are used by the REST service) and just add/modify them:

/**
 * Cache Prevention for Dojo xhr requests
 *
 * Adds no-cache header and enables dojo's preventCache feature
 * for every dojo xhr call. This prevents the caching of partial
 * refreshs.
 *
 * @author Sven Hasselbach
 * @version 0.3
 *
 **/
dojo.addOnLoad(
    function(){
        if( !dojo._xhr )
        dojo._xhr = dojo.xhr;

        dojo.xhr = function(){        
            try{
                var args = arguments[1];   
                args["preventCache"] = true;
                args["headers"] = { "cache-control": "no-cache" };
                arguments[1] = args;
          }catch(e){}

          dojo._xhr( arguments[0], arguments[1], arguments[2] );
        }
    }
)

http://openntf.org/XSnippets.nsf/snippet.xsp?id=cache-prevention-for-dojo-xhr-requests

I have the workaround in place already but will give this a try at some point.

Upvotes: 0

Related Questions