UTF-8: Hebrew charset from Ajax client to Tomcat server

Question

I am facing a problem when I transfer data [Hebrew chars] from browser to web service. There is no problem in Browser js side and java side but the problem occurring when I am transferring data between them.

AJAX - setRequestHeader

var hebrewData = 'יאש';
var encodeData =  encodeURIComponent( hebrew );

console.log('Encoding Before :', hebrewData, '
After :', encodeData);
var xmlhttp = new XMLHttpRequest();

// Method: POST; Encoding type: application/x-www-form-urlencoded (default):
xmlhttp.open("POST","http://localhost:8088/WebApplication/Service", true);          
xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded");
xmlhttp.send("data="+encodeData+"&userid="userid );

Web Service Java file save as UTF-8

String hebrew = "חוּט";     
System.out.println("Lan : "+hebrew);

I have tried these possible ways;

XMLHttpRequest.setRequestHeader("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");

as of java getParameter() uses ISO-8559-1 instead of UTF-8 changed it, Tomcat HTTP Connector with URIEncoding="UTF-8" (or) useBodyEncodingForURI="true"

In servlet « request.setCharacterEncoding("UTF-8");
tomcat server.xml «

but I am unable to get the correct output.

Yash · Accepted Answer

I have tried these possible ways to save data Form Java to DB, Servlet to DB, mentioned step by step clearly below.

➩ As Windows OS uses CP1252 character encoding by default. So, changed code page value to 65001 - Unicode (OR) windows-1255 - ANSI Hebrew

Java Program

public class My {
    public static void main(String args[]) throws java.io.UnsupportedEncodingException {    
        System.setProperty("file.encoding","UTF-8");
        System.out.println("Default Charset = " + java.nio.charset.Charset.defaultCharset());

        String పేరు = "Yash";
        System.out.println("UNICODE - Identifier[Variable Name] : " + పేరు);

        String hebrew = "יאש";
        System.out.println("UNICODE - Literal[Variable value] : "+ hebrew );

        System.out.println("windows-1255 : " + new String(hebrew.getBytes("UTF-8"), "UTF-8") );
    }   
}

Compiling Java program Manually which is written using Text Editor Notepad++.

Even on appending JAVA_OPTS does not help. When we compile a Java-File having Unicode characters, we get this error

error: unmappable character for encoding Cp1252
    ➩ As Windows OS uses CP1252 character encoding by default.

D:\Yash>set JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8
D:\Yash>javac -encoding UTF-8 -target 1.7 My.java 
    «  -encoding  Specify character encoding used by source files.
    «  -target     Generate class files to support lowest specified version[1.7] and above[1.8]
                            For Example: trywithresource feature of 7 will not work on 6.
D:\Yash>java -Dfile.encoding=UTF-8 My 
    «  -D= set a system property

O/P :- Unable to print UTF-8 characters on console command prompt, So go for Eclipse IDE 
Default Charset = UTF-8
UNICODE - Identifier[Variable Name] : Yash
UNICODE - Literal[Variable value] : ×™××©
windows-1255 : ×™××©

From Eclipse Editor

To support Unicode in Eclipse save Workspace|Class file Encoding with UTF-8

Class ➩ Project ► properties ► Run/Debug ➤ settings Eccoding - UTF 8 (OR)
WorkSpace ➜ Preferences ► General ► Workspace ► Text file encoding UTF-16

Or save the file as Save as UTF-8

MySQL DB « change Collation Name of a Database|Table to utf8_general_ci and Configuration settings file inorder to support Unicode.

Several other language interfaces to MySQL are based on the C client library. mysql-connector-java [5.1.34]

Test Query:

CREATE TABLE IF NOT EXISTS `unicodeinfo` (
  `id` int(5) NOT NULL AUTO_INCREMENT,
  `UserName` varchar(21) COLLATE utf8_unicode_ci NOT NULL,
  `Language` varchar(20) COLLATE utf8_unicode_ci NOT NULL,
  `Message` varchar(150) CHARACTER SET utf8 NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=6 ;

Sample Java Example to insert record.

public static void insertAutoIncrement_SQL(String UserName, String Language, String Message) {
    String DB_URL = "jdbc:mysql://localhost:3306/test", DB_User = "root", DB_Password = "";

    String insertSQL = "INSERT INTO `unicodeinfo`( `UserName`, `Language`, `Message`) VALUES (?,?,?)";
            //"INSERT INTO `unicodeinfo`(`id`, `UserName`, `Language`, `Message`) VALUES (?,?,?,?)";
    int primkey = 0 ;
    try {
        Class.forName("com.mysql.jdbc.Driver").newInstance();
        Connection conn = DriverManager.getConnection(DB_URL, DB_User, DB_Password);

        String columnNames[] = new String[] { "id" };

        PreparedStatement pstmt = conn.prepareStatement( insertSQL, columnNames );
        pstmt.setString(1, UserName );
        pstmt.setString(2, Language );
        pstmt.setString(3, Message );

        if (pstmt.executeUpdate() > 0) {
            // Retrieves any auto-generated keys created as a result of executing this Statement object
            java.sql.ResultSet generatedKeys = pstmt.getGeneratedKeys();
            if ( generatedKeys.next() ) {
                primkey = generatedKeys.getInt(1);
            }
        }
        System.out.println("Record updated with id = "+primkey);
    } catch (InstantiationException | IllegalAccessException | ClassNotFoundException | SQLException e) {
        e.printStackTrace();
    }
}

If you are using ConnectionPool DBCP use connectionProperties with value connectionProperties="useUnicode=yes;characterEncoding=utf8;"

Java Server Program deployed over Tomcat server:

In order to completely switch to using UTF-8, you need to make the following changes:
1. Set URIEncoding="UTF-8" on your in server.xml.
1. Use a character encoding filter with the default encoding set to UTF-8.
```
  CharacterEncoding
  org.apache.catalina.filters.SetCharacterEncodingFilter


  CharacterEncoding
  /msg/*
```
1. Change all your servlets to set the content type for responses and to include charset name in the content type to be UTF-8. Use response.setContentType("text/html; charset=UTF-8") or response.setCharacterEncoding("UTF-8").

@WebServlet(urlPatterns = { "/msg/locale", "/locale" }, loadOnStartup = 1 )
public class UnicodeTest extends HttpServlet {

    private static final long serialVersionUID = 5081877L;

    protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        //response.setCharacterEncoding("UTF-8");

        String unicodeMessage = URLDecoder.decode( request.getParameter("message"), "UTF-8");
                // request.getParameter("message"); // "×××©"
                //new String(request.getParameter("message").getBytes("ISO-8859-1"), "UTF-8");
        System.out.println("Unicode Message :"+ request.getParameter("message") );

        try {
            System.out.println("windows-1255 : " + new String(unicodeMessage.getBytes("UTF-8"), "UTF-8") );
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }

        UnicodeChars.insertAutoIncrement_SQL("Yash_777", "Telugu", unicodeMessage);

        PrintWriter writer = response.getWriter();
        if( unicodeMessage != null & !unicodeMessage.equals("") ) {
            writer.append("Received Message : "+ URLEncoder.encode( unicodeMessage, "UTF-8"));
        } else {
            writer.append("Not Received any Message.");
        }
        writer.flush();
        writer.close();
    }
}

Ajax Request using Javascript:

var methodHTTP = 'GET'; // "POST", "GET", "PUT"
var serverURL = 'http://localhost:8080/SteamingServlet/msg/locale';
var queryString = 'יאש';  // 'యశ్వంత్ ' //  'यश'
var enobjects = 'message='+ encodeURIComponent( queryString );

var xmlhttp = new XMLHttpRequest();

if( methodHTTP == 'GET' ) {  console.log( "XMLHttpRequest - GET" );

    xmlhttp.open(methodHTTP, serverURL+'?'+enobjects, true);

    xmlhttp.setRequestHeader("Content-type","application/x-javascript; charset=UTF-8");
    xmlhttp.setRequestHeader("Access-Control-Allow-Origin","*");

    xmlhttp.send( );
} else {  console.log( "XMLHttpRequest - POST" );

    xmlhttp.open(methodHTTP, serverURL, true);
    xmlhttp.setRequestHeader("Content-type","application/x-javascript; charset=UTF-8");
    xmlhttp.setRequestHeader("Access-Control-Allow-Origin","*");

    xmlhttp.send( enobjects );
}

xmlhttp.onreadystatechange = function() {
    if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
        console.log("Message :: ", xmlhttp.responseText);
        console.log("Decoded Message ::  ", decodeURIComponent( xmlhttp.responseText ) );

    } else {
        console.log("Status Code : ", xmlhttp.status);
    }
}

UTF-8: Hebrew charset from Ajax client to Tomcat server

Answers (1)

Related Questions