Character encoding issue for httpost between android app and spring rest service

Question

I am trying to make my android app talk with my spring REST web service. I am making httpost from android as below code. However I am getting the exception below. whatever I set for content type of my post request it did not work. if there is not non-english character in my post object, it works well. However if there there is şŞüÜöÖgG etc. it gives that error. I need your advice.

public SowHttpUtil executeConnection(Object obj) throws Exception {
    try{
        HttpPost httppost = new HttpPost(serviceUrl);
        if(obj != null){
            StringEntity se = new StringEntity(objToString(obj));
            se.setContentEncoding(HTTP.UTF_8);
            se.setContentType("application/json");
            httppost.setEntity(se);
        }
        HttpClient httpclient = new DefaultHttpClient();
        //httppost.setHeader("Accept", "application/json");
        //httppost.setHeader("Content-Type", "application/json;charset="+HTTP.UTF_8);
        //httppost.addHeader("Accept", "application/json");
        //httppost.addHeader("Content-Type", "application/json;charset="+HTTP.UTF_8);

        HttpResponse response = httpclient.execute(httppost); 
        HttpEntity entity = response.getEntity();
        if(response.getStatusLine().getStatusCode() == 200){
            InputStream inStream = entity.getContent();
            responseString = convertStreamToString(inStream);
        }else{
            Log.e("RESPONSE STATUS ERROR : " , "RSCode : "+response.getStatusLine().getStatusCode());
            throw new SowException(R.string.error4);            
        }
    } catch(HttpHostConnectException e){
        throw new SowException(R.string.error4); 
    }
    return this;
}

The exception I am getting at server side (spring):

Could not read JSON: Invalid UTF-8 start byte 0xfc
 at [Source: org.apache.catalina.connector.CoyoteInputStream@79e0123b; line: 3, column: 29]; nested exception is org.codehaus.jackson.JsonParseException: Invalid UTF-8 start byte 0xfc
 at [Source: org.apache.catalina.connector.CoyoteInputStream@79e0123b; line: 3, column: 29] 
 org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Invalid UTF-8 start byte 0xfc
 at [Source: org.apache.catalina.connector.CoyoteInputStream@79e0123b; line: 3, column: 29]; nested exception is org.codehaus.jackson.JsonParseException: Invalid UTF-8 start byte 0xfc
 at [Source: org.apache.catalina.connector.CoyoteInputStream@79e0123b; line: 3, column: 29]
    at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.readJavaType(MappingJacksonHttpMessageConverter.java:187)
    at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.read(MappingJacksonHttpMessageConverter.java:179)
    at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodArgumentResolver.readWithMessageConverters(AbstractMessageConverterMethodArgumentResolver.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.readWithMessageConverters(RequestResponseBodyMethodProcessor.java:183)
    at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.resolveArgument(RequestResponseBodyMethodProcessor.java:98)
    at org.springframework.web.method.support.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:79)
    at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:157)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:124)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:690)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:945)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:876)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:863)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:108)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.codehaus.jackson.JsonParseException: Invalid UTF-8 start byte 0xfc
 at [Source: org.apache.catalina.connector.CoyoteInputStream@79e0123b; line: 3, column: 29]
    at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433)
    at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:521)
    at org.codehaus.jackson.impl.Utf8StreamParser._reportInvalidInitial(Utf8StreamParser.java:2825)
    at org.codehaus.jackson.impl.Utf8StreamParser._reportInvalidChar(Utf8StreamParser.java:2819)
    at org.codehaus.jackson.impl.Utf8StreamParser._finishString2(Utf8StreamParser.java:1978)
    at org.codehaus.jackson.impl.Utf8StreamParser._finishString(Utf8StreamParser.java:1905)
    at org.codehaus.jackson.impl.Utf8StreamParser.getText(Utf8StreamParser.java:276)
    at org.codehaus.jackson.map.deser.std.StringDeserializer.deserialize(StringDeserializer.java:26)
    at org.codehaus.jackson.map.deser.std.StringDeserializer.deserialize(StringDeserializer.java:13)
    at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
    at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
    at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
    at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
    at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732)
    at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1923)
    at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.readJavaType(MappingJacksonHttpMessageConverter.java:184)
    ... 38 more

Answer 1

You should use

new StringEntity(content, HTTP.UTF_8);

instead.

The constructor for StringEntity calls getBytes() on the supplied string with the charset (check the source code). If no charset is provided, then the default one is used (and it's not UTF-8).

Calling setContentEncoding() afterwards changes the HTTP header, but not the content itself. Hence the mismatch (0xfc is not valid UTF-8)