CODEWITHSUNDEEP
javaspringspring-bootcharacter-encoding
Murillo Goulart
Murillo Goulart

Reputation: 214

How to configure Spring boot to convert a request encoded in a different encoding than UTF-8

Problem

I have a legacy application that uses Apache CXF (JAX-RS) and Spring, which accepts requests with the content type application/json; charset=Windows-1252 in the request body. Now, I'm migrating this application to Spring Boot 2 and I have this issue:

My Spring Boot application can't handle correctly this request:

{
    "text": "Apenas um teste técnico çâãéüûà"
}

It handles as:

{
    "text": "Apenas um teste técnico çâãéüûà"
}

If the client uses the content-type application/json; charset=UTF-8, everything works, but I have no control over my clients, So I can't ensure it.

I did a lot of tests with some server.servlet.encoding configs, CharacterEncodingFilter configs and RequestBodyAdvice, but I have had no success so far.

Below you can see the tests with trace enabled for Spring classes:

POST with charset UTF-8 - OK

2022-11-01 21:02:20.695 TRACE 12932 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : POST "/string-encoding/v1/my-string", parameters={}, headers={masked} in DispatcherServlet 'dispatcherServlet'
2022-11-01 21:02:20.695 TRACE 12932 --- [nio-8080-exec-1] o.s.b.f.s.DefaultListableBeanFactory     : Returning cached instance of singleton bean 'stringTestController'
2022-11-01 21:02:20.695 TRACE 12932 --- [nio-8080-exec-1] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.sap.string.encoding.controller.StringTestController#postString(HttpServletRequest, MyString)
2022-11-01 21:02:39.166 TRACE 12932 --- [nio-8080-exec-1] m.m.a.RequestResponseBodyMethodProcessor : Read "application/json;charset=UTF-8" to [MyString{text='Apenas um teste técnico çâãéüûà'}]
2022-11-01 21:02:39.166 TRACE 12932 --- [nio-8080-exec-1] o.s.web.method.HandlerMethod             : Arguments: [org.apache.catalina.connector.RequestFacade@48c529f2, MyString{text='Apenas um teste técnico çâãéüûà'}]
Encoding: UTF-8, Text: Apenas um teste técnico çâãéüûà
2022-11-01 21:02:39.166 DEBUG 12932 --- [nio-8080-exec-1] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/json', given [*/*] and supported [application/json, application/*+json, application/json, application/*+json]
2022-11-01 21:02:39.166 TRACE 12932 --- [nio-8080-exec-1] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [MyString{text='Apenas um teste técnico çâãéüûà'}]
2022-11-01 21:02:39.716 TRACE 12932 --- [nio-8080-exec-1] s.w.s.m.m.a.RequestMappingHandlerAdapter : Applying default cacheSeconds=-1
2022-11-01 21:02:39.716 TRACE 12932 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : No view rendering, null ModelAndView returned.
2022-11-01 21:02:39.716 DEBUG 12932 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed 200 OK, headers={masked}
2022-11-01 21:02:39.716 TRACE 12932 --- [nio-8080-exec-1] o.s.b.w.s.f.OrderedRequestContextFilter  : Cleared thread-bound request context: org.apache.catalina.connector.RequestFacade@48c529f2

cURL UTF-8

curl --request POST \
  --url http://localhost:8080/string-encoding/v1/my-string \
  --header 'Content-Type: application/json; charset=UTF-8' \
  --data '{
    "text": "Apenas um teste técnico çâãéüûà"
}'

POST with charset Windows-1252 - Wrong

2022-11-01 21:01:26.759 TRACE 12932 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet        : POST "/string-encoding/v1/my-string", parameters={}, headers={masked} in DispatcherServlet 'dispatcherServlet'
2022-11-01 21:01:26.767 TRACE 12932 --- [nio-8080-exec-2] o.s.b.f.s.DefaultListableBeanFactory     : Returning cached instance of singleton bean 'stringTestController'
2022-11-01 21:01:26.767 TRACE 12932 --- [nio-8080-exec-2] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.sap.string.encoding.controller.StringTestController#postString(HttpServletRequest, MyString)
2022-11-01 21:01:42.091 TRACE 12932 --- [nio-8080-exec-2] m.m.a.RequestResponseBodyMethodProcessor : Read "application/json;charset=Windows-1252" to [MyString{text='Apenas um teste técnico çâãéüûà'}]
2022-11-01 21:01:42.107 TRACE 12932 --- [nio-8080-exec-2] o.s.web.method.HandlerMethod             : Arguments: [org.apache.catalina.connector.RequestFacade@48c529f2, MyString{text='Apenas um teste técnico çâãéüûà'}]
Encoding: UTF-8, Text: Apenas um teste técnico çâãéüûà
2022-11-01 21:01:42.137 DEBUG 12932 --- [nio-8080-exec-2] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/json', given [*/*] and supported [application/json, application/*+json, application/json, application/*+json]
2022-11-01 21:01:42.137 TRACE 12932 --- [nio-8080-exec-2] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [MyString{text='Apenas um teste técnico çâãéüûà'}]
2022-11-01 21:01:43.526 TRACE 12932 --- [nio-8080-exec-2] s.w.s.m.m.a.RequestMappingHandlerAdapter : Applying default cacheSeconds=-1
2022-11-01 21:01:43.526 TRACE 12932 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet        : No view rendering, null ModelAndView returned.
2022-11-01 21:01:43.526 DEBUG 12932 --- [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet        : Completed 200 OK, headers={masked}
2022-11-01 21:01:43.526 TRACE 12932 --- [nio-8080-exec-2] o.s.b.w.s.f.OrderedRequestContextFilter  : Cleared thread-bound request context: org.apache.catalina.connector.RequestFacade@48c529f2

cURL Windows-1252

curl --request POST \
  --url http://localhost:8080/string-encoding/v1/my-string \
  --header 'Content-Type: application/json; charset=Windows-1252' \
  --data '{
    "text": "Apenas um teste técnico çâãéüûà"
}'

Question

My Spring boot application runs perfectly with UTF-8 and handles the requests correctly for these cases. I have only to ensure the backward compatibility as it works in the legacy application.

Do I need some customized filter to convert from Windows-1252 to UTF-8? Or make some configuration in the MappingJackson2HttpMessageConverter?

Upvotes: 3

Views: 1937

Answers (1)

Olivier
Olivier

Reputation: 18122

This log line:

Apenas um teste técnico çâãéüûÃ

clearly shows that a UTF-8 string was interpreted as windows-1252 (and not the other way around).

The explanation is this: your curl test is wrong. You're sending a UTF-8 string but claiming it is in windows-1252.

To do the test correctly, you should create a file with your JSON content, make sure it is encoded in windows-1252, and use a command like this:

curl -d @test-1252.json -H "Content-Type: application/json; charset=windows-1252" http://localhost:8080/string-encoding/v1/my-string

Upvotes: 2

Related Questions