'+' (plus sign) not encoded with RestTemplate using String url, but interpreted as ' ' (space)

Question

We are moving from Java 8 to Java 11, and thus, from Spring Boot 1.5.6 to 2.1.2. We noticed, that when using RestTemplate, the '+' sign is not encoded to '%2B' anymore (changes by SPR-14828). This would be okay, because RFC3986 doesn't list '+' as a reserved character, but it is still interpreted as a ' ' (space) when received in a Spring Boot endpoint.

We have a search query which can take optional timestamps as query parameters. The query looks something like http://example.com/search?beforeTimestamp=2019-01-21T14:56:50%2B00:00.

We can't figure out how to send an encoded plus sign, without it being double-encoded. Query parameter 2019-01-21T14:56:50+00:00 would be interpreted as 2019-01-21T14:56:50 00:00. If we were to encode the parameter ourselves (2019-01-21T14:56:50%2B00:00), then it would be received and interpreted as 2019-01-21T14:56:50%252B00:00.

An additional constraint is, that we want to set the base url elsewhere, when setting up the restTemplate, not where the query is being executed.

Alternatively, is there a way to force '+' not to be interpreted as ' ' by the endpoint?

I have written a short example demonstrating some ways of achieving stricter encoding with their drawbacks explained as comments:

package com.example.clientandserver;

import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.client.RestTemplateBuilder;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.DefaultUriBuilderFactory;
import org.springframework.web.util.UriComponentsBuilder;
import org.springframework.web.util.UriUtils;

import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;

@SpringBootApplication
@RestController
public class ClientAndServerApp implements CommandLineRunner {

    public static void main(String[] args) {
        SpringApplication.run(ClientAndServerApp.class, args);
    }

    @Override
    public void run(String... args) {
        String beforeTimestamp = "2019-01-21T14:56:50+00:00";

        // Previously - base url and raw params (encoded automatically). 
        // This worked in the earlier version of Spring Boot
        {
            RestTemplate restTemplate = new RestTemplateBuilder()
               .rootUri("http://localhost:8080").build();
            UriComponentsBuilder b = UriComponentsBuilder.fromPath("/search");
            if (beforeTimestamp != null) {
                b.queryParam("beforeTimestamp", beforeTimestamp);
            }
            restTemplate.getForEntity(b.toUriString(), Object.class);
            // Received: 2019-01-21T14:56:50 00:00
            //       Plus sign missing here ^
        }

        // Option 1 - no base url and encoding the param ourselves.
        {
            RestTemplate restTemplate = new RestTemplate();
            UriComponentsBuilder b = UriComponentsBuilder
                .fromHttpUrl("http://localhost:8080/search");
            if (beforeTimestamp != null) {
                b.queryParam(
                    "beforeTimestamp",
                    UriUtils.encode(beforeTimestamp, StandardCharsets.UTF_8)
                );
            }
            restTemplate.getForEntity(
                b.build(true).toUri(), Object.class
            ).getBody();
            // Received: 2019-01-21T14:56:50+00:00
        }

        // Option 2 - with templated base url, query parameter is not optional.
        {
            RestTemplate restTemplate = new RestTemplateBuilder()
                .rootUri("http://localhost:8080")
                .uriTemplateHandler(new DefaultUriBuilderFactory())
                .build();
            Map<String, String> params = new HashMap<>();
            params.put("beforeTimestamp", beforeTimestamp);
            restTemplate.getForEntity(
                "/search?beforeTimestamp={beforeTimestamp}",
                Object.class,
                params);
            // Received: 2019-01-21T14:56:50+00:00
        }
    }

    @GetMapping("/search")
    public void search(@RequestParam String beforeTimestamp) {
        System.out.println("Received: " + beforeTimestamp);
    }
}

Answer 1

Just want to add on!, You can try with this without configure every where u used RestTemplate in your project, config also support UTF-8 encoding respons:

public class PlusEncoderInterceptor implements ClientHttpRequestInterceptor {

@Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution)
        throws IOException {
    return execution.execute(new HttpRequestWrapper(request) {
        @NotNull
        @Override
        public URI getURI() {
            URI u = super.getURI();
            String strictlyEscapedQuery = StringUtils.replace(u.getRawQuery(), "+", "%2B");
            return UriComponentsBuilder.fromUri(u).replaceQuery(strictlyEscapedQuery).build(true).toUri();
        }
    }, body);
}
}

@Configuration
public class RestTemplateConfig {
@Bean
public RestTemplate restTemplate() {
    RestTemplate restTemplate = new RestTemplateBuilder().interceptors(new PlusEncoderInterceptor()).build();
    restTemplate.getMessageConverters().add(0, new StringHttpMessageConverter(StandardCharsets.UTF_8));
    return restTemplate;
}
}

Answer 2

To get around this kind of issue, I found it easier to build the URI by hand.

URI uri = new URI(siteProperties.getBaseUrl()
  + "v3/elements/"
  + URLEncoder.encode("user/" + user + "/type/" + type, UTF_8)
  + "/"
  + URLEncoder.encode(id, UTF_8)
);

restTemplate.exchange(uri, DELETE, new HttpEntity<>(httpHeaders), Void.class);

Answer 3

Thanks https://stackoverflow.com/users/4466695/gregor-eesmaa, it solved my issue. Just wanted to add that in case if you can format URL before calling RestTemplate, you can fix the URL at once (instead of replacing it in PlusEncoderInterceptor):

UriComponentsBuilder uriBuilder = UriComponentsBuilder.fromUriString("/search");
uriBuilder.queryParam("beforeTimestamp", "2019-01-21T14:56:50+00:00");
URI uriPlus = uriBuilder.encode().build(false).toUri();

// import org.springframework.util.StringUtils;
String strictlyEscapedQuery = StringUtils.replace(uriPlus.getRawQuery(), "+", "%2B");
URI uri = UriComponentsBuilder.fromUri(uriPlus)
        .replaceQuery(strictlyEscapedQuery)
        .build(true).toUri();

// prints "/search?beforeTimestamp=2019-01-21T14:56:50%2B00:00"
System.out.println(uri);

Then you can use in RestTemplate call:

RequestEntity<?> requestEntity = RequestEntity.get(uri).build();
ResponseEntity<String> responseEntity = restTemplate.exchange(requestEntity, String.class);

Answer 4

The issue has been discussed here as well.

Encoding of URI Variables on RestTemplate [SPR-16202]

A simpler solution is to set the encoding mode on the URI builder to VALUES_ONLY.

    DefaultUriBuilderFactory builderFactory = new DefaultUriBuilderFactory();
    builderFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.VALUES_ONLY);
    RestTemplate restTemplate = new RestTemplateBuilder()
            .rootUri("http://localhost:8080")
            .uriTemplateHandler(builderFactory)
            .build();

This achieved the same result as using the PlusEncodingInterceptor when using query parameters.

Answer 5

We realized the URL can be modified in an interceptor after the encoding is done. So a solution would be to use an interceptor, that encodes the plus sign in the query params.

RestTemplate restTemplate = new RestTemplateBuilder()
        .rootUri("http://localhost:8080")
        .interceptors(new PlusEncoderInterceptor())
        .build();

A shortened example:

public class PlusEncoderInterceptor implements ClientHttpRequestInterceptor {

    @Override
    public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
        return execution.execute(new HttpRequestWrapper(request) {
            @Override
            public URI getURI() {
                URI u = super.getURI();
                String strictlyEscapedQuery = StringUtils.replace(u.getRawQuery(), "+", "%2B");
                return UriComponentsBuilder.fromUri(u)
                        .replaceQuery(strictlyEscapedQuery)
                        .build(true).toUri();
            }
        }, body);
    }
}