CODEWITHSUNDEEP
javaspringlarge-filesresttemplate
user3971703
user3971703

Reputation:

unable to post large files using spring rest template w/ custom interceptor

I am trying to POST a large file from one microservice to another using spring rest template POST w/ custom interceptor as follows:

SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate = new RestTemplate(requestFactory);
restTemplate.getInterceptors().add({customInterceptor});
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("file", {InputStreamResource});
body.add("metadata", {JSON string});
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);
restTemplate.exchange({url}, HttpMethod.POST, requestEntity, ...);

(It makes no difference whether I use a SimpleClientHttpRequestFactory or HttpComponentsClientHttpRequestFactory)

Adding an interceptor results in the creation of an new InterceptingClientHttpRequestFactory (which wraps the original request factory) on a call to getRequestFactory.

This works fine for smaller files but for large files - since requests are never delegated to the original request factory, no streaming ever occurs and, hence, results in a java.lang.OutOfMemoryError: Java heap space exception.

Any help would be appreciated.

Upvotes: 4

Views: 1387

Answers (2)

HeatZync
HeatZync

Reputation: 330

I have also come across this issue, but switching to WebClient is not an option at this point in time so I opted for a work around by implementing a new ClientHttpRequestFactory (albeit via an AbstractClientHttpRequestFactoryWrapper); it mainly works due to the fact that we are only tweaking the headers and not the body.

SimpleClientHttpRequestFactory simpleRequestFactory =
    new SimpleClientHttpRequestFactory();

simpleRequestFactory.setConnectTimeout(10000);
simpleRequestFactory.setReadTimeout(60000);
simpleRequestFactory.setBufferRequestBody(false); // this enables streaming

SomeHeaderInterceptingClientHttpRequestFactory interceptingRequestFactory =
    new SomeHeaderInterceptingClientHttpRequestFactory(simpleRequestFactory);

RestTemplate restTemplate = new RestTemplate(interceptingRequestFactory);

and this is what the SomeHeaderInterceptingClientHttpRequestFactory looks like:

public class SomeHeaderInterceptingClientHttpRequestFactory
    extends AbstractClientHttpRequestFactoryWrapper {

    // you can have fields here that are initialized in the constructor
    // e.g. a service that supplies the header value that you want to populate

    public SomeHeaderInterceptingClientHttpRequestFactory(ClientHttpRequestFactory requestFactory) {
        super(requestFactory);
        // initialize fields
    }

    @Override
    protected ClientHttpRequest createRequest(URI uri,
                                              HttpMethod httpMethod,
                                              ClientHttpRequestFactory requestFactory)
                                              throws IOException {
        ClientHttpRequest request = requestFactory.createRequest(uri, httpMethod);
        HttpHeaders headers = request.getHeaders();

        headers.set("SOME_HEADER", "some value");

        return request;
    }
}

If you want to do something to the body then you could also try to implement a new ClientHttpRequest. You will then return a new instance of your newly introduced ClientHttpRequest in the createRequest method of your newly introduced ClientHttpRequestFactory.

Upvotes: 0

asinkxcoswt
asinkxcoswt

Reputation: 2554

It seems this problem of RestTemplate will not be fixed according to this issue

we're not going to support this, now that the WebClient is available and provides first-class support for streaming.

Note that WebClient is available in Spring 5.

Upvotes: 1

Related Questions