Spring Boot - Streaming Apache FileUpload - Stream ended unexpectedly

Question

I am trying to use Apache Streaming API with Spring Boot(disabled Spring Multipart resolver). But when I try to convert the InputStream to byte[] I get an exception - Stream ended unexpectedly. Any idea what is wrong? Code sample:

//Server side code - Spring Boot REST Controller
@PostMapping
public ResponseEntity<MyObjectDto> upload(HttpServletRequest request)
    {
    FileItemIterator itemIterator;
    MyObject myObject=null;
    if (!ServletFileUpload.isMultipartContent(request)) {
        throw new RuntimeException("Invalid content passed for upload");
    }
    ServletFileUpload servletFileUpload = new ServletFileUpload();
    itemIterator = servletFileUpload.getItemIterator(request);
    while (itemIterator.hasNext()) {
        FileItemStream fileItem = itemIterator.next();
        InputStream inputStream = fileItem.openStream();
        if (!fileItem.isFormField()) {
            byte[] bytes = IOUtils.toByteArray(inputStream);
           //...Pass bytes for further processing
           myObject =doSomething(bytes)
        }
    } 
    return ResponseEntity.ok(Mapper.map(myObject));
}


//Spring  multipart - disabled
spring.http.multipart.enabled=false

// Client code using Apache Streaming API

Request.Post(uri)
              .body(buildMultipartEntityNew(FileUtils.openInputStream(new File("src/test/resources/my.pdf"))))
              .execute()
              .handleResponse(responseHandler);

// Multipart Entity Builder
private static HttpEntity buildMultipartEntity(InputStream inputStream) throws JsonProcessingException {

    return MultipartEntityBuilder.create()
                                 .setMode(HttpMultipartMode.BROWSER_COMPATIBLE)
                                 .addBinaryBody("file", inputStream,
                                                ContentType.APPLICATION_OCTET_STREAM,
                                                "teamfile")
                                .build();
}

Exception:

org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:1005) at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:903) at java.io.InputStream.read(InputStream.java:101) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2146) at org.apache.commons.io.IOUtils.copy(IOUtils.java:2102) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2123) at org.apache.commons.io.IOUtils.copy(IOUtils.java:2078) at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:721)

Answer 1

I figured the problem. Incase someone runs in to similar issue first thing to check is any filters in the chain are trying to wrap/reconstruct the request(containing inputstream). If you know the exact filter you can disable it

 @Bean
    public FilterRegistrationBean registration(PreAuthenticationFilter filter) {
      FilterRegistrationBean registration = new FilterRegistrationBean(filter);
      registration.setEnabled(false);
      return registration;
}

Or you can test by disabling them all

    public class DefaultFiltersBeanFactoryPostProcessor implements BeanFactoryPostProcessor {

    @Override
    public void postProcessBeanFactory(ConfigurableListableBeanFactory bf)
        throws BeansException {
        DefaultListableBeanFactory beanFactory = (DefaultListableBeanFactory)bf;

        Arrays.stream(beanFactory.getBeanNamesForType(javax.servlet.Filter.class))
              .forEach(
                  name -> {
                      BeanDefinition definition =
                          BeanDefinitionBuilder.genericBeanDefinition(FilterRegistrationBean.class)
                                               .setScope(BeanDefinition.SCOPE_SINGLETON)
                                               .addConstructorArgReference(name)
                                               .addConstructorArgValue(new ServletRegistrationBean[] {})
                                               .addPropertyValue("enabled", false)
                                               .getBeanDefinition();

                      beanFactory.registerBeanDefinition(name + "FilterRegistrationBean", definition);
                  });
    }
}

In my case a custom security filter was unwrapping the request and reconstructing the inputstream and it is only reconstructing the inputstream partially and hence the error - MalformedStream.

I arrived at this by trying these options

1. Disabling all security

security:
  basic:
   enabled: false
management:
  security:
   enabled: false

2. Removing Autoconfigured Springs Default Security

spring:
 autoconfigure:
   exclude: org.springframework.boot.autoconfigure.security.SecurityAutoConfiguration

Once I knew security is the issue. I disabled security just for POSTs with inputstream and inputstream is coming along unscathed.

 @Override public void configure(WebSecurity web) throws Exception {
    web.ignoring()
       .requestMatchers(ignoreNonJsonMediaTypes())
       .and()
       .ignoring()
       .antMatchers(HttpMethod.POST);

    super.configure(web);
}

private MediaTypeRequestMatcher ignoreNonJsonMediaTypes() {
    MediaTypeRequestMatcher
        matcher =
        new MediaTypeRequestMatcher(new HeaderContentNegotiationStrategy(), MediaType.APPLICATION_JSON);
    matcher.setIgnoredMediaTypes(Collections.singleton(MediaType.ALL));
    return matcher;
}