CODEWITHSUNDEEP
javafile-upload
Sunil
Sunil

Reputation: 79

Tomcat 6.0 Large File Upload (> 2 GB)

I am having issues with HTTP file upload with files larger than 2GB. Both server and client are 64 bit so there must be no restriction of 2GB from the system perspective. I did the following:

  1. In Apache LimitRequestBody = 0 (http://httpd.apache.org/docs/2.0/mod/core.html#LimitRequestBody)
  2. In Tomcat Connector maxPostSize = 0 (http://tomcat.apache.org/tomcat-5.5-doc/config/ajp.html)

I am using apache commons file upload. I also tried to set max file size using ServerFileUpload setMaxFileSize method.

I am able to upload files less than 2GB (I tried 1.88GB file successfully). Please direct me, what am I missing here?

To be more specific ServletFileUpload.parseRequest method returns 0 FileItems when uploading large files

Here is the code snippet:

if (isMultipartForm()) {
try {
    DiskFileItemFactory factory = new DiskFileItemFactory();
    factory.setSizeThreshold(SIZE_THRESHOLD);//SIZE_THRESHOLD = 4MB
    ServletFileUpload upload = new ServletFileUpload(factory);
        //upload.setFileSizeMax(3000000000L); Tried this too
    upload.setProgressListener(progressListener);
    items = upload.parseRequest(request);
    if(items != null && items.size() == 0)
    return new CommandResultSet(false, "NoItemsFoundInRequest");
    return new CommandResultSet(true, "" + ( (items!=null) ? items.size() : ""));
} catch(FileUploadException e) {
    e.printStackTrace();
    System.out.println("Exception in MultipartFormManager. Can not parse request.");    
    return new CommandResultSet(false, e.getMessage());
    }
}

Upvotes: 5

Views: 12724

Answers (4)

Shahid
Shahid

Reputation: 31

If you are using common-fileupload 1.1 or old then you cannot upload morethan (1.9888)~2GB. The problem here is this jar is calling a method called getContentLength which is of int type, so your request can handle only size up to Integer.MAX_VALUE. In latest version of common-fileupload like 1.3 this was solved. Hope this helps.

Upvotes: 3

murwath
murwath

Reputation: 9

Jsp code :
<script src="../lib/app/configurator.data.ajax.js" type="text/javascript"></script>
<script src="../lib/ui/jquery.fileupload.js"></script>
<html>

<script language="Javascript">
var varb = '';
var test = 0;
var count = 1;
$(function () {
var maxChunkSize = 30000000; //SET CHUNK SIZE HERE
var your_global_var_for_form_submit = '';

var params = {
        year: threeDSelectedYear,
        series: threeDSelectedSeries
};
/*File upload bind with form, 'fileupload' is my form id. As sumit triggers
    for file ulaod will submit my form*/
/* replaceFileInput: true, */
$('#fileupload').fileupload({
maxChunkSize: maxChunkSize,
url: efccustom.getEfcContext()+'upload/uploadZip?year='+threeDSelectedYear+'&series='+threeDSelectedSeries,             //URL WHERE FILE TO BE UPLOADED
    error: function (jqXHR, textStatus, errorThrown) {
    // Called for each failed chunk upload
        $(".fileupload-loading").html("");
         $('.ANALYZE_DIALOG').dialog("close");       
    },

    success: function (data, textStatus, jqXHR) {
    /*This event will be called on success of upload*/
    count = parseInt($('#counter').val()) + 1;
    $('#counter').val(count);

    $('#ttk').val(count);
    data.ttk = 7;
    console.log(" count ",count , data);
    },

    submit: function (e, data) {
    /*This event will be called on submit here i am
                     adding variable to form*/
    //console.log($('#zip_uploaded_file').val());      

    //console.log(data.originalFiles[0].name);            
    test = data.originalFiles[0];
    $('#fname').val(data.originalFiles[0].name);
    $('#trequests').val(Math.ceil(data.originalFiles[0].size/maxChunkSize));
    $('#counter').val('1');
    },

    progress: function (e, data) {
    /*PROGRESS BAR CODE WILL BE HERE */
        $(".fileupload-loading").html('<img src="../public/themeroller/images/throbber.gif" alt="Uploading Please Wait..." title="Uploading Please Wait...." />');       
    },

    add: function (e, data) {
    $('.browsedFileName').html(data.originalFiles[0].name);
    your_global_var_for_form_submit = data;
    },

    done: function (e, data) {

        ajaxcall.Data._get('upload/extractZipNCreateJson',params,function(data2) {
            alert("file upload success ");
             $(".fileupload-loading").html("");
             $('.ANALYZE_DIALOG').dialog("close");
        });


    }

});
/*This is my button click event on which i am submitting my form*/
  $('#button').click(function(){
    your_global_var_for_form_submit.submit();
  });
});
</script>

<html>

<body>


<form id="fileupload" enctype="multipart/form-data"> 
<div class="row fileupload-buttonbar">
<div class="span7">
<!--<input type="file" name="files" id="file" /> -->

<input type="file" id="zip_uploaded_file"  name="zip_uploaded_file"  />
<input type="hidden" name="counter" id="counter" value="1" />
<input type="hidden" name="fname" id="fname" value="" />
<input type="hidden" name="trequests" id="trequests" value="1" />

<input type="hidden" name="ttk" id="ttk" value="1" />
<input type="button" id="button" name="button" value="submit" />
 <span class='browsedFileName'></span>
</div>
</div>
<!-- The loading indicator is shown during file processing -->
<div class="fileupload-loading"></div>
</form>
</body>


Controller COde :
@RequestMapping(value = "/uploadZip", method = RequestMethod.POST, consumes = "multipart/form-data")
    @ResponseBody
    public ResponseEntity<String>
     uploadZip(HttpServletRequest request, HttpServletResponse response)
            throws IOException, IllegalArgumentException {
        try {       
        String year = request.getParameter("year");
        String series = request.getParameter("series");
        log.info(" year " + year + " series " + series);
        boolean isMultipart = ServletFileUpload.isMultipartContent(request);
        if (isMultipart) {

            FileItemFactory factory = new DiskFileItemFactory();
            ServletFileUpload upload = new ServletFileUpload(factory);
            List items = upload.parseRequest(request);
            Iterator iterator = items.iterator();

            HttpSession session = request.getSession();
            UserContext userContext = (UserContext) session
                    .getAttribute(ApplicationConstant.USER_CONTEXT);
            String userName = userContext.getUser();
            String workSpaceInUse = userContext.getCurrentWorkSpace();           
            FileItem item = null;
            boolean fileNotExistFlag;
            String fileExtension;
            while (iterator.hasNext()) {
                item = (FileItem) iterator.next();

                String content = item.getContentType();
                log.info(" content "+content);
                log.info(" File Type Getting Uploaded :"+content);
                if (!item.isFormField()) {
            /* Here in IOUtils the Third Parameter true tells that the small chunks of data that comes need to be added to the same File */
            IOUtils.copy(fileItem.getInputStream(), new FileOutputStream(new File(threeDPath+"/"+year+"/"+series+"/"+uploadZipFileName),true));               
                }
            }
            return null;
        }   
        }
        catch(Exception e) {
            return handleError(new RuntimeException("Unexpected error while uploading ZIP", e));
        }
        return null;
    }

Upvotes: 0

StackOverflowed
StackOverflowed

Reputation: 5975

If you're expecting file uploads of those sizes, I wouldn't rely on a straight browser upload. A Java applet or maybe even a Flash file (not sure if possible, not a Flash person) would be my recommendation, so you can break the files into chunks. If the upload is interrupted, you can resume from where it was last left off.

Upvotes: 1

Mike
Mike

Reputation: 11

I certainly may new wrong but I have not found that even the 64 bit Browsers handle uploads of greater than 2GB. The problem is not the server but the browser. You will find strangely enough that most modern browsers will happily download files larger than 2GB from a standard server, no special configuration required.

Upvotes: 1

Related Questions