fine-uploader PHP Server Side Merge

Question

I'm been experimenting with Fine Uploader. I am really interested in the chunking and resume features, but I'm experiencing difficulties putting the files back together server side;

What I've found is that I have to allow for a blank file extension on the server side to allow the upload of the chunks, otherwise the upload will fail with unknown file type. It uploads the chunks fine with file names such as "blob" and "blob63" (no file extension) however is does not merge them back at completion of upload.

Any help or pointers would be appreciated.

$('#edit-file-uploader').fineUploader({
            request: {
                endpoint: 'upload.php'
            },
            multiple: false,
            validation:{
                allowedExtentions: ['stl', 'obj', '3ds', 'zpr', 'zip'],
                sizeLimit: 104857600  // 100mb * 1024 (kb) * 1024 (bytes)
            },
            text: {
                uploadButton: 'Select File'
            },
            autoUpload: false, 
            chunking: {
              enabled: true
            },
            callbacks: {
                 onComplete: function(id, fileName, responseJSON) {
                  if (responseJSON.success) {
                     /** some code here **??
                  }
           }

    });

And this is the server side script (PHP):

// list of valid extensions, ex. array("stl", "xml", "bmp")
$allowedExtensions = array("stl", "");
// max file size in bytes
$sizeLimit = null;

$uploader = new qqFileUploader($allowedExtensions, $sizeLimit);

// Call handleUpload() with the name of the folder, relative to PHP's getcwd()
$result = $uploader->handleUpload('uploads/');

// to pass data through iframe you will need to encode all html tags
echo htmlspecialchars(json_encode($result), ENT_NOQUOTES);

/******************************************/



/**
 * Handle file uploads via XMLHttpRequest
 */
class qqUploadedFileXhr {
    /**
     * Save the file to the specified path
     * @return boolean TRUE on success
     */
    public function save($path) {    
        $input = fopen("php://input", "r");
        $temp = tmpfile();
        $realSize = stream_copy_to_stream($input, $temp);
        fclose($input);

        if ($realSize != $this->getSize()){            
            return false;
        }

        $target = fopen($path, "w");        
        fseek($temp, 0, SEEK_SET);
        stream_copy_to_stream($temp, $target);
        fclose($target);

        return true;
    }

    /**
     * Get the original filename
     * @return string filename
     */
    public function getName() {
        return $_GET['qqfile'];
    }

    /**
     * Get the file size
     * @return integer file-size in byte
     */
    public function getSize() {
        if (isset($_SERVER["CONTENT_LENGTH"])){
            return (int)$_SERVER["CONTENT_LENGTH"];            
        } else {
            throw new Exception('Getting content length is not supported.');
        }      
    }   
}

/**
 * Handle file uploads via regular form post (uses the $_FILES array)
 */
class qqUploadedFileForm {

    /**
     * Save the file to the specified path
     * @return boolean TRUE on success
     */
    public function save($path) {
        return move_uploaded_file($_FILES['qqfile']['tmp_name'], $path);
    }

    /**
     * Get the original filename
     * @return string filename
     */
    public function getName() {
        return $_FILES['qqfile']['name'];
    }

    /**
     * Get the file size
     * @return integer file-size in byte
     */
    public function getSize() {
        return $_FILES['qqfile']['size'];
    }
}

/**
 * Class that encapsulates the file-upload internals
 */
class qqFileUploader {
    private $allowedExtensions;
    private $sizeLimit;
    private $file;
    private $uploadName;

    /**
     * @param array $allowedExtensions; defaults to an empty array
     * @param int $sizeLimit; defaults to the server's upload_max_filesize setting
     */
    function __construct(array $allowedExtensions = null, $sizeLimit = null){
        if($allowedExtensions===null) {
            $allowedExtensions = array();
        }
        if($sizeLimit===null) {
            $sizeLimit = $this->toBytes(ini_get('upload_max_filesize'));
        }

        $allowedExtensions = array_map("strtolower", $allowedExtensions);

        $this->allowedExtensions = $allowedExtensions;        
        $this->sizeLimit = $sizeLimit;

        $this->checkServerSettings();       

        if(!isset($_SERVER['CONTENT_TYPE'])) {
            $this->file = false;    
        } else if (strpos(strtolower($_SERVER['CONTENT_TYPE']), 'multipart/') === 0) {
            $this->file = new qqUploadedFileForm();
        } else {
            $this->file = new qqUploadedFileXhr();
        }
    }

    /**
     * Get the name of the uploaded file
     * @return string
     */
    public function getUploadName(){
        if( isset( $this->uploadName ) )
            return $this->uploadName;
    }

    /**
     * Get the original filename
     * @return string filename
     */
    public function getName(){
        if ($this->file)
            return $this->file->getName();
    }

    /**
     * Internal function that checks if server's may sizes match the
     * object's maximum size for uploads
     */
    private function checkServerSettings(){        
        $postSize = $this->toBytes(ini_get('post_max_size'));
        $uploadSize = $this->toBytes(ini_get('upload_max_filesize'));        

        if ($postSize < $this->sizeLimit || $uploadSize < $this->sizeLimit){
            $size = max(1, $this->sizeLimit / 1024 / 1024) . 'M';             
            die(json_encode(array('error'=>'increase post_max_size and upload_max_filesize to ' . $size)));    
        }        
    }

    /**
     * Convert a given size with units to bytes
     * @param string $str
     */
    private function toBytes($str){
        $val = trim($str);
        $last = strtolower($str[strlen($str)-1]);
        switch($last) {
            case 'g': $val *= 1024;
            case 'm': $val *= 1024;
            case 'k': $val *= 1024;        
        }
        return $val;
    }

    /**
     * Handle the uploaded file
     * @param string $uploadDirectory
     * @param string $replaceOldFile=true
     * @returns array('success'=>true) or array('error'=>'error message')
     */
    function handleUpload($uploadDirectory, $replaceOldFile = FALSE){
        if (!is_writable($uploadDirectory)){
            return array('error' => "Server error. Upload directory isn't writable.");
        }

        if (!$this->file){
            return array('error' => 'No files were uploaded.');
        }

        $size = $this->file->getSize();

        if ($size == 0) {
            return array('error' => 'File is empty');
        }

        if ($size > $this->sizeLimit) {
            return array('error' => 'File is too large');
        }

        $pathinfo = pathinfo($this->file->getName());
        $filename = $pathinfo['filename'];
        //$filename = md5(uniqid());
        $ext = @$pathinfo['extension'];        // hide notices if extension is empty

        if($this->allowedExtensions && !in_array(strtolower($ext), $this->allowedExtensions)){
            $these = implode(', ', $this->allowedExtensions);
            return array('error' => 'File has an invalid extension, it should be one of '. $these . '.');
        }

        $ext = ($ext == '') ? $ext : '.' . $ext;

        if(!$replaceOldFile){
            /// don't overwrite previous files that were uploaded
            while (file_exists($uploadDirectory . DIRECTORY_SEPARATOR . $filename . $ext)) {
                $filename .= rand(10, 99);
            }
        }

        $this->uploadName = $filename . $ext;

        if ($this->file->save($uploadDirectory . DIRECTORY_SEPARATOR . $filename . $ext)){
            return array('success'=>true);
        } else {
            return array('error'=> 'Could not save uploaded file.' .
                'The upload was cancelled, or server error encountered');
        }

    }    
}

Answer 1

In order to handle chunked requests, you MUST store each chunk separately in your filesystem.
How you name these chunks or where you store them is up to you, but I suggest you name them using the UUID provided by Fine Uploader and append the part number parameter included with each chunked request. After the last chunk has been sent, combine all chunks into one file, with the proper name, and return a standard success response as described in the Fine Uploader documentation. The original name of the file is, by default, passed in a qqfilename parameter with each request. This is also discussed in the docs and the blog.

It doesn't look like you've made any attempt to handle chunks server-side. There is a PHP example in the Widen/fine-uploader-server repo that you can use. Also, the documentation has a "server-side" section that explains how to handle chunking in detail. I'm guessing you did not read this. Have a look.) in the Widen/fine-uploader-server repo that you can use. Also, the documentation has a "server-side" section that explains how to handle chunking in detail. I'm guessing you did not read this. Have a look.

Note that, starting with Fine Uploader 3.8 (set to release VERY soon) you will be able to delegate all server-side upload handling to Amazon S3, as Fine Uploader will provide tight integration with S3 that sends all of your files directly to your bucket from the browser without you having to worry about constructing a policy document, making REST API calls, handling responses from S3, etc. I mention this as using S3 means that you never have to worry about handling things like chunked requests on your server again.