CODEWITHSUNDEEP
ruby-on-railsfile-uploadresque
Jonathan C.
Jonathan C.

Reputation: 227

Sending uploaded file to resque worker to be processed

I just started using resque to do some processing on some very large files in the background, and I'm having trouble figuring out how to pass a file to a resque worker. I use rails to handle the file upload, and rails creates an ActionDispatch::Http::UploadedFile object for each file uploaded from the form.

How do send this file to a resque worker? I tried sending a custom hash of just the pathname of the temporary file and original filename, but I can't reopen the temporary file in the resque worker anymore (just a normal Errno::ENOENT - No such file or directory) because rails seems to delete that temporary file after the request ends.

Upvotes: 6

Views: 2662

Answers (2)

user2297272
user2297272

Reputation: 119

I just spent two days trying to do this and finally figured it out. You need to Base64 encode the file so that it can be serialized into json. Then you need to decode it in the worker and create a new

ActionDispatch::Http::UploadedFile

Here's how to encode and pass to resque:

// You only need to encode the actual file, everything else in the 
// ActionDispatch::Http::UploadedFile object is just string or a hash of strings

file = params[:file] // Your ActionDispatch::Http::UploadedFile object 
file.tempfile.binmode
file.tempfile = Base64.encode64(file.tempfile.read)

Resque.enqueue(QueueWorker, params)

And Here's how to decode and convert back to an object within your worker

class QueueWorker
    @queue = :main_queue

    def self.perform(params)
        file = params['file']
        tempfile = Tempfile.new('file')
        tempfile.binmode
        tempfile.write(Base64.decode64(file['tempfile']))

        // Now that the file is decoded you need to build a new
        // ActionDispatch::Http::UploadedFile with the decoded tempfile and the other
        // attritubes you passed in.

        file = ActionDispatch::Http::UploadedFile.new(tempfile: tempfile, filename: file['original_filename'], type: file['content_type'], head: file['headers'])

        // This object is now the same as the one in your controller in params[:file]
    end
end

Upvotes: 5

klochner
klochner

Reputation: 8125

Http::UploadedFileisn't accessible once the request finishes. You need to write the file somewhere (or use s3 as temp storage). Pass resque the path to the file that you wrote.

Upvotes: 6

Related Questions