CODEWITHSUNDEEP
jsonrubyrestsinatra
TheFiddlerWins
TheFiddlerWins

Reputation: 922

Process the results of an HTTP request in the background - Ruby

I wrote a quick (hah) server to accept a JSON payload that contains a URL to download an archive from. After parsing the JSON I download the file, uncompress it and add metadata to each line. With a large file this can take a long time.

require 'thin'
require 'sinatra'
require 'json'

set :bind, '0.0.0.0'

#Listen on port 4567 at /core/action/parse-v1/parse-file for JSON post
post '/core/action/parse-v1/parse-file' do
request.body.rewind
request_payload = JSON.parse request.body.read

url = request_payload["s3Url"]
compressionType = request_payload["compressionType"]

Currently this all takes quite a while and holds the HTTP connection open the entire time. How do I pass the "stuff" part off immediately and let the HTTP server close its connection?

I've experimented a bit but I'm stuck.

Upvotes: 0

Views: 271

Answers (1)

Ross Geesman
Ross Geesman

Reputation: 33

If you want to return a response to the client before starting some long running process, maybe you need a queuing system? I'm not familiar with Sinatra and I don't know your use case but in Rails this is often accomplished with things like Delayed Job, Resque, or Sidekiq.

If those are too heavyweight for you, you could start another process directly with Ruby as is laid out in this answer.

Upvotes: 2

Related Questions