CODEWITHSUNDEEP
web-servicesazureasp.net-web-apirestful-architectureapi-design
user2058904
user2058904

Reputation:

Hosting Long Running Web Service Operations

I'm creating a web service to handle file processing (Which may be time consuming) and I'm trying to future proof it by making sure the web server doesn't end up with too many files in memory at once.

To achieve this, I'm allowing a user to upload the file, I immediately put it into Azure blob storage and will give the user an ID for that file.

In the mean time the file has been added into a processing queue in the background, (Handled by a separate application), which when done, will add the output file into azure and update the database to reflect the completed operation.

What I'm expecting the user to do is use the ID I gave them at the point of them uploading the file successfully, to call the web service to say "Is my file ready yet? if so, please give it to me".

Is this an acceptable practice?

Initially I had it so that the user would have to wait for the entire operation to finish, which was much cleaner, but it felt like it wouldn't scale very well.

What I was thinking was that if I separate out the file processing component from the request, and the file goes into a processing queue, it might get processed immediately, but it might also take several minutes (Depending on how busy the web service gets - worst case scenario, it could be hours if I don't implement auto scaling), and I was concerned about leaving a web request hanging around for potentially a very long time.

Am I over engineering this, or is this a sensible approach?

Upvotes: 0

Views: 44

Answers (1)

Chris Simon
Chris Simon

Reputation: 6505

This is a very sensible approach. In fact, REST specifically caters for this with the 202 Accepted response code.

See Asynchronous REST for more details.

To summarise:

  1. when the user submits the original request, don't just give them an ID for the file, return status code 202 Accepted and set the Location header to the URI of a temporary resource they can poll to obtain request state
    1. the body at this temporary resource might contain ETA to completion, or just that it is 'in progress'
  2. When the request is completed, this URI should return 303 See Other with the Location Header set to the URI where they can actually obtain the file.

Upvotes: 0

Related Questions