CODEWITHSUNDEEP
ruby-on-railsapiasynchronous
VAD
VAD

Reputation: 2401

Rails API, microservices, async/deferred responses

I have a Rails API which can handle requests from the clients. Clients use that API to perform analysis of their data. Client POSTs the data to API, API checks if that data have been analysed before. If so API just respond with analysis result. If the data haven't been analyzed before API:

  1. Tells client that analysis started.

  2. Establishes the connection with analyzing microservice.

  3. Performs asynchronous (or deferred or i don't know) request to the analyzing microservice and waiting for response. The analysis takes much time so neither the API nor the microservice should be blocked while doing it.

  4. When the response from analyzing microservice is returned API hands it to the client.

The main issue for me is to set up things such way that client could receive somehow the message "Your data had been sent to analysis" right after he performed the request. And then when analysis will be done client could receive its result.

The question is what approach I have to use in that case? Async responses, deferred responses, something else? And what known solutions could help me with that? Any gems?

I'm new to that stuff so I'm really sorry if I ask dumb questions.

Upvotes: 1

Views: 1199

Answers (2)

Kris
Kris

Reputation: 19948

If using HTTP you can only have one response to every request. To send multiple responses, i.e. "work in progress", then later the "results", you would need to use a different protocol, e.g. web sockets.

Since HTTP is so very common I'd stick with that in combination with background jobs. There are a couple of options which spring to mind.

  1. Polling: The API kicks off a background jobs (to call the microservice) and responds to the client with a URL which the client can ping periodically for the result. The URL would respond with some kind of "work in progress" status until the result is actually ready). The URL would need to include some kind of id so the API can lookup the background job.

The API would potentially have two URLS; /api/jobs/new and /api/jobs/<ID>. They would, in Rails, map to a controller new and show action.

  1. Webhooks: Have the client include a URL of its own in the request. Once the result is available have the background job hit the given URL with the result.

Either way, if using HTTP, you will not be able to handle the whole thing within a request/response, you will have to use some kind of background processing (so request to the microservice happens in a different process). You could look at Sidekiq, for example.

Here is an example for polling:

URL: example.com/api/jobs/new

  1. web app receives client request
  2. generates a unique id for the request, SecureRandom.uuid.
  3. starts a background job (Sidekiq) passing in the uuid and any other parameters needed
  4. respond with URL such as example.com/api/jobs/

--

background job

  1. sends request to microservice API and waits for response
  2. saves result to database with uuid

--

URL: example.com/api/jobs/UUID

  1. look in database for UUID, if not found respond that job is "in progress". If found return result found in database.

Upvotes: 2

leifg
leifg

Reputation: 9028

Depending on what kind of API you use. I assume your clients interact via HTTP.

If you want to build an asynchronous API over HTTP the first thing that you should do: accept the request, create a job, handle it in the background and immediately return.

For the client to get the response you have to 2 options:

  • Implement a status endpoint where clients can periodically poll the status of the job
  • Implement a callback via webhooks. So the client has to provide a URL which you then call after you're done.

A good start for background processing is the sidekiq gem or more general ActiveJob that ships with Rails.

Upvotes: 1

Related Questions