CODEWITHSUNDEEP
ruby-on-railsapiparallel-processingserver
Akshay
Akshay

Reputation: 830

How to perform parallel processing on a "single" rails API in server side?

There are a lot of methods to handle "multiple" API requests in server side. Parallel processing can be implemented here. But i would like to know how a single API can be parallely processed. For example: If an API request executes a method say method1.

    def method 1  
      .....
      ......
      .......
    end

If method1 is a long method which may take a long time for processing (include multiple loops and database queries), instead of processing it sequentially, is there a scope for parallel processing there?

One way would be using resque for creating background jobs. But is there any other way to do it and if so how should the code be written to accommodate the requirement.

And is there any server side method to do it which is not ruby specific?

Upvotes: 1

Views: 592

Answers (1)

max
max

Reputation: 102036

Note that there is a huge difference between event based servers and background jobs.

An event based server often runs on a single thread and uses callbacks for non-blocking IO. The most famous example is Node.js. For ruby there is the eventmachine library and various frameworks and simple HTTP servers.

An evented server can start processing one request and then switch to another request while the first is waiting for a database call for example.

Note that even if you have a event based server you can't be slow at processing requests. The user experience will suffer and clients will drop the connection.

Thats where background jobs (workers) come in, they let your web process finish fast so that it can send the response and start dealing with the next request. Slow processes like sending out emails or cleanup that don't have does not require user feedback or concurrency are farmed out to workers.

So in conclusion - if your application is slow then using parallel processing is not going to save you. Its not a silver bullet. Instead you should invest in optimising database queries and leveraging caching so that responses are fast.

While you could potentially run database queries or other operations in parallel in Rails, the greatly added complexity is probably not worth the performance gains.

What I mean here is that with what you are usually doing in Rails concurrency is not really applicable - you're fetching something from the DB and using it to make JSON or HTML. You can't really start rendering until you have the results back anyways. While you could potentially do something like fetch data and use it to render partials concurrently Rails does not support this out of the box since it would greatly increase the complexity while not offering much to the majority of the users of the framework.

As always - don't optimise prematurely.

Upvotes: 1

Related Questions