CODEWITHSUNDEEP
parallel-processing
madguy
madguy

Reputation: 45

Parallel processing of records from database table

I have a relational table that is being populated by an application. There is a column named o_number which can be used to group the records.

I have another application that is basically having a Spring Scheduler. This application is deployed on multiple servers. I want to understand if there is a way where I can make sure that each of the scheduler instances processes a unique group of records in parallel. If a set of records are being processed by one server, it should not be picked up by another one. Also, in order to scale, we would want to increase the number of instances of the scheduler application.

Thanks Anup

Upvotes: 1

Views: 522

Answers (1)

joker
joker

Reputation: 3752

This is a general question, so here's my general 2 cents on the matter.

You create a new layer managing the requesting originating from your application instances to the database. So, probably you will be building a new code/project running on the same server as the database (or some other server). The application instances will be talking to that managing layer instead of the database directly.

The manager will keep track of which records are requested hence fetch records that are yet to be processed upon each new request.

Upvotes: 2

Related Questions