CODEWITHSUNDEEP
persistent-storageorleans
user2995891
user2995891

Reputation: 111

Orleans - Bulk Data Import

I am currently learning about MS Orleans for our organization. I understand that Orleans Grains will remain syncronized with the DB as long as all DB updates are coming via the grains.

But what happens if there is some bulk process process (like processing of a data file) that updates / inserts / deletes records in DB?

Is there some process or pattern to use with Orleans to cater for this? Or do we need to process all bulk process via Grains? In case we process bulk operations via grains - do we go about it by updating each grain (seems very expensive if each grain updates itself into DB) or is there some bulk pattern to use to force all affected grains to 'refresh'?

Answer may be obvious. I didn't find anything in documentation about these scenarios.

We would be using Orleans as On-Premises installation with MS-SQL server.

Edit:

I am referring to a process that updates data of N grains. Having a single call that updates 1000 records is much better for sql then 1000 calls that update one record.. A concrete example would be stock update: each product stock would be a grain. Every 15 minutes or so, a file would arrive from 3rd party informing about stock quantity changes that happened outside of the application. This should update in the db and reflect in the grains. File may have 10k's records...

Upvotes: 2

Views: 599

Answers (1)

Gabi Kliot
Gabi Kliot

Reputation: 1383

You have a couple of options.

1) Upload via grains - the grains will cache the data and also store in the DB. This partially defeats your goal of efficient DB upload, so may not be what you want.

2) Bulk upload directly into the DB, but use grains to access and process/serve the data. The grains will read the data from the DB upon first request, cache it, serve for further requests. Also, all future data updates will go via grains. This is usually the most common pattern.

3) If you also need periodic data processing, even in the absence of serving requests, then after bulk uploading the grain, you can "bulk-kick" grains to start periodic processing. In this option you will write a controller loop (client logic for example) that will just call "Init" into the set of grains, one by one (but in parallel: call Init on X (X ~ 100) in parallel and await them all together) and then do the next batch. Grains will start a timer or a reminder upon Init.

Upvotes: 5

Related Questions