CODEWITHSUNDEEP
c#arrays
VoodooChild
VoodooChild

Reputation: 9784

Splitting array of objects and then process it in batches

What would be a good way to call the Execute method in batches of rulesObjs? Lets say the list have more than 10,000 objects and I want to call Execute with no more than 500 at a time.

    public static List<object> ExecutePolicy()
    {
        Policy policy = new Policy();

        List<object> rules = GetRules();

        object[] rulesObjs = rules.ToArray();

        // Call this method with array of object, but in batches.
        policy.Execute(rulesObjs);

        return rulesObjs.ToList();
    }

    private static List<object> GetRules()
    {
        // get the rules via some process
        return new List<object>();
    }
}

public sealed class Policy
{
    public void Execute(params object[] rules)
    {
        // Process rules...
    }
}

I do not have control over Execute() method.

Upvotes: 0

Views: 6252

Answers (4)

mdimai666
mdimai666

Reputation: 787

you could use new method

Enumerable.Chunk(items, 50)

like

List<object> rules = GetRules();

foreach (var batch in rules.Chunk(50))
{
    // Call this method with array of object, but in batches.
    policy.Execute(batch.ToArray());
}

return rules;

Upvotes: 0

Steve Konves
Steve Konves

Reputation: 2668

With a reference to System.Linq you can use skip and take:

int total = 10000;
int chunkSize = 500;
for (int i = 0; i < total; i += chunkSize )
{
    var chunk = rulesObjs.Skip(i).Take(chunkSize).ToArray();

    policy.Execute(chunk);
}

Upvotes: 3

Servy
Servy

Reputation: 203837

List<object> rules = GetRules();
int batchSize = 500;
int currentBatch = 0;

while (currentBatch * batchSize < rules.Count)
{
    object[] nextBatch = rules.Skip(currentBatch * batchSize)
        .Take(batchSize).ToArray();
    //use batch
    currentBatch++;
}

Upvotes: 7

Sam I am says Reinstate Monica
Sam I am says Reinstate Monica

Reputation: 31204

Well, if you have control over the Execute() method, the best way to do it would be to pass an index to that method so that it knows at which index of the array to start at. 

public void Execute(int startIndex, /*optional*/ int endIndex, params object[] rules)
{
    // Process rules...
}

Don't worry about passing too much data at once. Behind the scenes, your array is just a pointer, so you're only passing a reference anyways.


If you don't have control over the Execute() method, then you can make a new array for your section, using Array.Copy, and process that new array.

Upvotes: 2

Related Questions