CODEWITHSUNDEEP
phparrays
sam
sam

Reputation: 10084

Iterate through an array in batches using php

I've got an API I'm working with that accepts a call every 5 seconds, any more than that it wont respond. In each call it will accept 5 records in a batch. I've got a list of 1000s of records that I need to check using the api, so what I'm trying to do is send it my list of records in broken up into batches of 5 every 5 seconds.

I can get most of it to work, but the bit I can't figure is breaking down the list of records which is an array in batches, any idea how to do this?

This is the code I was using below, but it's outputting each individual part of the array every 5 seconds, rather than in batches of 5.

$my_array = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20];

    foreach ($my_array as $key => $value) {
        sleep (5);
        echo $value;
    }

Upvotes: 8

Views: 9812

Answers (4)

Andre
Andre

Reputation: 190

Just count to five (or whatever batch size is) when iterating your array to separate batches from each other. For this you can use the division remainder.

$my_array = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20];
$batch_size = 5;
$timeout = 5;
foreach ($my_array as $key => $value) {
    if (($key % $batch_size == 0) && $key !== 0){
        echo "---\n";
        #sleep($timeout);
    } 
    echo "$value\n";
}

if($key % $batch_size == 0 && $key !== 0 &&) means "divide current position by batch size. in case the remainder is 0 (and current position is not 0) this is the batch boundary."

Upvotes: 1

Rey0bs
Rey0bs

Reputation: 1217

You can create a specific iterator to iterate over batches :

class BatchIterator extends \IteratorIterator
{
    private array $batch = [];

    public function __construct(
        \Traversable $iterator,
        private readonly int $batchSize,
    ) {
        parent::__construct($iterator);
    }

    public function rewind(): void
    {
        parent::rewind();
        $this->next();
    }

    public function next(): void
    {
        $this->batch = [];
        for ($i = 0; $i < $this->batchSize && parent::valid(); $i++) {
            $this->batch[] = parent::current();
            parent::next();
        }
    }

    public function current(): array
    {
        return $this->batch;
    }

    public function valid(): bool
    {
        return (bool) $this->batch;
    }
}

And use it to iterate over your data:

$size = 5;
foreach (new BatchIterator(new ArrayIterator($my_array), $size) as $value) {
    // Here $value is an array of max. 5 items
}

Code sample created from Guzzle3 library: https://github.com/Guzzle3/iterator/blob/master/ChunkedIterator.php

Upvotes: 0

matt
matt

Reputation: 700

In case it helps anyone else, I wrote a function that will allow you to process an array in chunks. The description and details are here:

https://totaldev.com/php-process-arrays-batches/

The main difference between mine and array_chunk is that mine doesn't return an array of smaller arrays. It takes a user-defined function as a closure that will handle the small batches. Here is the function:

// Iterate through an array and pass batches to a Closure
function arrayBatch($arr, $batchSize, $closure) {
    $batch = [];
    foreach($arr as $i) {
        $batch[] = $i;

        // See if we have the right amount in the batch
        if(count($batch) === $batchSize) {
            // Pass the batch into the Closure
            $closure($batch);

            // Reset the batch
            $batch = [];
        }
    }

    // See if we have any leftover ids to process
    if(count($batch)) $closure($batch);
}

You can use it like this:

// Use array in batches
arrayBatch($my_array, 5, function($batch) {
    // Do whataver you need to with the $batch of 5 items here...

    sleep (5);
});

Upvotes: 4

exussum
exussum

Reputation: 18560

you could have a second loop of 

$batch_of = 5;
$batch = array_chunk($my_array, $batch_of);
foreach($batch as $b) {

    foreach ($b as $key => $value) {

        echo $value;
    }
   sleep (5);
}

Which would work as intended

Upvotes: 18

Related Questions