CODEWITHSUNDEEP
python
Jason
Jason

Reputation: 137

Limit loop to N iterations per minute

Could someone please give me some help with limiting a loop to N iterations per minute in Python.

Lets say I have

limit = 5
for items in recvData:
     # > limit iterations in past minute? -> sleep for 60 seconds from last iterations before proceeding? #
     ... do work ...

How would I do the time check / sleeping to give the correct flow. I'm not worried about blocking the executing thread/process while it waits.

Thanks

Upvotes: 0

Views: 4613

Answers (4)

Nicomoto
Nicomoto

Reputation: 483

This would be the most appropriate answer : What's a good rate limiting algorithm?

I especially like the second answer using the decorator!

Upvotes: 0

aaronasterling
aaronasterling

Reputation: 70984

It should be noted that this is not "hard real time" code. this will be off slightly because of OS scheduling and the like. That being said, unless you know that you need hard real time, this should suffice. 

import time

limit = 5
starttime = time.time()
for i, item in enumerate(recvData):
    if not i + 1 % limit:
        sleeptime =starttime + 60 - time.time()
        if sleeptime > 0:
            time.sleep(sleeptime)
        starttime = time.time()
    #processing code

Upvotes: 3

pszilard
pszilard

Reputation: 1962

This very much depends on the type of work you're doing inside the loop and on how accurate you want this mechanism to work.

ATM I can come up with 2 possible schemes:

  • If one iteration takes about constant time than you could calculate an average from the first few iterations and "sleep" for 1 - iterationTime afterwards.
  • Otherwise, you can poll the time and recalculate the average every step (or a few steps).

Depending on the standard deviation of your single loop execution times both scheme can work quite well, but if the execution times are very varying, neither of them will. Also, if you want evenly distributed loop cycles and not only keep the average/min you have to distribute the sleep-s and do one after each iteration.

I am not familiar enough with Python to know how expensive is to query the time and what other Python-specific issues might pop up with sleep-ing, though.

Upvotes: 0

Katriel
Katriel

Reputation: 123612

Use grouper from the itertools recipes, combined with a time check.

import itertools, datetime, time
limit = 5

def grouper(n, iterable, fillvalue=None):
    "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return itertools.izip_longest(fillvalue=fillvalue, *args)

for items in grouper(limit, recvData):
    prev_minute = datetime.datetime.now().minute

    for item in items:
        # do stuff

    _, _, _, _, minute, second, _ = datetime.datetime.now()
    if minute == prev_minute:
        time.sleep( 60 - second )

Upvotes: 0

Related Questions