CODEWITHSUNDEEP
pythonperformancegenerator
Martin McBride
Martin McBride

Reputation: 498

Efficiency of generators

I am writing some software to create a complex waveform (actually a soundwave) as an array. Starting with some primitive waveforms (sine waves etc), there will be functions which combine them to create more complex waves, and yet more functions which combine those waves, etc.

It might look like:

f(mult(sine(), env(square(), ramp()))

but a lot more complex.

One way of doing this would be to make each function a generator, so that the whole function tree executes once per element with each generator yielding a single value each time.

The array could have several million elements, and the function tree could easily be 10 deep. Would generators be ridiculously inefficient for doing this?

The alternative would be for each function to create and return an entire array. This would presumably be more efficient, but has disadvantages (messier implementation, no results available until the end of the calculation, could use a lot of memory).

They always say you shouldn't try to second guess Python efficiency, but will generators take a long time in this case?

Upvotes: 4

Views: 2798

Answers (2)

fferri
fferri

Reputation: 18950

In my opinion, generators are a good fit for this task.

Some signals have finite time (like an envelope, or a ramp), but some other signals are infinite (like oscillators).

Using generators you should no worry about this aspect, because -like the zip() function- a function combining (e.g. multiplying) an oscillator with an envelope, would only consume a finite amount of items from oscillator gen, because there's at least one generator which yields a finite number of samples.

Yet, using generators is very elegant and pythonic.

Recall that a generator like this:

def sine(freq):
    phase = 0.0
    while True:
        yield math.sin(phase)
        phase += samplerate/freq

is just syntactic sugar for a class like this:

class sine:
    def __init__(self, freq):
        self.freq = freq
        self.phase = 0.0

    def __iter__(self):
        return self

    def __next__(self):
        v = math.sin(self.phase)
        self.phase += samplerate/freq
        return v
        # for this infinite gen we never raise StopIteration()

so the performance overhead is not much than any other solution you can handcraft (like the block processing, commonly used in DSP algorithms).

Perhaps you could gain some efficiency if instead of yielding individual samples, you yield blocks of samples (for example 1024 samples at time).

Upvotes: 1

Marcin
Marcin

Reputation: 49856

Generators are lazy sequences. They are perfect for use when you have sequences which may be very long, as long as you can operate piecewise (either elementwise, or on reasonably sized chunks).

This will tend to reduce your peak memory use. Just don't ruin that by then storing all elements of the sequence somewhere.

Upvotes: 2

Related Questions