CODEWITHSUNDEEP
pythonnumpyscipyfft
Sergey Grechin
Sergey Grechin

Reputation: 966

needed: FFT implememtatin in Python using preallocated buffer to store results

I'm writing a python app which will do a lot of FFT conversions (audio analysis), my sampled audio are stored in float32 numpy arrays. At first, I though it would be straigtforward to find one of the out-of-the-box FFT functions, like this one:

http://docs.scipy.org/doc/numpy/reference/generated/numpy.fft.rfft.html#numpy.fft.rfft

however, I didn't find any functions which require a pre-allocated array to store FFT results (frequency domain).

A assume that it means each time I run fft, it will allocate another portion of memory to store the spectrum, which I see as a problem, considering the amount of times it will have to be run.

What I really need is a FFT implementation which I will be able to supply a preallocated array to fill in with results, to make memory consumption more manageable.

Do you have any ideas how to do it?

And are these memory concerns really such a big deal after all? (if it was, I guess, standard libs would have had to address it somehow).

There are some C libraries out there which I can use, but first I'd like to tru to find a native and no-hassle python way of accomlishing this.

Thanks!

Upvotes: 2

Views: 653

Answers (2)

Dennis Sakva
Dennis Sakva

Reputation: 1467

Don't worry about memory consumption. Previously allocated memory will be freed as soon as there are no references to it. And compared to the time needed to do FFT the time to do allocation-deallocation is negligible.

Here is a quick test I call 1000 times FFT and measure memory usage. There's no increase.

import psutil, os

process = psutil.Process(os.getpid())

print process.get_memory_info()[0] / float(2 ** 20)# print memory usage
for _ in range(10):
    FFTSize=2**np.random.randint(16,20)
    A=np.fft.fft(np.exp(2j * np.pi * np.arange(FFTSize) / FFTSize))
    A=A*2
    B=np.exp(A)
    C=np.log(B)

print process.get_memory_info()[0] / float(2 ** 20)

Upvotes: 1

TheBlackCat
TheBlackCat

Reputation: 10328

Memory shouldn't be a problem. Although exactly how memory is freed is a little complicated in Python, for practical purposes, if you del the array when you no longer need it, you don't need to worry about it anymore.

However, it is possible to write the FFT to an existing array (although it may have to store intermediate values, these will be quickly freed). You can do that like so:

>>> import numpy as np
>>> inarray = np.random.random(1000)
>>> targarray = np.empty(len(inarray)//2+1, dtype='complex')
>>> testarray = targarray  # just for checking the array wasn't overwritten
>>> targarray[:] = np.fft.rfft(inarray)
>>> targarray is testarray
True

If you really want to be able to directly write to a chosen array, you can use pyfftw:

>>> import numpy as np
>>> import pyfftw
>>> inarray = np.random.random(1000)
>>> targarray = np.empty(len(inarray)//2+1, dtype='complex')
>>> testarray = targarray  # just for checking the array wasn't overwritten
>>> pyfftw.FFTW(inarray, targarray).execute()
>>> targarray is testarray
True

Upvotes: 3

Related Questions