CODEWITHSUNDEEP
pythonmatlabaudiofftvisualization
StackReloader
StackReloader

Reputation: 11

Python or Matlab: synchronized audio playback with data visualization (waveform/STFT-spectrogram etc.)

I want to do a python or matlab project which visualizes audio data while being synchronized with playback.

In details, it means in a GUI, I have two main regions, one for data visulization and the other for audio playback. The form of data visulization can be defined specifically, e.g. as waveform or as a STFT-spectrogram. When I hit the button of audio playback, I can not only listen to the music, but also have a real-time cursor in the data visulization area, which is synchronized with the audio playback and indicates the time position of playback. And I would like to point out that I don't want it look like a digital oscilloscope which refreshes the spectrum or waveform for every buffer time. I want the data visualization as showed for the whole time range of audio, only the cursor to dynamically synchronized/ moved with audio playback.

So I want to ask you, do you know any existing project or packages that can realize similar function as I described? Or do you have any recommendation on how I can put the idea into reality from scratch?

Upvotes: 1

Views: 211

Answers (1)

Jussi Nurminen
Jussi Nurminen

Reputation: 2408

In principle, it's relatively straightforward. You'd probably want to use:

  • a GUI library such as PyQT
  • libraries for STFT and other maths; SciPy and NumPy are your friends here
  • an audio library to read and play back the audio data

Also you'll need to use threads, since you want to simultaneously play back audio and update your GUI etc. Thus, some understanding of multithreading is useful.

Though it's unproblematic in a sense, there's a lot of details to get right. If you don't have experience in some or all of these areas, there's a lot you need to learn. Of course, that could be a positive thing.

The biggest issue might be the visualization of the audio data. Matplotlib is a popular plotting library, but it's a bit tricky to integrate into a PyQt app, and probably the realtime requirement makes things even harder.

Upvotes: 1

Related Questions