Breadsnshreds
Breadsnshreds

Reputation: 1

How do I use FFMPEG/libav to access the data in individual audio samples?

The end result is I'm trying to visualise the audio waveform to use in a DAW-like software. So I want to get each sample's value and draw it. With that in mind, I'm currently stumped by trying to gain access to the values stored in each sample. For the time being, I'm just trying to access the value in the first sample - I'll build it into a loop once I have some working code.

I started off by following the code in this example. However, LibAV/FFMPEG has been updated since then, so a lot of the code is deprecated or straight up doesn't work the same anymore.

Based on the example above, I believe the logic is as follows:

  1. get the formatting info of the audio file
  2. get audio stream info from the format
  3. check that the codec required for the stream is an audio codec
  4. get the codec context (I think this is info about the codec) - This is where it gets kinda confusing for me
  5. create an empty packet and frame to use - packets are for holding compressed data and frames are for holding uncompressed data
  6. the format reads the first frame from the audio file into our packet
  7. pass that packet into the codec context to be decoded
  8. pass our frame to the codec context to receive the uncompressed audio data of the first frame
  9. create a buffer to hold the values and try allocating samples to it from our frame

From debugging my code, I can see that step 7 succeeds and the packet that was empty receives some data. In step 8, the frame doesn't receive any data. This is what I need help with. I get that if I get the frame, assuming a stereo audio file, I should have two samples per frame, so really I just need your help to get uncompressed data into the frame.

I've scoured through the documentation for loads of different classes and I'm pretty sure I'm using the right classes and functions to achieve my goal, but evidently not (I'm also using Qt, so I'm using qDebug throughout, and QString to hold the URL for the audio file as path). So without further ado, here's my code:

// Step 1 - get the formatting info of the audio file
    AVFormatContext* format = avformat_alloc_context();
    if (avformat_open_input(&format, path.toStdString().c_str(), NULL, NULL) != 0) {
        qDebug() << "Could not open file " << path;
        return -1;
    }

// Step 2 - get audio stream info from the format
    if (avformat_find_stream_info(format, NULL) < 0) {
        qDebug() << "Could not retrieve stream info from file " << path;
        return -1;
    }

// Step 3 - check that the codec required for the stream is an audio codec
    int stream_index =- 1;
    for (unsigned int i=0; i<format->nb_streams; i++) {
        if (format->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
            stream_index = i;
            break;
        }
    }

    if (stream_index == -1) {
        qDebug() << "Could not retrieve audio stream from file " << path;
        return -1;
    }

// Step 4 -get the codec context
    const AVCodec *codec = avcodec_find_decoder(format->streams[stream_index]->codecpar->codec_id);
    AVCodecContext *codecContext = avcodec_alloc_context3(codec);
    avcodec_open2(codecContext, codec, NULL);

// Step 5 - create an empty packet and frame to use
    AVPacket *packet = av_packet_alloc();
    AVFrame *frame = av_frame_alloc();

// Step 6 - the format reads the first frame from the audio file into our packet
    av_read_frame(format, packet);
// Step 7 - pass that packet into the codec context to be decoded
    avcodec_send_packet(codecContext, packet);
//Step 8 - pass our frame to the codec context to receive the uncompressed audio data of the first frame
    avcodec_receive_frame(codecContext, frame);

// Step 9 - create a buffer to hold the values and try allocating samples to it from our frame
    double *buffer;
    av_samples_alloc((uint8_t**) &buffer, NULL, 1, frame->nb_samples, AV_SAMPLE_FMT_DBL, 0);
    qDebug () << "packet: " << &packet;
    qDebug() << "frame: " <<  frame;
    qDebug () << "buffer: " << buffer;

For the time being, step 9 is incomplete as you can probably tell. But for now, I need help with step 8. Am I missing a step, using the wrong function, wrong class? Cheers.

Upvotes: 0

Views: 865

Answers (0)

Related Questions