CODEWITHSUNDEEP
c#windowsaudiopcmms-media-foundation
karthik vr
karthik vr

Reputation: 65

Media foundation: Writing G711 PCMU to mp4

We have a requirement (Windows UWP app) to store audio and video data received from cameras(RTP packets) in mp4.Video format is h264 and audio is g711 pcmu

We are using media foundation(c# using MF .Net) sinkWriter to write video data to mp4 which works fine. I would like to know how to write audio samples to mp4. I tried the following way:

private void SetupAudioMediaType(out IMFMediaType mediaType, in Guid audioSubType)
{    
    HResult hr = HResult.S_OK;

    hr = MFExtern.MFCreateMediaType(out mediaType);
    if (!hr.Succeeded())
    {
        Debug.Fail("MFCreateMediaType for audio failed " + hr.ToString());
        return;
    }

    hr = mediaType.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
    if (!hr.Succeeded())
    {
        Debug.Fail("Set MF_MT_MAJOR_TYPE media-out failed " + hr.ToString());
        return ;
    }
    hr = mediaType.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, audioSubType);
    if (!hr.Succeeded())
    {
        Debug.Fail("Set MF_MT_SUBTYPE media-out failed " + hr.ToString());
        return ;
    }

    hr = mediaType.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 48000);
    if (!hr.Succeeded())
    {
        Debug.Fail("Set MF_MT_AUDIO_SAMPLES_PER_SECOND media-out failed " + hr.ToString());
        return 
    }

    hr = mediaType.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 1);
    if (!hr.Succeeded())
    {
        Debug.Fail("Set MF_MT_AUDIO_NUM_CHANNELS media-out failed " + hr.ToString());
        return ;
    }

    hr = mediaType.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 8);
    if (!hr.Succeeded())
    {
        Debug.Fail("Set MF_MT_AUDIO_BITS_PER_SAMPLE media-out failed " + hr.ToString());
        return ;
    }

    return errorCode;
}



private MultiplexerErrorCode SetupAudio()
{
    HResult hr = HResult.S_OK;

    IMFMediaType mediaTypeOut = null;
    IMFMediaType mediaTypeIn = null;

    SetupAudioMediaType(out mediaTypeOut, MFMediaType.AAC);  // or mp3 (MP4 in windows support mp3 or aac)
    if (errorCode != MultiplexerErrorCode.Success)
    {
        Debug.Fail("setupAudioMediaType output failed:", errorCode.ToString());
    }
    else
    {
        hr = sinkWriter.AddStream(mediaTypeOut, out audioStreamIndex);
        if (!hr.Succeeded())
        {
            Debug.Fail("AddStream  audio  failed " + hr.ToString());
        }
        else
        {
            Guid PcmuAudioSubType = (new FourCC(7,0,0,0)).ToMediaSubtype();   //PCMU
            SetupAudioMediaType(out mediaTypeIn, PcmuAudioSubType);
            hr = sinkWriter.SetInputMediaType(audioStreamIndex, mediaTypeIn, null);
            if (!hr.Succeeded())
            {
                Debug.Fail("SetInputMediaType audio  failed " + hr.ToString());
            }
        }
    }

    return ;
}

SetInputMediaType returns error MF_E_INVALIDMEDIATYPE. From my analysis following are the reasons for the error

1) I think PCMU input type is not supported. It should be PCM. Is this understanding corect? If so does this mean I have to decode PCMU to PCM.If so is there any windows c# API which does this ? And once this decode is done , what are the bits per sample in the ouput pcm. Is it 16?

2)Even if I provide pcm as input type, SetInputMediaType returns MF_E_INVALIDMEDIATYPE error. Is it because the aac encoder supports only sampling rate of 44.1 & 48Khz. (mp3 supports 32 Khz,...). If my understanding is correct, how do I overcome the issue. Should I up-sample. If so how?

3)Is there any simpler way to write pcmu(8000 samples per second, 8 bits per sample) to mp4 along with the video frames

Upvotes: 2

Views: 1140

Answers (1)

mofo77
mofo77

Reputation: 1515

For 1) and 3) read this : muxing-only-audio-into-mp4-ffmpeg-failed

For 2)

  • decode G711 to PCM (CLSID_MULawCodecWrapper : 92B66080-5E2D-449E-90C4-C41F268E5514)
  • use an audio resampler (Audio Resampler DSP)
  • encode PCM resampled to AAC (AAC Encoder)

Perhaps CLSID_MULawCodecWrapper is able do to resampling. Perhaps you will have audio/video synchronization problems.

Upvotes: 2

Related Questions