CODEWITHSUNDEEP
c++videowebrtchololenswindows-mixed-reality
davidlll
davidlll

Reputation: 11

VideoStabilizationEnabled in webrtc for Hololens 2 not work

I am trying to enable the the video stabilization for the Hololens 2 in webrtc lib. I modified the following code in impl_webrtc_VideoCapturer.cpp in https://github.com/webrtc-uwp/webrtc-uwp-sdk/tree/master

mrcVideoEffectDefinition.VideoStabilizationEnabled(true);
mrcVideoEffectDefinition.VideoStabilizationBufferLength(15);

And there is an exception occured in "ConvertToI420" function below:

const int conversionResult = libyuv::ConvertToI420(
  videoFrame, videoFrameLength, buffer.get()->MutableDataY(),
  buffer.get()->StrideY(), buffer.get()->MutableDataU(),
  buffer.get()->StrideU(), buffer.get()->MutableDataV(),
  buffer.get()->StrideV(), 0, 0,  // No Cropping
  width, height, target_width, target_height, rotation_mode,
  frameInfo.fourcc);

I found that the videoFrameLength is not correct when VideoStabilizationEnabled set to true, With width = 896, height = 504, the videoFrameLength should be 677376 (896x504x3/2, yuv) for "ConvertToI420" function to work properly.

normal: videoFrameLength: 677376 (VideoStabilizationEnabled set to false)
wrong: videoFrameLength: 497664 (VideoStabilizationEnabled set to true) which is not correct.

Any idea? Thanks

Upvotes: 0

Views: 213

Answers (1)

davidlll
davidlll

Reputation: 11

I fixed it by editing the following, the resolution is changed after enable the the video stabilization, it needs to update the related variables.

in impl_webrtc_VideoCaptureMediaSink.cpp, get the buffer width and height and add callback to alert impl_webrtc_VideoCapturer.cpp

VideoCaptureStreamSink::SetCurrentMediaType(IMFMediaType *pMediaType)
{
  .....
  MFGetAttributeSize(pMediaType, MF_MT_FRAME_SIZE, &width, &height);
  _callback->OnResolutionChanged(width, height);
}

in impl_webrtc_VideoCaptureMediaSink.h, add new interface function

class ISinkCallback {
public:
  .....
  virtual void OnResolutionChanged(UINT32 width, UINT32 height) = 0;
};

in impl_webrtc_VideoCapturer.cpp, add below function to update resolution

void CaptureDevice::OnResolutionChanged(UINT32 width, UINT32 height) 
{
  OutputDebugString(L"OnResolutionChanged ");
  frame_info_.width = width;
  frame_info_.height = height;
}

Upvotes: 1

Related Questions