How to eliminate artifacting with fast update WriteableBitmap

Question

I'm trying to implement a video renderer with a WriteableBitmap.

My current implementation works fine for full screen 60fps content, but if I scrub the video (drag the playback back/forth to a new time) the renderer starts updating the WriteableBitmap after the second render cycle started, causing it to appear with artifacting (the frame rendering engine uses parallelism).

I tried implementing a double buffer to see if it could help, and it did to a certain point, but the artifacting continues.

public WriteableBitmap RenderedImage
{
    get => _frontBuffer;
    set => SetProperty(ref _frontBuffer, value);
}

public long CurrentRenderTime
{
    get => _currentRenderTime;
    set
    {
        if (SetProperty(ref _currentRenderTime, value))
            if (RenderedImage != null)
                _event.Set();
    }
}

private void RenderFrame()
{
    while (!_renderCancellationSource.IsCancellationRequested)
    {
        //Wait until the CurrentTime changes to render again.
        _event.WaitOne();

        //TEST: No noticeable difference with this lock.
        lock (Lock)
        {
            var currentTimestamp = CurrentRenderTime;

            //Draw the background and all tracks.
            DrawBackground(_backBufferAddress, _backStride, _backPixelWidth, _backPixelHeight);

            foreach (var track in Tracks)
                track.RenderAt(_backBufferAddress, _backStride, CurrentRenderTime);

            //Swap the back buffer with the front buffer.
            (_frontBuffer, _backBuffer) = (_backBuffer, _frontBuffer);
            (_frontBufferAddress, _backBufferAddress) = (_backBufferAddress, _frontBufferAddress);
            (_frontStride, _backStride) = (_backStride, _frontStride);
            (_frontPixelWidth, _backPixelWidth) = (_backPixelWidth, _frontPixelWidth);
            (_frontPixelHeight, _backPixelHeight) = (_backPixelHeight, _frontPixelHeight);

            //TEST: No difference with this check.
            if (currentTimestamp != CurrentRenderTime)
                continue;

            Application.Current.Dispatcher.Invoke(() =>
            {
                RenderedImage.Lock(); 
                RenderedImage.AddDirtyRect(new Int32Rect(0, 0, RenderedImage.PixelWidth, RenderedImage.PixelHeight));
                RenderedImage.Unlock();

                OnPropertyChanged(nameof(RenderedImage));

                _eventRenderer.Set();
            }, DispatcherPriority.Render);

            //TEST: No difference with this wait.
            _eventRenderer.WaitOne();
        }
    }
}

Lines marked with TEST are last-minute changes that provided no noticeable results.

---

More details about the implementation

The rendering must occur on another thread, and it's triggered by changes in the CurrentRenderTime by using a AutoResetEvent.

Pseudo-code:

Calculate timings, variable or fixed framerate.
Iterate through the timings.
Call render.
Wait (between-frame timing - time it took to render).

The WriteableBitmap is being displayed by a WPF Image element.

The playback mechanism is not the issue here, and anyway I use a fine-tuned timer resolution (PInvoking timeBeginPeriod).

This is the minimal example:

//This method runs in a separated thread.
private void RenderFrame()
{
    while (!_renderCancellationSource.IsCancellationRequested)
    {
        //Wait until the CurrentTime changes to render again.
        _event.WaitOne();

        lock (Lock)
        {
            //Draw the background and all tracks.
            DrawBackground(_backBufferAddress, _backStride, _backPixelWidth, _backPixelHeight);

            foreach (var track in Tracks)
                track.RenderAt(_backBufferAddress, _backStride, CurrentRenderTime);

            //Swap the back buffer with the front buffer.
            (_frontBuffer, _backBuffer) = (_backBuffer, _frontBuffer);
            (_frontBufferAddress, _backBufferAddress) = (_backBufferAddress, _frontBufferAddress);
            (_frontStride, _backStride) = (_backStride, _frontStride);
            (_frontPixelWidth, _backPixelWidth) = (_backPixelWidth, _frontPixelWidth);
            (_frontPixelHeight, _backPixelHeight) = (_backPixelHeight, _frontPixelHeight);

            //WriteableBitmap Lock/Unlock needs to run on the same thread as the UI.
            Application.Current.Dispatcher.Invoke(() =>
            {
                RenderedImage.Lock(); 
                RenderedImage.AddDirtyRect(new Int32Rect(0, 0, RenderedImage.PixelWidth, RenderedImage.PixelHeight));
                RenderedImage.Unlock();

                //Since I swapped the buffers, I need to tell WPF about that.
                OnPropertyChanged(nameof(RenderedImage));
            }, DispatcherPriority.Render);
        }
    }
}

---

Possible solution

One solution, but less performant would be to copy data between buffers, instead of switching them.

private void RenderFrame()
{
    while (!_renderCancellationSource.IsCancellationRequested)
    {
        _event.WaitOne();

        lock (Lock)
        {
            DrawBackground(_backBufferAddress, _backStride, _backPixelWidth, _backPixelHeight);

            foreach (var track in Tracks)
                track.RenderAt(_backBufferAddress, _backStride, Project.Width, Project.Height, CurrentRenderTime);

            Kernel32.CopyMemory(_frontBufferAddress, _backBufferAddress, (uint)(_backPixelHeight * _backStride));

            Application.Current.Dispatcher.Invoke(() =>
            {
                RenderedImage.Lock();
                RenderedImage.AddDirtyRect(new Int32Rect(0, 0, RenderedImage.PixelWidth, RenderedImage.PixelHeight));
                RenderedImage.Unlock();
            }, DispatcherPriority.Render);
        }
    }
}

Answer 1

You should most likely follow the workflow from the documentation:

For greater control over updates, and for multi-threaded access to the back buffer, use the following workflow.

1. Call the Lock method to reserve the back buffer for updates.
2. Obtain a pointer to the back buffer by accessing the BackBuffer property.
3. Write changes to the back buffer. Other threads may write changes to the back buffer when the WriteableBitmap is locked.
4. Call the AddDirtyRect method to indicate areas that have changed.
5. Call the Unlock method to release the back buffer and allow presentation to the screen.

keeping _backBufferAddress as a field is most likely incorrect, since it may change unless you have locked the bitmap. WriteableBitmap is already using double buffering internally, so your manual attempt at doing this is likely doing more harm than anything.

so your code should look something like

myWriteableBitmap.Lock();
try
{
   var backBufferPtr = (byte*)myWriteableBitmap.BackBuffer;
 
   DrawBackground(backBufferPtr , myWriteableBitmap.BackBufferStride, myWriteableBitmap.PixelWidth, myWriteableBitmap.PixelHeight);

    foreach (var track in Tracks)
        track.RenderAt(backBufferPtr , myWriteableBitmap.BackBufferStride, CurrentRenderTime);

    myWriteableBitmap.AddDirtyRect(new Int32Rect(0, 0, myWriteableBitmap.PixelWidth, myWriteableBitmap.PixelHeight));
}
finally
{
   myWriteableBitmap.Unlock();
}
            

Since you do not show anything how rendering is initiated it is difficult to tell if it is correct. Adding locks/checks/resetevents randomly are most likely not helpful, you need to understand the actual problem if you want to solve it. . If you want exact timings I would expect multi media timers to be the most correct solution, but as far as I know there are no managed API, so you may need to write or find a wrapper around the native API.