Choppy movement of large collection of items on a canvas

Question

This question is directly related to a question I recently posted, but I feel that the direction has changed enough to warrant a new one. I am trying to figure out the best way to move a large collection of images on a canvas in real-time. My XAML currently looks like this:

<UserControl.Resources>
    <DataTemplate DataType="{x:Type local:Entity}">
        <Canvas>
            <Image Canvas.Left="{Binding Location.X}"
                   Canvas.Top="{Binding Location.Y}"
                   Width="{Binding Width}"
                   Height="{Binding Height}"
                   Source="{Binding Image}" />
        </Canvas>
    </DataTemplate>
</UserControl.Resources>

<Canvas x:Name="content"
        Width="2000"
        Height="2000"
        Background="LightGreen">
    <ItemsControl Canvas.ZIndex="2" ItemsSource="{Binding Entities}">
        <ItemsControl.ItemsPanel>
            <ItemsPanelTemplate>
                <Canvas IsItemsHost="True" />
            </ItemsPanelTemplate>
        </ItemsControl.ItemsPanel>
    </ItemsControl>

The Entity class:

[Magic]
public class Entity : ObservableObject
{
    public Entity()
    {
        Height = 16;
        Width = 16;
        Location = new Vector(Global.rand.Next(800), Global.rand.Next(800));
        Image = Global.LoadBitmap("Resources/Thing1.png");
    }

    public int Height { get; set; }
    public int Width { get; set; }
    public Vector Location { get; set; }
    public WriteableBitmap Image { get; set; }        
}

To move the object:

private Action<Entity> action = (Entity entity) =>
{
    entity.Location = new Vector(entity.Location.X + 1, entity.Location.Y);
};

void Timer_Tick(object sender, EventArgs e)
{
    Task.Factory.StartNew(() =>
    {
        foreach (var entity in Locator.Container.Entities)
        {
            action(entity);
        }
    });
}

If I have fewer than about 400 entries in the Entities collection, movement is smooth, but I'd like to be able to increase that number by quite a bit. If I go above 400, movement becomes increasingly choppy. At first I thought it was an issue with the movement logic (which at this point isn't really much of anything), but I have found that that's not the problem. I added another collection with 10,000 entries and added that collection to the same timer loop as the first but did not include it in the XAML, and the UI didn't react any differently. What I find odd, however, is that if I add 400 entries to the collection and then 400 more with Image set to null, movement becomes choppy even though half of the items aren't drawn.

So, what can I do, if anything, to be able to draw and smoothly move more images on a canvas? Is this a situation where I may want to shy away from WPF & XAML? If you need more code, I will gladly post it.

---

Update: Per Clemens' suggestion, my Entity DataTemplate now looks like this:

<DataTemplate DataType="{x:Type local:Entity}">
    <Image Width="{Binding Width}"
           Height="{Binding Height}" 
           Source="{Binding Image}">
        <Image.RenderTransform>
            <TranslateTransform X="{Binding Location.X}" Y="{Binding Location.Y}" />
        </Image.RenderTransform>
    </Image>
</DataTemplate>

There may be a boost in performance by using this, but if there is it is very subtle. Also, I have noticed that if I use a DispatcherTimer for the loop and set it up as:

private DispatcherTimer dTimer = new DispatcherTimer();

public Loop()
{
    dTimer.Interval = TimeSpan.FromMilliseconds(30);
    dTimer.Tick += Timer_Tick;
    dTimer.Start();
}

void Timer_Tick(object sender, EventArgs e)
{
    foreach (var entity in Locator.Container.Entities)
    {
        action(entity);
    }
}

... The movement is smooth even with several thousand items, but very slow, regardless of the interval. If a DispatcherTimer is used and Timer_Tick looks like this:

void Timer_Tick(object sender, EventArgs e)
{
    Task.Factory.StartNew(() =>
    {
        foreach (var entity in Locator.Container.Entities)
        {
            action(entity);
        }
    });
}

... the movement is very choppy. What I find odd is that a Stopwatch shows that the Task.Factory takes between 1000 and 1400 ticks to iterate over the collection if there are 5,000 entries. The standard foreach loop takes over 3,000 ticks. Why would Task.Factory perform so poorly when it is twice as fast? Is there a different way to iterate through the collection and/or a different timing method that might allow for smooth movement without any major slowdowns?

---

Update: If anybody can help me improve the performance of real-time movement of objects on a canvas or can suggest another way in WPF to achieve similar results, 100 bounty awaits.

Answer 1

The issue here is the rendering/creation of so many controls.

The first question is whether you need to show all the images on the canvas. If so, I'm sorry but I can't help (if you need to draw all items then there's no way around it).

But if not all items are visible on the screen at one time - then you have hope in the shape of Virtualization. You'd need to write your own VirtualizingCanvas that inherits VirtualizingPanel and creates only the items that are visible. This will also allow you to recycle the containers which in turn will remove a lot of the load.

There's an example of a virtualizing canvas here.

Then you'd need to set the new canvas as your items panel, and set up the items to have the necessary information for the canvas to work properly.

Answer 2

That's an issue I had to solve when developping a very simple Library called Mongoose. I tried it with a 1000 images and its totally smooth (I don't have code that automatically moves images, I move them manually by drag and dropping on the Surface, but you should have the same result with code).

I wrote a quick sample you can run by using the library (you just need an attached view model with a collection of anything called PadContents) :

MainWindow.xaml

<Window x:Class="Mongoose.Sample.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:sys="clr-namespace:System;assembly=mscorlib"
        xmlns:col="clr-namespace:System.Collections;assembly=mscorlib"
        xmlns:mwc="clr-namespace:Mongoose.Windows.Controls;assembly=Mongoose.Windows"
        Icon="Resources/MongooseLogo.png"
        Title="Mongoose Sample Application" Height="1000" Width="1200">



    <mwc:Surface x:Name="surface" ItemsSource="{Binding PadContents}">
        <mwc:Surface.ItemContainerStyle>
            <Style TargetType="mwc:Pad">
                <Setter Property="Template">
                    <Setter.Value>
                        <ControlTemplate>
                            <Image Source="Resources/MongooseLogo.png" Width="30" Height="30" />
                        </ControlTemplate>
                    </Setter.Value>
                </Setter>
            </Style>
        </mwc:Surface.ItemContainerStyle>
    </mwc:Surface>

</Window>

MainWindow.xaml.cs

using System.Collections.ObjectModel;
using System.Windows;

namespace Mongoose.Sample
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
            DataContext = this;
        }

        public ObservableCollection<object> PadContents
        {
            get
            {
                if (padContents == null)
                {
                    padContents = new ObservableCollection<object>();
                    for (int i = 0; i < 500; i++)
                    {
                        padContents.Add("Pad #" + i);
                    }
                }
                return padContents;
            }
        }

        private ObservableCollection<object> padContents;
    }
}

And here is what it looks like for 1000 images :

The full code is available on Codeplex so even if you don't want to reuse the library, you can still check the code to see how achieved it.

I rely on a few tricks, but mostly the use of RenderTransform and CacheMode.

On my computer it's ok for up to 3000 images. If you want to do more, you'll probably have to think of other ways to achieve it though (maybe with some kind of virtualization)

Good luck !

EDIT:

By adding this code in the Surface.OnLoaded method :

var messageTimer = new DispatcherTimer();
messageTimer.Tick += new EventHandler(surface.messageTimer_Tick);
messageTimer.Interval = new TimeSpan(0, 0, 0, 0, 10);
messageTimer.Start();

And this method in the Surface class :

void messageTimer_Tick(object sender, EventArgs e)
{
    var pads = Canvas.Children.OfType<Pad>();
    if (pads != null && Layout != null)
    {
        foreach (var pad in pads)
        {
            pad.Position = new Point(pad.Position.X + random.Next(-1, 1), pad.Position.Y + random.Next(-1, 1));
        }
    }
}

You can see that it's totally ok to move each object separately. Here is a samll example with 2000 objects

Answer 3

Having so many controls move on the screen this frequently will never yield smooth results. You need to a completely different approach - rendering on your own. I'm not sure this would suit you, as now you will not be able to use control features per each item (e.g. to receive events, have tooltips or use data templates.) But with such a large amount of items, other approaches are impractical.

Here's a (very) rudimentary implementation of what that might look like:

Update: I've modified the renderer class to use the CompositionTarget.Rendering event instead of a DispatcherTimer. This event fires every time WPF renders a frame (normally around 60 fps). While this would provide smoother results, it is also more CPU intensive, so be sure to turn off the animation when it's no longer needed.

public class ItemsRenderer : FrameworkElement
{
    private bool _isLoaded;

    public ItemsRenderer()
    {
        Loaded += OnLoaded;
        Unloaded += OnUnloaded;
    }

    private void OnLoaded(object sender, RoutedEventArgs routedEventArgs)
    {
        _isLoaded = true;
        if (IsAnimating)
        {
            Start();
        }
    }

    private void OnUnloaded(object sender, RoutedEventArgs routedEventArgs)
    {
        _isLoaded = false;
        Stop();
    }

    public bool IsAnimating
    {
        get { return (bool)GetValue(IsAnimatingProperty); }
        set { SetValue(IsAnimatingProperty, value); }
    }

    public static readonly DependencyProperty IsAnimatingProperty =
        DependencyProperty.Register("IsAnimating", typeof(bool), typeof(ItemsRenderer), new FrameworkPropertyMetadata(false, (d, e) => ((ItemsRenderer)d).OnIsAnimatingChanged((bool)e.NewValue)));

    private void OnIsAnimatingChanged(bool isAnimating)
    {
        if (_isLoaded)
        {
            Stop();
            if (isAnimating)
            {
                Start();
            }
        }
    }

    private void Start()
    {
        CompositionTarget.Rendering += CompositionTargetOnRendering;
    }

    private void Stop()
    {
        CompositionTarget.Rendering -= CompositionTargetOnRendering;
    }

    private void CompositionTargetOnRendering(object sender, EventArgs eventArgs)
    {
        InvalidateVisual();
    }

    public static readonly DependencyProperty ImageSourceProperty =
        DependencyProperty.Register("ImageSource", typeof (ImageSource), typeof (ItemsRenderer), new FrameworkPropertyMetadata());

    public ImageSource ImageSource
    {
        get { return (ImageSource) GetValue(ImageSourceProperty); }
        set { SetValue(ImageSourceProperty, value); }
    }

    public static readonly DependencyProperty ImageSizeProperty =
        DependencyProperty.Register("ImageSize", typeof(Size), typeof(ItemsRenderer), new FrameworkPropertyMetadata(Size.Empty));

    public Size ImageSize
    {
        get { return (Size) GetValue(ImageSizeProperty); }
        set { SetValue(ImageSizeProperty, value); }
    }

    public static readonly DependencyProperty ItemsSourceProperty =
        DependencyProperty.Register("ItemsSource", typeof (IEnumerable), typeof (ItemsRenderer), new FrameworkPropertyMetadata());

    public IEnumerable ItemsSource
    {
        get { return (IEnumerable) GetValue(ItemsSourceProperty); }
        set { SetValue(ItemsSourceProperty, value); }
    }

    protected override void OnRender(DrawingContext dc)
    {
        ImageSource imageSource = ImageSource;
        IEnumerable itemsSource = ItemsSource;

        if (itemsSource == null || imageSource == null) return;

        Size size = ImageSize.IsEmpty ? new Size(imageSource.Width, imageSource.Height) : ImageSize;
        foreach (var item in itemsSource)
        {
            dc.DrawImage(imageSource, new Rect(GetPoint(item), size));
        }
    }

    private Point GetPoint(object item)
    {
        var args = new ItemPointEventArgs(item);
        OnPointRequested(args);
        return args.Point;
    }

    public event EventHandler<ItemPointEventArgs> PointRequested;

    protected virtual void OnPointRequested(ItemPointEventArgs e)
    {
        EventHandler<ItemPointEventArgs> handler = PointRequested;
        if (handler != null) handler(this, e);
    }
}


public class ItemPointEventArgs : EventArgs
{
    public ItemPointEventArgs(object item)
    {
        Item = item;
    }

    public object Item { get; private set; }

    public Point Point { get; set; }
}

Usage:

<my:ItemsRenderer x:Name="Renderer"
                  ImageSize="8 8"
                  ImageSource="32.png"
                  PointRequested="OnPointRequested" />

Code Behind:

Renderer.ItemsSource = Enumerable.Range(0, 2000)
            .Select(t => new Item { Location = new Point(_rng.Next(800), _rng.Next(800)) }).ToArray();

private void OnPointRequested(object sender, ItemPointEventArgs e)
{
    var item = (Item) e.Item;
    item.Location = e.Point = new Point(item.Location.X + 1, item.Location.Y);
}

You can use the OnPointRequested approach to get any data from the item (such as the image itself.) Also, don't forget to freeze your images, and pre-resize them.

A side note, regarding threading in the previous solutions. When you use a Task, you're actually posting the property update to another thread. Since you've bound the image to that property, and WPF elements can only be updated from the thread on which they were created, WPF automatically posts each update to the Dispatcher queue to be executed on that thread. That's why the loop ends faster, and you're not timing the actual work of updating the UI. It's only adding more work.

Answer 4

A few thoughts that come to mind:

1. Freeze your bitmaps.

2. Hard set the size of your bitmaps when you read them to be identical to the size you're displaying them in, and set the BitmapScalingMode to LowQuality.

3. Track your progress while updating your entities and cut out early if you can't and grab them next frame. This will require tracking their last frame too.

   // private int _lastEntity = -1;
   // private long _tick = 0;
   // private Stopwatch _sw = Stopwatch.StartNew();
   // private const long TimeSlice = 30;
   
   // optional: this._sw.Restart();
   var end = this._sw.ElapsedMilliseconds + TimeSlice - 1;
   
   this._tick++;
   var ee = this._lastEntity++;
   do {
       if (ee >= this._entities.Count) ee = 0;
   
       // entities would then track the last time
       // they were "run" and recalculate their movement
       // from 'tick'
       action(this._entities[ee], this._tick);
   
       if (this._sw.ElapsedMilliseconds > end) break;
   } while (ee++ != this._lastEntity);
   
   this._lastEntity = ee;
   

Answer 5

In a first optimization approach you may reduce the number of Canvases to just one by removing the Canvas from the DataTemplate and setting Canvas.Left and Canvas.Top in an ItemContainerStyle:

<DataTemplate DataType="{x:Type local:Entity}">
    <Image Width="{Binding Width}" Height="{Binding Height}" Source="{Binding Image}"/>
</DataTemplate>

<ItemsControl ItemsSource="{Binding Entities}">
    <ItemsControl.ItemsPanel>
        <ItemsPanelTemplate>
            <Canvas IsItemsHost="True" />
        </ItemsPanelTemplate>
    </ItemsControl.ItemsPanel>
    <ItemsControl.ItemContainerStyle>
        <Style TargetType="ContentPresenter">
            <Setter Property="Canvas.Left" Value="{Binding Location.X}"/>
            <Setter Property="Canvas.Top" Value="{Binding Location.Y}"/>
        </Style>
    </ItemsControl.ItemContainerStyle>
</ItemsControl>

Then you may replace setting Canvas.Left and Canvas.Top by applying a TranslateTransform:

<ItemsControl.ItemContainerStyle>
    <Style TargetType="ContentPresenter">
        <Setter Property="RenderTransform">
            <Setter.Value>
                <TranslateTransform X="{Binding Location.X}" Y="{Binding Location.Y}"/>
            </Setter.Value>
        </Setter>
    </Style>
</ItemsControl.ItemContainerStyle>

Now this could similarly be applied to the Image control in the DataTemplate instead of the item container. So you may remove the ItemContainerStyle and write the DataTemplate like this:

<DataTemplate DataType="{x:Type local:Entity}">
    <Image Width="{Binding Width}" Height="{Binding Height}" Source="{Binding Image}">
        <Image.RenderTransform>
            <TranslateTransform X="{Binding Location.X}" Y="{Binding Location.Y}"/>
        </Image.RenderTransform>                
    </Image>
</DataTemplate>

Answer 6

Try using TranslateTransform instead of Canvas.Left and Canvas.Top. The RenderTransform and TranslateTransform are efficient in scaling/moving existing drawing objects.