Synchronizing UIView animation speed with audio playback

Question

I have a vertical one-pixel view that I want to move over a grid that represents notes being played. Depending on the BPM (beats per minute) of the song the animation should be slower or faster. The notes are squeezed next to each other and have a width of 25px.

The way I'm doing this right now is by moving the view 1px to the right every time x samples have passed:

int bpm = 90;
int interval = (44100 * 60.0f / song.bpm) / 25.0f; // = 1176
// sample rate=44100 multiplied by 60 (seconds) divided by number of pixels

So with BPM=90 I move the UIView 1px every 1176 samples played. This works fairly well but gives me problems if the BPM is higher than 180. At that point the view is behind on the music that is played and when BPM goes up to 240 the view is way off.

How can I do this with the built in animation features of Views and base the animation speed on the calculations I posted above? I want to start the animation once the music starts playing so I don't need to manually move it pixel by pixel.

Answer 1

One solution: you have your background view as shown - the blue boxes. Make sure this view is opaque. Define a new UIView subclass and place it over what I call the background view - this view will be non-opaque - lets call it the marker view.

The marker view is going to know at any time at what x offset to draw a line. It will provide a drawRect method that will set the rect parameter to a clear color (alpha=0). You will then draw using Quartz calls the single vertical line (if some boolean says to do it).

When your controller starts up, it tells this view to not draw the line. When you start up, change the boolean to YES, give it the x coordinate, and then tell that view setNeedsDisplayInRect:CGRectMake(theXoffset, 0, 1, heightOfView)];

Even if there are some delays, when the view finally does draw the x value should be current, as you are updating that "in real time".

EDIT:

• By marker, I mean the vertical line - it "marks" the place no?

• alpha 0 means (in this case) most of your overlay view, the "marker" view, will be a clear color, meaning that each pixel is clear (alpha=0) so as to not block the view below it. The only pixels drawn by this view are the vertical line.

• the BOOL, "x", etc - area properties of the view. You may need a few more, but you will not need many. These properties then instruct the view what it should do when "drawRect:" is called.

• You will need to learn a small amount about Quartz (the technology Apple uses to actually draw). In drawRect, you will get the current CGContextRef, and using it in virtually all calls, set the width of line you want to draw (it defaults to 1), set the line color, move to some point, draw the line to some other point (this will be a "path"), then stroke the path. This will be very fast compared to most other solutions (but itself can be accelerated).

If this all seems to daunting then perhaps someone else will have a simpler solution, or you can offer up a bounty and see if someone (like me) will actually code the whole thing up for you.

EDIT2:

This is the code for the view that does the indicator line (I used a red line to better see it, this array sets the color: { 1, 0, 0, in R/G/B space)

#import "SlideLine.h"

#define LINE_WIDTH 1.0f

@implementation SlideLine
@synthesize showLine, x;

- (void)drawRect:(CGRect)rect
{
    if(!showLine) return;

    CGRect bounds = self.bounds;

    CGContextRef c = UIGraphicsGetCurrentContext();
    CGContextSetLineWidth(c, LINE_WIDTH);

    // draws two slightly transparent lines on either side to soften the look
    for(int i=0; i<3; ++i) {
        CGFloat val[4] = { 1, 0, 0, i==1 ? 1 : 0.5f}; // half alpha for each side line
        CGContextSetStrokeColor(c, val);

        CGFloat xx = x - LINE_WIDTH/2 - 1 + i;
        CGContextMoveToPoint(c, xx, 0);
        CGContextAddLineToPoint(c, xx, bounds.size.height);
        CGContextStrokePath(c);
    }
}

- (void)setShowLine:(BOOL)val
{
    showLine = val;
    [self setNeedsDisplay];
}

@end

I created an Xcode project that shows you how to use it, with controls to vary the speed etc:

EDIT3: So I started a modification to use animation, but unfortunately its not completely baked. The idea is to take the marker line (3 pixels wide) and make it a single view (3 pixels wide). A new SlidingLineController gets updates from its owner (would be driven by the audio callback) that tells the controller where x should be at what time (so its a rate). The controller then uses that rate to animate the marker so that at the designated time its where it should be. As updates come in, the animation rate may slightly speed up or slow down so that the absolute position is always close to where you want it to be.

While this would seem a better technique than the original, it actually looks more "choppy" than the original. Probably the optimal solution would be to use a layer (not a view) then animate it, and update its target and rate as its moving, instead of starting and restarting. Probably this Second Project could use the technique from the first to simply redraw itself at a rate of 30fps using the same concept of managing rate. Unfortunately I ran out of time to pursue this as I see now this is expiring shortly.

EDIT:4 Well, I did another update, redrawing at a rate of 30fps, using the controller as above, and its working pretty well - its not finished but you can see in the third spin that its pretty smooth

Answer 2

you might want to:

• declare an atomic int variable which stores the playback position in samples
• install an NSTimer during playback on the main thread at a suitable frequency (e.g. less than your screen's refresh rate).
• when rendering audio, advance the atomic int's position by the appropriate amount.
• when your timer fires, read the position written by the audio thread. if changed, do something appropriate to update your UI, such as calling setNeedsDisplay or reposition your song position view.

i suggest this because your attempt at optimization may backfire if you end up pushing to and from multiple renderers.

Answer 3

If I understood correctly, what you want to do is:

1. Draw a static background image.
2. Draw a vertical line on top of that which moves from left to right.
3. Synchronize the line's x position with the audio playback time.

There are two problems to solve:

• Drawing performance
• Audio synchronization

To get the drawing performance right there are two implementations to consider:

1. Using Core Animation layers and animation (much easier to implement than 2.)
2. Using OpenGL to draw the whole view (possibly a little faster and a little less latency than 1.)

Implementing the view in OpenGL is too much code to post as an example, but here's code for the Core Animation option:

// In the UIViewController:

- (CALayer *)markerLayer
{
    static NSString *uniqueName = @"com.mydomain.MarkerLayer";
    for (CALayer *layer in self.view.layer.sublayers) {
        if ([layer.name isEqualToString:uniqueName])
            return layer;
    }

    CALayer *markerLayer = [CALayer layer];
    markerLayer.backgroundColor = [UIColor whiteColor].CGColor;
    markerLayer.name = uniqueName;
    markerLayer.anchorPoint = CGPointZero;
    [self.view.layer addSublayer:markerLayer];
    return markerLayer;
}

- (void)startAnimationWithDuration:(NSTimeInterval)duration
{
    [CATransaction begin];
    [CATransaction setDisableActions:YES];

    CALayer *markerLayer = [self markerLayer];

    markerLayer.bounds = (CGRect){.size = {1, self.view.bounds.size.height}};
    markerLayer.position = CGPointZero;
    [CATransaction commit];

    CABasicAnimation *animation = [CABasicAnimation animationWithKeyPath:@"position.x"];
    animation.fromValue = @0.0f;
    animation.toValue = [NSNumber numberWithFloat:self.view.bounds.size.width];
    animation.duration = duration;
    [markerLayer addAnimation:animation forKey:@"audioAnimation"];
}

Audio synchronization is very much depending on the way you are playing it, for example using AVFoundation or Core Audio. To get precise synchronization you would need some kind of callback from the audio engine that tells you where exactly the playback is right now or when it did start.

Maybe it is enough to just call startAnimationWithDuration: immediately before you start the audio but that seems kind of fragile and depends on the latency of the audio framework.