First note played in AKSequencer is off

Question

I am using AKSequencer to create a sequence of notes that are played by an AKMidiSampler. My problem is, at higher tempos the first note always plays with a little delay, no matter what i do.

I tried prerolling the sequence but it won't help. Substituting the AKMidiSampler with an AKSampler or a AKSamplePlayer (and using a callback track to play them) hasn't helped either, though it made me think that the problem probably resides in the sequencer or in the way I create the notes.

Here's an example of what I'm doing (I tried to make it as simple as I could):

import UIKit
import AudioKit

class ViewController: UIViewController {

    let sequencer = AKSequencer()
    let sampler = AKMIDISampler()
    let callbackInst = AKCallbackInstrument()

    var metronomeTrack : AKMusicTrack?
    var callbackTrack : AKMusicTrack?

    let numberOfBeats = 8
    let tempo = 280.0

    var startTime : TimeInterval = 0

    override func viewDidLoad() {
        super.viewDidLoad()

        print("Begin setup.")

        // Load .wav sample in AKMidiSampler

        do {
            try sampler.loadWav("tick")
        } catch {
            print("sampler.loadWav() failed")
        }

        // Create tracks for the sequencer and set midi outputs

        metronomeTrack = sequencer.newTrack("metronomeTrack")
        callbackTrack = sequencer.newTrack("callbackTrack")
        metronomeTrack?.setMIDIOutput(sampler.midiIn)
        callbackTrack?.setMIDIOutput(callbackInst.midiIn)

        // Setup and start AudioKit

        AudioKit.output = sampler

        do {
            try AudioKit.start()
        } catch {
            print("AudioKit.start() failed")
        }

        // Set sequencer tempo

        sequencer.setTempo(tempo)

        // Create the notes

        var midiSequenceIndex = 0

        for i in 0 ..< numberOfBeats {

            // Add notes to tracks

            metronomeTrack?.add(noteNumber: 60, velocity: 100, position: AKDuration(beats: Double(midiSequenceIndex)), duration: AKDuration(beats: 0.5))
            callbackTrack?.add(noteNumber: MIDINoteNumber(midiSequenceIndex), velocity: 100, position: AKDuration(beats: Double(midiSequenceIndex)), duration: AKDuration(beats: 0.5))

            print("Adding beat number \(i+1) at position: \(midiSequenceIndex)")
            midiSequenceIndex += 1

        }

        // Set the callback

        callbackInst.callback = {status, noteNumber, velocity in

            if status == .noteOn {

                let currentTime = Date().timeIntervalSinceReferenceDate
                let noteDelay = currentTime - ( self.startTime + ( 60.0 / self.tempo ) * Double(noteNumber) )
                print("Beat number: \(noteNumber) delay: \(noteDelay)")

            } else if ( noteNumber == midiSequenceIndex - 1 ) && ( status == .noteOff)  {

                print("Sequence ended.\n")
                self.toggleMetronomePlayback()

            } else {return}

        }

        // Preroll the sequencer

        sequencer.preroll()

        print("Setup ended.\n")

    }

    @IBAction func playButtonPressed(_ sender: UIButton) {

        toggleMetronomePlayback()

    }


    func toggleMetronomePlayback() {

        if sequencer.isPlaying == false {

            print("Playback started.")
            startTime = Date().timeIntervalSinceReferenceDate
            sequencer.play()

        } else {

            sequencer.stop()
            sequencer.rewind()

        }

    }

}

Could anyone help? Thank you.

Answer 1

I had the same issue, preroll didn't help, but I have managed to solve it with a dedicated sampler for the first notes. I used a delay on the other sampler, about 0.06 of a second, works like a charm. Kind of a silly solution but it did the job and I could go on with the project :)

//This is for fixing AK bug that plays the first playback not in delay
    let fixDelay = AKDelay()
    fixDelay.dryWetMix = 1
    fixDelay.feedback = 0
    fixDelay.lowPassCutoff = 22000
    fixDelay.time = 0.06
    fixDelay.start()
    let preDelayMixer = AKMixer()
    let preFirstMixer = AKMixer()


    [playbackSampler,vocalSampler]  >>> preDelayMixer >>> fixDelay
    [firstNoteVocalSampler, firstRoundPlaybackSampler] >>> preFirstMixer
    [fixDelay,preFirstMixer] >>> endMixer

Answer 2

After a bit of testing I actually found out that it is not the first note that plays off but the subsequent notes that play in advance. Moreover, the amount of notes that play exactly on time when starting the sequencer depends on the set tempo.

The funny thing is that if the tempo is < 400 there will be one note played on time and the others in advance, if it is 400 <= bpm < 800 there will be two notes played correctly and the others in advance and so on, for every 400 bpm increment you get one more note played correctly.

So... since the notes are played in advance and not late, the solution that solved it for me is:

1) Use a sampler that is not connected directly to a track's midi output but has its .play() method called inside a callback.

2) Keep track of when the sequencer gets started

3) At every callback calculate when the note should play in relation to the start time and store what time it actually is, so you can then calculate the offset.

4) use the computed offset to dispatch_async after the offset your .play() method.

And that's it, I tested this on multiple devices and now all the notes play perfectly on time.

Answer 3

As Aure commented, the start up latency is a known problem. Even with preroll, there is still noticeable latency, especially at higher tempos.

But if you are using a looping sequence, I found that you can sometimes mitigate how noticeable the latency is by setting the 'starting point' of the sequence to a position after the final MIDI event, but within the loop length. If you can find a good position, you can get the latency effects out of the way before it loops back to your content.

Make sure to call setTime() before you need it (e.g., after stopping the sequence, not when you are ready to play) because the setTime()call itself can introduce about 200ms of wonkiness.

Edit: As an afterthought, you could do the same thing on a non-looping sequence by enabling looping and using an arbitrarily long sequence length. If you needed playback to stop at the end of the MIDI content, you could do this with an AKCallbackInstrument triggered by an MIDI event placed just after the final note.