Recording generated midi events: inner conversion from timestamps to midi file delta time

Question

I'm developing a small desktop aplication which gets a monophonic melody from the sound input port, and using pitch tracking techniques it converts it to midi messages to be dumped to a midi port or stored to a midi file.

The actual problem comes when I record this midi messages to a midi file, using the Receiver from the RealTimeSequencer.

The program generates, in another thread, a convenience class (TSMidiMessage) wich holds a MidiMessage (mostly ShortMessages) and a timestamp created taking the System.currentMilliSecs() value, and it's put into a BlockingQueue shared with this thread. So, as the Receiver gets a MidiMessage and a timestamp, expressed in milliseconds, it's now quite clear or well documented how the conversion of this timestamp to midi delta time it's done inside the Sequencer.

Also, looking inside the midi file with an hexadecimal editor, all the timestamps of the midi events are 0x00, so it explains why it doesn't show any note when imported to a score editor.

Below, there's a redux version of the relevant code:

public class MidiListener implements Runnable {
private BlockingQueue<TSMidiMessage[]> messageQueue = new ArrayBlockingQueue<TSMidiMessage[]>(10);

private volatile boolean run=false;
public volatile boolean start=false;

public static long startTime;

public void run() {
    run = true;

    TSMidiMessage[] messageBlock = null;

    Sequencer sequencer = null;
    Sequence sequence = null;
    Track pista1 = null;
    Receiver receiver = null;

    try{
        log.debug("Starting sequencer");
        sequencer = MidiSystem.getSequencer();
        receiver = sequencer.getReceiver();
        // almacenar mensaje midi
        log.debug("Starting sequence");
        sequence = new Sequence(Sequence.PPQ, cuantizeStringToPPQ(cuantizeMode));
        pista1 = sequence.createTrack();

        //****  General MIDI sysex -- turn on General MIDI sound set  ****
        // http://www.automatic-pilot.com/midifile.html
        log.trace("Loading General MIDI");
        byte[] b = {(byte)0xF0, 0x7E, 0x7F, 0x09, 0x01, (byte)0xF7};
        SysexMessage sm = new SysexMessage();
        sm.setMessage(b, 6);
        MidiEvent me = new MidiEvent(sm,(long)0);
        pista1.add(me);

        log.trace("Add tempo message");
        me =  new MidiEvent(midiTools.forgeTempoMessage(dataHolder.bpm), 0L);
        pista1.add(me);
        log.trace("Add signature message");
        me =  new MidiEvent(midiTools.forgeSignatureMessage(dataHolder.highMeasureNibble, dataHolder.lowMeasureNibble), 0L);
        pista1.add(me);


        //****  set omni on  ****
        ShortMessage mm = new ShortMessage();
        mm.setMessage(0xB0, 0x7D,0x00);
        me = new MidiEvent(mm,(long)0);
        pista1.add(me);

        //****  set poly on  ****
        mm = new ShortMessage();
        mm.setMessage(0xB0, 0x7F,0x00);
        me = new MidiEvent(mm,(long)0);
        pista1.add(me);

        mm = new ShortMessage();
        mm.setMessage(0xC0, 0x00, 0x00);
        me = new MidiEvent(mm,(long)0);
        pista1.add(me);

        sequencer.setSequence(sequence);
        sequencer.recordEnable(pista1, 0);
        sequencer.setTempoInBPM(dataHolder.bpm);

        sequencer.open();
        while(!this.start){
            // hold for a moment
        }
        log.debug("Start recording");
        startTime = System.currentTimeMillis();
        sequencer.startRecording();
    }catch (MidiUnavailableException e) {
        log.error("Midi device error", e);
    } catch (InvalidMidiDataException e) {
        log.error("Invalid midi data", e);
    }
    while(run){
        try {
            messageBlock = messageQueue.take();
            // Message load
            log.debug("Message load");

            if(messageBlock[0] == null)
                throw new InterruptedException("Thread interrupted");

            for(int i=0; i < messageBlock.length; i++){
                receiver.send(messageBlock[i].getMessage(), messageBlock[i].getTimeStamp());
            }

        } catch (InterruptedException e1) {
            log.debug("Thread interrupted");
            this.run = false;
        }
    }
    log.debug("Saving midi file");
    log.trace("Sequence "+sequence);
    log.trace("Length  "+sequence.getMicrosecondLength()+"ms");
    MetaMessage endOfTrack = new MetaMessage();
    try {
        endOfTrack.setMessage(0x2F, new byte[]{}, 0);
        receiver.send(endOfTrack, System.currentTimeMillis());
    } catch (InvalidMidiDataException e1) {
        log.error(e1);
    }
    sequencer.stopRecording();
    sequencer.recordDisable(pista1);
    sequencer.close();
    try {
        midiTools.renderMidiFile(sequence, mainControl.selectedFile);
    } catch (IOException e) {
        log.error("Error while saving midi file", e);
    }
}

}

And here it's an hex dump of a small generated midi file:

0000000: 4d54 6864 0000 0006 0001 0001 0004 4d54  MThd..........MT
0000010: 726b 0000 0268 00f0 057e 7f09 01f7 00ff  rk...h...~......
0000020: 5103 0b71 b000 ff58 0406 0324 0c00 b07d  Q..q...X...$...}
0000030: 0000 7f00 00c0 0083 b4b0 5990 4c5d 0080  ..........Y.L]..
0000040: 4c00 0090 4e44 0080 4e00 0090 5064 0080  L...ND..N...Pd..
0000050: 5000 0090 4b68 0080 4b00 0090 506c 0080  P...Kh..K...Pl..
0000060: 5000 0090 4d4f 0080 4d00 0090 582d 0080  P...MO..M...X-..
0000070: 5800 0090 473b 0080 4700 0090 4c39 0080  X...G;..G...L9..
0000080: 4c00 0090 5734 0080 5700 0090 4d2c 0080  L...W4..W...M,..
0000090: 4d00 0090 5731 0080 5700 0090 4b2e 0080  M...W1..W...K...
00000a0: 4b00 0090 5832 0080 5800 0090 4d2d 0080  K...X2..X...M-..
00000b0: 4d00 0090 4e30 0080 4e00 0090 4c2f 0080  M...N0..N...L/..
00000c0: 4c00 0090 4c3c 0080 4c00 0090 4f5d 0080  L...L<..L...O]..
00000d0: 4f00 0090 4a76 0080 4a00 0090 5264 0080  O...Jv..J...Rd..
00000e0: 5200 0090 505b 0080 5000 0090 4c60 0080  R...P[..P...L`..
00000f0: 4c00 0090 5059 0080 5000 0090 4d57 0080  L...PY..P...MW..
0000100: 4d00 0090 505c 0080 5000 0090 4e5a 0080  M...P\..P...NZ..
0000110: 4e00 0090 4d6a 0080 4d00 0090 4b63 0080  N...Mj..M...Kc..
0000120: 4b00 0090 5059 0080 5000 0090 4e5d 0080  K...PY..P...N]..
0000130: 4e00 0090 4d53 0080 4d00 0090 4c52 0080  N...MS..M...LR..
0000140: 4c00 0090 4b41 0080 4b00 0090 4c44 0080  L...KA..K...LD..
0000150: 4c00 0090 4a3f 0080 4a00 0090 4c4b 0080  L...J?..J...LK..
0000160: 4c00 0090 4e4e 0080 4e00 0090 4b5a 0080  L...NN..N...KZ..
0000170: 4b00 0090 4c52 0080 4c00 0090 4e5a 0080  K...LR..L...NZ..
0000180: 4e00 0090 504b 0080 5000 0090 4f56 0080  N...PK..P...OV..
0000190: 4f00 0090 5953 0080 5900 0090 4c55 0080  O...YS..Y...LU..
00001a0: 4c00 0090 4a55 0080 4a00 0090 4e54 0080  L...JU..J...NT..
00001b0: 4e00 0090 4b4f 0080 4b00 0090 4a4c 0080  N...KO..K...JL..
00001c0: 4a00 0090 4c4b 0080 4c00 0090 4b49 0080  J...LK..L...KI..
00001d0: 4b00 0090 4e3d 0080 4e00 0090 4f4b 0080  K...N=..N...OK..
00001e0: 4f00 0090 4e52 0080 4e00 0090 4d4b 0080  O...NR..N...MK..
00001f0: 4d00 0090 4b49 0080 4b00 0090 4f3c 0080  M...KI..K...O<..
0000200: 4f00 0090 4d42 0080 4d00 0090 4b47 0080  O...MB..M...KG..
0000210: 4b00 0090 4f41 0080 4f00 0090 4b4c 0080  K...OA..O...KL..
0000220: 4b00 0090 4d44 0080 4d00 0090 4e3e 0080  K...MD..M...N>..
0000230: 4e00 0090 4c44 0080 4c00 0090 5042 0080  N...LD..L...PB..
0000240: 5000 0090 4b3b 0080 4b00 0090 4c3c 0080  P...K;..K...L<..
0000250: 4c00 0090 4b3e 0080 4b00 0090 4c3b 0080  L...K>..K...L;..
0000260: 4c00 0090 4b3a 0080 4b00 0090 4e35 0080  L...K:..K...N5..
0000270: 4e00 0090 4922 0080 4900 00ff 2f00       N...I"..I.../.

It may relevant to mention that i'm using small (but I guess those arecorrect) values for tick resolution (expressed in PPQ), as I expect it would help to quantize the midi events, ant this may be one of the explanations of this problem. My other idea is that the timestamp data sent to the Receiver is not consistent to the inner working of it, as it must be expressed in other way rather than an epoch timestamp.

Thank you =)

---

Edit 1: After reading the Java Sound Api Progrramer's guide (chapter 10, paragraph "Time Stamps on Messages Sent to Devices"), I found this:

The time stamp that can optionally accompany messages sent between devices in the Java Sound API is quite different from the timing values in a standard MIDI file. The timing values in a MIDI file are often based on musical concepts such as beats and tempo, and each event's timing measures the time elapsed since the previous event. In contrast, the time stamp on a message sent to a device's Receiver object always measures absolute time in microseconds. Specifically, it measures the number of microseconds elapsed since the device that owns the receiver was opened.

So there i've found that the timestamps i was sending aren't expressed in the expected format, but I did a quick test, replacing those timestamps for a -1 value, meaning that it should ignore the timestamp value, and the result is the same, no delta timestamp in the events.

---

Edit 2: As I still working with it, I converted those epoch timestamp, to the cummulative timestamp format with microsecond resolution, expected from the Receiver. But this doesn't make it work (but improves the code =)). Also I tried to change the PP resolution to a higher value, 96, wich is supposed to work good enough as is also specified in the Java Sound guide.

Answer 1

Ok, so I'll answer my own question since I found a workaround of my problem.

If it doesn't work properly, why use a Receiver to get the MidiMessages with correct timing? We can bypass this and forge the MidiEvents manually, using the cummulative timestamp in microseconds and converting it to cummulative timestamp expressed in ticks, wich are proportional to the PPQ parameter.

private long microsecondTickToPPQTick(long msTick, Sequencer seq){
    long ret = msTick / 1000;
    double rawValue = ret / this.tickSize;

    double valueA  = (rawValue - Math.floor(rawValue)); 
    double valueB = ((Math.floor(rawValue)+1) - rawValue);
    double min = Math.min(valueA, valueB);

    if(min == valueA){
        ret =(long) Math.floor(rawValue);
    }else{
        ret =(long) (Math.floor(rawValue)+1);
    }

    log.info("MidiEvent's timestamp: "+ret);

    return ret;
}

This method also quantize the notes to the selected PPQ and it's consequent note resolution. You can find the tick size in milliseconds, just using this formula:

tickSize = (60.0/bpm)/(double)ppq;

And then you get the correct tick measure, and add the MidiEvent to the Sequence.

// startTime holds the value for the epoch time when the sequencer started recording.
long microTS = (messageBlock[i].getTimeStamp() - startTime)*1000;
long tick = microsecondTickToPPQTick(microTS, sequencer);
MidiEvent me = new MidiEvent(messageBlock[i].getMessage(), tick);
track1.add(me);