Korean_Of_the_Mountain
Korean_Of_the_Mountain

Reputation: 1579

Change images in a window via real time MIDI messages

I am trying to create a program that will

I've seen plenty of examples of being able to do the above separately, but am stuck on how to combine them all in one. I have not been able to find any examples of bindings to MIDI messages -- only for key and button presses.

Currently my code can open up a matplotlib window and display first image, but the code is stuck until the window is closed. After closing window, I can see MIDI notes in streaming output but no window for images is visible.

When I tried to use show(block=False) a window with no image would open, but I can at least see MIDI input stream. An image never shows up in window.

I'm open to trying other libraries (TKinter, PyQT5, etc) but I couldn't find any examples that they would be any different.

Here is my code so far

import matplotlib
import os, sys
import mido
import cv2

def get_images(path: str) -> list:
    """
    Recursively crawl through dir and load in all images
    """
    images = []
    image_files = glob.glob(os.path.join(path, '*'))
    print(image_files)

    for i in image_files:
        images.append(cv2.imread(i))

    return images


def get_midi_port():
    # Get a list of available input port names
    input_ports = mido.get_input_names()

    # Get first port that is not a Through type
    for p in input_ports:
        if "Midi Through" not in p:
            print(p)
            port = p
            return p

    if not input_ports or not p: 
        raise Exception("No MIDI input ports found.")

class MIDI_Images():
    def __init__(
        self, 
        path="."
    ):
        self.path = path
        self.loaded_images = get_images(self.path)
        self.curr = 0
        if auto_start:
            self.fig, self.ax = plt.subplots()
            self.display()
            plt.show(block=False) ## opens window but no image
            plt.pause(0.1)
            ### plt.show() ## shows image but code doesn't continue for MIDI notes
            
            ### Ingest MIDI messages
            try:
                p = get_midi_port()

                with mido.open_input(p) as inport:
                    inport.poll()
                    print(f"Listening for MIDI messages on port: {inport.name}")
                    while True:
                        for msg in inport.iter_pending():
                            print(msg)
                            self.curr = msg.note - 24 ## lowest note on my controller is 24
                            self.update_image()

            except Exception as e:
                print(f"Error: {e}")

    def update_image(self):
        """
        Load in next image
        """

        # advance to next image
        try:
            self.im.set_array(self.loaded_images[self.curr])
            self.display()
            ## self.fig.canvas.draw_idle()  ## waits for image window to close if uncommented
        except IndexError:
            print("Sorry no image in index: ", n)

    def display(self):
            """
            Orchestrating function to run
            """

            image = self.loaded_images[self.curr]

            self.im = self.ax.imshow(image)
            
            ### Examples of key bindings working just fine
            # self.fig.canvas.mpl_connect("key_press_event", self.next_image)


if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser(description='')

    parser.add_argument('--imagedir', action="store", dest='imagedir', default='')

    args = parser.parse_args()

    MIDI_Images(args.imagedir)


Upvotes: 0

Views: 48

Answers (1)

acw1668
acw1668

Reputation: 47163

Try executing the midi polling message loop in a child thread instead. Then you can use plt.show() instead of plt.show(block=False). Also you need to call plt.draw() after updating the image inside display().

Below is the required changes:

...
import threading

...

class MIDI_Images():
    def __init__(
        self,
        path="."
    ):
        self.path = path
        self.loaded_images = get_images(self.path)
        self.curr = 0
        if auto_start:  # note that auto_start is undefined in your code
            self.fig, self.ax = plt.subplots()
            self.display()
            # start the polling message loop in a child thread
            threading.Thread(target=self.poll_midi, daemon=1).start()
            plt.show()  # use blocking show()

    def poll_midi(self):
        try:
            p = get_midi_port()
            with mido.open_input(p) as inport:
                inport.poll()
                print(f"Listening for MIDI messages on port: {inport.name}")
                while True:
                    for msg in inport.iter_pending():
                        print(msg)
                        self.curr = msg.note - 24
                        self.update_image()
        except Exception as e:
            print(f"Error: {e}")

    def update_image(self):
        """
        Load in next image
        """

        # advance to next image
        try:
            self.im.set_array(self.loaded_images[self.curr])
            self.display()
        except IndexError:
            print("Sorry no image in index: ", self.curr)  # original is n which is undefined in your code

    def display(self):
            """
            Orchestrating function to run
            """
            image = self.loaded_images[self.curr]

            self.im = self.ax.imshow(image)
            plt.draw()  # update plot

...

Upvotes: 1

Related Questions