Is there a way to monitor and control macOS applications from another custom built application or script?

Question

This is a general question where I'm looking to be pointed in the right direction. I'm looking to be able to monitor the Mac application Logic Pro X. It's for a university research project that focuses on DAW control, I need to be able to monitor specific information from the DAW - primarily the displayed text infromation of Tracks and/or other aspects.

The control aspect is not such a major factor as this can be achieved using MIDI and/or keyboard shortcuts. But wondering if anyone knows of a way of monitoring some of the GUI text information from one application in another application.

Done some research online and can't seem to find much on the topic.

I've not done much in C based programming but know Python and Javascript quite well so this is going to be my next project to try to learn Objective-C.

Any help will be greatly appreciated.

EDIT - I have just found this, Use AppleScript to list the names of all UI elements in a window (GUI scripting)

would this approach using AppleScript work here?

EDIT - Areas of the GUI needed

Here's a screenshot of the text I need to access I have actually been able to get to all of these using atomacos in Python, the text is contained in the AXDescription of the correct buttons. However these aren't always at the same place in the hiearchy so I'm having to ignore errors and find the correct path. I'm also getting the text constantly in a while loop, it would be better if I could observe the properties to see if they change. Here is the Python code I'm using to get just the track names, I've also managed to get down inside the tracks to get the audio effect names.

import atomacos

logic = atomacos.getAppRefByBundleId('com.apple.logic10')
for w in logic.windows():
    title = w.AXTitle
    try:
        if title.split(' - ')[1] == 'Tracks':
            window = w
    except IndexError:
        continue
for section in window.AXChildren:
    if section.AXRoleDescription == 'group':
        try:
            if section.AXChildren[1].AXDescription == 'Tracks':
                tracks = section.AXChildren[1].AXChildren[0].AXChildren[1].AXChildren[0].AXChildren[0]
        except IndexError:
            continue
        except AttributeError:
            continue

while True:
    track_list = {}
    for track in tracks.AXChildren:
        try:
            for item in track.AXChildren:
                try:
                    if item.AXRoleDescription == 'text field':
                        track_list[item.AXDescription] = item
                except AttributeError:
                    continue
                except atomacos.errors.AXErrorIllegalArgument:
                    continue
            track_list.keys()

        except AttributeError:
            continue

Answer 1

Here's an example GUI script that shows you how to reliably find the two areas you marked, and gather information you want. I've shown how to find both the 'tracks' area and the 'Mixer area', but I only drilled down into the 'Tracks' area you give you examples of things work. If you have trouble piecing together the other area, let me know and I'll add more.

tell application "Logic Pro X" to activate

tell application "System Events"
    tell process "Logic Pro X"
        tell window "Untitled - Tracks"
            (*
                the following lines find the main 'tracks' group and the 'mixer' group, by looking for
                 stable features of their respective toolbars. the first looks for the 
                 'Horizontal Zoom' slider, the second for the 'Wide Channel Strips' radio button. 
                 the doubled 'whose' queries look confusing, but basically they say "Find the group 
                 whose first group is a toolbar, whose last (slider's/radiobutton's) description is...  
                 Once we have those, we can reliably find these area, regardless of whether other groups 
                 are opened or closed"
            *)
            set main_track_group to first group whose its (first group whose role description is "Toolbar")'s last slider's description is "Horizontal Zoom"
            try
                set mixer_group to first group whose its (first group whose role description is "Toolbar")'s last radio group's last radio button's description is "Wide Channel Strips"
            on error
                -- mixer group is not open, so open it
                keystroke "x"
                delay 2
                set mixer_group to first group whose its (first group whose role description is "Toolbar")'s first radio group's last radio button's description is "Wide Channel Strips"
            end try

            tell main_track_group
                set track_group to first group whose role description is not "Toolbar"
                tell track_group
                    tell first splitter group's second splitter group's first scroll area's first group
                        (*
                            And here we're at the list of tracks. Tracks can be addressed by index, like so:
                        *)
                        set track_item to third group
                        (* 
                            or you can find them by name, with a little trickery. The double 'whose' 
                            command is a little hard to parse, but it means "Find the first group that has a 
                            text field element with the descrition 'Bass'"
                        *)
                        set track_item to first group whose its (first UI element whose role description is "text field")'s description is "Audio Track"
                        tell track_item
                            (* 
                                once you have the track, you can access and set any values you could 
                                normally see or set in the interface, by finding the relevent UI element. 
                                all of these ui elements have their names in the 'description'
                                field. examples:
                            *)
                            -- log a list of all the visible UI elements
                            log (get description of every UI element)
                            -- get mute checkbox
                            set mute_check to first UI element whose role description is "checkbox" and description is "Mute"
                            -- log current state of the checkbox
                            log (get value of mute_check)
                            -- click the checkbox to toggle its state
                            click mute_check
                        end tell
                    end tell
                end tell
            end tell
        end tell
    end tell
end tell

If you want to constantly sample these over time, then your best bet is to create a stay-open application with an idle handler. The template for that is this:

global main_track_group, mixer_group -- global variables are available everywhere in the script

on run
    (*
        use this area to initialize stuff. In particular, initialize the two group variables 
        here so that you only have to do it once; whose commads are slow and expensive
    *)
end run

on idle
    (*
        use GUI scripting to extract the values here. the return value (below) is the idle time, in seconds: e.g. 'return 1' means that this hnalder will be called once a second. make it as small as you need to get the info you want, but not so small that this app is eating up processing time'
    *)
    return 1
end idle

on quit
    (*
do whatever cleanup you need to do here
*)
    continue quit
end quit

Save it as an application, with the Stay open after run handler checkbox checked.