CODEWITHSUNDEEP
objective-ccocoaaccessibility
Antony
Antony

Reputation: 41

Accessibility API for Big Sur 11.5 broken?

After replacing Big Sur OSX 11.0 with the latest 11.5, my app's AXObserverAddNotification methods fails. Here is sample code I tested:

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {

   NSLog(@"Hello world");

   AXUIElementRef frontMostApp = AXUIElementCreateApplication(82695); // '82695' is the PID reference to my Xcode 12.5.1 app running at front

   NSLog(@"frontMostApp: %@",frontMostApp);
    
   CFTypeRef frontMostWindow = NULL;

   AXError error = AXUIElementCopyAttributeValue(frontMostApp, kAXFocusedWindowAttribute, (CFTypeRef *)&frontMostWindow);

   if (error != kAXErrorSuccess) {

    NSLog(@"failed with error: %i",error);

}

'frontMostWindow' reference is never created and I get the error number -25204. It seems like the latest Big Sur 11.5 has revised the Accessibility API or perhaps there is some permission switch I am unaware of that would make things work. What am I doing wrong?

Upvotes: 1

Views: 351

Answers (3)

Antony
Antony

Reputation: 41

After over a year being unable to get latest developer copy of SpellSentry to work, I found that by providing Accessibility security clearance to the specific target application of the developer copy, the developer SpellSentry was able to interact with external app, TextEdit and Pages. The problem was that the security clearance that I was enabling targeted an existing copy of SpellSentry so that the developer edition was still being restricted. Also, when attempting to isolate the problem, I built a test application (posted above) that would also fail to interact with the third-part apps. Apparently only sandboxed apps that contain permissions for the third-party apps will work—things I did not include in the test apps. The upshot is that the accessibility API mechanisms still work in the latest OS, allowing application interaction with third-part apps so long as proper security clearance is provided in the code, and that the app’s specific output file is given permission to “control your computer” when user is prompted.

Upvotes: 0

Antony
Antony

Reputation: 41

Using similar code to the one posted, I am able to get the observer call-back mechanism to work only upon the main application's set of windows. When I attempt to evoke a callback on another app's windows, I get an error showing -25204 ("A fundamental error has occurred")--even when I provide this application Accessability permission within the Security and Privacy preferences. This problem has occurred as a result of installing the latest macOS Big Sur, version 11.5.2.

I suspect that Apple no longer will support Accessibility programming outside one's own application for security reasons, unless this they have created a bug in the new OS.

The following code works on an app that posts two windows, but does not work when evoking the windows of an external application such as TextEdit:

@interface AppDelegate : NSObject <NSApplicationDelegate>{

    IBOutlet NSWindow *window1;
    IBOutlet NSWindow *window2;
}

@end


@implementation AppDelegate

NSWorkspace *workspace;

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {

    [self installFocusedWindowChangedAXObserver];

}

static void focusedWindowChangedCallback(AXObserverRef observer, AXUIElementRef element, CFStringRef notification, void *self){
NSLog(@"mainWindowChangedCallback");

     NSBeep();    
}

- (AXError)installFocusedWindowChangedAXObserver {

    usleep(100000); // This pause allows me time to click on another application besides this one to test observer call-back mechanism if I wish. Otherwise, this test application will post window1 and window2 that will trigger the kAXFocusedWindowChangedNotification mechanism when they are alternately clicked upon and thus change the window focus.

    NSRunningApplication* frontMostApp = [[NSWorkspace sharedWorkspace] frontmostApplication];

    pid_t pid = [frontMostApp processIdentifier];
    NSLog(@"pid: %i",pid);

    AXObserverRef observer = NULL;
    
    AXError axErr = AXObserverCreate(pid, focusedWindowChangedCallback, &observer); // Note that the 2nd param is a 'AXObserverCallback' method (see documentation for method signature strictures)
    NSLog(@"axErr: %i",axErr);
    NSLog(@"observer: %@",observer);

    CFRunLoopAddSource(CFRunLoopGetCurrent(),
                   AXObserverGetRunLoopSource(observer),
                   kCFRunLoopDefaultMode);

    AXUIElementRef element = AXUIElementCreateApplication(pid); // pid of this app (the observer watcher)
    NSLog(@"element: %@",element);

    axErr = AXObserverAddNotification(observer, element, kAXFocusedWindowChangedNotification, (__bridge void *)(self));
    NSLog(@"axErr: %i",axErr);
    
    return axErr;
}

@end

Upvotes: 1

pietrorea
pietrorea

Reputation: 898

Try switching this line:

CFTypeRef frontMostWindow = NULL;

to:

AXUIElementRef frontMostWindow = NULL;

Internally, I bet AXUIElementCopyAttributeValue is doing a sizeof operation with the pointer you pass in to calculate the right amount of memory to allocate for the result (which is returned in a __AXUIElement struct, according to the header file).

My guess is that doing a sizeof() of a CFTypeRef does not give you the right size, so it doesn't work and you get that memory allocation error code.

Upvotes: 0

Related Questions