jnpdx
jnpdx

Reputation: 52555

ScreenCaptureKit delivers incorrect frames on macOS after "Move focus to next window" keyboard command

I've encountered a bug in macOS 14 while using ScreenCaptureKit. I've filed a Feedback (FB13561921), but in the meantime, I'm wondering if anyone has a way to circumvent this bug.

The bug

When using ScreenCaptureKit to get frames from a captured display, the frames become incorrect/unreliable after using the “Move focus to next window” (CMD `) keyboard command in an app with multiple open windows. The foregrounded window doesn’t appear foregrounded — it still appears behind the last focused window.

In other words, I expect to see the first image (and do on the actual screen), but see the 2nd on the captured screen.

enter image description here enter image description here

Steps to reproduce:

This bug happens on macOS 14.1-14.3. It does not appear to happen on macOS 13.6

Attempts I've made to work around this

The Question

Is there a workaround to get ScreenCaptureKit to produce valid updated frames of the newly-focused window after this key command has been used that doesn't create visual artifacts or require Accessibility like my "hack"?


Code for my Accessibility "hack" that isn't really workable due to visual artifacts:
func moveTheWindow(_ point: CGPoint) async {
    guard let originalCursorPosition = CGEvent(source: nil)?.location else {
        print("Failed to get the original cursor position.")
        return
    }

    let mouseDownPoint = point
    let mouseDragRightPoint = CGPoint(x: point.x + 1, y: point.y)
    let mouseDragLeftPoint = CGPoint(x: point.x, y: point.y)

    // Mouse down at the starting point
    if let mouseDown = CGEvent(mouseEventSource: nil, mouseType: .leftMouseDown, mouseCursorPosition: mouseDownPoint, mouseButton: .left) {
        mouseDown.post(tap: .cghidEventTap)
    }

    // Drag the window 1px to the right
    if let mouseDragRight = CGEvent(
        mouseEventSource: nil,
        mouseType: .leftMouseDragged,
        mouseCursorPosition: mouseDragRightPoint,
        mouseButton: .left
    ) {
        mouseDragRight.post(tap: .cghidEventTap)
    }

    // Mouse up to finish the first drag
    if let mouseUpRight = CGEvent(
        mouseEventSource: nil,
        mouseType: .leftMouseUp,
        mouseCursorPosition: mouseDragRightPoint,
        mouseButton: .left
    ) {
        mouseUpRight.post(tap: .cghidEventTap)
    }

    try? await Task.sleep(nanoseconds: NSEC_PER_SEC / 20)

    // Mouse down again for the second drag
    if let mouseDownAgain = CGEvent(
        mouseEventSource: nil,
        mouseType: .leftMouseDown,
        mouseCursorPosition: mouseDragRightPoint,
        mouseButton: .left
    ) {
        mouseDownAgain.post(tap: .cghidEventTap)
    }

    // Drag the window back 1px to the left
    if let mouseDragLeft = CGEvent(
        mouseEventSource: nil,
        mouseType: .leftMouseDragged,
        mouseCursorPosition: mouseDragLeftPoint,
        mouseButton: .left
    ) {
        mouseDragLeft.post(tap: .cghidEventTap)
    }

    // Mouse up to complete the second drag
    if let mouseUpLeft = CGEvent(mouseEventSource: nil, mouseType: .leftMouseUp, mouseCursorPosition: mouseDragLeftPoint,
                                 mouseButton: .left)
    {
        mouseUpLeft.post(tap: .cghidEventTap)
    }

    try? await Task.sleep(nanoseconds: NSEC_PER_SEC / 20)

    // Move the cursor back to the original position
    CGWarpMouseCursorPosition(originalCursorPosition)
}

Upvotes: 4

Views: 314

Answers (0)

Related Questions