PKCanvasView increase contentSize in all directions (infinite canvas)

Question

I'm looking to develop a CanvasView that can be extended in any direction (top, bottom, left, right) when a user draws near it close to the bounds of the canvas while keeping the user's frame of reference stable (like the example GIF below). This is simple to implement for the right and bottom directions, using a code snippet like the one below, inspired by this StackOverflow answer. However, I haven't found a good solution yet for extending it from above or the left.

    func canvasViewDrawingDidChange(_ canvasView: PKCanvasView) {
        let threshold: CGFloat = 200  // Distance from edge to trigger expansion
        let growValue: CGFloat = 400  // Amount to grow canvas size
        let drawingSize = canvasView.drawing.bounds
        
        if drawingSize.height >= (canvasView.contentSize.height - threshold) {
                canvasView.contentSize = CGSize(
                    width: canvasView.contentSize.width + growValue,
                    height: canvasView.contentSize.height + growValue
                )
            }
        }

This code works in the sense that I can swipe more and more to the right or bottom, and the canvas extends itself, allowing me to draw further away from the screen/increase the drawing. However, this does not happen when I swipe towards the top or the left of the screen, as that part is not extended.

One approach to extend the screen would be to offset the user's view (and possibly reposition the entire canvas), all upon a swipe near the bounds of the canvasView.contentSize. However, in attempts to do this, I found the experience not user-friendly. Popular drawing apps, including Apple's own Freeform, are able to seamlessly extend the canvas while keeping the user's frame/position stable (example GIF below). What would be an approach to replicate this?

Here's a code sample from the reposition + offset:

    func canvasViewDrawingDidChange(_ canvasView: PKCanvasView) {
      let threshold: CGFloat = 200  // Distance from edge to trigger expansion
      let growValue: CGFloat = 400  // Amount to grow canvas size
      let drawingSize = canvasView.drawing.bounds

      // Check all four edges
      if drawingSize.minX <= threshold {
        canvasView.contentSize.width += growValue
        canvasView.contentOffset.x -= growValue  // Adjust offset to keep frame stable
      }
      if drawingSize.minY <= threshold {
        canvasView.contentSize.height += growValue
        canvasView.contentOffset.y -= growValue  // Adjust offset to keep frame stable
      }
      if drawingSize.maxX >= (canvasView.contentSize.width - threshold) {
        canvasView.contentSize.width += growValue
      }
      if drawingSize.maxY >= (canvasView.contentSize.height - threshold) {
        canvasView.contentSize.height += growValue
      }
    }

Here is a GIF of the desired state, a screen recording from Apple's Freeform. In it, I draw the bounds and then extend the screen. In a new Freeform file, upon immediately zooming out to the max zoom (10%), it takes a split second to extend the canvas but it does so only from below and the right (with an empty canvas). Upon drawing, it can extend in any direction.

I think it is helpful to mention that I do not have much experience developing mobile applications in the Swift language, this being my first.