Find correspondences of points of intereset with known pixel coordinates across multiple camera views

Question

I am currently working on a project where I have nine points of interest identified through Hough lines intersection in each camera view. I have a total of three different views: left, mid, and right. While I have the pixel coordinates of these points, the correspondences between them across the three views are unknown. The cameras are calibrated, so the intrinsic and extrinsic parameters are known. My goal is to find the correspondences of these points across the three views to understand where a point in the left camera view appears in the right camera view or the mid camera view.

Here's an image showing three camera views with points of interest:

As of now, I have tried the following methods:

1. Computing the feature descriptors on these points and then brute force matching them. But it doesn't work well. One possible reason I think is that the region around these points look very similar.
2. Finding the intersection of rays passing through the projected points in 3D space using backward projection from 2D to 3D. However, it seems like this approach might be more complex than I initially anticipated since the points lie on the same plane. Here's a plot illustrating the rays from the three camera views: