distorted narrow lens image can't be undistorted with opencv?

Question

Overview:

The image was obtained using a ids camera on a wide field lens, roughly 15-25 images are loaded in, the output from the normal openCV functions were just giving me an identical image. Thus, the fisheye functions are used here without success.

            #obtains the object 3D vectors
            number_ro_co=self.find_modulus(number_of_impoints)
            objp = np.zeros((number_ro_co[0]*number_ro_co[1],3), np.float32)
            objp[:,:2] = np.mgrid[0:number_ro_co[0],0:number_ro_co[1]].T.reshape(-1,2)
            objp=np.float32(objp)

            for i in im_corners3:
                x,y = i.ravel()
                cv.circle(img,(x,y),3,255,-1)
            
            
            if np.any(im_corners):
                
                #appends the list with no nan values
                object_list.append(objp)
                
            else:
                cv.error()


        

        object_list=np.expand_dims(np.asarray(object_list),-2) 
        #obtains the intrinsic matrix
        ret, mtx, dist, rvecs, tvecs= cv.fisheye.calibrate(object_list, img_points, gray.shape[::-1],None,None)

        print(mtx)
        print(dist)
        h,w=gray.shape[:2]
        
        newcameramtx = cv.fisheye.estimateNewCameraMatrixForUndistortRectify(mtx, dist, (w,h),np.eye(3),None,1)

        print(newcameramtx)
    

        img0=cv.imread("path_removed")
        
        #obtains the refined intrinsic matrix using a scaling factor
        h, w = img0.shape[:2]
        

        #rectifies the image
        dst=cv.fisheye.undistortImage(img0,mtx,dist,None,newcameramtx)
        

        # crops the image
       
        plt.imshow(dst);plt.show()

Please look at the code and give me constructive advice.