Distort/resize polygon mask to match another polygon to match facial landmarks with python

Question

Need help on an issue I've been struggling with for a few weeks now...

The situation:

I have a diffusion model making treating faces. The result inside the face is good but it gets all blurry outside the face. See the exemple below (left part of the face):

Blurry Face

I'm trying to do some post processing treatment to blend the generated face in the original one to get rid of the blurry part.

The issue is that the model distords the face and makes it thinner which gives poor results when pasting the face back (see the double chin in the image below).

Double Chin

Applying a gaussian blur or blending the junction between original/predicted does not make it better because the gap is too large.

What I'm trying to do:

First: distort/resize the polygon of the predicted face (based on facial landmarks) to match the polygon of the original frame to minimize/remove the gap.

Then: apply a small blur or blend strategy.

I tried opencv methods to warp the mask but it is not precise and creates holes in the frame.

I've read that polynomial distortions might be the solution here, but it seems a lot more complex (both to implement and computational-wise).

Is there any way I can achieve this ?

If not, are there facial in painting models I could use to perform a guided recovery (masking the junction)?

Thanks for your help!