Filtering ORB matches due to average direction

Question

I'm trying to adjust a 2D Barcode. Therefore I have a reference Image containing only the edges of a barcode.

The second image is a barcode on a tube .

I want to use features to project the cornerpoints of the realworld image to the cornerpoints at the reference images.

So far the code works but the correspondenses found by BFMatcher() are not accurate even when using crosschecks.

Is there a way to filter the matches using some kind of average distance and direction threshold? So each features must have so distance and a similar direction. I'm currently using

matches = sorted(matches, key = lambda x:x.distance)