How to check if obtained homography matrix is good?

Question

This question was already asked, but I still don't get it. I obtain a homography matrix by calling cv::findHomography from a set of points. I need to check whether it's relevant or not.
The proposed method is to calculate maximum reprojection error for inliers and compare it with a threshold. But after such filtration I keep getting insane transformations with object bounding box transforming to almost a straight line or some strange non-convex quadrangle, with self-intersections etc.
What constraints can be used to check if the homography matrix itself is adequate?

Answer 1

Your question is mathematical. Given a matrix of 3x3 decide whether it represents a good rigid transformation. It is hard to define what is "good" but here are some clues that can help you

1. Homography should preserve the direction of polygonal points. Design a simple test. points (0,0), (imwidth,0), (width,height), (0,height) represent a quadrilateral with clockwise arranged points. Apply homography on those points and see if they are still clockwise arranged if they become counter clockwise your homography is flipping (mirroring) the image which is sometimes still ok. But if your points are out of order than you have a "bad homography"
2. The homography doesn't change the scale of the object too much. For example if you expect it to shrink or enlarge the image by a factor of up to X, just check this rule. Transform the 4 points (0,0), (imwidth,0), (width-1,height), (0,height) with homography and calculate the area of the quadrilateral (opencv method of calculating area of polygon) if the ratio of areas is too big (or too small), you probably have an error.
3. Good homography is usually uses low values of perspectivity. Typically if the size of the image is ~1000x1000 pixels those values should be ~0.005-0.001. High perspectivity will cause enormous distortions which are probably an error. If you don't know where those values are located read my post: trying to understand the Affine Transform . It explains the affine transform math and the other 2 values are perspective parameters.

I think that if you check the above 3 condition (condition 2 is the most important) you will be able to detect most of the problems. Good luck

Answer 2

Edit: This answer is irrelevant to the question, but the discussion may be helpful for someone who tries to use the matching results for recognition like I did!

This might help someone:

Point2f[] objCorners = { new Point2f(0, 0),
    new Point2f(img1.Cols, 0),
    new Point2f(img1.Cols, img1.Rows),
    new Point2f(0, img1.Rows) };

Point2d[] sceneCorners = MyPerspectiveTransform3(objCorners, homography);
double marginH = img2.Width * 0.1d;
double marginV = img2.Height * 0.1d;
bool homographyOK = isInside(-marginH, -marginV, img2.Width + marginH, img2.Height + marginV, sceneCorners);
if (homographyOK)
    for (int i = 1; i < sceneCorners.Length; i++)
        if (sceneCorners[i - 1].DistanceTo(sceneCorners[i]) < 1)
        {
            homographyOK = false;
            break;
        }
if (homographyOK)
    homographyOK = isConvex(sceneCorners);
if (homographyOK)
    homographyOK = minAngleCheck(sceneCorners, 20d);




     private static bool isInside(dynamic minX, dynamic minY, dynamic maxX, dynamic maxY, dynamic coors)
        {
            foreach (var c in coors)
                if ((c.X < minX) || (c.Y < minY) || (c.X > maxX) || (c.Y > maxY))
                    return false;
            return true;
        }      
        private static bool isLeft(dynamic a, dynamic b, dynamic c)
        {
            return ((b.X - a.X) * (c.Y - a.Y) - (b.Y - a.Y) * (c.X - a.X)) > 0;
        }
        private static bool isConvex<T>(IEnumerable<T> points)
        {
            var lst = points.ToList();
            if (lst.Count > 2)
            {
                bool left = isLeft(lst[0], lst[1], lst[2]);
                lst.Add(lst.First());
                for (int i = 3; i < lst.Count; i++)
                    if (isLeft(lst[i - 2], lst[i - 1], lst[i]) != left)
                        return false;
                return true;
            }
            else
                return false;
        }
        private static bool minAngleCheck<T>(IEnumerable<T> points, double angle_InDegrees)
        {
            //20d * Math.PI / 180d
            var lst = points.ToList();
            if (lst.Count > 2)
            {                
                lst.Add(lst.First());
                for (int i = 2; i < lst.Count; i++)
                {
                    double a1 = angleInDegrees(lst[i - 2], lst[i-1]);
                    double a2 = angleInDegrees(lst[i], lst[i - 1]);
                    double d = Math.Abs(a1 - a2) % 180d;

                    if ((d < angle_InDegrees) || ((180d - d) < angle_InDegrees))
                        return false;
                }
                return true;
            }
            else
                return false;
        }
        private static double angleInDegrees(dynamic v1, dynamic v2)
        {
            return (radianToDegree(Math.Atan2(v1.Y - v2.Y, v1.X - v2.X))) % 360d;
        }
        private static double radianToDegree(double radian)
        {
            var degree = radian * (180d / Math.PI);
            if (degree < 0d)
                degree = 360d + degree;

            return degree;
        }
        static Point2d[] MyPerspectiveTransform3(Point2f[] yourData, Mat transformationMatrix)
        {
            Point2f[] ret = Cv2.PerspectiveTransform(yourData, transformationMatrix);
            return ret.Select(point2fToPoint2d).ToArray();
        }