Accurate corners detection?

Question

I have this trivial reference image (others have objects in the white part) and I would like to detect the four corners to fix the perspective. I have tried several strategies:

Contours + Houghlines + intersections of lines

This strategy does not work because the hough lines are not correctly found. The white rectangle has a slight barrel deformation and the lines are not straight so I find multiple lines for each edges.

Contour + goodFeaturesToTrack

This looked promising, but with cv.goodFeaturesToTrack(u, 4, 0.5, 50) the corners are only found on the two bottom corners

Is there any other method to accurately detect the four corners of this rectangle ?

Take a look at [How to find corners on a Image using OpenCv](https://stackoverflow.com/questions/7263621/how-to-find-corners-on-a-image-using-opencv), [Approximating jagged edges as lines](https://stackoverflow.com/questions/59383119/approximating-jagged-edges-as-lines), [Detecting a lego baseplate in an image](https://stackoverflow.com/questions/37689009/detecting-a-lego-baseplate-in-an-image) — nathancy, Feb 24 '20 at 21:50

score 1 · Accepted Answer · answered Feb 22 '20 at 19:55

In your image, rectangle seems a proper rectangle to detect its contours. So fitted rectangle to this contour can fix the problem. My approach is using minAreaRect function:

Here is my code:

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>

using namespace cv;
using namespace std;


int main()
{

    Mat src; Mat src_gray;
    int thresh = 100;
    src = imread( "/ur/image/directory/image.png", 1 );
    Mat source = src.clone();
    cvtColor( src, src_gray, CV_BGR2GRAY );

    Mat threshold_output;
    vector<vector<Point> > contours;
    vector<Vec4i> hierarchy;

    threshold( src_gray, threshold_output, thresh, 255, THRESH_BINARY );
    findContours( threshold_output, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );

    vector<RotatedRect> minRect( contours.size() );

    for( int i = 0; i < contours.size(); i++ )
        minRect[i] = minAreaRect( Mat(contours[i]) );

    for( int i = 0; i< contours.size(); i++ )
    {

        Point2f rect_points[4]; minRect[i].points( rect_points );
        for( int j = 0; j < 4; j++ )
        {
            circle(source,rect_points[j],10,Scalar(255,255,0),2);
            line( source, rect_points[j], rect_points[(j+1)%4], Scalar(0,255,0), 2, 8 );
        }
    }

    imshow("output",source);

    waitKey(0);
    return(0);
}

Result:

Your approach is sweet. It accurately detect the corners only under the assumption this is a rectangle. Unfortunately sometimes I have some perspective distortions and `minAreaRect` is not the proper solution. — nowox, Feb 23 '20 at 15:36
Yes because I assumed its rectangle :). You may need to share a sample image to be able to see what kind of a distortionn. — Yunus Temurlenk, Feb 23 '20 at 15:39

score 0 · Answer 2 · answered Feb 22 '20 at 22:20

Python based solution similar to Yunus Temurlenk's answer. First Otsu thresholding, canny edge detection and dilation applied to get a good outline. Next, the largest contour is extracted by area and four points are extracted based on minimum area rectangle.

This code will also work without dilation step in this image. In that case the outline will be a tighter fit of the rectangle.

Code:

## Press ESC button to get next image

import cv2
import cv2 as cv
import numpy as np



#frame = cv2.imread('resources/pstr1.png')
frame = cv2.imread('resources/pstr2.png')


## keeping a copy of original
print(frame.shape)
original_frame = frame.copy()




## Show the original image
winName = 'Original'
cv.namedWindow(winName, cv.WINDOW_NORMAL)
cv.resizeWindow(winName, 800, 800)
cv.imshow(winName, original_frame)
cv.waitKey(0)



# Otsu's thresholding
grayimg = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
ret2,thresh_n = cv.threshold(grayimg,0,255,cv.THRESH_BINARY+cv.THRESH_OTSU)
thresh_n = cv2.cvtColor(thresh_n, cv2.COLOR_GRAY2BGR)
frame = thresh_n


## edge detection
frame = cv2.Canny(frame,100,200)


## dilate the edges
kernel = np.ones((5,5),np.uint8)
frame = cv2.dilate(frame,kernel,iterations = 1)




## Get largest contour from contours
contours, hierarchy = cv2.findContours(frame, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)



## Get minimum area rectangle and corner points
rect = cv2.minAreaRect(max(contours, key = cv2.contourArea))
box = cv2.boxPoints(rect)
print(box)




## draw anchor points on corner
z = 6
for b in box:
    cv2.circle(original_frame, tuple(b), z, 255, -1)



## show original image with corners
box2 = np.int0(box)
cv2.drawContours(original_frame,[box2],0,(0,0,255), 2)
cv2.imshow('Detected Corners',original_frame)
cv2.waitKey(0)
cv2.destroyAllWindows()

Input Image:

Output Image:

It was not needed here. I just added it to get a loose fit around rectangle and to connect edges in case of little gaps between detected edges. — B200011011, Feb 23 '20 at 12:04

Accurate corners detection?

2 Answers2

Linked