I have many images of yearbooks with people portraits and I'm trying to build an algorytm that will detect those portraits. At least, to detect correct rectangular portraits. Example 1 Example 2
I'm trying to investigate three directions:
- Face detection
- Dark rectangles detection (Since portraits are usually darker shapes on brighter background)
- People name extraction from OCR'ed texts
By combining results of three algorithms above, I hope to get some methodology, that will be applicable for many different yearbooks pages.
I would be very appreciate for any help for the rectangles detection. I started with Java and OpenCV 3.
Here is my code applied for an image:
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
Mat source = Imgcodecs.imread("Path/to/image", Imgcodecs.CV_LOAD_IMAGE_ANYCOLOR);
Mat destination = new Mat(source.rows(), source.cols(), source.type());
Imgproc.cvtColor(source, destination, Imgproc.COLOR_RGB2GRAY);
Imgproc.GaussianBlur(destination, destination, new Size(5, 5), 0, 0, Core.BORDER_DEFAULT);
int threshold = 100;
Imgproc.Canny(destination, destination, 50, 100);
Imgproc.Canny(destination, destination, threshold, threshold*3);
At this point, I have such result:
Trying to find contours from the edges above:
List<MatOfPoint> contourDetections = new ArrayList<>();
Mat hierarchy = new Mat();
// Find contours
Imgproc.findContours(destination, contourDetections, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
// Draw contours
Imgproc.drawContours(source, contours, -1, new Scalar(255,0,0), 2);
But not sure how to extract rectangles from those contours since many of lines are incomplete.
Getting back to edges and trying to find vertical and horizontal lines using HoughLinesP:
Mat lines = new Mat();
int thre = 50;
int minLineSize = 250;
int lineGap = 80;
int ignoreLinesShorter = 300;
Imgproc.HoughLinesP(destination, lines, 1, Math.PI/180, thre, minLineSize, lineGap);
for(int c = 0; c < lines.rows(); c++) {
double[] vec = lines.get(c, 0);
double x1 = vec[0],
y1 = vec[1],
x2 = vec[2],
y2 = vec[3];
// Filtering only verticat and horizontal lines
if(x1 == x2 || y1 == y2) {
// Filtering out short lines
if(Math.abs(x1 - x2) > ignoreLinesShorter || Math.abs(y1 - y2) > ignoreLinesShorter) {
Point start = new Point(x1, y1);
Point end = new Point(x2, y2);
// Draw line
Imgproc.line(source, start, end, new Scalar(0,0,255), 2);
}
}
}
Result:
Like with contours, I'm still not seeing correct rectangles that I could detect. Could you help me with a correct direction? Maybe there is an easier way to perform this task?