17

I have a fairly blurry 432x432 image of a Sudoku puzzle that doesn't adaptively threshold well (take the mean over a block size of 5x5 pixels, then subtract 2):

enter image description here

As you can see, the digits are slightly distorted, there are a lot of breakages in them, and a few 5s have fused into 6s and 6s into 8s. Also, there's a ton of noise. To fix the noise, I have to make the image even blurrier using a Gaussian blur. However, even a fairly large Gaussian kernel and adaptive threshold blockSize (21x21, subtract 2) fails to remove all the breakages and fuses the digits together even more:

enter image description here

I've also tried dilating the image after thresholding, which has a similar effect to increasing the blockSize; and sharpening the image, which doesn't do much one way or the other. What else should I try?

Community
  • 1
  • 1
1''
  • 23,546
  • 28
  • 128
  • 192
  • 1
    Have you seen [this thread?](http://stackoverflow.com/questions/10196198/how-to-remove-convexity-defects-in-sudoku-square) – karlphillip Nov 15 '12 at 03:26
  • I actually did look at this recently, but for the convexity code (which I didn't bother implementing in the end), not the normalization code. I'll definitely take a look at that now. – 1'' Nov 15 '12 at 03:32
  • 2
    Can you add non-image processing strategies by assuming a well-formed puzzle? If so you could apply some other rules that don't depend on image quality. For example, for cells where you have low confidence in the OCR result, you can check high-confidence digits in each of the three houses for the cell in question, which may allow you to constrain the possibilities for the low-confidence cells. You also could possibly partially solve the puzzle using high-confidence cells, which would add constraints. – DaveK Nov 15 '12 at 16:41
  • @DaveK I think this is a really creative solution. I like the idea of using nearby cells to improve confidence. And I don't think you'd even need to partially solve the puzzle, even with simple DFS solving takes very little time. Then you can simply take the first solution you get and assume it is correct. Throw in some parallel processing where each core takes a different assumed board and you'll have a solution in no time. – Bill Nov 16 '12 at 15:26
  • I'm using OpenCV's HoG and SVM to classify the digits. I'm not aware that it's possible to get confidences from that interface. Even if I could, I'd rather just improve the image quality going into the detection. Cool idea though, and I may consider it if all else fails. – 1'' Nov 16 '12 at 16:44

3 Answers3

21

A pretty good solution is to use morphological closing to make the brightness uniform and then use a regular (non-adaptive) Otsu threshold:

// Divide the image by its morphologically closed counterpart
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(19,19));
Mat closed = new Mat();
Imgproc.morphologyEx(image, closed, Imgproc.MORPH_CLOSE, kernel);

image.convertTo(image, CvType.CV_32F); // divide requires floating-point
Core.divide(image, closed, image, 1, CvType.CV_32F);
Core.normalize(image, image, 0, 255, Core.NORM_MINMAX);
image.convertTo(image, CvType.CV_8UC1); // convert back to unsigned int

// Threshold each block (3x3 grid) of the image separately to
// correct for minor differences in contrast across the image.
for (int i = 0; i < 3; i++) {
    for (int j = 0; j < 3; j++) {
        Mat block = image.rowRange(144*i, 144*(i+1)).colRange(144*j, 144*(j+1));
        Imgproc.threshold(block, block, -1, 255, Imgproc.THRESH_BINARY_INV+Imgproc.THRESH_OTSU);
    }
}

Result:

enter image description here

1''
  • 23,546
  • 28
  • 128
  • 192
7

Take a look at Smoothing Images OpenCV tutorial. Except GaussianBlur there are also medianBlur and bilateralFilter which you can also use to reduce noise. I've got this image from your source image (top right):

result image

Update: And the following image I got after removing small contours:

enter image description here

Update: also you can sharpen image (for example, using Laplacian). Look at this discussion.

ArtemStorozhuk
  • 8,542
  • 4
  • 30
  • 52
  • Thanks for the tips! This is better than the original but doesn't adequately solve the problem of breakages in the digits. – 1'' Nov 15 '12 at 17:03
-1

Always apply gaussian for better results.

cvAdaptiveThreshold(original_image, thresh_image, 255,
            CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY, 11, 2);
Sujay Kumar
  • 453
  • 6
  • 19