12

I'm trying to build a handwriting recognition system using python and opencv. The recognition of the characters is not the problem but the segmentation. I have successfully :

  • segmented a word into single characters
  • segmented a single sentence into words in the required order.

But I couldn't segment different lines in the document. I tried sorting the contours (to avoid line segmentation and use only word segmentation) but it didnt work. I have used the following code to segment words contained in a handwritten document , but it returns the words out-of-order(it returns words in left-to-right sorted manner) :

import cv2
import numpy as np
#import image
image = cv2.imread('input.jpg')
#cv2.imshow('orig',image)
#cv2.waitKey(0)

#grayscale
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
cv2.imshow('gray',gray)
cv2.waitKey(0)

#binary
ret,thresh = cv2.threshold(gray,127,255,cv2.THRESH_BINARY_INV)
cv2.imshow('second',thresh)
cv2.waitKey(0)

#dilation
kernel = np.ones((5,5), np.uint8)
img_dilation = cv2.dilate(thresh, kernel, iterations=1)
cv2.imshow('dilated',img_dilation)
cv2.waitKey(0)

#find contours
im2,ctrs, hier = cv2.findContours(img_dilation.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

#sort contours
sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])

for i, ctr in enumerate(sorted_ctrs):
    # Get bounding box
    x, y, w, h = cv2.boundingRect(ctr)

    # Getting ROI
    roi = image[y:y+h, x:x+w]

    # show ROI
    cv2.imshow('segment no:'+str(i),roi)
    cv2.rectangle(image,(x,y),( x + w, y + h ),(90,0,255),2)
    cv2.waitKey(0)

cv2.imshow('marked areas',image)
cv2.waitKey(0)

Please note that i am able to segment all the words here but they appear out order.Is there any way to sort these contours in order of top to bottom

OR

segment the image into separate lines so that each line can be segmented into words using above code?

Sidharth Ramesh
  • 396
  • 2
  • 3
  • 19
  • why don't you sort it by x and y then? – Tom Sep 18 '17 at 21:23
  • Actually i tried that but it just results in complete reordering of contours again. it doesnt return the contours in the correct order. I did left-to-right sorting followed by top-down sorting ,but contours are out of order. So i think each line would need to be separated. – Sidharth Ramesh Sep 19 '17 at 04:09
  • Making a list sorting contours based on x,y is the easiest i'd say. Add an image is you can. – I.Newton Sep 19 '17 at 04:30
  • @SidharthRamesh hi there. Do you have any contact (WA, Telegram ) ? – lucians Dec 05 '17 at 21:37
  • @Link you could email me at sidharthr44@gmail.com – Sidharth Ramesh Dec 07 '17 at 15:12
  • @SidharthRamesh How did separate word into single characters? Are there any references? – Vishnu Jayanand May 02 '18 at 04:07
  • 1
    @VishnuJayanand I updated the answer addressing your query, – Sidharth Ramesh May 03 '18 at 16:36
  • @SidharthRamesh Does reducing iterations work for separating handwritten characters? Also, how do you deal with text of different font sizes? – Vishnu Jayanand May 04 '18 at 08:07
  • 1
    @VishnuJayanand I dont understand your doubt on reducing iterations. This code will work for different font sizes because the cropped region for a line is calculated based on the largest character in that line. You can check the code with images having varying sized text. – Sidharth Ramesh Aug 08 '18 at 10:51

1 Answers1

12

I got the required segmentation by making a change to the above code on the line:

kernel = np.ones((5,5), np.uint8)

I changed it to :

kernel = np.ones((5,100), np.uint8)

Now i get the outputs as followingsegmented lines of input text This also works with handwritten text images with lines that are not perfectly horizontal:

EDIT : For getting individual characters out of a word, do the following :

  1. Resize the contour containing the word using the code as follows.

    im = cv2.resize(image,None,fx=4, fy=4, interpolation = cv2.INTER_CUBIC)

  2. Apply same contour detection process as in line segmentation, but with a kernel of size (5,5), i.e :

    kernel = np.ones((5,5), np.uint8)
img_dilation = cv2.dilate(im_th, kernel, iterations=1)
Sidharth Ramesh
  • 396
  • 2
  • 3
  • 19
  • Can you describe how to crop each bounding line into separate image. Any idea or resource. Your answer is helpful – susan097 Dec 25 '18 at 16:30
  • @Sushant you mean saving each of the cropped contours into separate images ? just save each of the detected contours as an image. thats it. – Sidharth Ramesh Jan 09 '19 at 09:54