Calibrating Webcam using Python and OpenCV Error

Question

Pretty new to all this and I'm trying to do the calibration for a Webcam following this guide and using the code below. I get the following error ..

OpenCV Error: Assertion failed (ni > 0 && ni == ni1) in collectCalibrationData, file /build/buildd/opencv-2.4.8+dfsg1/modules/calib3d/src/calibration.cpp, line 3193

cv2.error: /build/buildd/opencv-2.4.8+dfsg1/modules/calib3d/src/calibration.cpp:3193: error: (-215) ni > 0 && ni == ni1 in function collectCalibrationData

Can someone explain what this error is and how to fix it?

(Full error at the bottom)

import numpy as np
import cv2
import glob


criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# Arrays to store object points and image points from all the images.
objpoints = [] # 3d point in real world 
imgpoints = [] # 2d points in image plane.
images = glob.glob('*.png')


objp = np.zeros((6*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
objp = objp * 22


for fname in images:

    img = cv2.imread(fname)
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    ret = False
    # Find the chess board corners
    ret, corners = cv2.findChessboardCorners(gray, (6,9))
    # If found, add object points, image points (after refining them)
    if ret == True:
        objpoints.append(objp)
        cv2.cornerSubPix(gray, corners, (11,11), (-1,-1), criteria)

        imgpoints.append(corners)
        # Draw and display the corners 
        cv2.drawChessboardCorners(img, (6,9), corners, ret)
        cv2.imshow('img',img)
        cv2.waitKey(0)

cv2.waitKey(0)
for i in range (1,5):
    cv2.waitKey(1)
    cv2.destroyAllWindows()
    cv2.waitKey(1)


ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None)

OpenCV Error: Assertion failed (ni > 0 && ni == ni1) in collectCalibrationData, file /build/buildd/opencv-2.4.8+dfsg1/modules/calib3d/src/calibration.cpp, line 3193 Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 540, in runfile execfile(filename, namespace) File "/home/students/Test/test.py", line 49, in ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None) cv2.error: /build/buildd/opencv-2.4.8+dfsg1/modules/calib3d/src/calibration.cpp:3193: error: (-215) ni > 0 && ni == ni1 in function collectCalibrationData

score 50 · Accepted Answer · answered Apr 06 '16 at 04:48

I had this same problem, and your main mistake (which I know because I made it myself) is that you have changed checkerboard size (default in example is 7x6, yours is 6x9), but you have neglected to change the size in the initialization code at the top of the routine

objp = np.zeros((6*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)

To make this work with multiple checkerboard sizes you could adjust the code like so:

# checkerboard Dimensions
cbrow = 5
cbcol = 7

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((cbrow * cbcol, 3), np.float32)
objp[:, :2] = np.mgrid[0:cbcol, 0:cbrow].T.reshape(-1, 2)

.
.
.
ret, corners = cv2.findChessboardCorners(gray, (cbcol, cbrow), None)

this is indeed the real answer, it should be marked as the valid so others don´t make the same mistake as i did, didn't go to the bottom to check all answers... — Rui Sebastião, Jul 10 '17 at 17:51

score 4 · Answer 2 · answered Jul 12 '15 at 10:46

Digging into the source code:

for( i = 0; i < nimages; i++, j += ni )
{
    Mat objpt = objectPoints.getMat(i);
    Mat imgpt1 = imagePoints1.getMat(i);
    ni = objpt.checkVector(3, CV_32F);
    int ni1 = imgpt1.checkVector(2, CV_32F);
    CV_Assert( ni > 0 && ni == ni1 );
    ...

This: Assertion failed (ni > 0 && ni == ni1) means either that your object points array is of length zero, or that the object and image arrays are of different sizes.

To be clear: calibrateCamera() expects to be provided with not just an array of object points and another array of image points, but with an array of arrays of image and object points. If you only have one image (and therefore one pair of sets of image and object points) you can just wrap those arrays in a set of square brackets:

obj = [[x, y, z], [x1, y1, z1]] # wrong
img = [[x, y], [x1, y1]]        # wrong

obj = [[[x, y, z], [x1, y1, z1]]] # right
img = [[[x, y], [x1, y1]]]        # right

Thinking back to when I first followed this tutorial myself, it looks like the only thing you've changed is the file extension (from jpg to png) which suggests that you are using your own resources - and therefore that your problem could lie with the number of images you are using. I think it's possible that the image/images you are using just aren't having the checkerboard picked out successfully - and therefore your object point array never gets anything added to it. Try printing out the arrays before running calibrateCamera and maybe using the checkerboard images provided inside /samples/cpp

Marta Gómez · Answer 3 · 2016-12-19T15:55:41.943

I had the same issue and after a little research I found the problem in the answers here.

I resolved it changing the shape of objp:

objp = objp.reshape(-1,1,3)

Also I had another problem: the number of corners found by findChessboardCorners can be less than 7*6 (pattern size) so I just kept number of corners found 3D points:

corners2 = cv2.cornerSubPix(image=gray, corners=corners, 
               winSize=(11,11), zeroZone=(-1,-1),
               criteria=(cv2.TERM_CRITERIA_EPS + 
                         cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001))
imgpoints.append(corners2)
objpoints.append(objp[0:corners2.shape[0]])

after that the code worked fine :D

EDIT: I realized the number of corners can be less than pattern size if instead of using the retval (True or False) we check that corners is not None.

score 0 · Answer 4 · answered Jul 08 '15 at 12:57

So I have found that the error is due to imgpoints being a 1 long array when it should be as long as objpoints. I've found that if you use one image you can then replace imgpoints within the calibration function to corners directly. Hope that helps anyone with the same error.

(Have made some changes along the way and am still attempting to fix it to use multiple images)

import numpy as np
import cv2
import glob

# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# Arrays to store object points and image points from all the images.

imgpoints = [] # 2d points in image plane.
images = glob.glob('*.png')

for fname in images:

    img = cv2.imread(fname)
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    ret = False
    # Find the chess board c  orners
    ret, corners = cv2.findChessboardCorners(gray, (6,9))
    # If found, add object points, image points (after refining them)
    if ret == True:
        cv2.cornerSubPix(gray, corners, (11,11), (-1,-1), criteria)
        imgpoints.append(corners)
        # Draw and display the corners 
        cv2.drawChessboardCorners(img, (6,9), corners, ret)
        cv2.imshow('img',img)
        cv2.waitKey(0)

cv2.waitKey(0)
for i in range (1,5):
    cv2.waitKey(1)
    cv2.destroyAllWindows()
    cv2.waitKey(1)
imgpoints = np.array(imgpoints,'float32')
print len(corners), len(pattern_points)



pattern_size = (9, 6)
pattern_points = np.zeros( (np.prod(pattern_size), 3), np.float32)
pattern_points[:, :2] = np.indices(pattern_size).T.reshape(-1, 2).astype(np.float32)
pattern_points = np.array(pattern_points,dtype=np.float32)

ret, matrix, dist_coef, rvecs, tvecs = cv2.calibrateCamera([pattern_points], [corners], gray.shape[::-1], flags=cv2.CALIB_USE_INTRINSIC_GUESS)

score 0 · Answer 5 · answered Sep 14 '16 at 16:54

Thank you @s-low! Your source code is very helpful.

Another possible mistake for beginners is about the data type of objp.

When I assigned the objp = np.zeros((6*7,3), np.float32), I neglected the datatype assignment. In python 2.7, the default dtype is float64. So, when the code was calling the function 'cv2.calibrateCamera', there was a similar assertation error:

OpenCV Error: Assertion failed (ni >= 0) in collectCalibrationData, file /Users/jhelmus/anaconda/conda-bld/work/opencv-2.4.8/modules/calib3d/src/calibration.cpp, line 3169

So, just the source code ni = objpt.checkVector(3, CV_32F) gave me a clue that the matrix of objp has to be assigned to float32.

score 0 · Answer 6 · answered Dec 21 '16 at 19:59

If you followed the example in:

opencvdoc

Then, the problem is easy to solve, it is because the function:

Corners2 = cv2.cornerSubPix (gray, corners, (11,11), (-1,1), criteria)

In the current version of opencv returns Null

This one too:

Img = cv2.drawChessboardCorners (img, (7.6), corners2, ret)

So all you have to do is change those lines of code, this is my adaptation of the code:

from webcam import Webcam
import cv2
from datetime import datetime
import numpy as np

webcam = Webcam()
webcam.start()

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
objp = np.zeros((6*9,3), np.float32)
objp[:,:2] = np.mgrid[0:9,0:6].T.reshape(-1,2)
objpoints = []
imgpoints = []
i = 0

while i < 10:
    image = webcam.get_current_frame()
    gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
    ret, corners = cv2.findChessboardCorners(gray, (9,6), None)    
    print ret

    if ret == True:
        cv2.cornerSubPix(gray,corners,(11,11),(-1,-1),criteria)       
        imgpoints.append(corners)
        objpoints.append(objp)
        cv2.drawChessboardCorners(image, (9,6), corners,ret)
        i += 1


    cv2.imshow('grid', image)
    cv2.waitKey(1000)

cv2.destroyAllWindows()
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None)
np.savez("webcam_calibration_ouput_2", ret=ret, mtx=mtx, dist=dist, rvecs=rvecs, tvecs=tvecs)

Webcam class:

import cv2
from threading import Thread

class Webcam:

    def __init__(self):
        self.video_capture = cv2.VideoCapture(0)
        self.current_frame = self.video_capture.read()[1]

    # create thread for capturing images
    def start(self):
        Thread(target=self._update_frame, args=()).start()

    def _update_frame(self):
        while(True):
            self.current_frame = self.video_capture.read()[1]

    # get the current frame
    def get_current_frame(self):
        return self.current_frame

Note that the numbers 6 and 9 could change depending of the dimensions of your chess, the dimmensión of my chess was 10x7.

score 0 · Answer 7 · answered Mar 12 '17 at 12:25

I had the same problem while I was inputting my own jpg images, taken by self with my mobile cam. Initially, when I was just running the code available in the link shared by you, I found that rect was always set to FALSE. I figured out later that I was inputting the exact size of the checkerboard which made the code to not recognise the checkerboard pattern. What I mean is I used the checkerboard of size 8X6 and inputted 8X6 in the code, say for example just like below

objp = np.zeros((6*8,3), np.float32)
objp[:,:2] = np.mgrid[0:8,0:6].T.reshape(-1,2)

Because of which it couldn't recognise the pattern, when I reduced the row and column dimension each by 1 (or more than one, In my case, its 7X5) then boom, I got the parameters with printed images as output.

objp = np.zeros(((5*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:5].T.reshape(-1,2)

Also, if you are using this doc for OpenCV 3.0 beta doc there's a minor change that might need to be rectified, the difference of which you can spot by going here

score 0 · Answer 8 · answered Jun 23 '17 at 02:12

i got the same problem which leads me to Assertion failed (ni > 0 && ni == ni1); i think the real meaning of ni and ni1 should be learned which had been talked in the first answer. the objectpoint and imagepoints should be exactly the same size. no matter the number of mat or the size of mat. for me, my problem is that i have both 5 mats for imagepoints and objectpoints while for the size of the mat, the imagepoints is 36*2 while the objectpoints is 48*3. So i said they should be exactly the same except one is point2f and another is point3f.

i solved this problem after changing the size of the objectpoints to 36*3. — Nan, Jun 23 '17 at 02:13

Calibrating Webcam using Python and OpenCV Error

8 Answers8

Linked