3

I am writing a module for a casino surveillance software.

A bit of context:

I have several images from a video extracted at different times. I have a reference deck always at a fixed position that I know the size of. Assuming that the distortion, focal length of the camera and the distance is not known (this will be a software package sold commercially so these parameters are unknown).

Currently working:

I have applied canny edge detection and houghlines to get the pixel height of the two decks. Also the coordinates for all corners of the deck in the front is known since it is always in a fixed position with the same orientation.

The problem starts when the reference deck is not at the same distance as the deck I am trying to measure but closer to the camera. In this case since one of the decks are closer to the camera both decks may appear to have the same pixel height although the one in the back contains more cards.

Here is a visual example:

In reality both decks have 52 cards but the one closer to the camera measures 83 pixels where as the one in the back measures 63 pixels.

Sample Photo

How can I normalize the values I have for reference to the deck in the background? I have found a question which does something similar to calculate the size when moved within the image but this uses openGL which I am not familiar with (How do I reverse-project 2D points into 3D?). I am trying to achieve this using a computer vision/image manipulation library (I currently use ImageMagick for canny edge and hough lines) in Python(I am open to suggestions on using another programming language if you think python is not suitable).

EDIT:

To wrap up what I am trying to achieve: I want to know how the two objects compare height wise. How do I scale the one in the back to be the the size it would be if it was next to the one in the front so that I can compare the two?

PU2014
  • 311
  • 4
  • 17
  • 1
    Most probably, you want to scale using a transform. So, if you know the back of the cards and can find a transfer to a plane, you can estimate the distance to the camera. Take the distance as normalisation factor. For affine transforms and automatic fitting, look at e.g. Lukas-Kanade. – Dschoni Dec 20 '19 at 13:32
  • @Dschoni Thank you for your reply. Can you elaborate on finding a transfer to a plave and estimating the distance to camera? – PU2014 Dec 20 '19 at 13:42

2 Answers2

9

I'm not familiar with ImageMagick so I can't write the codes for you but I explained what your going to do:

There is something that we are sure about it: The real size of each card! You know that all cards have the same width and height (in real world), so the only thing you need to do is to find the width of upper card on the first stack and also the second stack in the image. Since these two numbers are related to each other, now you can have the ratios of the dimensions around the deck 1 and deck 2.

With these ratio you can normalize the heights of decks and compare these numbers with each other.

For example, consider width of each card (the red arrows) is 5 cm and this width in the image for the first deck is 80 pixels and for the second deck is 40 pixels. So the ratio is 2. Now For comparing those black lines, just multiply the second line's length with 2 and compare the result number with the first line's length.

enter image description here

You have lots of way to estimate the width of each card deck. Extracting the widths might be hard so I suggest to compare the width of the patterns (the yellow arrows) and for this, I would do such method:

1- Do a canny on image (because of the drawings on the card, you will have lots of line on the surface of the card)

2- Do a dilation to stick all those lines of drawing on the surface of the card and make a unit object

3- Do an erosion to shrink that object a little, and then catch the width of that object (this would be a good estimation of the width of the card)

MH304
  • 1,547
  • 7
  • 15
  • Thank you for your reply. Some casinos order custom made playing card so the size of the cards may not be standard. What can be done in such case? and can you explain steps 2 and 3 in more detail? maybe some examples? – PU2014 Dec 24 '19 at 09:58
  • @PU2014 I edited the answer. You DON'T need to know the real width of all cards in all casinos. The only thing you need is to be sure is that all cards in that casino has the same size. Step 2 is to make a unit object of the drawings on the card, when you do step 1, you will see why I suggest that. – MH304 Dec 24 '19 at 10:36
  • This is a clever solution, but unfortunately also incorrect. The two decks do not have the same angle with respect to the camera, and hence the relative sizes of the height, width and length of the decks are not comparable. You’d need to correct for projective distortion first, which involves calibrating the camera as mounted wrt the table surface. And if you have that calibration anyway, there is no longer a need to compare as proposed here. – Cris Luengo Dec 24 '19 at 14:24
  • @CrisLuengo Thanks for the good point. Of course as I mentioned it's an 'estimation', it's not completely precise, but it helps. Besides we know the real-world-shape of the cards are pure 'rectangle', so it's easy to compensate the projection without any calibration (with the help of the horizontal dimensions too). (All is estimation, even with calibration there is no way of finding the exact values of 3D dimensions in the image) – MH304 Dec 24 '19 at 14:47
  • @Meisam can you maybe give a pseudo code or an explanation on how to achieve the compensation of projection without any calibration? – PU2014 Dec 24 '19 at 16:16
  • @PU2014 You have to put a little time on this. But as a clue, consider the fact that all cards have an almost fixed height/width ratio (something around 1.6). Now define a hypothetical rectangle (the one in your CCD plane) with these points: (0,0),(0,16),(10,0),(10,16). Then find the projection between these points and each card's coordinates (cv.findHomography). That lets you to roughly estimate the calibration. – MH304 Dec 25 '19 at 11:28
0

I think, you should compare the area of your refrence deck with your target deck.
But first you should calculate the distance of the reference and target decks.
If your dataset has label or if your problem is supervise.
I think, if you use neural network, you can set the distance of two decks as the bias value.
Also, I think, you should calculate the distance of two nearest pixels of two decks as distance of two decks.

Alex 75
  • 1,507
  • 1
  • 20
  • 37
Mehdi Hamzeloee
  • 64
  • 3
  • Hi @Mehdi, I correct the grammar of your answer, but, I think the last sentence is not clear (at least not to me). Maybe you can explain better the last point? thanks – Alex 75 Jan 01 '20 at 11:42