3

I have a photograph containing multiple rectangles of various sizes and orientations. I am currently trying to find the distance from the camera to any rectangles present in the image. What is the best way to accomplish this?

For example, an example photograph might look like similar to this (although this is probably very out-of-proportion):

example image

I can find the pixel coordinates of the corners of any of the rectangles in the image, along with the camera FOV and resolution. I also know beforehand the length and width of any rectangle that could be in the image (but not what angle they face the camera). The ratio of length to width of each rectangular target that could be in the image is guaranteed to be unique. The rectangles and the camera will always be parallel to the ground.

What I've tried:

I hacked out a solution based on some example code I found on the internet. I'm basically iterating through each rectangle and finding the average pixel length and height.

I then use this to find the ratio of length vs. height, and compare it against a list of the ratios of all known rectangular targets so I can find the actual height of the target in inches. I then use this information to find the distance:

distance formula

...where actual_height is the real height of the target in inches, the IMAGE_HEIGHT is how tall the image is (in pixels), the pixel_height is the average height of the rectangle on the image (in pixels), and the VERTICAL_FOV is the angle the camera sees along the vertical axis in degrees (about 39.75 degrees on my camera).

I found this formula on the internet, and while it seems to work somewhat ok, I don't really understand how it works, and it always seems to undershoot the actual distance by a bit.

In addition, I'm not sure how to go about modifying the formula so that it can deal with rectangles that are very skewed from viewing them along an angle. Since my algorithm works by finding the proportion of the length and height, it works ok for rectangles 1 and 2 (which aren't too skewed), but doesn't work for rectangle 3, since it's very skewed, throwing the ratios completely off.

I considered finding the ratio using the method outlined in this StackOverflow question regarding the proportions of a perspective-deformed rectangle, but I wasn't sure how well that would work with what I have, and was wondering if it's overkill or if there's a simpler solution I could try.

Community
  • 1
  • 1
Michael0x2a
  • 41,137
  • 26
  • 119
  • 175
  • From what I understand two edges of the rectangle are not at the same distance from the camera. So what distance do you want to compute? The one to the center of the each rectangle? – Bogdan Jan 29 '13 at 09:37
  • You said "I also know beforehand the length and width of any rectangle that could be in the image" does that mean that you can specify all 4 corners of the rectangle in some coordinate system? Said another way, you know everything about the structure of the rectangle and you are trying to find its position/orientation. – Hammer Jan 29 '13 at 17:23
  • @Bogdan -- yeah, I'm looking for the distance to the center of the rectangle. – Michael0x2a Jan 29 '13 at 17:26
  • @Hammer -- That's correct -- that's a good way of putting it. I'm specifying the coordinates of the four corners based on the actual pixels they're located at on the image, where `(0,0)` is located at the bottom left corner and the top right corner is `(IMAGE_LENGTH, IMAGE_HEIGHT)` – Michael0x2a Jan 29 '13 at 17:27
  • @Michael0x2a the general solution to problems like this is to use a PnP algorithm like [solvePnP](http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html#cv-solvepnp) in openCV. Basically you provide points in 3d along with their current position in your image. The algorithm then returns the position of your camera (which is easy to turn into the position of the object) – Hammer Jan 29 '13 at 18:03
  • Consider asking cs-related questions in [cs.stackexchange.com](http://cs.stackexchange.com). – Realz Slaw Oct 14 '13 at 02:12
  • Did you find a solution? – Alex Black Mar 22 '16 at 22:59

1 Answers1

-1

FWIW I once did something similar with triangles (full 6DoF pose, not just distance).

Museful
  • 5,715
  • 2
  • 36
  • 53
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/low-quality-posts/11479009) – Brian Tompsett - 汤莱恩 Mar 04 '16 at 15:36
  • @BrianTompsett-汤莱恩 Thanks for the suggestion. Despite similarities, there is some distance between this question and what I linked. (They pertain to different generalizations of the same basic problem.) I'll make it a community wiki in case anyone is willing to reconcile them. If the distance is too great, we can delete the answer. – Museful Mar 04 '16 at 17:57