2

I have a flying drone with a downward facing camera and an IR sensor, and am working on SLAM for the drone. The drone only flies over a planar surface. To build a map of the drone's environment using the camera feed and opencv, you need to remove the camera's perspective from the captured images (introduced by height, roll, pitch, and yaw) so that distances in the images are accurate.

I've looked through openCV's methods for this, but they all seem to require a calibration of the camera beforehand to determine the camera's orientation using a script.

Is there any way to reliably transform points in the image so the spacing is constant, if you knew the orientation of the drone (and camera)? I have access to data on the drone's orientation from the IMU.

Jeru Luke
  • 13,413
  • 9
  • 57
  • 71
Luke Eller
  • 93
  • 5

2 Answers2

2

Unfortunately, inaccuracies in the drone's position and orientation will cause errors with your suggested approach.

The camera calibration is an annoying process, but is probably your best bet assuming you want a black-box solution.

But if you want to try it out, please see my comments below.

You can apply opencv's warp perspective function to try to remove the camera's perspective, but note that has several strong assumptions and may not be the best solution:

  • you have perfect information about the position of your drone
  • you know where the camera is located relative to the drone's body
  • you are flying over a perfectly planar surface

In this case you construct 3 transformation matrices to represent: the transform from the ground to the drone, the transform from the drone to the camera, and the transform from the ground to the camera by multiplying the previous two matrices.

T_{ground to camera} = T_{ground to drone}*T_{drone to camera}

The inverse of this matrix can be used as input to warpPerspective to transform the pixels to the ground frame. This a blog post that gives a good explanation of the function input and output parameters. The code is provided in c++ but should be relatively straightforward to implement in python.

Onasafari
  • 448
  • 3
  • 7
0

Open CV's perspectivetransform() functions seem appropriate, Transform a frame to be as if it was taken from above using OpenCV This question I think answers a lot of your questions. Compensation for rotation and tilt can be encoded in the 3x3 matrix to transform your image correctly.

Kevin Kang
  • 26
  • 4