If you are to design a real world application of a stereo vision algorithm, lets say for a UAV or a spacecraft which is computing elevation maps from the two images, is the fundamental matrix known a priori or will I have to calculate it alongside with the disparity map?
If the fundamental matrix can be obtained apriori, is it correct that knowledge of the calibration matrix and the projective matrices is sufficient to compute the matrix?
Regarding your first question:
In my experience, this depends on the mechanical design of your camera system, and on the use of a fixed focal length. If you are able to mount your cameras rigidly, and if your focal focal length does not change, then you can pre-calibrate the whole thing.
If the relative position of your cameras is likely to change (as they are mounted, for example, on a not perfectly rigid structure), or if you are zooming or using autofocus (!), then you must think about dynamic calibration (or about better fixing your cameras). The depth error induced by calibration error depends on the baseline of your stereo setup and the distance to your scene, so you can compute your tolerances.
Regarding your second question:
Yes, it is sufficient.
Your should be aware that there are many ways of computing an F-matrix. I highly recommend to look into Hearley & Zisserman, which is the de-facto reference for these topics.
