Camera Models

Below are all supported camera intrinsics models in MetriCal. If there is a model that you use that is not listed here, just contact us! We're always looking to expand our support.

Common Variables and Definitions

Variables	Description
$x_{c}$, $y_{c}$	Pixel coordinates in the image plane, with origin at the principal point
$X$, $Y$, $Z$	Feature coordinates in the world, in 3D Euclidean Space
$\hat X$, $\hat Y$, $\hat Z$	Corrected camera ray, in homogeneous coordinates centered on the camera origin
$\hat X_{dist}$, $\hat Y_{dist}$, $\hat Z_{dist}$	Distorted camera ray, in homogeneous coordinates centered on the camera origin

Modeling (3D → 2D) refers to projecting a 3D point in the world to a 2D point in the image, given the intrinsics provided. In other words, it models the effect of distortion on a 3D point. This is also known as "forward projection".

Correcting (2D → 3D) refers to the process of finding the camera ray that is created when intrinsics are applied to a 2D point in the image. When "undistorting" a pixel, this can be thought of as finding the corrected ray's point of intersection with the image plane. In other words, it corrects for the effect of distortion. This is also known as "inverse projection".

Unified refers to a model that can be used to both model and correct for distortion.

No Distortion

MetriCal keyword: no_distortion

This model is a classic pinhole projection with no distortion or affine effects. This model is most applicable when you're already correcting your images with a rectification process, or when you're using a camera with a very low distortion profile.

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)

De facto, nearly any camera that is already corrected for distortion uses this model. All models on this page are either pinhole models by design, or degrade to a pinhole model when no distortion is present.

$$\begin{align} x_{c} &= f \frac{X}{Z} + c_{x} \\ y_{c} &= f \frac{Y}{Z} + c_{y} \end{align}$$

OpenCV RadTan

Original Reference

OpenCV.org. Camera Calibration and 3D Reconstruction documentation. OpenCV 4.10-dev. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga7dfb72c9cf9780a347fbe3d1c47e5d5a

MetriCal keyword: opencv_radtan

Type: Modeling

This is based on OpenCV's default distortion model, which is a modified Brown-Conrady model. If you've ever used OpenCV, you've most certainly used this.

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$k_{1}$	first radial distortion term
$k_{2}$	second radial distortion term
$k_{3}$	third radial distortion term
$p_{1}$	first tangential distortion term
$p_{2}$	second tangential distortion term

Common Use Cases

OpenCV's adoption of this model has made it the de facto starting point for most calibration tasks. However, this does not mean it's multi-purpose. OpenCV RadTan is best suited for cameras with a field of view of 90° or less.

A good number of sensor packages use this model, including:

Model	Cameras With Model
Intel RealSense D435	All cameras
Intel RealSense D435i	All cameras
Intel RealSense D455	All cameras

Modeling

$$\begin{align} X' &= X / Z \\ Y' &= Y / Z \\ r^2 &= X'^2 + Y'^2 \\ r' &= (1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \\ X_{dist} &= r'X' + 2p_1X'Y' + p_2(r^2 + 2X'^2) \\ Y_{dist} &= r'Y' + 2p_2X'Y' + p_1(r^2 + 2Y'^2) \\ x_{c} &= f X_{dist} + c_{x} \\ y_{c} &= f Y_{dist} + c_{y} \\ \end{align}$$

Correcting

Correcting for OpenCV RadTan is a non-linear process. The most common method is to run a non-linear optimization to find the corrected point. This is the method used in MetriCal.

OpenCV Fisheye

Original Reference

OpenCV.org. Fisheye camera model documentation. OpenCV 4.10-dev. https://docs.opencv.org/3.4/db/d58/group__calib3d__fisheye.html

MetriCal keyword: opencv_fisheye

Type: Modeling

This model is based on OpenCV's Fisheye lens model, which is a modified Kannala-Brandt model. It has no tangential distortion terms, but is robust to wide-angle lens distortion.

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$k_{1}$	first radial distortion term
$k_{2}$	second radial distortion term
$k_{3}$	third radial distortion term
$k_{4}$	fourth radial distortion term

Common Use Cases

Any camera with a fisheye lens below 140° diagonal field of view will probably benefit from this model.

Modeling

$$\begin{align} X' &= X / Z \\ Y' &= Y / Z \\ r^2 &= X'^2 + Y'^2 \\ \theta &= atan(r) \\ \theta_d &= \theta (1 + k_1 \theta^2 + k_2 \theta^4 + k_3 \theta^6 + k_4 \theta^8) \\ X_{dist} &= (\frac{\theta_d}{r}) X' \\ Y_{dist} &= (\frac{\theta_d}{r}) Y' \\ x_{c} &= f X_{dist} + c_{x} \\ y_{c} &= f Y_{dist} + c_{y} \\ \end{align}$$

Correcting

Correcting for OpenCV Fisheye is a non-linear process. The most common method is to run a non-linear optimization to find the corrected point. This is the method used in MetriCal.

OpenCV Rational

Original Reference

OpenCV.org. Camera Calibration and 3D Reconstruction documentation. OpenCV 4.10-dev. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga7dfb72c9cf9780a347fbe3d1c47e5d5a

MetriCal keyword: opencv_rational

Type: Modeling

OpenCV Rational is the full distortion model used by OpenCV. It is an extension of the RadTan model, in which the radial distortion is modeled as a rational function.

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$k_{1}$	first radial distortion term
$k_{2}$	second radial distortion term
$k_{3}$	third radial distortion term
$p_{1}$	first tangential distortion term
$p_{2}$	second tangential distortion term
$k_{4}$	fourth radial distortion term
$k_{5}$	fifth radial distortion term
$k_{6}$	sixth radial distortion term

Common Use Cases

This model is mostly equivalent to OpenCV RadTan. Given the number of parameters, this model can overfit on the available data, making it tricky to generalize. However, if you know your camera benefits from this specification, it does offer additional flexibility.

As this is a modified version of OpenCV RadTan, we recommend its use for lenses with a field of view of 90° or less.

Modeling

$$\begin{align} X' &= X / Z \\ Y' &= Y / Z \\ r^2 &= X'^2 + Y'^2 \\ r' &= \frac{(1 + k_1 r^2 + k_2 r^4 + k_3 r^6)}{(1 + k_4 r^2 + k_5 r^4 + k_6 r^6)} \\ X_{dist} &= r' X' + 2p_1X'Y' + p_2(r^2 + 2X'^2) \\ Y_{dist} &= r' Y' + 2p_2X'Y' + p_1(r^2 + 2Y'^2) \\ x_{c} &= f X_{dist} + c_{x} \\ y_{c} &= f Y_{dist} + c_{y} \\ \end{align}$$

Correcting

Correcting for OpenCV Rational is a non-linear process. The most common method is to run a non-linear optimization to find the corrected point. This is the method used in MetriCal.

Pinhole with (Inverse) Brown-Conrady

Original Publication

A. E. Conrady, Decentred Lens-Systems, Monthly Notices of the Royal Astronomical Society, Volume 79, Issue 5, March 1919, Pages 384–390, https://doi.org/10.1093/mnras/79.5.384

MetriCal keyword: pinhole_with_brown_conrady

Type: Correcting

This model is the first of our inverse models. These models correct for distortion terms in the image space, rather than modeling the effects of distortion in the world space. If you're just looking to correct for distortion, this could be the model for you!

Notice that the model parameters are identical to those found in OpenCV RadTan. This is because both models are based off of the Brown-Conrady approach to camera modeling. Don't mix up one for the other!

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$k_{1}$	first radial distortion term
$k_{2}$	second radial distortion term
$k_{3}$	third radial distortion term
$p_{1}$	first tangential distortion term
$p_{2}$	second tangential distortion term

Common Use Cases

If you're just correcting for distortion, rather than modeling it, this model is a good choice.

Modeling

Modeling distortion for Inverse Brown-Conrady is a non-linear process. The most common method is to run a non-linear optimization to find the distorted point. This is the method used in MetriCal.

Correcting

$$\begin{align} r^2 &= x_{c}^2 + y_{c}^2 \\ r' &= (k_1 r^2 + k_2 r^4 + k_3 r^6) \\ x_{corr} &= r'x_{c} + p_1(r^2 + 2x_{c}^2) \cdot 2p_2x_{c}y_{c} \\ y_{corr} &= r'y_{c} + p_2(r^2 + 2y_{c}^2) + 2p_1x_{c}y_{c} \\ \hat X &= x_{c} - x_{corr} \\ \hat Y &= y_{c} - y_{corr} \\ \hat Z &= f \end{align}$$

Pinhole with (Inverse) Kannala-Brandt

Original Publication

J. Kannala and S. S. Brandt, "A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1335-1340, Aug. 2006, doi: 10.1109/TPAMI.2006.153.  https://oulu3dvision.github.io/calibgeneric/Kannala_Brandt_calibration.pdf

MetriCal keyword: pinhole_with_kannala_brandt

Type: Correcting

Inverse Kannala-Brandt follows the same paradigm as our other Inverse models: it corrects distortion in image space, rather than modeling it in world space.

Inverse Kannala-Brandt is close to the original Kannala-Brandt model, and therefore shares the same set of distortion parameters as OpenCV Fisheye. Don't get them mixed up!

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$k_{1}$	first radial distortion term
$k_{2}$	second radial distortion term
$k_{3}$	third radial distortion term
$k_{4}$	fourth radial distortion term

Common Use Cases

If you're just correcting for distortion, rather than modeling it, this model is a good choice for lenses with a field of view of 140° or less.

Modeling

Modeling distortion for Inverse Kannala-Brandt is a non-linear process. The most common method is to run a non-linear optimization to find the distorted point. This is the method used in MetriCal.

Correction

$$\begin{align} r^2 &= x_{c}^2 + y_{c}^2 \\ \theta &= atan(\frac{r}{f}) \\ r' &= \theta (1 + k_1 \theta^2 + k_2 \theta^4 + k_3 \theta^6 + k_4 \theta^8) \\ x_{corr} &= r' (\frac{x_{c}}{r}) \\ y_{corr} &= r' (\frac{y_{c}}{r}) \\ \hat X &= x_{c} - x_{corr} \\ \hat Y &= y_{c} - y_{corr} \\ \hat Z &= f \end{align}$$

EUCM

Original Publication

Khomutenko, B., Garcia, G., & Martinet, P. (2016). An Enhanced Unified Camera Model. IEEE Robotics and Automation Letters, 1(1), 137–144. doi:10.1109/lra.2015.2502921. https://hal.science/hal-01722264/document

MetriCal keyword: eucm

Type: Unified

EUCM stands for Enhanced Unified Camera Model (aka Extended Unified Camera Model), and is a riff on the Unified Camera Model. This model is "unified" because it offers direct calculation of both modeling and correcting for distortion. At lower distortion levels, this model naturally degrades into a pinhole model.

This model is also unique in that it primarily operates on camera rays, rather than requiring a certain focal length or pixel distance to mathematically operate. The Modeling and Correction operations below will convert camera rays to — and from — a distorted state. Users who wish to operate in the image plane should convert these homogeneous coordinates to a $\hat Z$ matching the focal length.

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$\alpha$	The distortion parameter that relates ellipsoid and pinhole projection
$\beta$	The distortion parameter that controls the ellipsoid shape

Common Use Cases

EUCM is (at time of writing) becoming a popular choice for cameras with strong radial distortion. Its ability to model distortion in a way that is both accurate and efficient makes it a good choice for many applications. It is capable of handling distortions for lenses with a field of view greater than 180°.

Modeling

$$\begin{align} d &= \sqrt{(\beta (\hat X^2 + \hat Y^2) + \hat Z^2)} \\ \gamma &= 1 - \alpha \\ s &= \frac{1}{\alpha d + \gamma\hat Z} \\ \hat X_{dist} &= \hat X s \\ \hat Y_{dist} &= \hat Y s \\ \hat Z_{dist} &= 1 \\ x_{c} &= f \hat X_{dist} + c_{x} \\ y_{c} &= f \hat Y_{dist} + c_{y} \\ \end{align}$$

Correction

$$\begin{align} (m_x, m_y) &= (\hat X_{dist} / \hat Z_{dist}, \hat Y_{dist} / \hat Z_{dist}) \\ r^2 &= m_x^2 + m_y^2 \\ \gamma &= 1 - \alpha \\ m_z &= \frac{1 - \beta \alpha^2 r^2}{\alpha \sqrt{1 - (\alpha \gamma)\beta r^2} + \gamma} \\ s &= \frac{1}{\sqrt{r^2 + m_z^2}} \\ \hat X &= m_x s \\ \hat Y &= m_y s \\ \hat Z &= m_z s \end{align}$$

Double Sphere

Original Publication

Usenko, V., Demmel, N., & Cremers, D. (2018). The Double Sphere Camera Model. 2018 International Conference on 3D Vision (3DV). doi:10.1109/3dv.2018.00069  https://arxiv.org/abs/1807.08957

MetriCal keyword: double_sphere

Type: Unified

Double Sphere is the newest model on this page. Like EUCM, it also offers direct computation of both modeling and correction on camera rays. However, it uses two spheres to model the effects of even strong radial distortion. At lower distortion levels, this model naturally degrades into a pinhole model.

Parameter	Description
$f$	focal length (px)
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$\xi$	The distortion parameter corresponding to distance between spheres
$\alpha$	The distortion parameter that relates second sphere and pinhole projection

Common Use Cases

Its use of projection through its "double spheres" makes it an ideal model for ultra-wide field of view lenses. The lenses tested in its original publication had fields of view ranging from 122° to 195°!

Modeling

$$\begin{align} d_{1} &= \sqrt{(\hat X^2 + \hat Y^2 + \hat Z^2)} \\ Z_{mod} &= \xi d_{1} + \hat Z \\ d_2 &= \sqrt{(\hat X^2 + \hat Y^2 + Z_{mod}^2)} \\ s &= \frac{1}{\alpha d_2 + (1 - \alpha) Z_{mod}} \\ \hat X_{dist} &= \hat X s \\ \hat Y_{dist} &= \hat Y s \\ \hat Z_{dist} &= 1 \\ x_{c} &= f \hat X_{dist} + c_{x} \\ y_{c} &= f \hat Y_{dist} + c_{y} \\ \end{align}$$

Correcting

$$\begin{align} (m_x, m_y) &= (\hat X_{dist} / \hat Z_{dist}, \hat Y_{dist} / \hat Z_{dist}) \\ r^2 &= m_x^2 + m_y^2 \\ m_z &= \frac{(1 - \alpha^2 r^2)}{\alpha \sqrt{1 - (2\alpha - 1) r^2} + (1 - \alpha)} \\ s &= \frac{m_z \xi + \sqrt{m_z^2 + (1-\xi^2) r^2}}{m_z^2 + r^2} \\ \hat X &= m_x s \\ \hat Y &= m_y s \\ \hat Z &= m_z s - \xi \end{align}$$

Omnidirectional (Omni)

Original Publication

Mei, C. and Rives, P. Single View Point Omnidirectional Camera Calibration from Planar Grids. Proceedings 2007 IEEE International Conference on Robotics and Automation, Rome, Italy, 2007, pp. 3945-3950, doi: 10.1109/ROBOT.2007.364084. https://www.robots.ox.ac.uk/~cmei/articles/single_viewpoint_calib_mei_07.pdf

MetriCal keyword: omni

Type: Modeling

The Omnidirectional camera model is designed to handle the unique distortion profile of a catadioptric camera system, or one that combines mirrors and lenses together. These systems are often used for camera systems that seek to capture a full 360° field of view (or close to it).

In "A Unifying Theory for Central Panoramic Systems and Practical Implications" (Geyer, Daniilidis), the authors show that all mirror surfaces can be modeled with a projection from an imaginary unit sphere onto a plane perpendicular to the sphere center and the conic created by the mirror. The Omnidirectional camera model codifies this relationship mathematically.

Yes, it's a little confusing

We won't go into it here, but the papers linked above explain it well.

The Omnidirectional camera model also implements the same radial and tangential distortion terms as OpenCV RadTan. However, while OpenCV RadTan uses 3 radial distortion terms, this only uses 2. The reason for this? Everyone else did it (even the authors' original implementation), so now it's convention.

Parameter	Description
$f$	generalized focal length (px). This term is not a "true" focal length, but rather the camera focal length scaled by a collinear factor $\eta$ that represents the effect of the mirror
$c_{x}$	principal point in x (px)
$c_{y}$	principal point in y (px)
$\xi$	the distance between the unit sphere and the "projection" sphere upon which the focal plane is projected
$k_{1}$	first radial distortion term
$k_{2}$	second radial distortion term
$p_{1}$	first tangential distortion term
$p_{2}$	second tangential distortion term

Common Use Cases

The omnidirectional camera model is best used for extreme fields of view. Anything greater than 140° would be well-served by this model.

Modeling

$$\begin{align} \mathbf{X} &= [X, Y, Z] \\ \mathbf{X}_S &= \frac{\mathbf{X}}{\lVert \mathbf{X} \lVert} \\ \mathbf{X}_C &= [X_S, Y_S, Z_S + \xi] \\ X' &= X_C / Z_C \\ Y' &= Y_C / Z_C \\ r^2 &= X'^2 + Y'^2 \\ r' &= 1 + k_1 r^2 + k_2 r^4 \\ X_{dist} &= r' X' + 2p_1X'Y' + p_2(r^2 + 2X'^2) \\ Y_{dist} &= r' Y' + 2p_2X'Y' + p_1(r^2 + 2Y'^2) \\ x_{c} &= f X_{dist} + c_{x} \\ y_{c} &= f Y_{dist} + c_{y} \\ \end{align}$$

Correcting

Though the Omnidirectional model technically has a unified inversion, the introduction of the radial and tangential distortion means that correcting is a non-linear process. The most common method is to run a non-linear optimization to find the corrected point. This is the method used in MetriCal.