# Covariance

Both components and constraints include ** covariance**, a measure of
uncertainty in a Plex. The inclusion of covariance as a core idea is one of the
biggest differentiators of the Tangram Vision Platform.

## The Role Of Covariance

Many calibration pipelines will have some notion of whether or not some parameter is "fixed" or
"variable." Fixed parameters are treated as being *perfectly observed*:

- Their values are known
- They have no error
- They are not optimized during a calibration

Variable parameters are the opposite by being *perfectly unobserved*:

- Their values are not known at all
- They may have any amount of error
- They are always optimized during a calibration

This creates a dichotomy where we either have zero (0%) information about some quantity, or we have perfect information (100%) about some quantity. This is simply not true; most of the time, there is a reasonable, but not extreme, amount of uncertainty in any given system.

As you might have guessed, ** covariance** provides a method of modeling this uncertainty.

## Describing Covariance

While we may not know the *exact* values of every parameter in our calibration, we can typically
make an educated guess. It is common practice to state what we know about a parameter like this:

** <parameter> is 1000.0 <units>, ± 0.010 <units>**.

This ± 0.010 tolerance gives us a way to initialize a parameter's ** variance-covariance**
(shortened to just

*covariance*here). Many of the Tangram Vision Platform's processes take this information into account as a statistical prior. Rather than "fixing" any of our quantities, we update these values through the optimization process. This guarantees that we will never converge to values with variance / precision that is worse than what is specified by our priors.

Warning

When using covariance, know that this value is standard deviation *squared*. In our example
above, if our standard deviation is ± 0.010 units, then our covariance is (±
0.010)^{2} units^{2}.

The Tangram Vision Platform incorporates the concept of covariance for all observable quantities in our calibration process. This does add some complexity to the system, but provides the benefit of statistical rigor in our calibration pipeline.

## Component-wise Concerns

### Cameras

#### Intrinsic Covariance

The covariance matrix for a camera component's intrinsics cannot be described generally by a single size matrix. This will depend on the model that one chooses for a given component. For example:

- The pinhole projection model (\(f\), \(c_x\), \(c_y\)) and Brown-Conrady distortion (\(k_1\), \(k_2\) \(k_3\), \(p_1\), \(p_2\)) would have an 8×8 covariance matrix.
- The pinhole projection model (\(f\), \(c_x\), \(c_y\)) and Kannala-Brandt distortion (\(k_1\), \(k_2\), \(k_3\), \(k_4\)) would have a 7×7 covariance matrix.