Spatial Constraints

Spatial Constraint Traversal - Python Example

Since all spatial constraints in a plex form an undirected (maybe even fully connected) graph, it can be confusing to figure out how to traverse that graph to find the "best" extrinsics between two components. MetriCal itself provides the Shape command to help with this (with a few helpful options in the Report command), but sometimes it's useful to do your own thing.

To help would-be derivers, the spatial_constraint_traversal repository demonstrates the right way to derive optimal extrinsics straight from the Plex JSON via the magic of python.

It is common to ask for the spatial relationship or extrinsics between two given components. A Plex incorporates this information in the form of what is called spatial constraints. A spatial constraint can be broken down into:

Field	Type	Description
Extrinsics	An extrinsics object	The extrinsics describing the "To" from "From" transformation.
Covariance	A matrix of floats	The 6×6 covariance of the extrinsics described by this constraint.
From	UUID	The UUID of the component that describes the "From" or base coordinate frame.
To	UUID	The UUID of the component that describes the "To" coordinate frame, which we are transforming into. This can be considered the "origin" of the extrinsics matrix

For a single-camera system, a plex is a simple affair. For a complicated multi-component system, plexes can become incredibly complex and difficult to parse. Unlike other calibration systems, MetriCal creates fully connected graphs whenever it can. Everything is related!

Spatial Covariance

Spatial covariance is generally presented as a 6×6 matrix relating the variance-covariance of an se3 lie group:

$$\begin{bmatrix} v_1 & v_2 & v_3 & \omega_1 & \omega_2 &\omega_3 \end{bmatrix}$$

When traversing for spatial constraints within the Plex, the constraint returned will always contain the extrinsic with the minimum overall covariance. This ensures that users will always get the extrinsic that has the smallest covariance (thus, the highest confidence / precision), even if multiple spatial constraints exist between any two components.

Coordinate Bases in MetriCal

It's common to represent a transform in a certain convention, such as FLU (X-Forward, Y-Left, Z-Up) or RDF (X-Right, Y-Down, Z-Forward). One might then wonder what the default coordinate system is for MetriCal. Short answer: it entirely depends on the data you're working with.

In MetriCal, spatial constraints are designed to transform observations (not components!) from the from frame to the to frame. In the case of a camera-LiDAR extrinsics transform $\Gamma_{L}^{C}$, the solved extrinsics will move LiDAR observations $p_{L}$ (which may be in FLU) to a camera observations's coordinate frame (which may be in RDF):

$$p_{C} = \Gamma_{L}^{C} \cdot p_{L}$$

This makes it simple to move observations from one component to another for sensor fusion tasks. This is what is meant when MetriCal is said to have no "default" component coordinate system; it operates directly on the provided observations!