MetriCal Results MCAP

Every calibration outputs a results MCAP file. By default, this file is named results.mcap. This file contains:

• Metadata about the software that generated it
• The input plex and object-space passed to the calibration
• The optimized plex representing the calibrated system
• The optimized object space (with any updated spatial constraints for a given object space)
• Metrics derived over the dataset that was calibrated. These are split into three categories:
  1. Pre-calibration metrics, computed from the data ingestion phase of the software
  2. Residual metrics for each cost relationship constructed as part of the calibration
  3. Summary statistics for the adjustment, computed from the residual metrics

The total sum of this data exists in the MCAP in different places. Generally speaking, an MCAP file stores data in one of three different kinds of records within the format:

1. Metadata
2. Channel Messages
3. Attachments

We break down how the above information is organized across these three "sections" of an MCAP below. See the MCAP specification for more information on the MCAP format.

Results MCAP Structure

Metadata

MetriCal writes the following metadata out to every results.mcap:

• Command: The program used to generate the MCAP (this should typically be "metrical")
• Version: The version of MetriCal used to perform calibration
• Arguments: The command line arguments used during calibration

These can be extracted from the MCAP as a JSON object using the mcap CLI tool:

mcap get metadata --name metrical results.mcap

Channel Messages

Channel messages in the results.mcap can be broken into pre-calibration, residual, and summary metrics. These messages are not written to the results MCAP on an unsuccessful calibration unless the --override-diagnostics flag is provided.

Pre-Calibration Metrics

Pre-Calibration metrics are generated after MetriCal has performed data ingestion, detection, and run the various quality and motion filters associated with the input dataset. These metrics are listed below:

Pre-Calibration Metric
Topic Filter Statistics
Binned Feature Counts
Circle Coverage

Residual Metrics

Residual metrics are generated for each and every cost or observation added to the calibration. The most immediately familiar residual metric might be reprojection error, but similar metrics can be derived for other modalities and observations as well. A full list of these is linked below:

Residual Metric	Produced by
Circle Edge Misalignment	All Camera-LiDAR pairs
Circle Misalignment	All Camera-LiDAR pairs
Composed Relative Extrinsics Error	All Components and Object pairs
Differenced Pose Trajectory Error	Local Navigation Systems
IMU Preintegration Error	All IMUs
Image Reprojection	All Cameras
Interior Points to Plane Error	All Camera-LiDAR pairs
Object Inertial Extrinsics Error	All IMUs
Paired 3D Point Error	All LiDAR-LiDAR pairs

Summary Statistics

Of all the metrics output in a results.mcap file, the Summary Statistics for a calibration run the most risk of being misinterpreted. Always bear in mind that these figures represent broad, global mathematical strokes, and should be interpreted holistically along with the rest of the metrics of a calibration. All of the summary metrics are generated downstream of the residual metrics; Consequently, if you have interest in generating different summary metrics one should first look at the residual metrics as they contain the raw data that these are generated downstream from.

Summary Statistic	Produced
Overall	Every calibration
Camera	Per Camera
Camera-LiDAR	Per Camera-LiDAR pair
LiDAR-LiDAR	Per LiDAR-LiDAR pair
IMU	Per IMU

These are the same summary statistics that are output in the console logs.

Message Schema Definitions

See our tangram-protobuf-messages repo for the exact message schemas we support for all of our channel message metrics.

Attachments

Input Plex

This is the plex that was used during calibration to initialize the priors of the system. This plex is not refined, notably in that it usually denotes the beginning state of your system prior to calibration.

The input plex can be extracted from the results.mcap by using the mcap CLI tool:

mcap get attachment --name input-plex results.mcap > input-plex.json

This can often be helpful if you want to:

1. Debug what the state of the system was when calibration was invoked; OR
2. Re-use the input plex from a particular run for a subsequent calibration

Optimized Plex

The optimized Plex is a description of the now-calibrated System. This Plex is typically more "complete" and information-rich than the input Plex, since it is based off of the real data used to calibrate the System.

The optimized plex can be extracted from the results.mcap by using the mcap CLI tool:

mcap get attachment --name optimized-plex results.mcap > optimized-plex.json

Input Object-Space

This is the object-space that was used during calibration to initialize the priors of the object-space definition. This object-space is not refined, notably that it defines the state of the objects / targets in the scene prior to optimization.

The input object-space can be extracted from the results.mcap by using the mcap CLI tool:

mcap get attachment --name input-object-space results.mcap > input-object-space.json

This can often be helpful if you want to:

1. Debug which object-space was used for a particular calibration
2. Better understand downstream effects on the optimized object-space or how error was pushed into the object-space during calibration

Optimized Object-Space

MetriCal will optimize over the object spaces used in every calibration. For example, If your object space consists of a checkerboard, MetriCal will directly estimate how flat (or not) the checkerboard actually is using the calibration data.

Additionally, the optimized object-space will contain optimized spatial constraints, which can be an easy check to validate whether the scene geometry you observed was correctly reconstructed.

The purpose of this is to be used with modes like consolidate-object-spaces or simply re-using your optimized object-space in a subsequent calibrate mode command. You can extract the object-space from the results.mcap by using the mcap CLI tool:

mcap get attachment --name optimized-object-space results.mcap > optimized-object.json

Plex and Object-Space Schemas

See the individual pages for plexes and object-spaces for more information on the schemata for these types.