Calibrate Mode

Purpose

• Calibrate a multi-modal system based on pre-recorded data.
• Visualize the calibration data capture and detection process.

Usage

metrical calibrate [OPTIONS] \
    <INPUT_DATA_PATH> \
    <PLEX_OR_RESULTS_PATH> \
    <OBJECT_SPACE_OR_RESULTS_PATH>

Concepts

The Calibrate command runs a full bundle adjustment over the input calibration data. It requires three main arguments:

• The input data
• An inital plex (usually derived from Init mode). This represents a naive guess at the state of your system.
• An object space file. This tells MetriCal what fiducials, targets, or other features to look for during calibration.

We mean it when we say this runs a full bundle adjustment. MetriCal will optimize both the plex values (the calibration of your system, effectively) and the object space values. This means that MetriCal is robust to bent boards, misplaced targets, etc. All of these values will be solved for during the adjustment, and your calibration results will be all the better for it.

We use ANSI terminal codes for colorizing output according to an internal assessment of metric quality. Generally speaking:

• Cyan: spectacular
• Green: good
• Orange: okay, but generally poor
• Red: bad

Note that these judgements are applied internally to the software and have limits that may change release-to-release. We advise that you determine your own internal limits for each independent metric.

Cached Detections

Extracting detections from a dataset usually takes the lion's share of time during a run. To make this less onerous on subsequent runs, Calibrate mode will automatically create a cache of detections (in JSON format) and save them to the same directory as the input dataset.

Preserving UUIDs for Cached Detections

Detections rely on matching UUIDs between its data and the components in the plex to correctly attribute observations. This means that generating a new Init plex from scratch will require new detections, since the UUIDs will have changed. MetriCal does its best to preserve UUIDs between Init runs by using any previous Init plex as a seed. This allows one to play with different models without having to re-run the detection stage of the calibration.

Motion Filtering

Calibrate and Evaluate modes both run a motion filter over any and all images and point clouds in the input dataset. This filter removes any features that might be affected by motion blur, rolling shutter, false detections, or other artifacts. This is a critical step in the calibration process, as it ensures that the data used for calibration is the best we can get.

There are three states that the motion filter can assign to a component:

• In Motion: The movement between subsequent detections is above the motion threshold for this modality.
• Still: The movement between subsequent detections is below the motion threshold for this modality.
• Still (Redundant): The component is Still and the movement between detections is small enough that they can be considered the same detection.

The motion filter works at the sync group level, not component-by-component. You can see how this decision affects the filter's behavior over time. If one component is moving, the motion filter labels all other components as moving as well, regardless of their actual detected motion status. This decision makes intuitive sense: in a calibration dataset, all components (or all object spaces) should be moving together.

If you have a dataset that is a series of snapshots, rather than continuous motion, you can disable the motion filter using the --disable-motion-filter flag.

Image Motion Filtering

The image motion filter is based on the average flow of detected features between subsequent frames. Background motion is not considered, so you can still use this filter when taking data in a busy place.

• Any motion that results in an average pixel shift higher than the camera motion threshold (--camera-motion-threshold) is considered In Motion. The default value for the camera motion threshold is 1 pixel/frame, but this can be adjusted using the flag.
• Any motion that results in an average pixel shift lower than the camera motion threshold is considered Still.
• If two frames are both still and the second frame's features have shifted less than 1 pixel on average, the second frame is considered Still (Redundant). These redundant frames add little information to the calibration process and are removed.

Lidar Motion Filtering

The lidar motion filter is based on the movement of the detected center point of a circular markerboard between subsequent observations.

• Any motion that results in the center point moving more than the lidar motion threshold is considered In Motion. The default value for lidar is 0.1m (a pretty generous threshold!).
• Any motion that results in the center point moving less than the lidar motion threshold is considered Still.
• There is no Still (Redundant) state for lidar data.

Extracting Optimized Values from a Results JSON

There is a lot of data in a results JSON output from MetriCal. Luckily, it's all JSON, and it's fairly easy to pick apart. Use jq to extract this data into its own file for easy analysis and comparison to the original inputs.

Install jq using apt:

sudo apt install jq

Then use jq to extract the relevant optimized data.

jq .plex results.json > optimized_plex.json
jq .object_space results.json > optimized_obj.json

Examples

Run a calibration

metrical calibrate             \
    --output-json results.json \
    $DATA $PLEX $OBJSPC

Visualize detections

Make sure you have a Rerun server running!

metrical calibrate             \
    --render                   \
    --output-json results.json \
    $DATA $PLEX $OBJSPC

Remap topics to different components in a plex

With the --topic-to-component flag, you can map topics to component names or UUIDs in the input plex. This prevents laborious hand-tuning of a plex JSON whenever a dataset is recorded with a different topic name.

For example, if we called our folder camera_1 and that corresponded to some component named ir_SN5XXX in the plex, then we could do the following:

metrical calibrate                           \
    --topic-to-component camera_1:ir_SN5XXX  \ # our corrected topic-component relation
    --output-json results.json               \ # the output file
    $DATA $PLEX $OBJSPC

Save calibration log output

Log output from the program is output on stderr. If you want to record this for later, you can actually save a log by using the --report-path flag. This will save a .log file and, if possible, a HTML version of the log as well.

metrical calibrate --report-path metrical.log $DATA $PLEX $OBJSPC

Disable the motion filter for motion-heavy datasets

By default, MetriCal will run a motion filter over the input dataset and remove observations with too much motion, i.e. where it appears that relative motion was too large. For observation streams with inconsistent or infrequent observations, this can result in filtering everything! It can also be a sign of malformed timestamps.

Filtering is recommended when running a dataset with generally quick observation frequencies (>15Hz) and consistent timestamps. Otherwise, you should disable the filter with the --disable-motion-filter flag.

metrical calibrate --disable-motion-filter $DATA $PLEX $OBJSPC

Arguments

[INPUT_DATA_OR_DETECTIONS_PATH]

The dataset with which to calibrate, or the detections from an earlier run. Users can pass:

1. ROS1 bags, in the form of a .bag file.
2. MCAP files, with an .mcap extension.
3. A top-level directory containing a set of nested directories for each topic.
4. A detections JSON with all the cached detections from a previous run.

In all cases, the topic/folder name must match a named component in the plex in order to be matched correctly. If this is not the case, there's no need to edit the plex directly; instead, one may use the --topic_to_component flag.

[PLEX_OR_RESULTS_PATH]

The path to the input plex. This can be a MetriCal results JSON or a plex JSON.

[OBJECT_SPACE_OR_RESULTS_PATH]

A path pointing to a description of the object space for the adjustment. This can be a MetriCal results JSON or an object space JSON.

Options

Universal Options

As with every mode, all universal options are supported (though not all may be used).

-y, --overwrite-detections

Overwrite the detections at this location, if they exist.

--disable-motion-filter

Disable data filtering based on motion from any data source. This is useful for datasets that are a series of snapshots, rather than continuous motion.

--camera-motion-threshold [CAMERA_MOTION_THRESHOLD]

Default: 1.0 (pixels/observation)

This threshold is used for filtering camera data based on detected feature motion in the image. An image is considered "still" if the average delta in features between subsequent frames is below this threshold. The units for this threshold are in pixels/frame.

--lidar-motion-threshold [LIDAR_MOTION_THRESHOLD]

Default: 0.1 (meters/observation)

This threshold is used for filtering lidar data based on detected feature motion in the point cloud's detected circle center. A point cloud is considered "still" if the average delta in metric space between subsequent detected circle centers is below this threshold. The units for this threshold are in meters/observation.

-o, --results-path [RESULTS_PATH]

Default: path/to/dataset/[name_of_dataset].results.json

The output path to save the final results of the program, in JSON format.

--optimization-profile [OPTIMIZATION_PROFILE]

Default: standard

The optimization profile of the bundle adjustment. This is a high-level setting that controls the optimization's maximum iterations, relative cost threshold, and absolute cost threshold.

Possible values:

• performance: Use this argument when speed is necessary
• standard: This is a balanced profile between speed and accuracy
• minimize-error: This profile maximizes accuracy. For some small datasets, this profile may be just as fast as the standard profile. However, for larger datasets, this profile may take significantly longer to converge

--opt-max-iterations [OPT_MAX_ITERATIONS]

The maximum iteration count on the bundle adjustment. This is a hard limit on the number of iterations for the bundle adjustment. If the optimization does not converge within this number of iterations, the optimization will stop and return the current state of the optimization.

--opt-relative-threshold [OPT_RELATIVE_THRESHOLD]

The relative cost termination threshold for the bundle adjustment. "Relative cost" is the difference in cost between iterations. If the relative cost between iterations is below the threshold, the calibration will exit

--opt-absolute-threshold [OPT_ABSOLUTE_THRESHOLD]

The absolute cost termination threshold for the bundle adjustment. "Absolute cost" is the value of the cost at a given iteration of the optimization. In practice, this value is hardly ever reached; the optimization will hit a local minima well above this value and exit

-m, --topic-to-component [TOPIC_TO_COMPONENT]

A mapping of ROS topic/folder names to component names/UUIDs in the input plex.

MetriCal only parses data that has a topic-component mapping. Ideally, topics and components share the same name. However, if this is not the case, use this flag to map topic names from the dataset to component names in the plex.

--preserve-input-constraints

Preserve the spatial constraints that were input for this dataset.

The default behavior is to discard any constraints that were not derived from the data (e.g. spatial constraints from sync groups) or optimized. Pass this flag if the output plex should include any constraints that were input but not found or optimized.

--disable-ore-inference

Disable inference of object relative extrinsic constraints in the bundle adjustment.

The default behavior is to infer object relative extrinsic constraints in the bundle adjustment. Pass this flag if object relative extrinsic constraints should not be inferred; this may save a substantial amount of time in datasets using many object spaces.

-r, --render

Whether to visualize the current mode using Rerun. Run

rerun --memory-limit=1GB

...in another process to start a visualization server in Rerun. Read more about configuring Rerun here.

--render-socket [RENDER_SOCKET]

The web socket address on which Rerun is listening. This should be an IP address and port number separated by a colon, e.g. --render-socket="127.0.0.1:3030". By default, Rerun will listen on socket host.docker.internal:9876. If running locally (not via Docker), Rerun's default port is 127.0.0.1:9876

When running Rerun from its CLI, the IP would correspond to its --bind option and the port would correspond to its --port option.

Deprecated Options

These options are now deprecated and will be removed in the next major version of MetriCal.

-i, --interactive

This flag is now a no-op.

-d, --disable-filter

Replaced with --disable-motion-filter.

-t, --stillness-threshold [STILLNESS_THRESHOLD]

Replaced with --camera-motion-threshold.

-x, --max-iterations [MAX_ITERATIONS]

Replaced with --opt-max-iterations.

-T (short argument)

Short arg -T replaced by -m.