Rolling Shutter Calibration Guide

Rolling shutter is the bane of calibration. It compounds every data collection issue that can occur during camera calibration. MetriCal can negate many of the effects of rolling shutter, but first it needs to know that a camera is rolling shutter. This guide will show you how to make that distinction.

Example Dataset: Camera ↔ IMU

We're going to use our Camera ↔ IMU guide's dataset as a proof-of-concept. In that guide, we only specify calibration of the IMU and both IR cameras, which are global shutter. However, there's a third, color camera on that rig which presents rolling shutter artifacts. Look at that wobble!

Get the dataset here:

📁Download: Camera-IMU Example Dataset

Marking Rolling Shutter Cameras

Rolling shutter cameras are designated in two places, only one of which matters for your setup:

• A camera_pixel_readout field in the system specification.
• A pixel_readout field of a camera component in the plex. This is auto-populated from the system spec during plex learn. This helps carry over the shutter status of the camera during steady state production runs.

There are two valid settings for that field: "Global" (designating a global shutter camera) and "Rolling" (designating rolling shutter).

Here's an example system spec file that designates the /rs/color/image_raw camera as rolling shutter:

camera_imu_box_rolling_shutter_spec.json

{
    "camera_pixel_readout": {
        "/rs/color/image_raw": "Rolling"
    }
    // ...and then other fields as needed.
}

We then feed this into our manifest via the plex learn command:

camera_imu_box_rolling_shutter_manifest.toml

...

[stages.learn]
command = "plex-learn"
input-plex = "{{create.output-plex}}"
dataset = "{{variables.dir}}/camera_imu_box.mcap"
system-specification = "{{variables.dir}}/camera_imu_box_rolling_shutter_spec.json"
topic-to-model = [
    ["/rs/color/image_raw", "opencv-radtan"],
    ["*infra*", "opencv-radtan"],
    ["*imu*", "scale-shear"]
]
output-plex = "{{auto}}"
...

Nice, you did it. Run everything else as normal!

What Gets Solved

Note that MetriCal won't actually profile your rolling shutter effects in any way; it will only work to negate their effects from the rest of the calibration. This negation adds a noticeable time to any calibration, so be aware of that when setting this up. However, you will get markedly better results in both intrinsics and extrinsics.

Troubleshooting

If you encounter errors during calibration, please refer to our Errors and Troubleshooting documentation.

Remember that all measurements for your targets should be in meters, and you should ensure visibility of as much of the target as possible when collecting data.