Skip to main content
Version: dev-latest

Recalibrating RealSense Sensors

Get the Code

The full code for this tutorial can be found in the MetriCal Sensor Calibration Utilities repository on GitLab.

Objectives​

  • Record proper data from all RealSense sensors at the same time
  • Calibrate several RealSense sensors at once using MetriCal
  • Flash calibrations to each sensor

One of the most popular applications of MetriCal is calibrating multiple Intel RealSense sensors at once. RealSense is one of the most popular 3D sensing technologies, and it's common to see multiple RealSense deployed on a system (even if they aren't all being used!).

As with any sensor system, the calibration on a RealSense can drift over time, whether due to wear and tear, installation changes, or environmental factors. MetriCal can help you keep your RealSense sensors calibrated and accurate with minimal effort.

This tutorial will focus on the D4xx series.

Recording Data​

MetriCal is meant for convenience, so we recommend recording all sensors at the same time, in the same dataset. There is no need for multiple recordings!

Every RealSense device streams rectified data unless set to a specific frame format and resolution. In other words, most data will have already applied the current calibration to the frame! We need to avoid this in our own setup. Here are the settings you'll want to apply for a calibration dataset:

Device ModelIR Stream FormatStream Resolution
D400, D410, D415Y161920x1080
All other modelsY161280x800
Device ModelColor Stream FormatStream Resolution
D415, D435, D435iYUY21920x1080
D455YUY21280x800
Easy Recording with RealSense-Rust

Tangram Vision is also the maintainer of the RealSense-Rust crate, which makes it easy to run and record RealSense data from the safety of Rust.

The record_bag example in the repository is already configured with the above settings. Use that to quickly record a calibration ROSbag for MetriCal.

Avoiding Motion Blur​

Whether or not a camera is global shutter (as many are on the RealSense line), motion blur can still affect image quality. This is the last thing you want when calibrating a camera, as it can lead to inaccurate results regardless of whether you're moving the sensor rig or the targets. To avoid motion blur, make sure to stand still every second or so during calibration. MetriCal will filter out frames in motion automatically when processing the data.

For more on this and other tips for calibrating cameras, see the camera data capture documentation for MetriCal.

Identifying the Sensors​

Make sure you've recorded the sensor data in a way that makes each stream identifiable. If you're recording in a ROSbag or MCAP format, it's enough to prefix each device topic with its serial or name, like sensor1/color/image_raw and sensor2/color/image_raw.

If recording to a folder of images, keep in mind that MetriCal will expect all component folders to be in a single directory. This means that if you have two RealSense sensors, you'll need to record the data from both to the same folder:

observations/
    |- sensor1-color/
        |- 0001.png
        |- 0002.png
        |- ...
    |- sensor1-ir-left/
        |- 0001.png
        |- 0002.png
        |- ...
    |- sensor1-ir-right/
        |- 0001.png
        |- 0002.png
        |- ...
    |- sensor2-color/
        |- 0001.png
        |- 0002.png
        |- ...
    ...

See the input data formats documentation for more.

The Right Target​

We at Tangram Vision always recommend using a ChArUco markerboard for camera calibration. In fact, you can use multiple boards, which would ensure good coverage and depth of field! See our documentation on using multiple fiducials for more.

Running MetriCal​

Once you have the data all ready, running MetriCal is as easy as running any other dataset:

metrical init -m *:opencv_radtan $DATA $INIT_PLEX
metrical calibrate -o realsense_cal.json $DATA $INIT_PLEX $OBJ
# Get our calibrated plex from the results using jq
jq .plex realsense_cal.json > realsense_cal_plex.json

Notice that we're setting all cameras (all topics: *) to use the opencv_radtan model. This is the model that Intel uses for all cameras on the RealSense line.

And that's about it! This whole process can take from 10 seconds to 3 minutes, depending on how much data you used.You should now have a set of calibrations for all components, which we'll then flash to the sensors in the next step.

First, though, if you're worried that the data you captured isn't the best for a calibration procedure...

Seeding Extrinsics (Optional)​

You can seed the calibration using the --preset_device option during the Init phase:

metrical init -m *:opencv_radtan \
    --preset-device RealSense435:["sensor1-ir-left","sensor1-ir-right","sensor1-color"] \
    ...
    $DATA $INIT_PLEX

This tells the calibration to use the extrinsics from the RealSense 435 as a starting point for the topics "sensor1-ir-left", "sensor1-ir-right", and "sensor1-color". You can pass multiple --preset-device options to seed multiple sensors. Note that order matters when using this option.

Flashing the Calibration to the Sensors​

Now here's the nifty part: We can script this entire process! The results of MetriCal can be flashed directly to the RealSense sensors using an OSS tool created by Tangram Vision called RealSense Cal Flasher. If you haven't, clone the repository and run through the "Setup" instructions.

Sensor Config​

Once you've set things up, it's time to create your sensor config. This is another JSON we use especially for the RealSense Cal Flasher. It maps topics from our plex to the correct RealSense sensor using the serial number. Here's an example:

{
  "sensor_serial_one": {
    "left": "name_of_the_left_component_in_plex",
    "right": "name_of_the_right_component_in_plex",
  },
  "sensor_serial_two": {
    ...
  }
  ...
}

Fill that bad boy in, then run the flasher:

./rs-cal-flasher realsense_cal_plex.json <path_to_camera_config>.json

All of your RealSense sensors should now be calibrated and ready to go!

Summary​

  1. Take a calibration dataset
  2. Run the following script:
metrical init -m *:opencv_radtan $DATA $INIT_PLEX
metrical calibrate -o realsense_cal.json $DATA $INIT_PLEX $OBJ
# Get our calibrated plex from the results using jq
jq .plex realsense_cal.json > realsense_cal_plex.json
./rs-cal-flasher realsense_cal_plex.json <path_to_sensor_config>.json

Boom. Done.