Valid Data Formats

MCAP Files

MCAP Files

MCAP is a flexible serialization format that supports a wide range of options and message encodings. This includes the capability to encode ROS1, ROS2/CDR serialized, Protobuf, Flatbuffer, and more.

Header Time vs Log Time

By default, MetriCal uses the header timestamp in each message for synchronization purposes. If your current workflow uses the log time instead, use the --use-log-time flag when running metricalcalibrate.

Don't use this as a replacement for fixing your header timestamps, though! MetriCal always knows...

Message Encodings

MetriCal only supports a limited subset of the total well-known message encodings in MCAP.

Message Type	ROS1 Encodings	ROS2 with CDR Serialization	Protobuf Serialization Schemas
Image	sensor_msgs/Image	sensor_msgs/msgs/Image	RawImage.proto
CompressedImage	sensor_msgs/CompressedImage	sensor_msgs/msgs/CompressedImage	CompressedImage.proto
PointCloud	sensor_msgs/PointCloud2	sensor_msgs/msgs/PointCloud2	PointCloud.proto
IMU	sensor_msgs/Imu	sensor_msgs/msgs/Imu	--
Odometry	nav_msgs/Odometry	nav_msgs/msgs/Odometry	--
H264 Video	--	--	CompressedVideo.proto

Valid Image Encodings

Image type	Encoding
Ordered pixels	mono8, mono16, rgb8, bgr8, rgba8, bgra8, rgb16, bgr16, rgba16, bgra16
8-bit, multi-channel	8UC1, 8UC2, 8UC3, 8UC4, 8SC1, 8SC2, 8SC3, 8SC4
16-bit, multi-channel	16UC1, 16UC2, 16UC3, 16UC4, 16SC1, 16SC2, 16SC3, 16SC4
32-bit, multi-channel	32UC1, 32UC2, 32UC3, 32UC4, 32SC1, 32SC2, 32SC3, 32SC4
64-bit, multi-channel	64UC1, 64UC2, 64UC3, 64UC4, 64SC1, 64SC2, 64SC3, 64SC4
Bayer images	bayer_rggb8, bayer_bggr8, bayer_gbrg8, bayer_grbg8, bayer_rggb16, bayer_bggr16, bayer_gbrg16, bayer_grbg16
YUYV images	uyvy, UYVY, yuv422, yuyv, YUYV, yuv422_yuy2

ROS1 Bags

ROS1 Bags

ROS1 itself is no longer supported. However, that doesn't mean you can't use MetriCal! Instead, just convert your ROS1 bags to MCAP using the MCAP CLI tool:

mcap convert [your].bag [your].mcap

Then, use the resulting MCAP file with MetriCal as normal.

Folders

Folders

If your system does not use ROS or encode data as an MCAP file at all, you can still use MetriCal by providing it with recursive folders of structured data. This approach is a little easier to get started with compared to using MCAP files, but can leave a lot of performance (and data size, as MCAP files are often compressed-by-default) on the table.

Message Types

Type	Guidelines
Image	Must be in JPEG or PNG format
Point Cloud	Must be in PCD format. See more on valid PCD messages below

Folder format description

MetriCal assumes that the folder layout looks something like the following:

.
└── data/   <--- passed as $DATA argument in CLI
    ├── topic_1/
    ├── topic_2/
    └── topic_3/

Where each directory contains inputs whose file names correspond to timestamps (canonically, in nanoseconds). For example, if we had a message topic named camera_1, we might have the following example tree of files:

.
├── camera_1/
│   ├── 1643230518150000000.png
│   ├── 1643230523197000000.png
│   ├── 1643230526125000000.png
│   ├── 1643230529419000000.png
│   ├── 1643230532161000000.png
│   └── 1643230537869000000.png
...

The folder names should correspond to the default topic names (e.g. such as in a ROS bag). Every topic does not need to have the same number of messages, or even share exactly the same timestamps as other topics. However, these timestamps will be assumed to be synced according to what is provided in the plex.

Beware Epoch Ordering

Make sure that your files are actual timestamps in units of nanoseconds. MetriCal does support sequentially ordering images (e.g., 0.png, 1.png, etc.); however, this often leads to a reduction in information in how MetriCal derives data synchronization across topics. In particular, it means MetriCal will assume that each "epoch" in the ordering is synced perfectly.

This can result in poor or degraded performance compared to using real timestamp data, if you are not careful.

PointCloud2 messages and PCD Files

MetriCal has partial support for v0.7 of the pcd file format for reading point cloud data. Since pcd is a very flexible format, we impose the following restrictions on pointclouds we can process:

• Fields named "x", "y" and "z" are all present (referred to as Geometric data)
  • Geometric data fields are floating point (f32, float) values
• One of the fields "intensity", "i", or "reflectivity" is present (referred to as Intensity data)
  • Intensity data contains data of one of the following types
    • Unsigned 8 bit integer (u8, uint8_t)
    • Unsigned 16 bit integer (u16, uint16_t)
    • Unsigned 32 bit integer (u32, uint32_t)
    • 32 bit floating point (f32, float)
• Each of the Geometric data and Intensity data fields contains precisely 1 count

PCD Restrictions

In addition to LiDAR / point cloud data needing to include the fields described above, we make some additional restrictions for PCD files when reading in data in the folder format:

• The pcd data format is either ascii or binary
  • Explicitly, "binary_compressed" is not yet supported

As an example, a pcd with the header below would be supported.

# .PCD v.7 - Point Cloud Data file format
VERSION .7
FIELDS x y z intensity
SIZE 4 4 4 4
TYPE F F F F
COUNT 1 1 1 1
WIDTH 213
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS 213
DATA binary

Whereas, a pcd with the header below would not be supported

# .PCD v.7 - Point Cloud Data file format
VERSION .7
FIELDS x y z intensity
SIZE 4 4 4 4
TYPE F F F I
COUNT 3 3 3 1
WIDTH 213
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS 213
DATA binary_compressed