[Humble]Timestamp syncing with Lidar and IMU

[li-dahua]

Hi,I would want to sync the time stamp of a stereo of blackfly s cameras with IMU and Lidar. But I found the time stamp is output by the camera sdk directly.
How can I set timestamp?
Thank you.

It seems these parameter do not allow me to change it.
'adjust_timestamp': True,
'chunk_mode_active': True,
'chunk_selector_timestamp': 'Timestamp',
'chunk_enable_timestamp': True,

System details

• operating system/version: Ubuntu 22.04
• ROS version (please prepend to issue title in square brackets, e.g. [noetic]): Humble
• camera model:Blacklfy s-U3-31S4C

[berndpfrommer]

The camera driver has no way to get a time stamp from the IMU or the lidar. But what you can do is trigger the cameras with a hardware pulse signal from the IMU (if you have an IMU that has a sync out pulse). So this way you know that the camera frames correspond to one of the IMU frames, but you still don't know which of the IMU frames triggered the camera frames, and there is no easy way to figure this out. Certain calibration softwares (for instance Kalibr) can figure out what the delay is by comparing camera and IMU motion, that's the only way I'm aware of. Alternatively, you can set the IMU to really low trigger frequencies such that you can manually see what the delay is between IMU and camera frames. Either way you do it, you still have to hope and pray that the delay remains constant across data sets.
I have not ever synchronized a LIDAR to other sensors, so I cannot help you with that task.
If you run the synchronized_camera_driver it will figure out which of the stereo camera frames belong together, and put the same header time stamps on them. To do that it has to guess which frames were generated by the same sync pulse. Between two cameras that is relatively easy because the frames are usually well separated in time. With an IMU at high frequency this is harder.

[li-dahua]

Is it possible to sync Blackfly S GigE and other devices using PTP? I have 2 more GigE cameras and a livox Lidar with built-in IMU that allows PTP synchronization.
https://github.com/Livox-SDK/Livox-SDK/wiki/livox-device-time-synchronization-manual

[berndpfrommer]

Ok, the blackflys gige can do it. I don't have that camera though, only USB3

[berndpfrommer]

I can't find any documentation about how PTP is supposed to be used with the Spinnaker SDK. I suspect ptp will simply affect chunk time.
I suggest you use Spinview to enable PTP somehow. Then trigger the cameras externally from the lidar or another pulse generator. Enable chunk mode and chunk time stamps. Once you have all of that working, hack the ros driver to print out the raw chunk time stamp, and compare between the two cameras. Are they almost identical? Related to epoch time? If yes then we can modify the driver to use the chunk time directly.

[li-dahua]

Hi, I manage to set-up PTP clock in spinview and sync the timestamp with IMU and lidar. But I unable to get video or image from ros node.

[berndpfrommer]

Does it publish if you run at slightly lower frame rates?
Can you double check that the topic is correct by doing ros2 node list and then ros2 node info <name_of_your_driver_node and then ros2 topic info <name_of_published_topic> while you have the ros2 topic echo connected to it?

[li-dahua]

It is not output any image from the single node.

But when I run sync node of 2 camera. It can be run. The problem is the some frames are keeping being drop in the process.

[berndpfrommer]

The output from the sync driver tells me that you have "incomplete" frames. That means your network is dropping packets, and the frames are incomplete, and thus not published by the driver.
Check your network setup, i.e. MTU = 9000 on all network interfaces involved, and that your camera is set to Jumbo packets as well (9000 bytes).

[berndpfrommer]

Added a new issue #203 to keep track of the need to implement PTP support.