Cartographer crashes /odom messages out of order #597
+114
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added a flat_world_odom_node. Cartographer crashed not only when an /imu message arrives out of order, but also when an /odom message is out of
order. I add some diagnose files here so you can verify the issue. I will also add an python script to debug the captured data.
Python Script The reported message is in german.
Odom Diagnose
Imu Diagnose
A little background:
I'm using cartographer with a kinect camera on the turtlebot3 burger
& ros melodic. I'm a beginner with ros. Using cartographer in 2D mode
worked pretty fine. When i switched to 3D mode, the cartographer crashed
a lot with the following error:
[FATAL] [1591025245.229967139]: F0601 17:27:25.000000 20863
pose_extrapolator.cc:229] Check failed: time >= imu_tracker->time()
(637266220451080319 vs. 637266220451179130)
I found out, that the flat_world_imu_node orders the message by timestamp
by ignoring messages with a lower timestamp. I wrote a script checking all
the topics for this kind of behaviour and found out, that odom is not always
in the right order. When you take a look at the two files in the next commit
called odom_diag.txt and imu_diag.txt you will find an error on Seq 260784 in
odom_diag.txt. The error is in the nanoseconds timestamp. Compare Seq 260784 and Seq 260783. The messages are transmitted via 802.11ac. I captured the messages using rostopic echo.
I don't know if it helps anyone with the same problem but i hope so. Maybe
rename the node before implementing this it is a quick and dirty fix for my bachelor thesis. If you are interested i can rename the node.
Greetings from germany
Happy Pride / Black Lives Matter