Repository navigation
blf: interpret object timestamps as signed 64-bit values - #2101
Open
FlorianMickler wants to merge 3 commits into
Open
FlorianMickler wants to merge 3 commits into
FlorianMickler wants to merge 3 commits into
Conversation
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
Change BLF object header timestamp fields from uint64 (Q) to int64 (q) for both V1 and V2 object headers. Observed CANape-generated BLF files containing messages before the recording start timestamp. When interpreted as unsigned values, these timestamps wrap around and appear approximately 2^64 ns in the future, resulting in invalid timestamps (e.g. year 2611) and NaT values in pandas. Using signed 64-bit timestamps correctly preserves negative recording-relative offsets while still providing a usable range of approximately ±292 years at nanosecond resolution. Root cause: A timestamp of -60 s was decoded as 2^64 ns - 60 s which produced timestamps approximately 584 years in the future. The resulting values exceeded the valid pandas datetime range and appeared as NaT.
FlorianMickler
force-pushed
the
blf_fix_prerecord_timestamps
branch
from
September 11, 2026 07:03
dc7fe99 to
783b902
Compare
Add regression tests covering messages that occur before the recording start timestamp. The first test performs a BLFReader -> BLFWriter -> BLFReader roundtrip using the existing BLF files from the test suite. Besides verifying timestamp preservation, this provides a convenient framework for reproducing and validating real-world BLF files when investigating format compatibility issues. The second test verifies that messages with negative recording-relative timestamps survive a write/read cycle without modification. These tests reproduce the failure modes seen working with Vector CANape blf recordings which are fixed by the previous and following change.
BLFWriter currently clamps negative recording-relative timestamps to zero before writing them to the BLF stream. This breaks BLF files containing messages that occur before the recording start timestamp. A practical example is Vector CANape recordings with a configured pre-recording buffer. When extracting a single CAN channel from such a measurement, the pre-recording messages have negative offsets relative to the BLF header start timestamp. Clamping these offsets to zero loses information and prevents a lossless BLFReader -> BLFWriter roundtrip. Preserving the original header start timestamp is desirable, as it provides a stable recording reference and can be used to identify or group related BLF fragments originating from a split measurement. Write the signed timestamp value unchanged instead of clamping it to zero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change BLF object header timestamp fields from uint64 (Q) to int64 (q) for both V1 and V2 object headers.
Observed CANape-generated BLF files containing messages before the recording start timestamp. When interpreted as unsigned values, these timestamps wrap around and appear approximately 2^64 ns in the future, resulting in invalid timestamps (e.g. year 2611) and NaT values in pandas.
Using signed 64-bit timestamps correctly preserves negative recording-relative offsets while still providing a usable range of approximately ±292 years at nanosecond resolution.
Root cause:
A timestamp of -60 s was decoded as
2^64 ns - 60 s
which produced timestamps approximately 584 years in the future. The resulting values exceeded the valid pandas datetime range and appeared as NaT.
Summary of Changes
this fixes a bug observed with pre-recording buffers in some CANape blf recordings.
Related Issues / Pull Requests
no issue opened.
Type of Change
Checklist
tox). (blf related ones: )PASS [ 4.004ms] test/logformats_test.py::TestBlfFileFormat::test_append_mode
PASS [ 877.1us] test/logformats_test.py::TestBlfFileFormat::test_can_error_frame_ext
PASS [ 1.020ms] test/logformats_test.py::TestBlfFileFormat::test_can_fd_message
PASS [ 828.9us] test/logformats_test.py::TestBlfFileFormat::test_can_fd_message_64
PASS [ 838.3us] test/logformats_test.py::TestBlfFileFormat::test_can_message
PASS [ 721.1us] test/logformats_test.py::TestBlfFileFormat::test_can_message_2
PASS [ 2.659ms] test/logformats_test.py::TestBlfFileFormat::test_file_like_context_manager
PASS [ 2.611ms] test/logformats_test.py::TestBlfFileFormat::test_file_like_explicit_stop
PASS [ 651.6us] test/logformats_test.py::TestBlfFileFormat::test_issue_1905
PASS [ 2.605ms] test/logformats_test.py::TestBlfFileFormat::test_path_like_context_manager
PASS [ 2.172ms] test/logformats_test.py::TestBlfFileFormat::test_path_like_explicit_stop
PASS [ 355.9us] test/logformats_test.py::TestBlfFileFormat::test_timestamp_to_systemtime
Additional Notes