feat(cache): add HDF5 baseline inference cache layer (#567)#568
Open
AdityaX18 wants to merge 3 commits intoJdeRobot:masterfrom
Open
feat(cache): add HDF5 baseline inference cache layer (#567)#568AdityaX18 wants to merge 3 commits intoJdeRobot:masterfrom
AdityaX18 wants to merge 3 commits intoJdeRobot:masterfrom
Conversation
Add perceptionmetrics/utils/cache.py with CacheWriter, CacheReader, and is_cache_valid. Serialises preprocessed image tensors and detection predictions to HDF5 after one clean baseline eval run. Downstream perturbation conditions reuse the cache, eliminating N*P*I redundant forward passes on clean baseline. This is Layer 1 (disk cache write/read) only. Integration into torch_detection.py eval() is a follow-up PR. - CacheWriter: context manager, writes tensors + preds per image - CacheReader: validates model_hash + schema_version on open, lazy access - is_cache_valid: O(1) guard for eval loop short-circuit - Zero-detection images write empty (0,4)/(0,)/(0,) datasets Tests: round-trip, stale-hash, is_cache_valid, zero-det, metadata, image_ids. Adds h5py>=3.10,<4 dependency. Closes JdeRobot#567
Author
|
@dpascualhe, I've finished the HDF5 cache implementation (Layer 1) as we discussed in #567. Ready for review when you have a moment! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #567
Adds
perceptionmetrics/utils/cache.py— a standalone HDF5-backed cache layerfor baseline inference outputs, implementing Layer 1 of the two-layer cached
architecture discussed in #567.
Problem
The current eval loop re-runs model inference for every perturbation condition.
For N images × P perturbation types × I intensities, this produces N·P·I
forward passes on the clean baseline — all redundant after the first run.
Example: COCO val2017 (5,000 images), 5 perturbation types, 5 intensities →
125,000 forward passes. With this cache: 5,000 baseline passes once, reused
across all conditions.
What this PR adds
perceptionmetrics/utils/cache.py:CacheWriter— context manager; writes preprocessed image tensors(C, H, W)and detection predictions (bboxes, labels, scores) to HDF5 after the clean
baseline eval loop.
CacheReader— validatesmodel_hash+schema_versionon open; provideslazy per-image tensor and prediction access.
is_cache_valid(path, model_hash) → bool— O(1) guard for the eval loopshort-circuit.
compute_model_hash(model, file_path) → str— SHA-256 of checkpoint file,or numel-proxy fallback.
tests/test_cache.py: round-trip, stale-hash, is_cache_valid,zero-detection, metadata, and image_ids tests (6 passing).
pyproject.toml: addsh5py = ">=3.10,<4".HDF5 schema
cache.hdf5
├── metadata/ (model_name, coco_split, model_hash, timestamp, schema_version)
├── tensors/{img_id} float32 (C, H, W), chunks=(C,H,W)
└── preds/{img_id}/
├── bboxes float32 (N_det, 4)
├── labels int64 (N_det,)
└── scores float32 (N_det,)
Zero-detection images write empty
(0,4)/(0,)datasets — consumers needno branch on group existence.
Out of scope (follow-up PR)
Integration into
torch_detection.py'seval()— the read/short-circuitpath that uses
is_cache_validbefore inference.Testing
All 6 tests pass.