Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use binary diff to persist finalized states #7005

Open
wants to merge 57 commits into
base: feature/differential-archive
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
ca67174
Add binary diff codec
nazarhussain Aug 6, 2024
cf634b2
Add snapshot strategy
nazarhussain Aug 6, 2024
23185b9
Rename the utils file
nazarhussain Aug 6, 2024
8b8c1d8
Add diff strategy
nazarhussain Aug 6, 2024
7460803
Rename historical sate util file
nazarhussain Aug 6, 2024
2a11791
Add skipped slot historical state
nazarhussain Aug 6, 2024
cb804da
Add worker support
nazarhussain Aug 6, 2024
6a22962
Update the archiver
nazarhussain Aug 6, 2024
eacc51e
Fix the types
nazarhussain Aug 6, 2024
63fa909
Add vcdiff package
nazarhussain Aug 6, 2024
be596b2
Add unit tests for multiple diff
nazarhussain Aug 8, 2024
ed75730
Add diff state tests
nazarhussain Aug 8, 2024
09dbcd2
Add support to fetch last db state
nazarhussain Aug 8, 2024
d83bed9
Disable storing anchor state
nazarhussain Aug 8, 2024
2f033f2
Extract historical state metrics
nazarhussain Aug 8, 2024
ba93548
Update metrics usage
nazarhussain Aug 8, 2024
b32d942
Add diff and snapshot size metrics
nazarhussain Aug 8, 2024
30fd368
Add doc for historical state
nazarhussain Aug 8, 2024
33608dc
Rename skip strategy to block replay
nazarhussain Aug 12, 2024
07a9124
Add diff layer configurations
nazarhussain Aug 12, 2024
36b1dd9
Implement the state diff layers
nazarhussain Aug 12, 2024
4e70fcd
Use diff layers
nazarhussain Aug 13, 2024
d82c444
Remove the obsolete stateArchiveEpochFrequency option
nazarhussain Aug 13, 2024
8a9d879
Rename stateArchive to stateSnapshotArchive
nazarhussain Aug 13, 2024
1fb0af9
Split state snapshot and diff archives
nazarhussain Aug 13, 2024
e7c2c24
Use state diff repository
nazarhussain Aug 13, 2024
212772f
Use historical regen object
nazarhussain Aug 13, 2024
b51cd37
Fix lint errors
nazarhussain Aug 13, 2024
f183653
Refactored per suggested feedback
nazarhussain Aug 15, 2024
d3215b1
Add documentation details with more explanation
nazarhussain Aug 15, 2024
ebe5264
Add logs to historical state
nazarhussain Aug 15, 2024
7bb569a
Add perf tests and few analysis about the defaults
nazarhussain Aug 15, 2024
d88dfb2
Add unit tests for diffs
nazarhussain Aug 15, 2024
7345c61
Remove docs from this PR
nazarhussain Aug 24, 2024
16ee9f9
Change the defaul options for the diff layers
nazarhussain Aug 24, 2024
9212e88
Rename an attribute
nazarhussain Sep 12, 2024
765f8dc
Fix some code after rebase
nazarhussain Sep 12, 2024
e43be08
Add dashbaord for hisotical state
nazarhussain Sep 12, 2024
363b384
Add more logging
nazarhussain Sep 13, 2024
3979fcb
Add unit tests for last stored state
nazarhussain Sep 16, 2024
a085e7b
Change default value for diff layers
nazarhussain Sep 16, 2024
b78082b
Add more logging for debugging
nazarhussain Sep 16, 2024
d22df00
Fix code between merges
nazarhussain Oct 10, 2024
d3441b2
Use @chainsafe/xdelta3-node package
nazarhussain Oct 10, 2024
4566f49
Merge branch 'feature/differential-archive' into nh/state-diff-2
nazarhussain Feb 3, 2025
c2ac67c
Fix building
nazarhussain Feb 3, 2025
5dadd14
Remove unused type
nazarhussain Feb 3, 2025
ed7fa1f
Restructure the historical state module
nazarhussain Feb 4, 2025
0661cb4
Update types usage in the cli
nazarhussain Feb 4, 2025
c135c5d
Use all observers
nazarhussain Feb 4, 2025
e860770
Fix observer callback
nazarhussain Feb 4, 2025
38f2451
Add state diff docs
nazarhussain Aug 24, 2024
3239084
Apply suggestions from code review
nazarhussain Feb 4, 2025
8ec035b
Apply suggestions from code review
nazarhussain Feb 4, 2025
2772e01
Arrange utility functions
nazarhussain Feb 4, 2025
1f8886a
Rearrange tests
nazarhussain Feb 4, 2025
0b5f8e4
Update the differential operation logic
nazarhussain Feb 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -83,4 +83,7 @@ packages/cli/.git-data.json
.last_build_unixsec
dictionary.dic

temp/
temp/

# AI Agents
.aider.*
Loading
Loading