-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Summary
Nodes running in pruning mode (default) discard old blocks to bound disk growth. To support syncing new nodes from genesis without requiring every node to be a full archive node, we need an external block serving service.
Motivation
- Block archive storage now prunes by default (retention window of ~100k blocks)
- Chain-index (SQLite) serves RPC/explorer queries independently of the block archive
- Only seed/archive nodes need full block history
- New nodes joining the network need a way to sync from genesis without requiring every peer to store all blocks
Design
An external block provider service that:
- Stores all blocks from genesis (potentially using commonware's
immutable::Archivewith freezer for minimal memory + compression) - Serves block data over a well-defined API (HTTP/gRPC) to syncing nodes
- Can be operated independently of the P2P network -- e.g., a centralized service, cloud storage, or a dedicated archive node
Sync flow
New Node Block Provider
| |
|-- request blocks [0, N) --->|
|<--- stream ArchivedBlock ---|
| |
|-- apply blocks to STF --- |
|-- build chain-index --- |
| |
|-- join P2P, continue from |
| latest block |
Considerations
- The provider does not need to be trusted for correctness -- the node validates state roots against the chain's state transition function
- Provider could be backed by object storage (S3/R2) serving pre-compressed block segments
- Protocol should support range requests and resumable sync
- Consider using commonware's
immutable::Archive(freezer) as the backing store for the provider -- it's designed for exactly this workload (write-once, read-many, minimal memory, compressed)
Acceptance Criteria
- Define block provider API (gRPC service definition or HTTP endpoints)
- Implement block provider server backed by full block archive
- Implement client-side sync that fetches blocks from provider and applies them
- Support resumable sync (track last synced block)
- Node config option to specify block provider URL for initial sync
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels