Skip to content

Per-block littidx flush + single shard (gated on #3645) #3660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Kbhat1 merged 1 commit into from
Jun 29, 2026
Merged

Per-block littidx flush + single shard (gated on #3645) #3660

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 13 additions & 7 deletions sei-db/ledger_db/receipt/litt_receipt_store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,12 @@ import (
// metadata (m:latest / m:earliest).
//
// Durability: a background flusher bounds litt durability lag to
// littFlushInterval without putting fsync on the commit path; Close flushes the
// remainder. Accepted tradeoff: a hard crash can lose up to littFlushInterval
// of the most recent receipt bodies (litt buffers in memory, no WAL) while the
// index still lists them — tolerable for auxiliary, non-consensus RPC data,
// since reads return not-found for a missing body. Retention: receipt values
// littFlushInterval (~one block) without putting fsync on the commit path;
// Close flushes the remainder. A hard crash can lose up to littFlushInterval of
// the most recent receipt bodies while the index still lists them — tolerable
// for auxiliary, non-consensus RPC data, since reads return not-found for a
// missing body. The tight interval is only affordable because litt flushes its
// keymap asynchronously off the control loop. Retention: receipt values
// expire via litt's per-table TTL (time based), tag keys are pruned by block
// range, and reads enforce the KeepRecent floor, so visible retention never
// exceeds KeepRecent regardless of GC timing.
Expand Down Expand Up @@ -80,7 +81,12 @@ const (
receiptBackendLittIdx = "littidx"

littReceiptTableName = "receipts"
littFlushInterval = 100 * time.Millisecond
// littFlushInterval is roughly one flush per block at Giga throughput (a
// block is ~7ms), bounding crash loss to about a single block. Flushing this
// often is only cheap because litt flushes its keymap asynchronously, off the
// control loop; without that, a per-block flush regresses write throughput
// badly (≈-48% observed before the async keymap landed).
littFlushInterval = 5 * time.Millisecond
Comment on lines +84 to +89

@cody-littley cody-littley Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we requiring one flush per block for crash recovery safety? If so, doing it via wall clock time seems a bit fragile.

// littTTLPerBlock converts the KeepRecent block count into litt's wall-clock
// TTL (KeepRecent * littTTLPerBlock). Set above Giga block times so the TTL
// over-retains; only a sustained block time above this would expire a body
Expand Down Expand Up @@ -123,7 +129,7 @@ func newLittReceiptStore(cfg dbconfig.ReceiptStoreConfig, storeKey sdk.StoreKey)
return nil, fmt.Errorf("failed to open littdb: %w", err)
}
tableConfig := litt.DefaultTableConfig(littReceiptTableName)
tableConfig.ShardingFactor = 16 // spread writes across shards (cores spare)
tableConfig.ShardingFactor = 1 // single shard: flushing one file is cheaper; sharding mainly helps across multiple disks
receipts, err := values.BuildTable(tableConfig)
if err != nil {
_ = values.Close()
Expand Down
Loading