Skip to content

Commit 1f3a8ff

Browse files
committed
Incorporate Javier's review comments
1 parent 052d511 commit 1f3a8ff

File tree

1 file changed

+47
-37
lines changed

1 file changed

+47
-37
lines changed

docs/website/contents/for-developers/Whitepaper.md

+47-37
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,9 @@ The following table outlines the informal components of the Haskell reference im
3131
| `Mempool` | Maintain a list of valid transactions to include in new blocks |
3232
| `ChainSync` and `BlockFetch` clients | Gather candidate chains from peers |
3333
| `ChainSync` and `BlockFetch` servers | Provide peers with access to `ChainDB` (see Storage Layer) |
34-
| `ChaineSel` | Among all forks that can be fetched from peers, must identify the best chain. |
34+
| `ChainSel` | Among all forks that can be fetched from peers, must identify the best chain. |
3535
| N2N `TxSubmission`, N2C `TxSubmission` servers | Accept transactions from peers and store them into `Mempool` for further processing |
36-
| | |
37-
| | |
38-
| | |
36+
3937

4038
## Responsibilities of the Storage Layer
4139

@@ -47,9 +45,13 @@ flowchart TD
4745
A("Storage") -- Random Access --> C("Volatile Chain")
4846
A("Storage") -- Random Access --> D("Recent Leder States")
4947
50-
subgraph noteB ["Efficient Rollback"]
48+
subgraph noteB ["Hot - Efficient Rollback"]
5149
C
5250
D
51+
52+
subgraph noteB ["Cold - Efficient Storage"]
53+
B
54+
5355
end
5456
5557
```
@@ -61,10 +63,11 @@ In the Haskell reference implementation, the Storage layer is implemented by the
6163
| Component | Responsibility |
6264
|:------------|:----------------------------------------------------------------------------------------------------------|
6365
| ImmutableDB | Store definitive blocks on disk and provide iterator access to them for new syncing peers. |
64-
| VolatileDB | Store non-definitive blocks on disk, provide them to peers (random access?), |
66+
| VolatileDB | Store non-definitive blocks on disk, providing random access to peers, |
6567
| | efficiently switch to a different suffix of the chain if needed |
6668
| LedgerDB | Maintaining the last `k` (2160 on Cardano mainnet) ledger states in memory to facilitate chain selection. |
67-
| | |
69+
70+
**TODO**: refile requirements on access patterns for Immutable and Volatile databases. Read/write access? When do we need random access and where we do not? What does it mean specifically to efficiently switch chains?
6871

6972
## Requirements and Constraints for the Consensus and Storage Layers
7073

@@ -96,7 +99,7 @@ Transmit chains as fast as possible so that blocks arrive to the next forger in
9699

97100
## Requirements imposed onto the Ledger layer
98101

99-
TODO
102+
**TODO**
100103

101104
# Single-era Consensus Layer
102105

@@ -110,16 +113,14 @@ The protocol state is also independently maintained alongside each candidate cha
110113

111114
### Details
112115

113-
It disconnects if the peer violates the mini protocol, if the peer sends an invalid header, or if switching to their chain would require rolling back more than kcp blocks of the local selection.
116+
It disconnects if the peer violates the mini protocol, if the peer sends an invalid header, or if switching to their chain would require rolling back more than `k` blocks of the local selection.
114117

115-
*TODO*(georgy): what is kcp here?
118+
It's able to validate headers past the intersection with the local selection because the parts of the ledger state necessary to validate a header were completely determined some positive number of slots ago on that header's chain; see _forecasting_ within the ledger rules and _snapshots_ within the ledger state.
116119

117-
It's able to their validate headers past the intersection with the local selection because the parts of the ledger state necessary to validate a header were completely determined some positive number of slots ago on that header's chain; see _forecasting_ within the ledger rules and _snapshots_ within the ledger state.
118-
119-
ChainSync blocks while the candidate chain is past the forecast range of the local selection.
120+
The `ChainSync` protocol stalls while the candidate chain is past the forecast range of the local selection.
120121
The forecast range must be great enough for the peer to rescue the local node from the worse-case scenario.
121-
The Praos paper bounds this to needing at most kcp+1 headers from the peer, which that paper also bounds to requiring at most scg slots of forecast range.
122-
The Genesis paper is also satisfied by a forecast range of scg.
122+
The Praos paper bounds this to needing at most `k`+1 headers from the peer, which that paper also bounds to requiring at most the stability window slots of forecast range.
123+
The Genesis paper is also satisfied by a forecast range of the stability window.
123124
(The ChainSync server is much simpler than the client; see _followers_ below.)
124125

125126
## ChainSync server
@@ -128,7 +129,7 @@ ChainSync server provides an iterator into the ChainDB for downstream peers to b
128129

129130
#### Details
130131

131-
Moreover — because the node must serve the whole chain and not only the historical chain — each ChainSync server actually uses a _follower_ abstraction, which is implemented via iterators and additionally supports the fact that ChainSel might have to rollback a follower if it's already within kcp of the local selection's tip.
132+
Moreover — because the node must serve the whole chain and not only the historical chain — each ChainSync server actually uses a _follower_ abstraction, which is implemented via iterators and additionally supports the fact that ChainSel might have to rollback a follower if it's already within `k` of the local selection's tip, i.e. if it is streaming the volatile part of the chain.
132133
(Even the pipelining signaling from ChainSel to ChainSync clients is implemented via a follower, one that follows the so-called _tentative chain_ instead of just the actual local selection.)
133134

134135
## BlockFetch client and client coordinator
@@ -160,7 +161,7 @@ It is therefore an important observation that the ChainDB does not require the D
160161

161162
## LedgerDB
162163

163-
In both ChainSel and ChainSync, rollbacks require maintenance of/access to the past kcp+1 states, not only the tip's state — access to any such state must be fast enough to avoid disrupting the semantics of the worst-case delay parameter Delta assumed in the Praos paper's security argument.
164+
In both ChainSel and ChainSync, rollbacks require maintenance of/access to the past `k`+1 states, not only the tip's state — access to any such state must be fast enough to avoid disrupting the semantics of the worst-case delay parameter Delta assumed in the Praos paper's security argument.
164165

165166
In addition to validation in ChainSel and ChainSync, these ledger states are how the node handles a fixed set of queries used by wallets, CLI tools, etc via the LocalStateQuery mini protocol.
166167

@@ -176,7 +177,7 @@ This mini protocol leverages the fact that the Mempool is a sequence as opposed
176177

177178
Whenever the wall clock enters a new slot, the node checks whether its configured minting credentials (if any) were elected by the protocol state and forecasted ledger state to lead this slot.
178179
If so, it mints a block that extends its local selection (or its immediate predecessor if the tip somehow already occupies this slot or greater) and contains the longest prefix of the Mempool's transactions that can fit.
179-
That block is then sent to ChainSel, as if it had been fetched.
180+
That block is then sent to ChainSel, as if it had been fetched, to make sure other nodes will be able to adopt it. The only reason why a block could be rejected at this point is if there was a mismatch between the block application and Mempool transaction validation logic.
180181

181182
When (if) the node selects that block, the Mempool will react as it does for any change in the local selection: it discards any transactions that are no longer valid in the updated hypothetical block the Mempool continuously represents.
182183
Because every Ouroboros transaction _consumes_ at least one UTxO, the transactions in a newly minted and selected block will definitely be discarded.
@@ -189,28 +190,20 @@ A transaction, though, might be validated against a different chain/ledger state
189190
But many "static" checks don't need to be repeated, since they'd fail regardless of the ledger state.
190191

191192
Despite block reapplication being much faster than initial validation, the node should not need to reapply their entire historical chain whenever it is restarted.
192-
The node instead occasionally persists its oldest ledger state (ie the kcp+1st).
193-
On startup, the node only needs to deserialize that snapshotted ledger state and then replay the best chain amongs the persisted blocks it has that extends this ledger state in order to re-establish its kcp+1 ledger states.
193+
The node instead occasionally persists its oldest ledger state in the `LedgerDB` (i.e. the `k+1`st state, the immutable tip of the chain).
194+
On startup, the node only needs to deserialize that snapshotted ledger state and then replay the best chain amongs the persisted blocks it has that extends this ledger state in order to re-establish its `k`+1 ledger states in the `LedgerDB` (and `k` volatile blocks in its selected chain).
194195

195196
# Multi-era Considerations
196197

197-
In Cardano, an _era_ is an informal concept that unites a set of Cardano features, such as ledger rules or consensus protocols. In Cardano, the on-chain governance ultimately decides when to adopt backwards-incompatible changes, by incrementing the major component of the protocol version. The first such _era transition_ switched to the Praos protocol.
198+
In Cardano, an _era_ is an informal concept that unites a set of Cardano features, such as ledger rules or consensus protocols. See [Cardano Features](https://github.com/cardano-foundation/CIPs/blob/master/CIP-0059/feature-table.md) for the current list of eras.
198199

199-
In this section, we discuss the challenges imposed by the need to support multiple Cardano eras. We start with the table outlining the different aspects that need to considered when implementing a multi-era node. We than transition to discussing possible approaches to supporting a multiple-era blockchain network in the node software. Finally, we discuss the Hard Fork Combinator --- the approach implemented in the Haskell reference implementation.
200+
In Cardano, the on-chain governance ultimately decides when to adopt backwards-incompatible changes, by incrementing the major component of the protocol version. The first such _era transition_ switched to the Praos protocol.
200201

201-
202-
| Responsibility | Timing | Description |
203-
|:-------------------------|:---------------------------------------------------|:-----------------------------------------------------------------------------------------------------|
204-
| Process historical chain | slot number of the era boundaries known statically | switch and translate between the statically known historical sequence of revisions of block formats, |
205-
| | | ledger rules and protocols |
206-
| Transition to a new era | slot number of the era boundary unknown | during the era transition: |
207-
| | | * switch the consensus protocol, block format, ledger rules |
208-
| | | * translate transactions received from prevoios-era peers |
209-
| | | into the format of the current era for them included in a new block |
202+
In this section, we discuss the challenges imposed by the need to support multiple Cardano eras. We start with the possible approaches to handle era changes. We then outline the different aspects that need to considered when implementing a multi-era node. Then we transition to discussing possible approaches to supporting a multiple-era blockchain network in the node software. Finally, we discuss the Hard Fork Combinator --- the approach implemented in the Haskell reference implementation.
210203

211204
## Approaches to handle protocol changes
212205

213-
With the blokchain network evolving, the block format and ledger rules are bound to change. In Cardano, every significant change starts a new "era". There are several ways to deal with multiple eras in the node software, associated here with some of the DnD alignments:
206+
With the blockchain network evolving, the block format and ledger rules are bound to change. In Cardano, every significant change starts a new "era". There are several ways to deal with multiple eras in the node software, associated here with some of the DnD alignments:
214207

215208
* Chaotic Evil: the node software only ever implements one era. When the protocol needs to be updated, all participants must update the software or risk being ejected from the network. Most importantly, the decision to transition to the new era needs to happen off-chain.
216209
Pros: the simplest possible node software.
@@ -225,6 +218,20 @@ Pros: enables massive code reuse and forces the code to be structured in the way
225218

226219
Cons: abstraction is a double-edged sword and may be difficult to encode in some programming languages. Engineers require extended onboarding to be productive.
227220

221+
We argue that Cardano has been the True Neutral so far, which allowed to maintain the stability of the network while allowing it to change and evolve.
222+
223+
## Responsibilities of a multi-era node
224+
225+
226+
| Responsibility | Timing | Description |
227+
|:-------------------------|:---------------------------------------------------|:-----------------------------------------------------------------------------------------------------|
228+
| Process historical chain | slot number of the era boundaries known statically | switch and translate between the statically known historical sequence of revisions of block formats, |
229+
| | | ledger rules and protocols |
230+
| Transition to a new era | slot number of the era boundary unknown | during the era transition: |
231+
| | | * switch the consensus protocol, block format, ledger rules |
232+
| | | * translate transactions received from prevoios-era peers |
233+
| | | into the format of the current era for them included in a new block |
234+
228235
## Replaying history of multiple eras
229236

230237
Cardano has the peculiarity of being a multi-era network, in which at given
@@ -236,9 +243,12 @@ the ledger eras.
236243

237244
## The Hard Fork Combinator: a uniform way to support multiple eras}
238245

239-
*Hard Fork Combinator (HFC)* is mechanism that handles era transitions, including changes to ledger rules, block formats and even consensus protocols. It automates translations between different eras and provides a minimal interface for defining specific translations that obey rigorous laws. The HFC was primarily introduced to handle the fundamental complexity that arises when the translation between wall clocks and slot onsets depends on the ledger state (since on-chain governance determines when the era transition happens).
240-
It also handles the comparatively simple bookkeeping of the protocol, ledger, codecs, and so on changing on the era boundary — ie a variety of blocks, transactions, etc co-existing on the same chain in a pre-specified sequence but not according to a pre-specified schedule.
241-
Lastly, it handles the fact that ticking and forecasting can cross era boundaries, which requires translation of ledger states, protocol states, and pending transactions from one era to the next.
242-
The HFC cannot automatically infer the implementation of these necessities, but it automates as much as possible against a minimal interface that requires the definition of the specific translations etc.
246+
*Hard Fork Combinator (HFC)* is a mechanism that handles era transitions, including changes to ledger rules, block formats and even consensus protocols. It automates translations between different eras and provides a minimal interface for defining specific translations that obey rigorous laws. The HFC was primarily introduced to handle the fundamental complexity that arises when the translation between wall clocks and slot onsets depends on the ledger state (since on-chain governance determines when the era transition happens).
247+
248+
The HFC also handles the comparatively simple bookkeeping of the protocol, ledger and codecs changing on the era boundary --- i.e. a variety of blocks, transactions, etc. co-existing on the same chain in a pre-specified sequence but not according to a pre-specified schedule.
249+
250+
Lastly, the HFC handles the fact that ticking and forecasting can cross era boundaries, which requires translation of ledger states, protocol states, and pending transactions from one era to the next.
251+
252+
The HFC cannot automatically infer the implementation of these necessities, but it automates as much as possible against a minimal interface that requires the definition of the specific translations.
243253

244-
# Glossary
254+
The HFC lifts types parameterised by eras into sum types which can be manipulated in an abstract way. As so, for example, it lifts the `LedgerState` of an era (for example, `ShellyLedgerState`) into a `HardForkLedgerState` which is a sum type of `LedgerState`s of all Cardano eras. Such sum types fulfil the same interfaces as the single-era types. The consensus layer is designed around such interfaces, allowing the same code to run a multi-era network without modifications. These sum types are paired with the functions and mechanisms that make possible to transition to a different constructor in the type (translating between eras) or combine properties of those different summands (the forecasting window over an era transition is a combination of the forecasting window of the old era and the new era). This uniform interface for handling single-era states and mixed-era states is the essence of the Hard Fork Combinator.

0 commit comments

Comments
 (0)