You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/website/contents/for-developers/ProtocolVersioning.md
+58-56Lines changed: 58 additions & 56 deletions
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@
3
3
The Cardano protocol will change over time.
4
4
The community innovates applications and updates feature priorities, researchers refine Ouroboros, and developers improve the code.
5
5
6
-
This document describes how Cardano accomodates these changes, as a community, as a software engineering project, and as a distributed system.
6
+
This document describes how Cardano accomodates these changes as a community, as a software engineering project, and as a distributed system.
7
7
8
8
## The Protocol Version
9
9
@@ -12,12 +12,12 @@ Each change to the Cardano protocol advances through the following steps before
12
12
- Someone (you!) specifies a change that is backwards-incompatible but worthwhile.
13
13
For example, it could be a [Cardano Improvement Proposal](https://cips.cardano.org/) (CIP).
14
14
- The Technical Steering Committee and Ouroboros researchers address the worth, safety, and future consequences of the proposed solution.
15
-
- Cardano developers prepare a software release that includes the changes but only enables for chains on which the protocol version has been incremented.
16
-
- The community uses on-chain governance to increment the protocol version, thereby adopting the new rules.
15
+
- Cardano developers prepare a software release that includes the changes but only enables them for chains on which the protocol version has been incremented.
16
+
- The community uses on-chain governance to increment the protocol version, thereby putting the change into effect.
17
17
(Stake pool operators must not vote to increment the protocol version before updating their software!)
18
18
19
19
Cardano philosophy mandates that the community has the final say regarding if and when to adopt some protocol change.
20
-
They accordingly control the final step above: the community increments the protocol version on mainnet, not the developers.
20
+
They accordingly control the final step above: developers determine the exact meaning of each protocol version, but only the community increments the protocol version on mainnet.
21
21
22
22
It is crucial that the community does not increment the protocol version before enough nodes have upgraded their software.
23
23
Each protocol version increment corresponds to a _hard fork_, because the changes are backwards-incompatible: nodes running the old code will not be able to adopt blocks minted by the new code.
@@ -30,86 +30,81 @@ So every protocol change must remain conditional within all future releases, ins
30
30
31
31
## When to Increment the Protocol Version?
32
32
33
-
Fundamentally, the protocol version must be incremented if some change to the protocol that an old node and a new node that received the same messages otherwise might disagree on which block is the best valid block, aka maintain consensus.
33
+
Fundamentally, the protocol version must be incremented for some change to the protocol when an old node and a new node that received the same messages might disagree on which block is the best valid block, aka maintain consensus.
34
34
A more precise and actionable answer will be given after introducing more context.
35
35
36
-
### The Abstraction Ladder
36
+
### The Cardano Abstraction Ladder
37
37
38
38
For the sake of this discussion, Cardano fixes a ladder of abstractions between the following two extrema.
39
39
40
40
-*Opacity* (bottom rung).
41
-
The Cardano node exchanges opaque bytestrings with peers in the network and with local users (eg wallets).
42
-
At any given time, the node can determine whether such a bytestring is valid, and that predicate depends on past bytestrings.
41
+
The Cardano node exchanges bytestrings with peers in the network and with local users (eg wallets).
42
+
At any given time, the node can determine whether such a bytestring is valid, and that predicate depends on previously exchanged bytestrings and the passage of time.
43
43
44
44
-*Transparency* (top rung).
45
45
Users interact with Cardano nodes to interpret the current state of the network and to evolve it according to their goals.
46
46
47
-
An analogous spectrum applies to any useful computer system, and there are longstanding traditions of how to organize the rungs between these two.
47
+
An analogous spectrum applies to any computer system that humans interact with, and there are longstanding traditions of how to organize the rungs between these two.
48
48
The Cardano node is internally organized as the following ladder.
49
+
The overall intent is for every Cardano use case to be satisfied by exchanging messages with nodes, so messages are the focus of some foundational rungs.
49
50
50
51
- Opaque Rung (bottom).
51
52
- CBOR Rung.
52
53
- The _generic data model_ of [RFC 8949 Concise Binary Object Representation](https://cbor.io/).
53
-
- Cardano Message Data Model Rung.
54
-
- A data model for Cardano messages that is sufficiently abstract for Cardano developers to easily interpret them in a consistent and useful way (eg algebraic data types).
55
-
- Note that these messages include transaction IDs, transactions, block headers, blocks, queries, query replies, etc, but not the node's internal state [^state-data-model].
56
-
- Ideally, every use case is satisfied by exchanging messages with the node.
57
-
- Some fields within this model are hashes computed in terms of the CBOR Rung.
58
-
- Cardano Consensus and Ledger Rung.
59
-
- Received messages must be validated and invalid messages must not be sent.
60
-
In practice, this requires the node to maintain as state a summary of all relevant messages received that is sufficient to determine which messages to send (eg how to respond to queries).
61
-
- All validity predicates are defined in terms of the Cardano Message Data Model Rung.
62
-
- When to mint a new block, which chain it should extend, and which transactions it should include.
63
-
- Cardano Networking Rung.
64
-
- Maintaining a robust connection to the rest of the Cardano network that keeps throughput high and latency low.
65
-
- Cardano Executable Rung.
66
-
-[The Cardano node](https://github.com/IntersectMBO/cardano-node)'s configuration interface, the [Cardano Command-Line Interface](https://github.com/IntersectMBO/cardano-cli), etc.
67
-
- This rung also contains alternatives to that CLI and/or extensions of it developed by the broader community, including wallets, indexers, Mithril, decentralized applications (aka dapps), Layer 2, etc.
54
+
- Message Data Rung.
55
+
- A data model for the content of Cardano messages that is sufficiently abstract for Cardano developers to easily interpret them in a consistent and useful way (eg algebraic data types).
56
+
- Note that this content includes transaction IDs, transactions, block headers, blocks, queries, query replies, etc, but not the node's internal state [^state-data-model].
57
+
- Some notable fields within this data model are hashes computed in terms of the CBOR Rung.
58
+
- Message Validity Rung.
59
+
- The key difference from thee Message Data Rung is that this rung excludes messages with invalid content.
60
+
Users usually only consider valid messages, and the honest node should certainly never send invalid messages.
61
+
- In practice, this rung requires the honest node to maintain as state a summary of all relevant messages that is sufficient to determine which messages to send [^theory-coalgebra].
62
+
- Query replies are an awkward fit for this rung, since the node only ever sends them without ever receiving them.
63
+
But the honest node must only send valid messages, which in this case means a correct reply.
64
+
- Another important category of messages the honest node must send are messages that indicate which chain it has selected.
65
+
This rung also therefore must resolve Praos's inherent short forks.
66
+
- There are other rungs, such as the rung that abstracts from the set of nodes to the network as a whole, but they are beyond the scope of this document.
68
67
- Transparent Rung (top).
69
68
70
-
### The Message Data Model Onion
69
+
### The Cardano Message Data Onion
71
70
72
-
The Cardano Message Data Model Rung is organized as the layers of an onion.
73
-
Some are some other rungs, but their structure is beyond the intended scope of this document.
71
+
The Message Data Rung is organized as the layers of an onion.
72
+
So are some other rungs, but their structure is beyond the scope of this document.
- The content of the few queries and their responses that either do not depend on the ledger era or else regard when prior era transitions happened, which the Ledger Layer does not record.
81
-
- An additional envelope wrapping block headers, blocks, transactions, almost all queries, and almost all query responses that indicates a particular era.
82
-
The era tag behaves as an additional field [^era-tag-parse]of block header or a transaction, which affects the validity but not the identifying hash.
83
-
- In an Ouroboros Peras implementation, also votes and certificates.
79
+
- An additional envelope wrapping block headers, blocks, transactions, almost all queries, and almost all query responses.
80
+
The envelope of a block header or a transaction indicates a particular era.
81
+
The _era tag_ behaves as an additional field [^era-tag-parse], affecting the validity of the object but not its identifying hash.
- ChainSync, BlockFetch, TxSubmission, LocalStateQuery, etc; see "Chapter 3 Mini Protocols" within the [Ouroboros Network Specification](https://ouroboros-network.cardano.intersectmbo.org/pdfs/network-spec/network-spec.pdf)[^network-pdf-link].
86
84
- The content of each mini protocol message, excluding the payload.
87
-
- In an Ouroboros Peras and/or Ouroboros Leios implementation, additional mini protocols.
88
85
89
86
### Precise Answer
90
87
91
88
The protocol version must be incremented if an old node and a new node would otherwise disagree on any of the following.
92
89
93
-
-*Chain Validity Predicates*, the validity predicates for headers, blocks, and/or transactions within the Cardano Consensus and Ledger Rung.
94
-
As of Peras, this would also include votes and certificates.
95
-
As of Leios, this would also include input blocks and endorsement blocks.
96
-
-*Chain Order*, the [partial order](https://en.wikipedia.org/wiki/Partially_ordered_set) that determines one chain is better than another within the Cardano Consensus and Ledger Rung.
90
+
-*Chain Validity Predicates*, the validity predicates within the Message Validity Rung for block headers and blocks.
91
+
-*Chain Order*, the [partial order](https://en.wikipedia.org/wiki/Partially_ordered_set) within the Message Validity Rung that determines one chain is better than another.
97
92
Tiebreakers are a gray area, since Praos does not inherently rely on them.
98
-
As of Peras, this would depend on voting quorums/certificates.
99
-
-*Chain Data Model*, the parts of the Cardano Message Data Model Rung used by the Chain Validity Predicates and the Chain Order.
93
+
-*Chain Data Model*, the parts of the Message Data Rung used to define the Chain Validity Predicates and the Chain Order.
100
94
- A change to the data model of the node's maintained state --- as opposed to its messages --- would not inherently trigger this.
101
95
- However, if such a change were meaningful (ie not just a refactor/optimization/etc), then some Chain Validity Predicates would necessarily also change.
102
96
-*Chain Codecs*, the parts of the mapping from the CBOR Rung to the Chain Data Model that affect the hashes therein.
103
-
Two nodes won't even be able to exchange transactions, headers, and/or blocks if they disagree on their identifying hashes.
97
+
Two nodes won't even be able to exchange transactions, block headers, and/or blocks if they disagree on their identifying hashes.
104
98
105
99
The most common change in Cardano's protocol so far has been the addition of a new kind/variant of transaction, which triggers all of these except the Chain Order.
106
-
The Chain Order has not yet changed --- it's always been length.
100
+
The Chain Order has only changed from Praos's length to Ouroboros Genesis's length-soon-after-the-intersection, which is slight.
107
101
On the other hand, the tiebreaker has changed, and out of caution those changes were bundled up with unrelated changes that independently required a protocol version increment.
108
102
109
-
It's also worth noting that the Consensus Layer and Network Layer or the Cardano Message Data Model Rung can change without without incrementing the protocol version.
110
-
The most common example that queries can be changed without triggering any of the above (queries do not contain any of the Chain Data Model).
111
-
Mini protocol changes are similar, except for their Chain Data Model payloads.
112
-
Such changes are conditional on the per-connection version negotiated by the Handshake mini protocol instead of the protocol version.
103
+
It's also worth noting that the Consensus Layer and Network Layer of the Message Data Rung can change without incrementing the protocol version.
104
+
The most common example is that queries can be changed without triggering any of the above, since queries do not contain any of the Chain Data Model.
105
+
Another example is that the structure of the Consensus envelope could change.
106
+
Or the mini protocols --- excluding their payload --- could change.
107
+
All such changes are conditional on the per-connection version negotiated by the Handshake mini protocol instead of the protocol version.
113
108
114
109
## Ledger Eras
115
110
@@ -120,31 +115,38 @@ The [sequence of eras](https://github.com/cardano-foundation/CIPs/blob/master/CI
120
115
A Cardano node implementation or other tooling implemented in a language prone to dynamic typing might find it convenient to ignore the era tags for the most part.
121
116
However, the era tags must still be validated according to the intended function from Cardano protocol versions to Ledger eras, since a strongly-typed node would fail to accept a block with the wrong era tag.
122
117
123
-
Protocol changes that do not also introdue a new era are known as "intra-era hard forks"
118
+
Protocol changes that do not also introduce a new era are known as _intra-era hard forks_.
124
119
These are certainly sound if they only change the Chain Validity Predicate, but they generally could do more.
125
-
So far, Cardano has forbidden intra-era hard forks from changing the Chain Codecs.
120
+
But so far, Cardano has forbidden intra-era hard forks from changing the Chain Codecs.
126
121
This could be allowed in either of two ways.
127
122
128
-
- The Consensus envelope could store the protocol version instead of the era tag.
129
-
At that point, the era tag would be an internal implementation detail of strongly-typed node implementations, and could even freely differ between them.
130
-
The only downside is that the Consensus envelope's byte size might grow slightly, but it'll still be dwarfed by the Ledger Layer portion of the wrapped header/block/transaction.
131
-
The code would also need to store the mapping from protocol version to era tags, but that is known at compile-time --- there is the slight complication that it might vary between chains (eg testnets), but they already tend to have different configuration files.
132
-
- The Consensus envelope could be entirely eliminated if the object's slot could be always be determined even without knowing the protocol version.
123
+
-*NoEnvelope*.
124
+
The Consensus envelope could be entirely eliminated if an object's slot could be always be determined even without knowing the protocol version.
133
125
The slot could then be used to determine the appropriate protocol version, based on the node's state.
134
-
The downside here is significant: programs that do not maintain the node's state (since that is costly to do) would not necessarily be able to parse headers, blocks, and transactions, since they wouldn't necessarily know the mapping from slot to protocol version (since it's determined by on-chain governance).
126
+
The downside here is significant: programs that do not maintain the node's state (since that is costly to do) would not necessarily be able to parse block headers, blocks, and transactions, since they wouldn't necessarily know the mapping from slot to protocol version (since it's determined by on-chain governance).
127
+
-*HideEras*.
128
+
The Consensus envelope could store the protocol version instead of the era tag.
129
+
At that point, the era tag would be an internal implementation detail of strongly-typed node implementations, and could even freely differ between them.
130
+
The only downside is that the Consensus envelope's byte size might grow slightly, but it'll still be dwarfed by the Ledger Layer portion of the wrapped block header/block/transaction.
131
+
- Notably, the Ledger already maintains an invariant that the CBOR of UTxO are forward-compatible, so the storage of each UTxO does not need to include an era tag.
132
+
That's the only object in the system that might be of comparable size as the era tag and numerous enough to incur significant costs.
133
+
- The code would also need to store the mapping from protocol version to era tags, but that is known at compile-time --- there is the slight complication that the mapping might vary between chains (eg testnets), but they already tend to have different configuration files.
135
134
136
-
It might be plausible for the second option to restrict stateless programs to assume they're downstream of a node that had annotated the blocks and/or transactions with the correct protocol version.
137
-
At that point, though, this option is basically the same as the first, except that nodes would exclude the Consensus envelope when exchanging with other nodes.
135
+
It might be plausible for the NoEnvelope option to restrict stateless programs to assume they're downstream of a node that had annotated the blocks and/or transactions with the correct protocol version.
136
+
At that point, though, this option is basically the same as HideEras, except that nodes would exclude the Consensus envelope when exchanging with other nodes.
138
137
139
138
[^state-data-model]: *Remark*.
140
-
The state maintained by the Cardano Consensus and Ledger Rung also has a data model, but, in contrast to messages, it is an internal implementation detail of the node.
139
+
The state maintained by the Message Validity Rung also has a data model, but, in contrast to messages, it is an internal implementation detail of the node.
141
140
Different node implementations could have different state data models, but still implement the same protocol, ie send/receive/accept/reject the exact same messages.
142
141
142
+
[^theory-coalgebra]: *Remark*.
143
+
In theory, on the other hand, that summary state could merely be the sequence of valid received and sent messages along with their timestamps.
144
+
143
145
[^era-tag-parse]: *Remark*.
144
146
In particular, it's the only part of the block or transaction whose codecs does not depend on the era.
145
147
146
148
[^mux-layer]: *Disclaimer*.
147
-
This Network Layer of the Cardano Message Data Model Onion actually contains an additional layer, the Mux Layer, but that is transparent to the rest of system and beyond the intended scope of this doucment.
149
+
This Network Layer of the Cardano Message Data Onion actually contains an additional layer, the Mux Layer, but that is transparent to the rest of system and beyond the scope of this doucment.
148
150
149
151
[^network-pdf-link]: *Note*.
150
152
If that link breaks, the document can usually be obtained via the "The Shelley Networking Protocol" hyperlink in the `README` file of the [`ouroboros-network` repository](https://github.com/IntersectMBO/ouroboros-network).
0 commit comments