BIP54 "Consensus Cleanup" implementation #99

darosior · 2025-10-21T12:06:05Z

Based on top of #98, this implements the BIP54 consensus rules.

Each of the 4 mitigations comes with its own unit test in a new src/test/bip54_tests.cpp module. The unit tests can be used to generate JSON test vectors for the BIP (opt-in). For those unit tests that require mainnet blocks, the JSON test vectors were generated separately and included in this PR as data files.

Besides the unit test for each mitigation, this contains a fuzz harness for the sigop accounting logic as well as a functional test which goes over all mitigations in pseudo realistic situations (for instance, mimic a Timewarp attack and a Murch-Zawy attack to demonstrate the new timestamp restrictions would prevent that). The fuzz target for the sigop accounting logic was designed to make it possible to seed it from the BIP test vectors. This branch implements that.

The chains of headers for the timestamp restrictions test vectors were generated using this unit test (note the premined headers to allow modifications without re-generating all headers from scratch). The small chains of blocks for the coinbase restrictions test vectors were generated using this unit test.

In the assumeutxo functional tests, the final test case with alternated UTxO data tests the error raised when deserializing a snapshot that contains a coin with an amount not in range (<0 or >MAX_MONEY). The current malleation uses an undocumented byte string and offset which makes it hard to maintain. In addition, the undocumented offset is set surprisingly high (39 bytes is well into the serialization of the amount which starts at offset 36). Similarly the value is surprisingly small, presumably one was adjusted for the other. But there is no comment explaining how they were chosen, why not in a clearer manner and what they are supposed to represent. Instead replace this seemingly magic value with a clear one, MAX_MONEY + 1, serialize the whole value for the amount field at the correct offset, and document the whole thing for the next person around.

This test case is brittle as it asserts a specific error string, when the error string depends on data in the snapshot not controlled by the test (i.e. not injected by the test before asserting the error string). This can be fixed in a more involved way as per Bitcoin Core PR 32117, but since this PR is now closed in Core, in the meantime just disable the brittle test in inquisition (see discussion in Bitcoin Inquisition PR 90).

We don't set the nSequence as it will be set directly in the block template generator in a following commit.

The Consensus Cleanup soft fork proposal includes enforcing that coinbase transactions set their locktime field to the block height, minus 1 (as well as their nSequence such as to not disable the timelock). If such a fork were to be activated by Bitcoin users, miners need to be ready to produce compliant blocks at the risk of losing substantial amounts mining would-be invalid blocks. As miners are unfamously slow to upgrade, it's good to make this change as early as possible. Although Bitcoin Core's GBT implementation does not provide the "coinbasetxn" field, and mining pool software crafts the coinbase on its own, updating the Bitcoin Core mining code is a first step toward convincing pools to update their (often closed source) code. A possible followup is also to introduce new fields to GBT. In addition, this first step also makes it possible to test future Consensus Cleanup changes. The changes to the seemingly-unrelated RBF tests is because these tests assert an error message which may vary depending on the txid of the transactions used in the test. This commit changes the coinbase transaction structure and therefore impact the txid of transactions in all tests. The change to the "Bad snapshot" error message in the assumeutxo functional test is because this specific test case reads into the txid of the next transaction in the snapshot and asserts the error message based it gets on deserializing this txid as a coin for the previous transaction. As this commit changes this txid it impacts the deserialization error raised.

This encapsulates the soft fork configuration logic as set by the `-testactivationheight` (for buried deployments) and `-vbparams` (for version bits deployments) options which for the moment are regtest-only, in order to make them available on other networks as well in the next commit. Can be reviewed using git's --color-moved option.

DrahtBot · 2025-10-21T21:56:47Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#98 (Make deployment options available on all networks, not just regtest by darosior)
#94 (Allow configuring target block time for a signet by benthecarman)
#90 (Consensus Cleanup preparation: backport "miner: timelock the coinbase to the mined block's height" by darosior)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

src/consensus/consensus.h

src/validation.cpp

ariard

Just going over the introduced consensus code fix by fix for now.

src/policy/policy.cpp

ariard · 2025-12-01T04:04:32Z

src/consensus/tx_verify.cpp

+    for (const auto& txin: tx.vin) {
+        const auto& prev_txo{inputs.AccessCoin(txin.prevout).out};
+
+        // Unlike the existing block wide sigop limit which counts sigops present in the block


I think this comment can be improved.

The sentence “unlike the existing block wide sigop limit which counts sigops present in the block itself” sounds to say that currently with the MAX_BLOCK_SIGOPS_COST limit (evaluated L2685 in src/validation.cpp with GetTransactionSigOpCost), only sigops that are included (but not necessarily committed) in a block are accounted for, namely covering (1) txin.scriptSig and txout.scriptPubkey of legacy transaction, (2) pay-to-script hash full script and (3) a witness-spending transaction’s witnessScript.

With BIP54, now the new limit is applying to data elements that might have been brought into the chain by previous blocks, so it’s not only a self-contained check that is applied on the block, but a stateful check where the spent scriptPubkeys sigops are also checked.

As a suggestion for a better formulation:

Bitcoin consensus knows 2 types of signature operators cost limit: - the BIP141 limits applying to (1) original transaction scriptSig and scriptPubkey, (2) pay-to-script-hash and (3) BIP141 spends’s witnessScript. All the script elements checked by this are self-contained in the block undergoing verification. - the BIP54 limits applying to all types of transaction input’s scriptSig and their spent scriptPubkey. The script elements checked by this limit can be scattered among any block, however they are only checked during the block verifcation where they can be potentially executed.

This comment already exists (here and in the Bitcoin Core repo). If we want to improve it then it should be done there directly.

On the specific of your suggestions, it is incorrect: there are more than 2 types of cost limits. There is the legacy sigop limit for bare scripts, that only accounts for sigops when they are created (output scriptpubkeys) or where they are unexpected (input scriptsigs). Then there is the BIP 16 (P2SH) limit that accounts for sigops when they are executed (in the redeemScript). Then there is the BIP 141 that also accounts for sigops when they are executed, but with a 4x higher limit. Finally there is the BIP 54 limit, which tightens the legacy and BIP 16 limits and whose main difference is to be at the transaction level.

This comment already exists (here and in the Bitcoin Core repo). If we want to improve it then it should be done there directly.

Yeah this can be done if the comment is far better. Note I got your point that there are more 2 types of cost limits, and the enumeration you’re laying out is matching the one in my comment “(1) original transaction scriptSig and scriptPubkey, (2) pay-to-script-hash and (3) BIP141 spends’s witnessScript”, what I think it would be worthy to highlight for the comment is while the current block wide limit are on the whole 4 MB limited executed block, the new static limit is on the spent scriptpubkey + scriptSig for legacy / original transactions. This is already described by the comment, though in a very rough way (“present in the block itself vs potentially executed”).

Now to be clear, I don’t think it matters that much what we put in this comment. However, the dissociation of the 2 limits, and the cost model to trigger them or at the contrary stay under them I do think it matters from an adversarial perspective. Played a bit on “block sig overflow” attacks to break time-sensitive second-layers in the past and the cost results were far from being uninteresting.

(— to be clear yes the conversation is a bit beyond the scope of the pull request, I think we’ll go back to the full-node hashing cost versus UTXO memory we surfaced offline, but I’m not exactly sure if it’s without direct interest for the engineering validation cost of a full-node).

src/consensus/consensus.h

This allows to set `-testactivationheight` and `-vbparams` on all networks instead of exclusively on regtest.

Prior commits are preparatory work. Following commits is the implementation of BIP54.

Move the function that checks whether a transaction respects the BIP54 sigops rule to the consensus folder (along with the accompanying constant), as it will be made consensus-critical in the next commit. Can be reviewed with git's --color-moved option.

darosior · 2025-12-04T19:20:32Z

Thanks for kicking CI @ajtowns. I fixed the missing annotations that DEBUG_LOCKORDER complained about.

instagibbs

Focused primarily on BIP compliance, everything looks good to me

src/consensus/consensus.h

src/validation.cpp

src/consensus/consensus.h

src/validation.cpp

ariard

Did a first review on the 64-byte fix, answering opened comments.

src/consensus/consensus.h

src/validation.cpp

src/consensus/consensus.h

Sjors · 2025-12-11T10:44:55Z

test/functional/feature_block.py

            self.next_block(blockname)
            badtx = template.get_tx()
-            if TxTemplate != invalid_txs.InputMissing:
+            if TxTemplate != invalid_txs.InputMissing and TxTemplate != invalid_txs.SizeExactly64:


In 8ee48e4 consensus: make 64 bytes transactions invalid: I'm confused by this test change. Was the test incorrect before?

I managed to apply this change to 7ccdcc1 [test] Separate 64B and 63B tx size tests instead and it still passes. So that's a better place, though a comment would be useful.

It is necessary because sign_tx() will populate the scriptSig and the transaction won't be 64-byte anymore. I don't think 7ccdcc1 would be a better place for this change, since the failure is due to enforcing the check by consensus which is only done in the commit where the change is made.

src/validation.cpp

Sjors · 2025-12-12T08:51:25Z

The consensus changes, with respect to their specification in BIP54, look good to me:

84e1043 validation: make BIP54 sigops check consensus-critical
87e9e65 consensus: prevent timewarp attacks with a 2h grace period
c6c1b18 consensus: prevent negative difficulty adjustment intervals
00745a2 consensus: mandate coinbase transactions be timelocked to block height
8ee48e4 consensus: make 64 bytes transactions invalid

I think this PR is well organised (starting at ---). Each consensus change is first prepared, done in a minimal commit, and then followed by new tests.

I'll study the test changes in more detail later.

When BIP54 is active, make sure transaction in blocks do not violate the BIP54 limit on the number of potentially-executed legacy sigops.

In Taproot the signature commits to the list of spent outputs.

Test the newly introduced limit with various combinations of inputs and outputs types, historical transactions, and exercise some implementation-specific edge cases. Record each test case and optionally write them to disk as JSON to generate the BIP test vectors.

The fuzz target was specifically crafted to support seeding it with the BIP54 test vectors generated by the unit test in the previous commit.

We are going to introduce the timewarp fix for mainnet with a greater grace period. Rename the MAX_TIMEWARP value for testnet to differentiate them.

The test vectors were generated using https://github.com/darosior/bitcoin/tree/bip54_miner

That is, enforce nLockTime be set to height-1 and nSequence not be set to final.

The test vectors were generated using https://github.com/darosior/bitcoin/tree/bip54_miner

… vectors) This adds tests exercising the bounds of the checks on the invalid transaction size, for various types of transactions (legacy, Segwit, bytes in input/output to get to 64 bytes) as well as sanity checking against some known historical violations. Thanks to Chris Stewart for digging up the historical violations to this rule.

It's not a standardness limit anymore, it was made consensus.

The previously introduced unit tests extensively test the specific implementation of each mitigation. This functional test complements them by sanity checking all mitigations in a "black box" manner. For the added timestamp constraints, it mimicks how they would get exploited (by implementing pseudo timewarp and Murch-Zawy attacks) and demonstrates those exploits are not possible anymore after BIP54 activates.

ariard

See second comment on nSequence == SEQUENCE_FINAL`.

ariard · 2025-12-20T04:31:54Z

src/validation.cpp

        }
    }

+    // Make sure the coinbase transaction is timelocked to the block's height.


nit: I think the comment can be “Make sure the coinbase transaction is timelocked to the block’s height and its nSequence field must be set to 0xff_ff_ff_ff.”.

ariard · 2025-12-20T05:04:09Z

src/validation.cpp

+            return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "bad-cb-locktime", "block height mismatch in coinbase nLockTime");
+        }
+        if (block.vtx[0]->vin[0].nSequence == CTxIn::SEQUENCE_FINAL) {
+            return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "bad-cb-sequence", "locktime is disabled for coinbase");


I think there can be a comment pointing either to IsFinalTx() or reproducing the comment from there: “If nLocktime isn’t satisfied by nBlockHeight / nBlockTime, a transaction is still considered final if all inputs’ nSequence == (SEQUENCE_FINAL) 0xff_ff_ff_ff, in which case the nLocktime is ignored”.

By the way on reading again IsFinalTx logic, the threshold test to check that the transaction’s nLocktime is valid w.r.t block’s height is a <. So if you have coinbase at block 99 (i.e BIP54 nHeight - 1), the current height is 100 and the IsFinalTx() check is applied (in consensus path byContextualCheckBlock()), the transaction should be considered as final on the check L21-L22 in IsFinalTx and the input nsequence will never be inspected.

On the other way, setting the coinbase transaction nLocktime to current block nHeight and still with nSequence set to SEQUENCE_FINAL should lead to have the transaction considered as final IIUC IsFinalTx (L32 - L35).

BIP54 does not really document why nHeight - 1 is picked up. True, there is footnote 3 reminding that is the latest block at which is invalid, but if the idea with this choose of value is to have the coinbase transaction being invalid up to the block for which it is generated, there is no need for it. Post BIP54 activation, if the transaction’s nLocktime is not nHeight - 1 it should fail on the check above (L4302). If the nLocktime is inferior to current height, it will be considered as valid by IsFinalTx second check, and if it’s not and if the nSequence == SEQUENCE_FINAL it will be considered as final (i.e valid) and if the nSequence != SEQUENCE_FINAL the coinbase transaction will be rejected.

So in my understanding, asking for the nSequence of the coinbase tx input to be mandatory set to SEQUENCE_FINAL remove a footgun corner case for miners (but in practice their softwares can still deviate), but it’s not per logically needed with BIP54’s “block height mismatch in coinbase nLocktime”.

ariard · 2025-12-20T05:35:04Z

Finished a first round of review of the code for the 4 consensus fixes:

“difficulty adjustment exploits”
“long block validation time”
“merkle tree malleability with 64-byte transaction”
“possibility of duplicate coinbase transactions”

Will review more the fixes at the conceptual level and the breadth / depth of their respective test coverage.

DrahtBot · 2025-12-21T08:53:06Z

🐙 This pull request conflicts with the target branch and needs rebase.

Sjors

See inline suggestion for renaming the bip_active argument to check_bip54 and making it a bit more clear in the tests if they're checking standardness (pre activation) or consensus (post activation).

The other consensus commit changes for the 19d159e push (compared to my previous review) look fine.

Sjors · 2025-12-23T04:35:00Z

src/policy/policy.cpp

 *
 * Note that only the non-witness portion of the transaction is checked here.
 *
 * We also check the total number of non-witness sigops across the whole transaction, as per BIP54.


In d1d435f validation: make BIP54 sigops check consensus-critical: this BIP54 comment can be dropped again.

Sjors · 2025-12-23T04:36:50Z

src/policy/policy.cpp

        return true; // Coinbases don't use vin normally
    }

-    if (!Consensus::CheckSigopsBIP54(tx, mapInputs)) {


In d1d435f validation: make BIP54 sigops check consensus-critical: note to other reviewers: this doesn't prematurely drop the standardness check before activation.

Sjors · 2025-12-23T04:40:46Z

src/validation.cpp

    }

    // The mempool holds txs for the next block, so pass height+1 to CheckTxInputs
-    if (!Consensus::CheckTxInputs(tx, state, m_view, m_active_chainstate.m_chain.Height() + 1, ws.m_base_fees)) {


In d1d435f validation: make BIP54 sigops check consensus-critical: let's add a comment here that we treat BIP54 as active for the mempool before activation.

Or perhaps we should pick a different name for this argument.

Sjors · 2025-12-23T04:41:32Z

src/consensus/tx_verify.h

 * Check whether all inputs of this transaction are valid (no double spends and amounts)
 * This does not modify the UTXO set. This does not check scripts and sigs.
 * @param[out] txfee Set to the transaction fee if successful.
+ * @param[in] bip54_active Whether to perform the BIP54 sigops check.


In d1d435f validation: make BIP54 sigops check consensus-critical: let's rename this to check_bip54, because this check is used by the mempool too when bip54 is not active.

Sjors · 2025-12-23T04:44:14Z

test/functional/mempool_sigoplimit.py


    def test_legacy_sigops_stdness(self):
-        self.log.info("Test a transaction with too many legacy sigops in its inputs is non-standard.")
+        self.log.info("Test a transaction with too many legacy sigops in its inputs is invalid.")


In d1d435f validation: make BIP54 sigops check consensus-critical: whether it's non-standard or invalid depends on whether the fork is active. It's fine for the error message to be the same in either case, but the test log statement should be clear as to which of these two scenarios it's testing.

theStack and others added 11 commits October 19, 2025 09:39

test: introduce output amount (de)compression routines

27a692a

test: introduce VARINT (de)serialization routines

9245c13

qa: timelock coinbase transactions created in functional tests

61295ef

contrib: timelock coinbase transactions in signet miner

88292c3

qa: timelock coinbase transactions created in fuzz targets

4b16dcc

qa: use prev height as nLockTime for coinbase txs created in unit tests

ecb9cf5

We don't set the nSequence as it will be set directly in the block template generator in a following commit.

qa: sanity check mined block have their coinbase timelocked to height

a26b657

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from ee557fc to bc95a67 Compare October 21, 2025 12:49

darosior mentioned this pull request Oct 21, 2025

BIP54: Consensus Cleanup test vectors bitcoin/bips#2015

Merged

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from bc95a67 to 8d513a0 Compare October 21, 2025 13:23

This was referenced Oct 21, 2025

Allow configuring target block time for a signet #94

Open

Prevent 64 byte txs at consensus level #24

Closed

ariard reviewed Nov 24, 2025

View reviewed changes

src/consensus/consensus.h Outdated Show resolved Hide resolved

src/validation.cpp Outdated Show resolved Hide resolved

src/validation.cpp Outdated Show resolved Hide resolved

ariard reviewed Dec 1, 2025

View reviewed changes

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from 8d513a0 to 00383c4 Compare December 2, 2025 18:42

chainparams: make deployment configuration options always available

85e4009

This allows to set `-testactivationheight` and `-vbparams` on all networks instead of exclusively on regtest.

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from 00383c4 to 98a9a5e Compare December 4, 2025 19:18

darosior added 3 commits December 4, 2025 14:19

======= Consensus Cleanup BEGINS HERE =======

642f16d

Prior commits are preparatory work. Following commits is the implementation of BIP54.

binana: add Consensus Cleanup

64ab309

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from 98a9a5e to 224ef91 Compare December 4, 2025 19:19

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from 224ef91 to 5e06850 Compare December 5, 2025 14:35

instagibbs reviewed Dec 9, 2025

View reviewed changes

src/consensus/consensus.h Show resolved Hide resolved

src/validation.cpp Outdated Show resolved Hide resolved

src/consensus/consensus.h Outdated Show resolved Hide resolved

src/validation.cpp Show resolved Hide resolved

ajtowns added this to the 29.x milestone Dec 9, 2025

ariard reviewed Dec 11, 2025

View reviewed changes

src/consensus/consensus.h Show resolved Hide resolved

src/validation.cpp Show resolved Hide resolved

src/validation.cpp Show resolved Hide resolved

Sjors reviewed Dec 11, 2025

View reviewed changes

src/consensus/consensus.h Outdated Show resolved Hide resolved

Sjors reviewed Dec 11, 2025

View reviewed changes

Sjors reviewed Dec 12, 2025

View reviewed changes

src/validation.cpp Show resolved Hide resolved

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from 5e06850 to babef47 Compare December 17, 2025 18:58

darosior and others added 16 commits December 18, 2025 10:29

validation: make BIP54 sigops check consensus-critical

d1d435f

When BIP54 is active, make sure transaction in blocks do not violate the BIP54 limit on the number of potentially-executed legacy sigops.

qa: add to utilities a version of SignSignature for Taproot inputs

78ac6fb

In Taproot the signature commits to the list of spent outputs.

fuzz: add a fuzz target for the BIP54 sigops check

f7a1277

The fuzz target was specifically crafted to support seeding it with the BIP54 test vectors generated by the unit test in the previous commit.

Rename testnet4 timewarp constant

b6292f5

We are going to introduce the timewarp fix for mainnet with a greater grace period. Rename the MAX_TIMEWARP value for testnet to differentiate them.

consensus: prevent timewarp attacks with a 2h grace period

446b27a

consensus: prevent negative difficulty adjustment intervals

9c5da76

qa: BIP54 test vectors for timewarp and Murch-Zawy

18dda34

The test vectors were generated using https://github.com/darosior/bitcoin/tree/bip54_miner

consensus: mandate coinbase transactions be timelocked to block height

3be9b5d

That is, enforce nLockTime be set to height-1 and nSequence not be set to final.

qa: BIP54 test vectors for restrictions on coinbase transactions

5ccb177

The test vectors were generated using https://github.com/darosior/bitcoin/tree/bip54_miner

Avoid creating <= 64-byte transactions in most functional tests.

6116b63

[test] Separate 64B and 63B tx size tests

0dbe1ce

consensus: make 64 bytes transactions invalid

de0756a

qa: rename constant for BIP54 sigops limit

3fde9fa

It's not a standardness limit anymore, it was made consensus.

darosior force-pushed the 2509_inquisition_consensus_cleanup branch from babef47 to 19d159e Compare December 18, 2025 15:37

This was referenced Dec 18, 2025

Make deployment options available on all networks, not just regtest #98

Open

Consensus Cleanup preparation: backport "miner: timelock the coinbase to the mined block's height" #90

Merged

ariard reviewed Dec 20, 2025

View reviewed changes

DrahtBot added the Needs rebase label Dec 21, 2025

Sjors reviewed Dec 23, 2025

View reviewed changes

BIP54 "Consensus Cleanup" implementation #99

Are you sure you want to change the base?

BIP54 "Consensus Cleanup" implementation #99

Conversation

darosior commented Oct 21, 2025

Uh oh!

DrahtBot commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Conflicts

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ariard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

darosior commented Dec 4, 2025

Uh oh!

instagibbs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ariard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sjors Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Sjors commented Dec 12, 2025

Uh oh!

ariard left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ariard commented Dec 20, 2025

Uh oh!

DrahtBot commented Dec 21, 2025

Uh oh!

Sjors left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

DrahtBot commented Oct 21, 2025 •

edited

Loading

Sjors Dec 11, 2025 •

edited

Loading