BAD BLOCK 91255861

[yangj888]

Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: INFO [08-07|11:19:48.264] Skip duplicated bad block number=91,255,861 hash=b482cf..4e4821
Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: ERROR[08-07|11:19:48.264]
Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: ########## BAD BLOCK #########
Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: Block: 91255861 (0xb482cf26cab5f839981c767eeb36795e3f063a86479f609ae2ab33f8664e4821)
Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: Error: invalid gas used (remote: 26264507 local: 25634720)
Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: Platform: github.com/ethereum/go-ethereum@(devel) go1.26.3 amd64 linux
Aug 07 11:19:48 wallet-prod-nodepolygon-02 bor[2603983]: Chain config: &params.ChainConfig{ChainID:137, HomesteadBlock:0, DAOForkBlock:, DAOForkSupport:true, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:3395000, MuirGlacierBlock:3395000, BerlinBlock:14750000, LondonBlock:23850000, ArrowGlacierBlock:, GrayGlacierBlock:, MergeNetsplitBlock:, ShanghaiBlock:50523000, CancunBlock:54876000, PragueBlock:73440256, VerkleBlock:, OsakaBlock:, TerminalTotalDifficulty:, DepositContractAddress:0x0000000000000000000000000000000000000000, EnableVerkleAtGenesis:false, Bor:(*params.BorConfig)(0x7da0360), Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(nil), BlobScheduleConfig:(*params.BlobScheduleConfig)(nil)}

bor version: v2.9.0
heimdall version: v0.10.0

[Kosechok]

the same issue but with another block

INFO [08-13|13:59:33.583] Imported new chain segment number=91,949,698 hash=513e5b..aa85a3 blocks=1 txs=109 mgas=31.072 elapsed=226.427ms mgasps=137.228 triediffs=250.25MiB triedirty=135.17MiB
INFO [08-13|13:59:34.924] Imported new chain segment number=91,949,699 hash=1a4c5c..a9354f blocks=1 txs=111 mgas=45.992 elapsed=215.942ms mgasps=212.981 triediffs=250.34MiB triedirty=136.25MiB
INFO [08-13|13:59:35.857] Whitelisting new milestone from heimdall block=91,949,696 hash=543218..49a4d4
INFO [08-13|13:59:37.857] Whitelisting new milestone from heimdall block=91,949,698 hash=513e5b..aa85a3
ERROR[08-13|13:59:39.630]
########## BAD BLOCK #########
Block: 91949700 (0x37b0d8bdbce7284988c266da32d6c4b1715944bc1468617c295074e560c66d5e)
Error: missing gas target or base fee change denominator in extra data
Platform: geth v1.14.14-0.20260702151440-920556e8f785+dirty go1.26.3 amd64 linux
VCS: 920556e-20260702 (dirty)
Chain config: &params.ChainConfig{ChainID:137, HomesteadBlock:0, DAOForkBlock:, DAOForkSupport:true, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:3395000, MuirGlacierBlock:3395000, BerlinBlock:14750000, LondonBlock:23850000, ArrowGlacierBlock:, GrayGlacierBlock:, MergeNetsplitBlock:, ShanghaiBlock:50523000, CancunBlock:54876000, PragueBlock:73440256, VerkleBlock:, OsakaBlock:, TerminalTotalDifficulty:, DepositContractAddress:0x0000000000000000000000000000000000000000, EnableVerkleAtGenesis:false, Bor:(*params.BorConfig)(0x7e6a1a0), Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(nil), BlobScheduleConfig:(*params.BlobScheduleConfig)(nil)}
Receipts:
##############################

WARN [08-13|13:59:39.630] Synchronisation failed, dropping peer peer=15cfb3af9c20c18ef56f2f5422c3fbdc7b61de7564cca1a94a2a32abaffcd78b err="retrieved hash chain is invalid: missing gas target or base fee change denominator in extra data" mode=full
INFO [08-13|13:59:39.856] Whitelisting milestone deferred err="chain out of sync"
INFO [08-13|13:59:41.296] Skip duplicated bad block number=91,949,700 hash=37b0d8..c66d5e
ERROR[08-13|13:59:41.297]
########## BAD BLOCK #########
Block: 91949700 (0x37b0d8bdbce7284988c266da32d6c4b1715944bc1468617c295074e560c66d5e)
Error: missing gas target or base fee change denominator in extra data
Platform: geth v1.14.14-0.20260702151440-920556e8f785+dirty go1.26.3 amd64 linux
VCS: 920556e-20260702 (dirty)
Chain config: &params.ChainConfig{ChainID:137, HomesteadBlock:0, DAOForkBlock:, DAOForkSupport:true, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:3395000, MuirGlacierBlock:3395000, BerlinBlock:14750000, LondonBlock:23850000, ArrowGlacierBlock:, GrayGlacierBlock:, MergeNetsplitBlock:, ShanghaiBlock:50523000, CancunBlock:54876000, PragueBlock:73440256, VerkleBlock:, OsakaBlock:, TerminalTotalDifficulty:, DepositContractAddress:0x0000000000000000000000000000000000000000, EnableVerkleAtGenesis:false, Bor:(*params.BorConfig)(0x7e6a1a0), Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(nil), BlobScheduleConfig:(*params.BlobScheduleConfig)(nil)}
Receipts:
##############################

WARN [08-13|13:59:41.297] Synchronisation failed, dropping peer peer=2160093b5e3c3ebefca3daef7189e26db331171f78471bb71b0a343fb3104198 err="retrieved hash chain is invalid: missing gas target or base fee change denominator in extra data" mode=full
INFO [08-13|13:59:41.599] Skip duplicated bad block number=91,949,700 hash=37b0d8..c66d5e

bor version: v2.9.0
heimdall version: v0.10.0

[darkhorse-spb]

same here, block 91949700, 6 nodes were affected simultaneously

########## BAD BLOCK #########
Block: 91949700 (0x37b0d8bdbce7284988c266da32d6c4b1715944bc1468617c295074e560c66d5e)
Error: missing gas target or base fee change denominator in extra data

bor version: v2.9.0
heimdall version: v0.10.0

[riyue121]

INFO [08-13|22:33:51.960] Skip duplicated bad block number=91,949,700 hash=37b0d8..c66d5e
ERROR[08-13|22:33:51.960]
########## BAD BLOCK #########
Block: 91949700 (0x37b0d8bdbce7284988c266da32d6c4b1715944bc1468617c295074e560c66d5e)
Error: missing gas target or base fee change denominator in extra data
Platform: github.com/ethereum/go-ethereum@(devel) go1.26.3 amd64 linux
Chain config: &params.ChainConfig{ChainID:137, HomesteadBlock:0, DAOForkBlock:, DAOForkSupport:true, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:3395000, MuirGlacierBlock:3395000, BerlinBlock:14750000, LondonBlock:23850000, ArrowGlacierBlock:, GrayGlacierBlock:, MergeNetsplitBlock:, ShanghaiBlock:50523000, CancunBlock:54876000, PragueBlock:73440256, VerkleBlock:, OsakaBlock:, TerminalTotalDifficulty:, DepositContractAddress:0x0000000000000000000000000000000000000000, EnableVerkleAtGenesis:false, Bor:(*params.BorConfig)(0x7da0360), Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(nil), BlobScheduleConfig:(*params.BlobScheduleConfig)(nil)}
Receipts:
##############################

[riyue121]

bor version: v2.9.0

[dl-my]

I encountered the same problem.
bor version: v2.9.0
heimdalld version 0.10.0

[ieshuaganocry]

same here, even tried bor 2.9.1:
https://hub.docker.com/r/0xpolygon/bor?tag=2.9.1

[ieshuaganocry]

Root cause & proposed fix for bor#2351 — BAD BLOCK 91,949,700 "missing gas target or base fee change denominator in extra data"

(Analysis ready to post as a comment on #2351)

TL;DR

Block 91,949,700 (mainnet) is not missing its Giugliano fields. Its producer serialized
extraData with the TxDependency list element omitted (3-element RLP list instead of 4).
Stock verifiers (v2.9.0, v2.9.1, v2.9.1-beta2, develop) decode extra with
blockExtraDataRawTxDeps, whose TxDependency rlp.RawValue silently swallows the GasTarget
element, shifting fields: BFCD's value lands in GasTarget and BFCD comes out nil →
errMissingGiuglianoFields → deterministic BAD BLOCK on every stock full node, network-wide.

The likely producer-side origin is an encode path that re-encodes the raw variant struct with
an empty TxDependency rlp.RawValue: go-ethereum's RLP encoder writes RawValue verbatim, so
an empty RawValue contributes zero bytes and the element vanishes from the wire format.

Both claims reproduce byte-for-byte against the canonical on-chain blocks (below), and a
kind-aware fallback decoder recovers the rejected block's actual fields
(gasTarget=112,000,000, bfcd=64).

Evidence

Canonical extraData payloads (vanity/seal stripped), from public RPC:

block	payload (hex)	RLP structure
91,949,699 ✅ accepted	c8 80 c0 84016e3600 40	[validators=∅, txDeps=∅list, gasTarget=24_000_000, bfcd=64] — 4 elements
91,949,700 ❌ rejected	c7 80 84 06acfc00 40	[validators=∅, 112_000_000, 64] — 3 elements, txDeps omitted

Decoding the 3-element payload with the current verification struct
(core/types/block.go: blockExtraDataRawTxDeps):

TxDependency (RawValue)  <- 0x8406acfc00   (swallows the gas-target item)
GasTarget                <- 64             (actually the producer's BFCD)
BaseFeeChangeDenominator <- nil            → errMissingGiuglianoFields (consensus/bor/bor.go:505)

Reproduction (go-ethereum rlp, actual bytes)

type WithDeps struct { // = BlockExtraData (normal producer path)
    ValidatorBytes           []byte
    TxDependency             [][]uint64
    GasTarget                *uint64 `rlp:"optional"`
    BaseFeeChangeDenominator *uint64 `rlp:"optional"`
}
type RawDeps struct { // = blockExtraDataRawTxDeps (raw variant)
    ValidatorBytes           []byte
    TxDependency             rlp.RawValue
    GasTarget                *uint64 `rlp:"optional"`
    BaseFeeChangeDenominator *uint64 `rlp:"optional"`
}

gtA, bfA := uint64(24_000_000), uint64(64)
a, _ := rlp.EncodeToBytes(&WithDeps{nil, nil, &gtA, &bfA})
// a = c880c084016e360040  == accepted block 91,949,699 payload ✅

gtB, bfB := uint64(112_000_000), uint64(64)
b, _ := rlp.EncodeToBytes(&RawDeps{nil, nil, &gtB, &bfB})
// b = c7808406acfc0040    == REJECTED block 91,949,700 payload ✅
// (empty rlp.RawValue encodes to zero bytes -> the TxDependency element disappears)

So a producer path that decodes extra into the raw variant and re-encodes it (with
TxDependency empty) emits exactly the malformed-looking — but recoverable — 3-element form.
Note miner/worker.go (v2.9.0 lines ~1706–1760) performs a decode→modify→re-encode cycle of
header.Extra; any refactor of such a cycle to the raw struct (cf. d707ad362 "decode validator
bytes without expanding TxDependency") reproduces this on the producing side.

Proposed fix (verification side — required to sync past the canonical block)

Make GetValidatorBytesAndBaseFeeParams disambiguate by the RLP kind of the second list
element instead of assuming TxDependency is always present:

type blockExtraDataNoDeps struct {
    ValidatorBytes           []byte
    GasTarget                *uint64 `rlp:"optional"`
    BaseFeeChangeDenominator *uint64 `rlp:"optional"`
}

func (h *Header) GetValidatorBytesAndBaseFeeParams(chainConfig *params.ChainConfig) ([]byte, *uint64, *uint64) {
    if len(h.Extra) < ExtraVanityLength+ExtraSealLength {
        return nil, nil, nil
    }
    if !chainConfig.IsCancun(h.Number) {
        return h.Extra[ExtraVanityLength : len(h.Extra)-ExtraSealLength], nil, nil
    }
    payload := h.Extra[ExtraVanityLength : len(h.Extra)-ExtraSealLength]

    // Peek the kind of the 2nd list element to detect whether TxDependency is present.
    if content, _, err := rlp.SplitList(payload); err == nil {
        if _, _, rest, err := rlp.Split(content); err == nil && len(rest) > 0 {
            if k, _, _, err := rlp.Split(rest); err == nil && k != rlp.List {
                // TxDependency omitted by producer: [validators, gasTarget?, bfcd?]
                var d blockExtraDataNoDeps
                if err := rlp.DecodeBytes(payload, &d); err == nil {
                    return d.ValidatorBytes, d.GasTarget, d.BaseFeeChangeDenominator
                }
                return nil, nil, nil
            }
        }
    }

    var blockExtraData blockExtraDataRawTxDeps
    if err := rlp.DecodeBytes(payload, &blockExtraData); err != nil {
        return nil, nil, nil
    }
    if !txDependencyValidPreValencia(chainConfig, h.Number, blockExtraData.TxDependency) {
        return nil, nil, nil
    }
    return blockExtraData.ValidatorBytes, blockExtraData.GasTarget, blockExtraData.BaseFeeChangeDenominator
}

Validated against both canonical payloads:

91,949,699: form=with-deps  gasTarget=24000000   bfcd=64  ✅ (behavior unchanged)
91,949,700: form=no-deps    gasTarget=112000000  bfcd=64  ✅ (block now passes the Giugliano presence check)

The same tolerant parse should be applied in DecodeBlockExtraData and any other extra-data
consumers.

Proposed fix (producer side — stop emitting the ambiguous form)

Wherever header.Extra is rebuilt from a decoded raw struct, never encode an empty
rlp.RawValue for TxDependency — normalize nil/empty to an explicit empty list (0xc0)
before rlp.EncodeToBytes, or use the [][]uint64 struct for re-encoding. A guard in the
encode path (or a unit test asserting the 4-element wire shape) prevents regression.

Operator note

Restart, debug_setHead, and upgrading among current stable/beta tags do not help — the
rejection is deterministic in all published versions; the block is canonical and finalized
(milestones progressed past it), so every stock full node walls at 91,949,699 until a patched
release ships.

---

Analysis: extraData bytes fetched from public RPC; encode/decode reproduced with
github.com/ethereum/go-ethereum/rlp; sources examined: bor v2.9.0 (920556e), v2.9.1-beta2,
develop; 0xPolygon/erigon v3.7.2 (not the producer — it doesn't produce blocks).

[ieshuaganocry]

Did you modify the code, recompile, and test it? Did that fix the issue?

Yes, i will use this monkey-patch until the moment when Bor maintainers will release the patch. Fix is working:
currentBlock=91,951,620
highestBlock=91,952,675

it's still catching up, but bad block is behind

[pratikspatil024]

Upgrading to v2.10.0 will solve the bad block on block 91949700