docs: cip-Privval Interface Extension for Arbitrary Message Signing #337

evan-forbes · 2025-07-25T16:43:48Z

Overview

adds CIP-40

for editors

assign CIP number and rename file
add to cips/README.md - to render on CIPs table
add to cips/SUMMARY.md - to render on the site

gemini-code-assist

Summary of Changes

Hello @evan-forbes, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new Celestia Improvement Proposal (CIP-40) that outlines an extension to the CometBFT privval.Message interface. The primary goal is to enable validators to securely sign arbitrary off-chain message hashes using their existing consensus keys, thereby facilitating the development of high-throughput, low-latency gossiping protocols while maintaining backward compatibility.

Highlights

New CIP Document: Adds CIP-40, titled "Privval Interface Extension for Arbitrary Message Signing," which details a standard for extending the CometBFT privval interface.
Interface Extension: Specifies the addition of SignP2PMessageRequest and SignedP2PMessageResponse protobuf messages to the privval.Message interface, allowing for the signing of generic P2P message hashes.
Off-chain Protocol Support: Enables validators to leverage their consensus keys and Key Management Systems (KMS) to sign messages for off-chain protocols, such as Full Mesh Overlay (FMO) and Vacuum!, without affecting on-chain state.
Design Principles: Emphasizes hash-based signing, inclusion of chain_id for security, an optional unique_id for flexibility, and clarifies that double-signing protection is not required for these off-chain messages.
Security and Compatibility: Addresses key reuse, signature verification, DoS protection, and key compromise, while ensuring full backward compatibility with existing KMS implementations.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This PR introduces CIP-40, which proposes an extension to the CometBFT privval interface for signing arbitrary messages. The proposal is well-structured and detailed. My review includes a few suggestions to improve the document's clarity and completeness, such as standardizing the author format, adding relevant discussion and implementation links, and clarifying the definition of an external type used in the protobuf specification.

cips/cip-040.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

cips/cip-040.md

rootulp · 2025-07-25T17:17:27Z

cips/cip-040.md

+
+- Maintain consistent message size regardless of original content length
+- Ensure cryptographic properties of the signed data
+- Simplify KMS implementations by avoiding message content validation


Does each validator get the raw message in addition to the hash of the message before signing?

If not, is the validator blindly signing over a hash without access to the raw message?

this is only for the KMS, so the validator is the one verifying and serializing the message

the KMS doesn't have any verification, no. this is different from votes, which have a "water mark" to prevent double signing

cips/cip-040.md

cmwaters

Looks good, just concerned about the digest we sign over

cips/cip-040.md

cmwaters · 2025-07-28T08:44:57Z

cips/cip-040.md

+
+## Backwards Compatibility
+
+This proposal is fully backwards compatible. Existing KMS implementations will continue to function normally, as the new message types use previously unused field numbers in the protobuf oneof union. That being said, all repos that import the interface MUST update their implementions to at least use a noop.


Is it fully backwards compatible. I know that adding fields won't be breaking, but Celestia's block propagation depends on this. Users have to update their KMS in order to continue running a consensus node. You can't run a new consensus node with an older version of KMS right?

Users have to update their KMS in order to continue running a consensus node.

technically speaking, this is not currently the case. they just can't propose blocks 😅 . we're discussing if this needs to be changed in celestiaorg/celestia-core#2243

old KMS will still work, and new KMS will work with old versions of comet. the compatibility is a bit messy, if you have other suggestions I'm open to not using the term backwards compatible

I see, perhaps they can continue to run and they just have noisy logs

cmwaters · 2025-07-28T08:48:14Z

cips/cip-040.md

+### Field Specifications
+
+- `chain_id`: The chain identifier to prevent cross-chain signature reuse.
+- `raw_bytes`: The digest that is signed over.


Shouldn't the digest sign over the chain_id, raw_bytes and unique_id (if present)?

Else a rogue consensus node could make the KMS double sign votes and proposals. I know we don't care about double signing these messages but it should not allow double signing of existing messages that have double signing protection

I see good point, we can reword this field doc to clarify that this digest is not the entire digest and that the unique id and chain id are also included.

we can also reword the unique field id to mention that its not really optional, since its included in the sign bytes

Unique ID Field: The optional unique_id field allows protocols to differentiate between different message types or contexts without requiring separate interface extensions.

see

message SignRawBytesRequest { string chain_id = 1; bytes raw_bytes = 2; string unique_id = 3; }

I forgot that uniqueID is actually optional since its a string and therefore can be "", same with chain-id

a rogue consensus node could make the KMS double sign votes and proposals.

then this makes sense to me if we're using strings across the board cc @rach-id

the fix here is to not make unique-id optional

That's a very good point 👍 making it optional will have the same vulnerability. Maybe we should generate a nonce on the KMS, prepend it before signature, return it along with the signed bytes. This would guarantee uniqueness and add random data even if the payload is trying to mimic another message.

making it optional will have the same vulnerability

yeah defo, I think the fix is make it not optional

do you think that fixes the issue? we could use a nonce, but the nonce must be known by all nodes verifying the signature as they must be the ones adding it to the sign bytes in order to get the benefits

see 89e64eb

The nonce can be included in the compact block and can be used during verification of the signature.

I meant making it required will not fully fix the issue as there could be edge cases where some vote bytes, for example, coincide with the protobuf encoding overhead (the wire type + the length) which would result in the same thing. It's very less likely but could be possible

the protobuf thing is interesting, and might also apply to nonces then as well if they are an issue no?

either way, the chain-id is being added by the kms, so as long as that's the case I think we're fine and the original concern also isn't valid

hum, I think we're fine without changing the unique_id with an int. Check the sign bytes:

the unique_id is required, and also we add a domain prefix to the sign bytes. So the issue shouldn't happen at all. Apologies I thought I remembered the implementation, but I forgot the prefix thing

Co-authored-by: Callum Waters <[email protected]>

precisely

Co-authored-by: CHAMI Rachid <[email protected]>

cips/cip-040.md

jcstein

LGTM aside from unresolved comments from @rach-id and @evan-forbes

cips/cip-040.md

cmwaters

Thanks for addressing my suggestions. LGTM!

cmwaters · 2025-07-30T10:33:42Z

cips/cip-040.md

+### Sign Bytes Construction
+
+The actual bytes that are signed MUST be constructed by concatenating:
+
+```text
+sign_bytes = chain_id + unique_id + raw_bytes
+```


👍 thanks

cmwaters · 2025-07-30T10:35:02Z

cips/cip-040.md

+
+## Backwards Compatibility
+
+This proposal is fully backwards compatible. Existing KMS implementations will continue to function normally, as the new message types use previously unused field numbers in the protobuf oneof union. That being said, all repos that import the interface MUST update their implementions to at least use a noop.


I see, perhaps they can continue to run and they just have noisy logs

…elestiaorg#337) Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Josh Stein <[email protected]> Co-authored-by: CHAMI Rachid <[email protected]>

docs: cip-40

82e71d5

gemini-code-assist bot reviewed Jul 25, 2025

View reviewed changes

chore: add discussion link

b46d868

gemini-code-assist bot reviewed Jul 25, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

cips/cip-040.md Outdated Show resolved Hide resolved

cips/cip-040.md Show resolved Hide resolved

cips/cip-040.md Outdated Show resolved Hide resolved

Update cips/cip-040.md

93e85e6

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

jcstein requested review from cmwaters, ebuchman, jcstein and rootulp July 25, 2025 16:59

jcstein assigned evan-forbes Jul 25, 2025

chore: add to readme

0e67ad6

jcstein mentioned this pull request Jul 25, 2025

Celestia Core Devs Call 36 #331

Closed

fix merge

f296c60

rootulp approved these changes Jul 25, 2025

View reviewed changes

fix: update to use raw message instead of hashes

3b76664

evan-forbes mentioned this pull request Jul 25, 2025

docs: add CIP-Matcha Network Upgrade #338

Merged

3 tasks

rach-id reviewed Jul 25, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

cips/cip-040.md Show resolved Hide resolved

cmwaters requested changes Jul 28, 2025

View reviewed changes

evan-forbes and others added 2 commits July 28, 2025 06:59

Apply suggestion from @cmwaters

bdd5f58

Co-authored-by: Callum Waters <[email protected]>

chore: update to force inclusion of uint and specify encoding more

89e64eb

precisely

jcstein changed the title ~~docs: cip-40~~ docs: cip-Privval Interface Extension for Arbitrary Message Signing Jul 28, 2025

Apply suggestions from code review

a51495f

Co-authored-by: CHAMI Rachid <[email protected]>

jcstein reviewed Jul 28, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

jcstein reviewed Jul 28, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

jcstein reviewed Jul 28, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

jcstein reviewed Jul 28, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

jcstein reviewed Jul 28, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

jcstein reviewed Jul 28, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

jcstein added 2 commits July 28, 2025 19:49

Apply suggestions from code review

ca70671

chore: add to summary.md

d31ba8c

jcstein approved these changes Jul 28, 2025

View reviewed changes

chore: mdlint fix

2f57ff9

jcstein reviewed Jul 29, 2025

View reviewed changes

cips/cip-040.md Outdated Show resolved Hide resolved

Apply suggestions from code review

7c701ea

rach-id mentioned this pull request Jul 30, 2025

Force the inclusion of the uniqueID in signing over raw messages via privval celestiaorg/celestia-core#2250

Closed

cmwaters approved these changes Jul 30, 2025

View reviewed changes

cmwaters merged commit d5cef18 into celestiaorg:main Jul 30, 2025
2 checks passed


		## Backwards Compatibility

		This proposal is fully backwards compatible. Existing KMS implementations will continue to function normally, as the new message types use previously unused field numbers in the protobuf oneof union. That being said, all repos that import the interface MUST update their implementions to at least use a noop.

docs: cip-Privval Interface Extension for Arbitrary Message Signing #337

docs: cip-Privval Interface Extension for Arbitrary Message Signing #337

Uh oh!

Conversation

evan-forbes commented Jul 25, 2025 • edited by jcstein Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

for editors

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evan-forbes Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cmwaters left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evan-forbes Jul 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jcstein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cmwaters left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

evan-forbes commented Jul 25, 2025 •

edited by jcstein

Loading

evan-forbes Jul 25, 2025 •

edited

Loading

evan-forbes Jul 28, 2025 •

edited

Loading