From f2ea4acdd987ae42347680179fe1102e53ead9ee Mon Sep 17 00:00:00 2001 From: metricaez Date: Thu, 18 Dec 2025 18:13:35 -0300 Subject: [PATCH 1/7] feat: pub-sub-rfc --- text/0160-pub-sub-mechanism.md | 162 +++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 text/0160-pub-sub-mechanism.md diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md new file mode 100644 index 000000000..28d6d7aab --- /dev/null +++ b/text/0160-pub-sub-mechanism.md @@ -0,0 +1,162 @@ +# RFC-0160: PubSub Mechanism + +| | | +| --------------- | ------------------------------------------------------------------------------------------- | +| **Start Date** | 18 - 12 - 2025 | +| **Description** | PubSub mechanism via XCM data publishing and relay state proof propagation. | +| **Authors** | metricaez | + +## Summary + +Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via verified relay chain storage proofs. + +## Motivation + +As pointed out in issue #606, when parachains need to exchange information, the only available options are repeated point-to-point XCM messages or ad-hoc off-chain infrastructure. This RFC is motivated by the desire to propose an alternative based on a relay-assisted publish/subscribe mechanism that enables one-to-many, verifiable, cross-parachain data distribution. + +## Stakeholders + +Runtime developers, oracles, bridges, and general rollup product and application builders. + +## Explanation + +This RFC defines a relay-assisted publish/subscribe mechanism that enables parachains to share arbitrary key-value data through the relay chain without relying on repeated XCM messages or off-chain protocols. The core idea is to use the relay chain as a common, verifiable distribution layer, allowing data to be published once and consumed by many parachains with bounded and predictable costs. +Parachains publish data via a new XCM v5 `Publish` instruction. Published entries are stored on the relay chain in a per-publisher child trie managed by the broadcaster pallet. Publishers must be registered, and both the number of keys and their sizes are bounded to prevent unbounded state growth. Each published item is addressed by a fixed-size key. The suggestion is to backport the new instruction to v5 as v6 discussion advances. + +```rust +Publish { data: PublishData }  +PublishData = BoundedVec<(PublishKey, BoundedVec), MaxPublishItems> +``` + +On the consumer side, subscribing parachains declare their interests at the runtime level by specifying which publishers and keys they want to subscribe to. This information is exposed to the collator through a new dedicated runtime API (`KeyToIncludeInRelayProofApi`), allowing the collator to know which relay-chain storage keys must be proven for a given block. During block production, the collator fetches relay-chain state proofs for those keys, including child-trie entries, and embeds them in the ParachainInherentData. + +```rust +/// A relay chain storage key to be included in the storage proof. +#[derive(Clone, Debug, Encode, Decode, TypeInfo, PartialEq, Eq)] +pub enum RelayStorageKey { + /// Top-level relay chain storage key. + Top(Vec), + /// Child trie storage key. + Child { + /// Unprefixed storage key identifying the child trie root location. + /// Prefix `:child_storage:default:` is added when accessing storage. + /// Used to derive `ChildInfo` for reading child trie data. + storage_key: Vec, + /// Key within the child trie. + key: Vec, + }, +} + +/// Request for proving relay chain storage data. +/// +/// Contains a list of storage keys (either top-level or child trie keys) +/// to be included in the relay chain state proof. +#[derive(Clone, Debug, Encode, Decode, TypeInfo, PartialEq, Eq, Default)] +pub struct RelayProofRequest { + /// Storage keys to include in the relay chain state proof. + pub keys: Vec, +} + +/// API for specifying which relay chain storage data to include in storage proofs. +/// +/// This API allows parachains to request both top-level relay chain storage keys +/// and child trie storage keys to be included in the relay chain state proof. +pub trait KeyToIncludeInRelayProofApi { + /// Returns relay chain storage proof requests. + /// + /// The returned `RelayProofRequest` contains a list of storage keys where each key + /// can be either: + /// - `RelayStorageKey::Top`: Top-level relay chain storage key + /// - `RelayStorageKey::Child`: Child trie storage, containing the child trie identifier + /// and the key to prove from that child trie + /// + /// The collator generates proofs for these and includes them in the relay chain state proof. + fn keys_to_prove() -> RelayProofRequest; +} +``` + +The parachain verifies the relay-chain state proofs as part of `set_validation_data` of `parachain-system`. A processing hook extracts the relevant key–value pairs from the proofs and only triggers updates when the corresponding child-trie root has changed since the previous block, avoiding redundant child trie data extractions which are the most significant computation-demanding process of the feature. Verified updates are then delivered to the runtime through a subscription handler, enabling consumption of cross-chain data. + +```rust +/// Processor for relay chain proof keys. +/// +/// This allows parachains to process data from the relay chain state proof, +/// including both child trie keys and main trie keys that were requested +/// via `KeyToIncludeInRelayProofApi`. +type RelayProofKeysProcessor: ProcessRelayProofKeys; + +/// Process keys from verified relay chain state proofs. +/// +/// This trait allows processing of relay chain storage data from the verified proof. +pub trait ProcessRelayProofKeys { + /// Process keys from a verified relay state proof. + fn process_relay_proof_keys(verified_proof: &RelayChainStateProof) -> Weight; +} + +``` + +```rust +/// Define subscriptions and handle received data. +pub trait SubscriptionHandler { + /// List of subscriptions as (ParaId, keys) tuples. + /// Returns (subscriptions, weight) where weight is the cost of computing the subscriptions. + fn subscriptions() -> (Vec<(ParaId, Vec>)>, Weight); + + /// Called when subscribed data is updated. + /// Returns the weight consumed by processing the data. + fn on_data_updated(publisher: ParaId, key: Vec, value: Vec) -> Weight; +} +``` + +This design intents to address the issue #606 by reusing existing `cumulus` and relay-chain infrastructure. + + +## Drawbacks + +- The relay-chain state passed during collator block construction increases, and its size and impact must be carefully monitored. +- Processing relay state proofs introduces additional overhead for data extraction and verification in the runtime. +- `KeyToIncludeInRelayProofApi` currently works for child tries using the default route; making it fully generic would require extending `ChildInfo` with `TypeInfo` or a workaround, which should be evaluated separately. + + +## Testing, Security, and Privacy + +### Testing + +The implementation is covered by unit tests that exercise the publish instruction and handler, broadcaster pallet, subscription handling, and relay-state proof processing logic. In addition, a dedicated testing branch provides a full end-to-end reference implementation based on Rococo Relay and testing Parachain, with publisher and subscriber parachains interacting through the relay chain, which can be run locally using Zombienet - https://github.com/blockdeep/polkadot-sdk/tree/feat/pubsub-rev1225-dev + +### Security + +Data consumed by parachains is verified against relay-chain state and validators proof, preventing forgery by collators. + +### Privacy + +Published data is stored in relay-chain child tries and is therefore publicly observable. + +### Implementation pitfalls. + +Care must be taken to avoid requesting keys unnecessarily, as this directly impacts proof size, building and runtime execution cost. + + +## Performance, Ergonomics, and Compatibility + +### Performance + +This proposal is a necessary trade-off rather than a pure optimization, adding relay-chain state proofs and runtime verification to avoid repeated point-to-point XCM requests and off-chain systems. Overhead is limited by proving only explicitly subscribed keys and enforcing strict bounds, while update handling based on child-trie root changes is managed by the Subscriber. + +Does not present any overhead when not implemented. Runtimes can decide simply not to implement the API and pallet broadcaster rr subscriber and there will be no performance impact. + +### Ergonomics + +Runtime developers interact with the mechanism through explicit runtime configuration. + +### Compatibility + +The proposal does not break existing XCM semantics or runtime interfaces. All changes are additive and opt-in, allowing parachains to adopt the mechanism incrementally. + +## Future Directions and Related Material + +This RFC enables a range of follow-up work around scalable cross-parachain data distribution. A straightforward application is price oracles, and there has also been interest in using the mechanism for broader state propagation patterns, including bridging-related operations. + +At the same time, this RFC represents an initial design. Additional use cases, optimal configurations, limitations, and alternative patterns are expected to emerge as the mechanism is adopted and tested in practice. + +Future RFCs may refine the API surface, extend child-trie handling, introduce additional safeguards or ergonomics, storage management, optimize proof generation and processing, or specialize the mechanism for particular use cases. From 803ab577c19db6ad63892c1fcebd570df15b5b6e Mon Sep 17 00:00:00 2001 From: metricaez Date: Mon, 19 Jan 2026 09:58:30 -0300 Subject: [PATCH 2/7] feat: better naming and design suggestions --- text/0160-pub-sub-mechanism.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md index 28d6d7aab..683f143a5 100644 --- a/text/0160-pub-sub-mechanism.md +++ b/text/0160-pub-sub-mechanism.md @@ -8,11 +8,11 @@ ## Summary -Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via verified relay chain storage proofs. +Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via relay chain storage proofs. ## Motivation -As pointed out in issue #606, when parachains need to exchange information, the only available options are repeated point-to-point XCM messages or ad-hoc off-chain infrastructure. This RFC is motivated by the desire to propose an alternative based on a relay-assisted publish/subscribe mechanism that enables one-to-many, verifiable, cross-parachain data distribution. +As pointed out in issue #606 (https://github.com/paritytech/polkadot-sdk/issues/606), when parachains need to exchange information, the only available options are repeated point-to-point XCM messages or ad-hoc off-chain infrastructure. This RFC is motivated by the desire to propose an alternative based on a relay-assisted publish/subscribe mechanism that enables one-to-many, verifiable, cross-parachain data distribution. ## Stakeholders @@ -24,8 +24,7 @@ This RFC defines a relay-assisted publish/subscribe mechanism that enables parac Parachains publish data via a new XCM v5 `Publish` instruction. Published entries are stored on the relay chain in a per-publisher child trie managed by the broadcaster pallet. Publishers must be registered, and both the number of keys and their sizes are bounded to prevent unbounded state growth. Each published item is addressed by a fixed-size key. The suggestion is to backport the new instruction to v5 as v6 discussion advances. ```rust -Publish { data: PublishData }  -PublishData = BoundedVec<(PublishKey, BoundedVec), MaxPublishItems> +Publish { key: PublishKey, data: BoundedVec, ttl: u32 }  ``` On the consumer side, subscribing parachains declare their interests at the runtime level by specifying which publishers and keys they want to subscribe to. This information is exposed to the collator through a new dedicated runtime API (`KeyToIncludeInRelayProofApi`), allowing the collator to know which relay-chain storage keys must be proven for a given block. During block production, the collator fetches relay-chain state proofs for those keys, including child-trie entries, and embeds them in the ParachainInherentData. @@ -42,9 +41,9 @@ pub enum RelayStorageKey { /// Prefix `:child_storage:default:` is added when accessing storage. /// Used to derive `ChildInfo` for reading child trie data. storage_key: Vec, - /// Key within the child trie. + trie_key: Vec, key: Vec, - }, + item_key: Vec, } /// Request for proving relay chain storage data. @@ -58,7 +57,7 @@ pub struct RelayProofRequest { } /// API for specifying which relay chain storage data to include in storage proofs. -/// +/// Runtime API for specifying which relay chain storage data to include in storage proofs. /// This API allows parachains to request both top-level relay chain storage keys /// and child trie storage keys to be included in the relay chain state proof. pub trait KeyToIncludeInRelayProofApi { @@ -160,3 +159,4 @@ This RFC enables a range of follow-up work around scalable cross-parachain data At the same time, this RFC represents an initial design. Additional use cases, optimal configurations, limitations, and alternative patterns are expected to emerge as the mechanism is adopted and tested in practice. Future RFCs may refine the API surface, extend child-trie handling, introduce additional safeguards or ergonomics, storage management, optimize proof generation and processing, or specialize the mechanism for particular use cases. +Future RFCs may refine the API surface, extend child-trie handling, introduce additional safeguards or ergonomics, storage management, optimize proof generation and processing, or specialize the mechanism for particular use cases. \ No newline at end of file From 940a5fce8ba053566529a640e69953c53119ff0c Mon Sep 17 00:00:00 2001 From: metricaez Date: Wed, 21 Jan 2026 10:39:04 -0300 Subject: [PATCH 3/7] fix: typo --- text/0160-pub-sub-mechanism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md index 683f143a5..8d13fbce6 100644 --- a/text/0160-pub-sub-mechanism.md +++ b/text/0160-pub-sub-mechanism.md @@ -142,7 +142,7 @@ Care must be taken to avoid requesting keys unnecessarily, as this directly impa This proposal is a necessary trade-off rather than a pure optimization, adding relay-chain state proofs and runtime verification to avoid repeated point-to-point XCM requests and off-chain systems. Overhead is limited by proving only explicitly subscribed keys and enforcing strict bounds, while update handling based on child-trie root changes is managed by the Subscriber. -Does not present any overhead when not implemented. Runtimes can decide simply not to implement the API and pallet broadcaster rr subscriber and there will be no performance impact. +Does not present any overhead when not implemented. Runtimes can decide simply not to implement the API and pallet broadcaster nor subscriber and there will be no performance impact. ### Ergonomics From 426166d97a4dae2f3e00029e1bd039000d284958 Mon Sep 17 00:00:00 2001 From: metricaez Date: Wed, 21 Jan 2026 11:13:12 -0300 Subject: [PATCH 4/7] fix: RelayStorageKey Child key naming --- text/0160-pub-sub-mechanism.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md index 8d13fbce6..92baeb10f 100644 --- a/text/0160-pub-sub-mechanism.md +++ b/text/0160-pub-sub-mechanism.md @@ -40,10 +40,9 @@ pub enum RelayStorageKey { /// Unprefixed storage key identifying the child trie root location. /// Prefix `:child_storage:default:` is added when accessing storage. /// Used to derive `ChildInfo` for reading child trie data. - storage_key: Vec, trie_key: Vec, - key: Vec, item_key: Vec, + } } /// Request for proving relay chain storage data. From 5a62b109d27049e0cb2c4bb9414e99f30388df84 Mon Sep 17 00:00:00 2001 From: metricaez Date: Wed, 21 Jan 2026 11:14:44 -0300 Subject: [PATCH 5/7] choir: add comment --- text/0160-pub-sub-mechanism.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md index 92baeb10f..2cda3b36e 100644 --- a/text/0160-pub-sub-mechanism.md +++ b/text/0160-pub-sub-mechanism.md @@ -41,6 +41,7 @@ pub enum RelayStorageKey { /// Prefix `:child_storage:default:` is added when accessing storage. /// Used to derive `ChildInfo` for reading child trie data. trie_key: Vec, + /// Key within the child trie. item_key: Vec, } } From 61e9c9ba14533ee09a0d241196a599f20142c5e7 Mon Sep 17 00:00:00 2001 From: Emiliano <84690100+metricaez@users.noreply.github.com> Date: Wed, 21 Jan 2026 11:23:54 -0300 Subject: [PATCH 6/7] choir: spacing and wording from review Co-authored-by: Francisco Aguirre --- text/0160-pub-sub-mechanism.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md index 2cda3b36e..a21c4b665 100644 --- a/text/0160-pub-sub-mechanism.md +++ b/text/0160-pub-sub-mechanism.md @@ -56,7 +56,7 @@ pub struct RelayProofRequest { pub keys: Vec, } -/// API for specifying which relay chain storage data to include in storage proofs. +/// Runtime API for specifying which relay chain storage data to include in storage proofs. /// Runtime API for specifying which relay chain storage data to include in storage proofs. /// This API allows parachains to request both top-level relay chain storage keys /// and child trie storage keys to be included in the relay chain state proof. @@ -158,5 +158,5 @@ This RFC enables a range of follow-up work around scalable cross-parachain data At the same time, this RFC represents an initial design. Additional use cases, optimal configurations, limitations, and alternative patterns are expected to emerge as the mechanism is adopted and tested in practice. -Future RFCs may refine the API surface, extend child-trie handling, introduce additional safeguards or ergonomics, storage management, optimize proof generation and processing, or specialize the mechanism for particular use cases. +Future RFCs may refine the API surface, extend child-trie handling, introduce additional safeguards or ergonomics, storage management, optimize proof generation and processing, or specialize the mechanism for particular use cases. Future RFCs may refine the API surface, extend child-trie handling, introduce additional safeguards or ergonomics, storage management, optimize proof generation and processing, or specialize the mechanism for particular use cases. \ No newline at end of file From d8c615ec03380c62f07e959ab6b5d3cd9044e328 Mon Sep 17 00:00:00 2001 From: metricaez Date: Wed, 21 Jan 2026 11:27:12 -0300 Subject: [PATCH 7/7] choir: remove irrelevant text --- text/0160-pub-sub-mechanism.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0160-pub-sub-mechanism.md b/text/0160-pub-sub-mechanism.md index a21c4b665..3900fd668 100644 --- a/text/0160-pub-sub-mechanism.md +++ b/text/0160-pub-sub-mechanism.md @@ -114,7 +114,6 @@ This design intents to address the issue #606 by reusing existing `cumulus` and - The relay-chain state passed during collator block construction increases, and its size and impact must be carefully monitored. - Processing relay state proofs introduces additional overhead for data extraction and verification in the runtime. -- `KeyToIncludeInRelayProofApi` currently works for child tries using the default route; making it fully generic would require extending `ChildInfo` with `TypeInfo` or a workaround, which should be evaluated separately. ## Testing, Security, and Privacy