RFC-0160: PubSub Mechanism #160

metricaez · 2025-12-18T21:19:40Z

This RFC proposes a relay-assisted publish/subscribe mechanism that enables parachains to share arbitrary key-value data through the relay chain without relying on repeated XCM messages or off-chain protocols.

bkchr · 2026-01-14T14:14:47Z

text/0160-pub-sub-mechanism.md

+
+## Summary
+
+Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via verified relay chain storage proofs. 


Suggested change

Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via verified relay chain storage proofs.

Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via relay chain storage proofs.

bkchr · 2026-01-14T14:15:23Z

text/0160-pub-sub-mechanism.md

+
+## Motivation
+
+As pointed out in issue #606, when parachains need to exchange information, the only available options are repeated point-to-point XCM messages or ad-hoc off-chain infrastructure. This RFC is motivated by the desire to propose an alternative based on a relay-assisted publish/subscribe mechanism that enables one-to-many, verifiable, cross-parachain data distribution.


Please provide a full link to the issue.

bkchr · 2026-01-14T21:56:08Z

text/0160-pub-sub-mechanism.md

+Publish { data: PublishData } 
+PublishData = BoundedVec<(PublishKey, BoundedVec<u8, MaxPublishValueLength>), MaxPublishItems>


Suggested change

Publish { data: PublishData }

PublishData = BoundedVec<(PublishKey, BoundedVec<u8, MaxPublishValueLength>), MaxPublishItems>

Publish { key: PublishKey, data: BoundedVec<u8, MaxPublishValueLength>, ttl: u32 }

No need to publish multiple things at once and if this is needed, these messages can just be batched.

ttl 0 means infinite and otherwise the number in blocks the data is valid. In the end it is up to the subscriber to decide if they still process the data or not.

Yeah we could include multiple of these instructions in one same XCM

bkchr · 2026-01-14T21:58:09Z

text/0160-pub-sub-mechanism.md

+## Explanation
+
+This RFC defines a relay-assisted publish/subscribe mechanism that enables parachains to share arbitrary key-value data through the relay chain without relying on repeated XCM messages or off-chain protocols. The core idea is to use the relay chain as a common, verifiable distribution layer, allowing data to be published once and consumed by many parachains with bounded and predictable costs.
+Parachains publish data via a new XCM v5 `Publish` instruction. Published entries are stored on the relay chain in a per-publisher child trie managed by the broadcaster pallet. Publishers must be registered, and both the number of keys and their sizes are bounded to prevent unbounded state growth. Each published item is addressed by a fixed-size key. The suggestion is to backport the new instruction to v5 as v6 discussion advances.


Key doesn't need to be fixed size.

"Each published item is addressed by a size-capped key." ?

bkchr · 2026-01-14T21:59:00Z

text/0160-pub-sub-mechanism.md

+On the consumer side, subscribing parachains declare their interests at the runtime level by specifying which publishers and keys they want to subscribe to. This information is exposed to the collator through a new dedicated runtime API (`KeyToIncludeInRelayProofApi`), allowing the collator to know which relay-chain storage keys must be proven for a given block. During block production, the collator fetches relay-chain state proofs for those keys, including child-trie entries, and embeds them in the ParachainInherentData.
+
+```rust
+/// A relay chain storage key to be included in the storage proof.
+#[derive(Clone, Debug, Encode, Decode, TypeInfo, PartialEq, Eq)]
+pub enum RelayStorageKey {
+	/// Top-level relay chain storage key.
+	Top(Vec<u8>),
+	/// Child trie storage key.
+	Child {
+		/// Unprefixed storage key identifying the child trie root location.
+		/// Prefix `:child_storage:default:` is added when accessing storage.
+		/// Used to derive `ChildInfo` for reading child trie data.
+		storage_key: Vec<u8>,
+		/// Key within the child trie.
+		key: Vec<u8>,
+	},
+}
+
+/// Request for proving relay chain storage data.
+///
+/// Contains a list of storage keys (either top-level or child trie keys)
+/// to be included in the relay chain state proof.
+#[derive(Clone, Debug, Encode, Decode, TypeInfo, PartialEq, Eq, Default)]
+pub struct RelayProofRequest {
+	/// Storage keys to include in the relay chain state proof.
+	pub keys: Vec<RelayStorageKey>,
+}
+
+/// API for specifying which relay chain storage data to include in storage proofs.
+///
+/// This API allows parachains to request both top-level relay chain storage keys
+/// and child trie storage keys to be included in the relay chain state proof.
+pub trait KeyToIncludeInRelayProofApi {
+    /// Returns relay chain storage proof requests.
+    ///
+    /// The returned `RelayProofRequest` contains a list of storage keys where each key
+    /// can be either:
+    /// - `RelayStorageKey::Top`: Top-level relay chain storage key
+    /// - `RelayStorageKey::Child`: Child trie storage, containing the child trie identifier
+    ///   and the key to prove from that child trie
+    ///
+    /// The collator generates proofs for these and includes them in the relay chain state proof.
+    fn keys_to_prove() -> RelayProofRequest;
+}
+```
+
+The parachain verifies the relay-chain state proofs as part of `set_validation_data` of `parachain-system`. A processing hook extracts the relevant key–value pairs from the proofs and only triggers updates when the corresponding child-trie root has changed since the previous block, avoiding redundant child trie data extractions which are the most significant computation-demanding process of the feature. Verified updates are then delivered to the runtime through a subscription handler, enabling consumption of cross-chain data.
+
+```rust
+/// Processor for relay chain proof keys.
+///
+/// This allows parachains to process data from the relay chain state proof,
+/// including both child trie keys and main trie keys that were requested
+/// via `KeyToIncludeInRelayProofApi`.
+type RelayProofKeysProcessor: ProcessRelayProofKeys;
+
+/// Process keys from verified relay chain state proofs.
+///
+/// This trait allows processing of relay chain storage data from the verified proof.
+pub trait ProcessRelayProofKeys {
+	/// Process keys from a verified relay state proof.
+	fn process_relay_proof_keys(verified_proof: &RelayChainStateProof) -> Weight;
+}
+
+```
+
+```rust
+/// Define subscriptions and handle received data.
+pub trait SubscriptionHandler {
+	/// List of subscriptions as (ParaId, keys) tuples.
+	/// Returns (subscriptions, weight) where weight is the cost of computing the subscriptions.
+	fn subscriptions() -> (Vec<(ParaId, Vec<Vec<u8>>)>, Weight);
+
+	/// Called when subscribed data is updated.
+	/// Returns the weight consumed by processing the data.
+	fn on_data_updated(publisher: ParaId, key: Vec<u8>, value: Vec<u8>) -> Weight;
+}
+```
+
+This design intents to address the issue #606 by reusing existing `cumulus` and relay-chain infrastructure.


IMO not important for the XCM changes here. Because these are just implementation details.

can you clarify this one ?

bkchr · 2026-01-14T21:59:25Z

text/0160-pub-sub-mechanism.md

+
+- The relay-chain state passed during collator block construction increases, and its size and impact must be carefully monitored.
+- Processing relay state proofs introduces additional overhead for data extraction and verification in the runtime.
+- `KeyToIncludeInRelayProofApi` currently works for child tries using the default route; making it fully generic would require extending `ChildInfo` with `TypeInfo` or a workaround, which should be evaluated separately.


Suggested change

- `KeyToIncludeInRelayProofApi` currently works for child tries using the default route; making it fully generic would require extending `ChildInfo` with `TypeInfo` or a workaround, which should be evaluated separately.

franciscoaguirre · 2026-01-16T20:48:34Z

text/0160-pub-sub-mechanism.md

+## Explanation
+
+This RFC defines a relay-assisted publish/subscribe mechanism that enables parachains to share arbitrary key-value data through the relay chain without relying on repeated XCM messages or off-chain protocols. The core idea is to use the relay chain as a common, verifiable distribution layer, allowing data to be published once and consumed by many parachains with bounded and predictable costs.
+Parachains publish data via a new XCM v5 `Publish` instruction. Published entries are stored on the relay chain in a per-publisher child trie managed by the broadcaster pallet. Publishers must be registered, and both the number of keys and their sizes are bounded to prevent unbounded state growth. Each published item is addressed by a fixed-size key. The suggestion is to backport the new instruction to v5 as v6 discussion advances.


We can't add new instructions to XCMv5, it would have to go into v6.
Data can be published via Transact meanwhile.

franciscoaguirre · 2026-01-16T20:49:12Z

text/0160-pub-sub-mechanism.md

+Publish { data: PublishData } 
+PublishData = BoundedVec<(PublishKey, BoundedVec<u8, MaxPublishValueLength>), MaxPublishItems>


Yeah we could include multiple of these instructions in one same XCM

franciscoaguirre · 2026-01-16T20:50:17Z

text/0160-pub-sub-mechanism.md

+
+```rust
+Publish { data: PublishData } 
+PublishData = BoundedVec<(PublishKey, BoundedVec<u8, MaxPublishValueLength>), MaxPublishItems>


PublishKey is not defined?

franciscoaguirre · 2026-01-16T20:51:55Z

text/0160-pub-sub-mechanism.md

+		/// Unprefixed storage key identifying the child trie root location.
+		/// Prefix `:child_storage:default:` is added when accessing storage.
+		/// Used to derive `ChildInfo` for reading child trie data.
+		storage_key: Vec<u8>,


Suggested change

storage_key: Vec<u8>,

trie_key: Vec<u8>,

franciscoaguirre · 2026-01-16T20:52:04Z

text/0160-pub-sub-mechanism.md

+		/// Used to derive `ChildInfo` for reading child trie data.
+		storage_key: Vec<u8>,
+		/// Key within the child trie.
+		key: Vec<u8>,


Suggested change

key: Vec<u8>,

item_key: Vec<u8>,

text/0160-pub-sub-mechanism.md

franciscoaguirre · 2026-01-16T20:58:44Z

text/0160-pub-sub-mechanism.md

+
+This proposal is a necessary trade-off rather than a pure optimization, adding relay-chain state proofs and runtime verification to avoid repeated point-to-point XCM requests and off-chain systems. Overhead is limited by proving only explicitly subscribed keys and enforcing strict bounds, while update handling based on child-trie root changes is managed by the Subscriber.
+
+Does not present any overhead when not implemented. Runtimes can decide simply not to implement the API and pallet broadcaster rr subscriber and there will be no performance impact. 


text/0160-pub-sub-mechanism.md

franciscoaguirre · 2026-01-16T21:00:59Z

text/0160-pub-sub-mechanism.md

+## Explanation
+
+This RFC defines a relay-assisted publish/subscribe mechanism that enables parachains to share arbitrary key-value data through the relay chain without relying on repeated XCM messages or off-chain protocols. The core idea is to use the relay chain as a common, verifiable distribution layer, allowing data to be published once and consumed by many parachains with bounded and predictable costs.
+Parachains publish data via a new XCM v5 `Publish` instruction. Published entries are stored on the relay chain in a per-publisher child trie managed by the broadcaster pallet. Publishers must be registered, and both the number of keys and their sizes are bounded to prevent unbounded state growth. Each published item is addressed by a fixed-size key. The suggestion is to backport the new instruction to v5 as v6 discussion advances.


I presume publishers should put down a deposit when publishing to the relay?

They must register as publishers and reserve a deposit on the relay.

bkchr · 2026-01-21T12:47:09Z

text/0160-pub-sub-mechanism.md

+Parachains publish data via a new XCM v5 `Publish` instruction. Published entries are stored on the relay chain in a per-publisher child trie managed by the broadcaster pallet. Publishers must be registered, and both the number of keys and their sizes are bounded to prevent unbounded state growth. Each published item is addressed by a fixed-size key. The suggestion is to backport the new instruction to v5 as v6 discussion advances.
+
+```rust
+Publish { key: PublishKey, data: BoundedVec<u8, MaxPublishValueLength>, ttl: u32 } 


We should probably make PublishKey directly a Hash. This saves bandwidth and computation time on the relay chain.

WDYT @franciscoaguirre?

Co-authored-by: Francisco Aguirre <[email protected]>

burdges · 2026-01-21T19:33:39Z

At some level scalability requires off-chain infrastructure, but whether you need this depends upon how much you & everyone else sends.

An XCM message goes into relay chain blocks, which then get synced by every RC node and every collator of every parachain. It follows that XCM protocols are always more global broadcast announcements to all nodes of all parachains and XCM protocol being called "pub-sub" feels dubious to me.

All parachains place their state roots into their candidate receipts, so message can authenticate via some method like relay parents. If you want long delays and accept delivery failures then you can simply insert a proof of some parachain data into the RC DHT, so then subscribing means checking the right DHT locations. If you want more assurances, then we can have gossip protocols that say which messages go where, but how you want RC nodes involved changes things.

Anyways XCM is a protocol that works, so use it when you need it. Also XCMs having a recipient group is much less stupid than XCMs having only one recipient, so this maybe a major improvement over the current XCM. I anyways wanted to point out that the "subscribe" part feels a bit overstated here.

feat: pub-sub-rfc

f2ea4ac

anaelleltd added the Proposed Is awaiting 3 formal reviews. label Dec 23, 2025

bkchr reviewed Jan 14, 2026

View reviewed changes

franciscoaguirre reviewed Jan 16, 2026

View reviewed changes

feat: better naming and design suggestions

803ab57

bkchr reviewed Jan 21, 2026

View reviewed changes

metricaez and others added 5 commits January 21, 2026 10:39

fix: typo

940a5fc

fix: RelayStorageKey Child key naming

426166d

choir: add comment

5a62b10

choir: spacing and wording from review

61e9c9b

Co-authored-by: Francisco Aguirre <[email protected]>

choir: remove irrelevant text

d8c615e


		## Summary

		Add a publish/subscribe mechanism that allows parachains to publish data to the relay chain and other parachains to consume it via verified relay chain storage proofs.


		## Motivation

		As pointed out in issue #606, when parachains need to exchange information, the only available options are repeated point-to-point XCM messages or ad-hoc off-chain infrastructure. This RFC is motivated by the desire to propose an alternative based on a relay-assisted publish/subscribe mechanism that enables one-to-many, verifiable, cross-parachain data distribution.

		Publish { data: PublishData }
		PublishData = BoundedVec<(PublishKey, BoundedVec<u8, MaxPublishValueLength>), MaxPublishItems>

	Publish { data: PublishData }
	PublishData = BoundedVec<(PublishKey, BoundedVec<u8, MaxPublishValueLength>), MaxPublishItems>
	Publish { key: PublishKey, data: BoundedVec<u8, MaxPublishValueLength>, ttl: u32 }


		This proposal is a necessary trade-off rather than a pure optimization, adding relay-chain state proofs and runtime verification to avoid repeated point-to-point XCM requests and off-chain systems. Overhead is limited by proving only explicitly subscribed keys and enforcing strict bounds, while update handling based on child-trie root changes is managed by the Subscriber.

		Does not present any overhead when not implemented. Runtimes can decide simply not to implement the API and pallet broadcaster rr subscriber and there will be no performance impact.

RFC-0160: PubSub Mechanism #160

Are you sure you want to change the base?

RFC-0160: PubSub Mechanism #160

Uh oh!

Conversation

metricaez commented Dec 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

burdges commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

burdges commented Jan 21, 2026 •

edited

Loading