refactor: Ensure event and error processing backward and forward compatibility

AGENTS.md

@@ -120,6 +120,22 @@ The client uses an actor-based architecture (`/client/actors-framework`) with sp
    - Run relevant tests for your changes
    - Update TypeScript types if runtime APIs changed: `pnpm typegen`
 
+## Event/Error Encoding Stability
+
+StorageHub pallets enforce strict encoding stability rules to ensure clients can decode events and errors across runtime upgrades. **These rules are critical.**
+
+### Rules for StorageHub Pallets
+
+The following pallets have encoding stability requirements: `Providers`, `FileSystem`, `ProofsDealer`, `Randomness`, `PaymentStreams`, `BucketNfts`.
+
+1. **Pallet indices are immutable**: The `#[runtime::pallet_index(N)]` value must never change once deployed.
+
+2. **Event/error variant indices are pinned**: All event and error variants use explicit `#[codec(index = N)]` attributes. These indices must never change or be reused.
+
+3. **Field signatures are stable**: The fields (types, count, order) of existing event/error variants must never change.
+
+4. **Breaking changes require new variants**: If you need to change an event or error, add a new variant with a `Vx` suffix (e.g., `NewStorageRequestV2`) using the next available index. Keep the old variant for backward compatibility.
+
 ## Key Development Notes
 
 - The project uses a monorepo structure with both Rust (Cargo workspace) and TypeScript (pnpm workspace)

CLAUDE.md

@@ -120,6 +120,22 @@ The client uses an actor-based architecture (`/client/actors-framework`) with sp
    - Run relevant tests for your changes
    - Update TypeScript types if runtime APIs changed: `pnpm typegen`
 
+## Event/Error Encoding Stability
+
+StorageHub pallets enforce strict encoding stability rules to ensure clients can decode events and errors across runtime upgrades. **These rules are critical.**
+
+### Rules for StorageHub Pallets
+
+The following pallets have encoding stability requirements: `Providers`, `FileSystem`, `ProofsDealer`, `Randomness`, `PaymentStreams`, `BucketNfts`.
+
+1. **Pallet indices are immutable**: The `#[runtime::pallet_index(N)]` value must never change once deployed.
+
+2. **Event/error variant indices are pinned**: All event and error variants use explicit `#[codec(index = N)]` attributes. These indices must never change or be reused.
+
+3. **Field signatures are stable**: The fields (types, count, order) of existing event/error variants must never change.
+
+4. **Breaking changes require new variants**: If you need to change an event or error, add a new variant with a `Vx` suffix (e.g., `NewStorageRequestV2`) using the next available index. Keep the old variant for backward compatibility.
+
 ## Key Development Notes
 
 - The project uses a monorepo structure with both Rust (Cargo workspace) and TypeScript (pnpm workspace)

api-augment/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@storagehub/api-augment",
-  "version": "0.3.2",
+  "version": "0.3.3",
   "description": "",
   "scripts": {
     "scrape": "pnpm tsx scripts/scrapeMetadata.ts",

client/common/src/blockchain_utils.rs

@@ -51,35 +51,54 @@ pub enum ModuleErrorDecodeError {
 
 /// Get the events storage element for a given block.
 ///
-/// TODO: Version-aware decoding should be implemented in the client layer in the future to support
-/// processing blocks from different runtime versions.
+/// # Event Decoding Strategy
+///
+/// This function decodes `System.Events` locally using the compiled runtime's event types.
+/// If a runtime upgrade introduces breaking event changes, the SCALE decode will fail and return an error.
+///
+/// To ensure backward compatibility for historical blocks, Runtimes which implement StorageHub pallets _**must**_ have fixed pallet indices via `#[runtime::pallet_index(N)]`
+/// but this cannot be enforced at the StorageHub pallet level.
+///
+/// StorageHub pallets do enforce the following constraints on event and error variants:
+/// - Fixed variant indices via `#[codec(index = N)]`
+/// - Append-only variants (breaking changes use `Vx` suffixes)
+///
+/// # Errors
+///
+/// Returns `EventsRetrievalError::DecodeError` if the events payload exists but cannot be
+/// SCALE-decoded into `StorageHubEventsVec<Runtime>`. A decode failure likely indicates an
+/// incompatible runtime upgrade and the client may need to be upgraded.
 pub fn get_events_at_block<Runtime: StorageEnableRuntime>(
     client: &Arc<StorageHubClient<Runtime::RuntimeApi>>,
     block_hash: &H256,
 ) -> Result<StorageHubEventsVec<Runtime>, EventsRetrievalError> {
     // Get the events storage.
-    let raw_storage_opt = client.storage(*block_hash, &StorageKey(EVENTS_STORAGE_KEY.clone()))?;
+    let raw_storage = client
+        .storage(*block_hash, &StorageKey(EVENTS_STORAGE_KEY.clone()))?
+        .ok_or(EventsRetrievalError::StorageNotFound)?;
 
-    // Decode the events storage.
-    raw_storage_opt
-        .map(|raw_storage| StorageHubEventsVec::<Runtime>::decode(&mut raw_storage.0.as_slice()))
-        .transpose()?
-        .ok_or(EventsRetrievalError::StorageNotFound)
+    StorageHubEventsVec::<Runtime>::decode(&mut raw_storage.0.as_slice()).map_err(|e| {
+        error!(
+            target: "blockchain-utils",
+            "Failed to decode System.Events at block {:?}. This likely indicates a breaking change in a possible runtime upgrade since an event was likely 
+            added or even worse an existing event was removed or updated and cannot be decoded. Underlying error: {:?}",
+            block_hash, e
+        );
+        EventsRetrievalError::DecodeError(e)
+    })
 }
 
 /// Decode a [`sp_runtime::ModuleError`] into [`StorageEnableErrors`].
 ///
 /// This uses compile-time pallet indices to determine which pallet the error originated from,
 /// then decodes the error bytes into the appropriate error variant.
 ///
-/// # Note
+/// # Error Decoding Strategy
 ///
-/// This uses compile-time pallet indices. For version-aware decoding (processing blocks
-/// from different runtime versions), metadata-based decoding should be implemented in
-/// the client layer in the future.
+/// This function relies on fixed pallet indices and error variant indices to
+/// decode the error into the appropriate error variant.
 ///
-/// TODO: Version-aware decoding should be implemented in the client layer in the future to support
-/// processing blocks from different runtime versions.
+/// If a runtime upgrade changes the ordering of error variants or removes an error variant, decoding will fail.
 pub fn decode_module_error<Runtime: StorageEnableRuntime>(
     module_error: sp_runtime::ModuleError,
 ) -> Result<StorageEnableErrors<Runtime>, ModuleErrorDecodeError> {

client/src/tasks/msp_upload_file.rs

@@ -1495,12 +1495,9 @@ where
                 match decode_module_error::<Runtime>(module_error) {
                     Ok(decoded) => Some(decoded),
                     Err(e) => {
-                        error!(
-                            target: LOG_TARGET,
-                            "Failed to decode module error: {:?}",
-                            e
-                        );
-                        return Err(anyhow!("Failed to decode module error: {:?}", e));
+                        let err_msg = format!("Failed to decode module error. This likely indicates a breaking change in a possible runtime upgrade since a new error variant was encountered and cannot be decoded. Underlying error: {:?}", e);
+                        error!(target: LOG_TARGET, "{}", err_msg);
+                        return Err(anyhow!(err_msg));
                     }
                 }
             }

pallets/bucket-nfts/src/lib.rs

@@ -63,38 +63,58 @@ pub mod pallet {
     #[pallet::pallet]
     pub struct Pallet<T>(_);
 
+    /// # Event Encoding/Decoding Stability
+    ///
+    /// All event variants use explicit `#[codec(index = N)]` to ensure stable SCALE encoding/decoding
+    /// across runtime upgrades.
+    ///
+    /// These indices must NEVER be changed or reused. Any breaking changes to errors must be
+    /// introduced as new variants (append-only) to ensure backward and forward compatibility.
     #[pallet::event]
     #[pallet::generate_deposit(pub(super) fn deposit_event)]
     pub enum Event<T: Config> {
         /// Notifies that access to a bucket has been shared with another account.
+        #[codec(index = 0)]
         AccessShared {
             issuer: T::AccountId,
             recipient: AccountIdLookupTargetOf<T>,
         },
         /// Notifies that the read access for an item has been updated.
+        #[codec(index = 1)]
         ItemReadAccessUpdated {
             admin: T::AccountId,
             bucket: BucketIdFor<T>,
             item_id: T::ItemId,
         },
         /// Notifies that an item has been burned.
+        #[codec(index = 2)]
         ItemBurned {
             account: T::AccountId,
             bucket: BucketIdFor<T>,
             item_id: T::ItemId,
         },
     }
 
-    // Errors inform users that something went wrong.
+    /// # Error Encoding/Decoding Stability
+    ///
+    /// All error variants use explicit `#[codec(index = N)]` to ensure stable SCALE encoding/decoding
+    /// across runtime upgrades.
+    ///
+    /// These indices must NEVER be changed or reused. Any breaking changes to errors must be
+    /// introduced as new variants (append-only) to ensure backward and forward compatibility.
     #[pallet::error]
     pub enum Error<T> {
         /// Bucket is not private. Call `update_bucket_privacy` from the file system pallet to make it private.
+        #[codec(index = 0)]
         BucketIsNotPrivate,
         /// Account is not the owner of the bucket.
+        #[codec(index = 1)]
         NotBucketOwner,
         /// No collection corresponding to the bucket. Call `update_bucket_privacy` from the file system pallet to make it private.
+        #[codec(index = 2)]
         NoCorrespondingCollection,
         /// Failed to convert bytes to `BoundedVec`
+        #[codec(index = 3)]
         ConvertBytesToBoundedVec,
     }