refactor(storage): enforce API limits via stackable guard layer (#1630)

Replace the transactions-only WithMax* opts with a single, stackable guard
decorator layer applied service-wide at the multiplexed driver seam, addressing
review feedback that the per-constructor opts approach did not scale.

- New token/services/storage/db/guard package: Policy (loaded once from
  token.storage.maxPayloadSize / maxPageSize), payload-size checks, a
  row-capping LimitIterator, and interface-embedding decorators for the
  transaction, token, endorser, identity, and wallet stores.
- multiplexed.Driver wraps each NewXxx result with the guard, so every backing
  driver (sqlite, postgres) and store is covered by one mechanism.
- Remove the maxPayloadSize/maxPageSize fields, WithMax* options and inline
  checks from the SQL transaction store and its config/driver wiring.
- pagination.ValidateLimited now also caps keyset page size and guards a
  typed-nil offset.
- Writes (payload size) and streaming/paginated reads (row cap) are guarded;
  QueryMovements stays uncapped (balance totals). Materialized slice/map reads
  are a tracked follow-up.
- Update storage/configuration docs to describe the service-wide limits.

Signed-off-by: AkramBitar <akram@il.ibm.com>

Author: AkramBitar

docs/configuration.md

@@ -589,10 +589,10 @@ to `<params>_<short_code>` (params still apply when provided; short-code overrid
 
 ### Optional: token.storage.maxPayloadSize
 
-Maximum serialised size, in bytes, accepted by a single transaction-store write
-(`AddTokenRequest`, `AddTransaction`, `AddMovement`). Writes whose payload exceeds
-this size are rejected before reaching the database. A value of `0` disables the
-check. When the key is absent, the store default (4 MiB) applies.
+Maximum serialised size, in bytes, accepted by a single storage-service write
+(applies across the transaction, token, endorser, identity, and wallet stores).
+Writes whose payload exceeds this size are rejected before reaching the database.
+A value of `0` disables the check. When the key is absent, the default (4 MiB) applies.
 
 ```yaml
 token:
@@ -604,19 +604,18 @@ token:
 
 ### Optional: token.storage.maxPageSize
 
-Maximum number of rows a single transaction-store read may return. It limits the
-pages accepted by transaction queries and caps token-request queries so an
-unlimited scan cannot exhaust database resources. When the key is absent, the
-store default (1000) applies.
+Maximum number of rows a single storage-service read may return. It bounds the pages
+accepted by paginated queries and caps streaming iterator reads so an unlimited scan
+cannot exhaust database resources. When the key is absent, the default (1000) applies.
 
 ```yaml
 token:
   storage:
     maxPageSize: 1000
 ```
 
-See [Transaction Store API Limits](services/storage.md#transaction-store-api-limits)
-for details, including why movement/balance queries are intentionally not row-capped.
+See [Storage API Limits](services/storage.md#storage-api-limits) for details,
+including why movement/balance queries are intentionally not row-capped.
 
 ---
 

docs/services/storage.md

@@ -315,28 +315,50 @@ Cleanup behavior is controlled by the configuration section. See the [Configurat
 
 See the [Configuration Guide](../configuration.md), Section `Optional: token.tms.<name>.services.network.fabric.recovery`, for detailed parameter descriptions and tuning recommendations.
 
-## Transaction Store API Limits
-
-The transaction store (used by both **TTXDB** and **AuditDB**) limits its writes and
-reads so one huge write or one runaway query can't eat up the database. The checks
-live in the single SQL implementation
-(`token/services/storage/db/sql/common/transactions.go`), so the owner and auditor
-stores are both covered.
-
-**Write size limit.** `AddTokenRequest`, `AddTransaction`, and `AddMovement` reject a
-call whose payload is larger than `maxPayloadSize` (default 4 MiB; `0` turns the check
-off). `AddTokenRequest` measures the raw token request plus its metadata;
-transaction and movement records add up their field lengths instead, to keep the
-write path fast.
-
-**Read size limits.** `QueryTransactions` needs a page with a size limit (`nil` and
-`pagination.None()` are rejected; callers page through the full result set).
-`QueryTokenRequests` adds a hard `LIMIT` equal to `maxPageSize` (default 1000).
-`QueryMovements` is left uncapped on purpose: it feeds balance totals, and dropping
-rows there would quietly return wrong balances. It is already narrowed by its required
-filters.
-
-Both limits have built-in defaults. You can override them in configuration
-(`token.storage.maxPayloadSize` / `maxPageSize`) or in code with the
-`WithMaxPayloadSize` / `WithMaxPageSize` store options. See the
-[Configuration Guide](../configuration.md) for the config keys.
+## Storage API Limits
+
+The storage service limits its writes and reads so one huge write or one runaway
+query can't exhaust the database (issue #1630, CWE-400 / CWE-770). The limits are
+enforced by a single, stackable **guard** decorator layer
+(`token/services/storage/db/guard/`) applied once at the multiplexed driver
+(`token/services/storage/db/multiplexed/driver.go`) as each store is created, so
+every backing driver (SQLite, PostgreSQL) and every store is covered by the same
+mechanism. The decorators embed the store interfaces and override only the methods
+that need a check, delegating everything else — new layers (metrics, tracing) can be
+stacked the same way, with the concrete SQL store at the bottom.
+
+A single `Policy` (`MaxPayloadSize`, `MaxPageSize`) drives every check. It is loaded
+once from configuration; absent keys fall back to the built-in defaults, and an
+explicit `0` disables that check.
+
+**Write size limit.** Writes whose serialised payload exceeds `maxPayloadSize`
+(default 4 MiB; `0` disables) are rejected before reaching the database. Covered
+writes include the transaction store's `AddTokenRequest` / `AddTransaction` /
+`AddMovement` / `AddTransactionEndorsementAck`, the token store's `StoreToken` /
+`StorePublicParams` / `StoreCertifications`, the endorser store's
+`AddValidationRecord`, the identity store's `StoreIdentityData` / `StoreSignerInfo` /
+`RegisterIdentityDescriptor` / `AddConfiguration`, and the wallet store's
+`StoreIdentity`.
+
+**Read size limits.**
+- Paginated reads (`QueryTransactions`) require a bounded page: `nil` and
+  `pagination.None()` are rejected, and a page size larger than `maxPageSize`
+  (default 1000) is rejected. Callers page through the full result set.
+- Streaming iterator reads are capped at `maxPageSize` rows via a limiting iterator
+  that errors (rather than silently truncating) once the cap is exceeded. Covered
+  reads include the token store's unspent/spendable/unsupported iterators, the
+  endorser store's `QueryValidations`, the transaction store's `QueryTokenRequests`,
+  and the identity store's `IteratorConfigurations`.
+- `QueryMovements` is left uncapped on purpose: it feeds balance totals, and dropping
+  rows there would quietly return wrong balances. It is already narrowed by its
+  required filters.
+
+You can override the limits in configuration (`token.storage.maxPayloadSize` /
+`token.storage.maxPageSize`) — see the [Configuration Guide](../configuration.md).
+
+**Not yet covered (tracked follow-up).** Reads that materialise a full slice/map
+before returning (e.g. `ListUnspentTokens`, `QueryTokenDetails`,
+`ConfigurationsByID`, `GetWalletIDs`) cannot be bounded by a wrapper alone — the SQL
+has already loaded everything — so they need a SQL-level `LIMIT` or conversion to
+iterators. The opaque `Keystore.Put` value and input-size caps for variadic id lists
+(`DeleteTokens`, `GetTokens`) are in the same follow-up.

token/services/storage/db/guard/decorator_test.go

@@ -0,0 +1,82 @@
+/*
+Copyright IBM Corp. All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package guard_test
+
+import (
+	"database/sql"
+	"testing"
+
+	"github.com/DATA-DOG/go-sqlmock"
+	driver "github.com/LFDT-Panurus/panurus/token/services/storage/db/driver"
+	"github.com/LFDT-Panurus/panurus/token/services/storage/db/guard"
+	sqlcommon "github.com/LFDT-Panurus/panurus/token/services/storage/db/sql/common"
+	"github.com/LFDT-Panurus/panurus/token/services/storage/db/sql/query/pagination"
+	"github.com/stretchr/testify/require"
+)
+
+func ownerStore(t *testing.T, db *sql.DB, p guard.Policy) driver.TokenTransactionStore {
+	t.Helper()
+	store, err := sqlcommon.NewOwnerTransactionStore(db, db, sqlcommon.TableNames{
+		Movements:             "MOVEMENTS",
+		Transactions:          "TRANSACTIONS",
+		Requests:              "REQUESTS",
+		TransactionEndorseAck: "TRANSACTION_ENDORSE_ACK",
+	}, nil, nil)
+	require.NoError(t, err)
+
+	return guard.WrapOwnerTransaction(store, p)
+}
+
+// TestGuardRejectsOversizeWrite verifies the guard rejects an oversize write
+// on the atomic transaction before it reaches the database.
+func TestGuardRejectsOversizeWrite(t *testing.T) {
+	db, mockDB, err := sqlmock.New()
+	require.NoError(t, err)
+
+	store := ownerStore(t, db, guard.Policy{MaxPayloadSize: 20, MaxPageSize: 1000})
+	mockDB.ExpectBegin()
+
+	w, err := store.NewTransactionStoreTransaction()
+	require.NoError(t, err)
+
+	err = w.AddTokenRequest(t.Context(), "tx", make([]byte, 100), nil, nil, nil)
+	require.Error(t, err)
+	require.Contains(t, err.Error(), "exceeds maximum")
+	require.NoError(t, mockDB.ExpectationsWereMet())
+}
+
+// TestGuardDisabledWritePassesThrough verifies a zero payload limit disables
+// the check and the write proceeds to the database.
+func TestGuardDisabledWritePassesThrough(t *testing.T) {
+	db, mockDB, err := sqlmock.New()
+	require.NoError(t, err)
+
+	store := ownerStore(t, db, guard.Policy{MaxPayloadSize: 0, MaxPageSize: 1000})
+	mockDB.ExpectBegin()
+	mockDB.ExpectExec("INSERT INTO REQUESTS").WillReturnResult(sqlmock.NewResult(1, 1))
+
+	w, err := store.NewTransactionStoreTransaction()
+	require.NoError(t, err)
+
+	require.NoError(t, w.AddTokenRequest(t.Context(), "tx", make([]byte, 100), nil, nil, nil))
+	require.NoError(t, mockDB.ExpectationsWereMet())
+}
+
+// TestGuardRejectsUnlimitedQuery verifies the read guard rejects nil and
+// unlimited (None) pagination without querying the database.
+func TestGuardRejectsUnlimitedQuery(t *testing.T) {
+	db, _, err := sqlmock.New()
+	require.NoError(t, err)
+
+	store := ownerStore(t, db, guard.DefaultPolicy())
+
+	_, err = store.QueryTransactions(t.Context(), driver.QueryTransactionsParams{}, nil)
+	require.Error(t, err)
+
+	_, err = store.QueryTransactions(t.Context(), driver.QueryTransactionsParams{}, pagination.None())
+	require.Error(t, err)
+}

token/services/storage/db/guard/endorser.go

@@ -0,0 +1,60 @@
+/*
+Copyright IBM Corp. All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package guard
+
+import (
+	"context"
+
+	tokendriver "github.com/LFDT-Panurus/panurus/token/driver"
+	driver "github.com/LFDT-Panurus/panurus/token/services/storage/db/driver"
+)
+
+// WrapEndorser wraps an endorser store with the guard policy.
+func WrapEndorser(s driver.EndorserStore, p Policy) driver.EndorserStore {
+	if s == nil {
+		return s
+	}
+
+	return &guardedEndorserStore{EndorserStore: s, policy: p}
+}
+
+type guardedEndorserStore struct {
+	driver.EndorserStore
+	policy Policy
+}
+
+func (g *guardedEndorserStore) QueryValidations(ctx context.Context, params driver.QueryValidationRecordsParams) (driver.ValidationRecordsIterator, error) {
+	it, err := g.EndorserStore.QueryValidations(ctx, params)
+	if err != nil {
+		return nil, err
+	}
+
+	return LimitIterator(it, g.policy.MaxPageSize, "QueryValidations"), nil
+}
+
+func (g *guardedEndorserStore) NewEndorserStoreTransaction() (driver.EndorserStoreTransaction, error) {
+	w, err := g.EndorserStore.NewEndorserStoreTransaction()
+	if err != nil {
+		return nil, err
+	}
+
+	return &guardedEndorserStoreTx{EndorserStoreTransaction: w, policy: g.policy}, nil
+}
+
+type guardedEndorserStoreTx struct {
+	driver.EndorserStoreTransaction
+	policy Policy
+}
+
+func (w *guardedEndorserStoreTx) AddValidationRecord(ctx context.Context, txID string, tokenRequest []byte, meta map[string][]byte, ppHash tokendriver.PPHash) error {
+	size := len(txID) + len(tokenRequest) + BlobMapSize(meta)
+	if err := CheckPayload("AddValidationRecord", size, w.policy.MaxPayloadSize); err != nil {
+		return err
+	}
+
+	return w.EndorserStoreTransaction.AddValidationRecord(ctx, txID, tokenRequest, meta, ppHash)
+}

token/services/storage/db/guard/identity.go

@@ -0,0 +1,76 @@
+/*
+Copyright IBM Corp. All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package guard
+
+import (
+	"context"
+
+	tokendriver "github.com/LFDT-Panurus/panurus/token/driver"
+	idriver "github.com/LFDT-Panurus/panurus/token/services/identity/driver"
+	driver "github.com/LFDT-Panurus/panurus/token/services/storage/db/driver"
+)
+
+// WrapIdentity wraps an identity store with the guard policy.
+func WrapIdentity(s driver.IdentityStore, p Policy) driver.IdentityStore {
+	if s == nil {
+		return s
+	}
+
+	return &guardedIdentityStore{IdentityStore: s, policy: p}
+}
+
+type guardedIdentityStore struct {
+	driver.IdentityStore
+	policy Policy
+}
+
+func (g *guardedIdentityStore) IteratorConfigurations(ctx context.Context, configurationType string) (idriver.IdentityConfigurationIterator, error) {
+	it, err := g.IdentityStore.IteratorConfigurations(ctx, configurationType)
+	if err != nil {
+		return nil, err
+	}
+
+	return LimitIterator(it, g.policy.MaxPageSize, "IteratorConfigurations"), nil
+}
+
+func (g *guardedIdentityStore) AddConfiguration(ctx context.Context, wp idriver.IdentityConfiguration) error {
+	size := len(wp.ID) + len(wp.Type) + len(wp.URL) + len(wp.Config) + len(wp.Raw)
+	if err := CheckPayload("AddConfiguration", size, g.policy.MaxPayloadSize); err != nil {
+		return err
+	}
+
+	return g.IdentityStore.AddConfiguration(ctx, wp)
+}
+
+func (g *guardedIdentityStore) StoreIdentityData(ctx context.Context, id []byte, identityAudit []byte, tokenMetadata []byte, tokenMetadataAudit []byte) error {
+	size := len(id) + len(identityAudit) + len(tokenMetadata) + len(tokenMetadataAudit)
+	if err := CheckPayload("StoreIdentityData", size, g.policy.MaxPayloadSize); err != nil {
+		return err
+	}
+
+	return g.IdentityStore.StoreIdentityData(ctx, id, identityAudit, tokenMetadata, tokenMetadataAudit)
+}
+
+func (g *guardedIdentityStore) StoreSignerInfo(ctx context.Context, id tokendriver.Identity, info []byte) error {
+	if err := CheckPayload("StoreSignerInfo", len(id)+len(info), g.policy.MaxPayloadSize); err != nil {
+		return err
+	}
+
+	return g.IdentityStore.StoreSignerInfo(ctx, id, info)
+}
+
+func (g *guardedIdentityStore) RegisterIdentityDescriptor(ctx context.Context, descriptor *idriver.IdentityDescriptor, alias tokendriver.Identity) error {
+	size := len(alias)
+	if descriptor != nil {
+		size += len(descriptor.Identity) + len(descriptor.AuditInfo) + len(descriptor.SignerInfo)
+	}
+	if err := CheckPayload("RegisterIdentityDescriptor", size, g.policy.MaxPayloadSize); err != nil {
+		return err
+	}
+
+	return g.IdentityStore.RegisterIdentityDescriptor(ctx, descriptor, alias)
+}

token/services/storage/db/guard/iterator.go

@@ -0,0 +1,51 @@
+/*
+Copyright IBM Corp. All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+
+package guard
+
+import (
+	"github.com/hyperledger-labs/fabric-smart-client/pkg/utils/errors"
+	"github.com/hyperledger-labs/fabric-smart-client/platform/common/utils/collections/iterators"
+)
+
+// LimitIterator wraps a streaming iterator so that reading more than max items
+// fails instead of materialising an unbounded result set. It returns an error
+// on the read that would exceed max (rather than silently truncating), so a
+// caller that legitimately needs the full set learns it must paginate.
+//
+// max <= 0 disables the cap, and a nil iterator is returned unchanged, so the
+// wrapper is safe to apply unconditionally. op names the read for the error.
+func LimitIterator[A any](it iterators.Iterator[*A], max int, op string) iterators.Iterator[*A] {
+	if it == nil || max <= 0 {
+		return it
+	}
+
+	return &limitedIterator[A]{Iterator: it, max: max, op: op}
+}
+
+type limitedIterator[A any] struct {
+	iterators.Iterator[*A]
+	max   int
+	count int
+	op    string
+}
+
+func (l *limitedIterator[A]) Next() (*A, error) {
+	next, err := l.Iterator.Next()
+	if err != nil {
+		return nil, err
+	}
+	if next == nil {
+		// exhausted
+		return nil, nil
+	}
+	l.count++
+	if l.count > l.max {
+		return nil, errors.Errorf("%s exceeded maximum of %d rows", l.op, l.max)
+	}
+
+	return next, nil
+}