fix: cut Get Orders read amplification on hot GSI partitions (#696)

* fix: cut Get Orders read amplification on hot GSI partitions

GET /dutch-auction/orders is throwing "Throughput exceeds the current
capacity of your table or index" in prod. The Orders table is
PAY_PER_REQUEST, so there is no table-level ceiling to hit -- the limit
being reached is the ~3000 RCU/s DynamoDB caps a single partition at.
Filler polling concentrates on one GSI partition key (`1_open` on
chainId_orderStatus-createdAt-all), and two multipliers sit on top of it.

Bound the retry loop: MAX_QUERY_RETRY 10 -> 3. `orderType` is applied as
a post-query filter rather than a key condition, so each retry is another
full 50-item page read against the same partition, billed in full even
when it matches no rows. Worst case per request drops from 11 reads to 4.

Add a 250ms cache over the list-query path. Fillers poll the same
chainId/orderStatus shapes continuously, so repeats inside that window
are byte-identical; collapsing them cuts reads on the hot key by roughly
TTL/handler-latency. The key covers every input that can change the
result set, and callers get a copied array so a cached entry cannot be
mutated downstream. Hit/miss metrics are emitted so the real rate is
visible before deciding whether a shared cache is also needed.

The cache is opt-in per repository, not a repository default: the
unimind cron writes orders and re-reads them expecting fresh data, so
only the read-only get-orders path passes it in. TTL is configurable via
GET_ORDERS_CACHE_TTL_MS, and 0 disables it without a code change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address review — cache entry expiry, kill switch, retry visibility

Stamp cache entries from when the query finished rather than when it
started. A throttled partition retries with backoff, so a query can
outlast the 250ms TTL; dating the entry from the query start wrote it
already expired and silently disabled the cache under exactly the load
it exists to absorb. Covered by a regression test that fails against the
previous behaviour.

Plumb GET_ORDERS_CACHE_TTL_MS through the beta and prod stage envVars.
CDK writes the complete Environment.Variables map on every deploy, so a
console-set variable it does not know about is dropped by the next one —
the documented rollback path did not survive a deploy.

Emit GetOrdersRetryExhausted when a type-filtered query gives up with a
cursor still outstanding, so the cost of the lower MAX_QUERY_RETRY is
measurable rather than assumed.

Correct the copyQueryResult comment: the copy is shallow, so it protects
the array but not the order objects, which must be treated as read-only.

Build the query-cache tests on a locally constructed QueryCache instead
of the exported singleton, whose TTL comes from the environment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address review round 2 — dedupe multi-status merges, per-endpoint caches, drop retry cut

- Dedupe getOrdersForMultipleStatuses by orderHash: per-status sub-queries are
  independent cache entries, so one can serve a stale page listing the same order
  under a second status; keep the non-OPEN copy since transitions flow away from OPEN
- Opt the read-only GET /limit-orders lambda into the query cache — it polls the
  same hot single-status GSI partitions on the LimitOrders table
- Move cache construction out of the generic repository into per-endpoint modules;
  hit/miss metrics are now named by the cache instance (GetOrdersQueryCache*,
  GetLimitOrdersQueryCache*) instead of hardcoded in the shared base class
- Gate cache bookkeeping on enabled so the GET_ORDERS_CACHE_TTL_MS=0 kill switch
  also silences the miss metric (a disabled cache no longer looks broken), and skip
  key building entirely on uncached repositories
- Revert MAX_QUERY_RETRY to 10 and drop the GetOrdersRetryExhausted metric: the
  cache absorbs the hot-partition read amplification, and the deeper retry budget
  keeps type-filtered pages full for cursor-ignoring pollers
- Extract the five duplicated per-type retry loops into fetchOrderPages
- Document GET_ORDERS_CACHE_TTL_MS in CLAUDE.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: fix comments

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alan Wu <60207036+alanhwu@users.noreply.github.com>
Co-authored-by: Alan Wu <alanwu100@gmail.com>

Author: 3 people

CLAUDE.md

@@ -46,6 +46,9 @@ Required for deployment:
 - `RPC_HEADER_SECRET` - Value sent as the `x-internal-service-secret` header on all RPC requests (see `RPC_HEADERS` in `lib/util/constants.ts`). Omitted when unset.
 - `FAILED_EVENT_DESTINATION_ARN` - Failed event SNS ARN
 
+Optional:
+- `GET_ORDERS_CACHE_TTL_MS` - TTL for the read-path query cache on the get-orders/get-limit-orders Lambdas (default 250; set to `0` to disable the cache).
+
 For tests:
 - `UNISWAP_API` - Deployed API URL (e2e tests)
 - `LABS_COSIGNER` - Valid EVM address (unit tests)

bin/app.ts

@@ -200,10 +200,18 @@ export class APIPipeline extends Stack {
           .secretValueFromJson('ACTIVE_ORDER_EVENT_DESTINATION_ARN_BETA')
           .toString(),
         POSTED_ORDER_DESTINATION_ARN: resourceArnSecret.secretValueFromJson('POSTED_ORDER_DESTINATION_BETA').toString(),
-        UNIMIND_RESPONSE_DESTINATION_ARN: resourceArnSecret.secretValueFromJson('UNIMIND_RESPONSE_DESTINATION_ARN_BETA').toString(),
-        UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN: resourceArnSecret.secretValueFromJson('UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN_BETA').toString(),
-        CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN: resourceArnSecret.secretValueFromJson('CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN_BETA').toString(),
+        UNIMIND_RESPONSE_DESTINATION_ARN: resourceArnSecret
+          .secretValueFromJson('UNIMIND_RESPONSE_DESTINATION_ARN_BETA')
+          .toString(),
+        UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN: resourceArnSecret
+          .secretValueFromJson('UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN_BETA')
+          .toString(),
+        CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN: resourceArnSecret
+          .secretValueFromJson('CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN_BETA')
+          .toString(),
         THROTTLE_PER_FIVE_MINS: '3000',
+        // Get Orders query cache TTL. Set to '0' and deploy to disable the cache.
+        GET_ORDERS_CACHE_TTL_MS: '250',
         REGION: 'us-east-2', //needed in checkOrderStatusHandler to kick off step function retries
         LABS_COSIGNER: labsCosignerBeta.secretValue.toString(),
         LABS_PRIORITY_COSIGNER: labsPriorityCosignerBeta.secretValue.toString(),
@@ -237,10 +245,18 @@ export class APIPipeline extends Stack {
           .secretValueFromJson('ACTIVE_ORDER_EVENT_DESTINATION_ARN_PROD')
           .toString(),
         POSTED_ORDER_DESTINATION_ARN: resourceArnSecret.secretValueFromJson('POSTED_ORDER_DESTINATION_PROD').toString(),
-        UNIMIND_RESPONSE_DESTINATION_ARN: resourceArnSecret.secretValueFromJson('UNIMIND_RESPONSE_DESTINATION_ARN_PROD').toString(),
-        UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN: resourceArnSecret.secretValueFromJson('UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN_PROD').toString(),
-        CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN: resourceArnSecret.secretValueFromJson('CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN_PROD').toString(),
+        UNIMIND_RESPONSE_DESTINATION_ARN: resourceArnSecret
+          .secretValueFromJson('UNIMIND_RESPONSE_DESTINATION_ARN_PROD')
+          .toString(),
+        UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN: resourceArnSecret
+          .secretValueFromJson('UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN_PROD')
+          .toString(),
+        CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN: resourceArnSecret
+          .secretValueFromJson('CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN_PROD')
+          .toString(),
         THROTTLE_PER_FIVE_MINS: '3000',
+        // Get Orders query cache TTL. Set to '0' and deploy to disable the cache.
+        GET_ORDERS_CACHE_TTL_MS: '250',
         REGION: 'us-east-2', //needed in checkOrderStatusHandler to kick off step function retries
         LABS_COSIGNER: labsCosignerProd.secretValue.toString(),
         LABS_PRIORITY_COSIGNER: labsPriorityCosignerProd.secretValue.toString(),
@@ -385,6 +401,7 @@ envVars['UNIMIND_PARAMETER_UPDATE_DESTINATION_ARN'] = process.env['UNIMIND_PARAM
 envVars['CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN'] = process.env['CLOUDWATCH_LOGS_FIREHOSE_ROLE_ARN'] || ''
 envVars['LABS_COSIGNER'] = process.env['LABS_COSIGNER'] || ''
 envVars['LABS_PRIORITY_COSIGNER'] = process.env['LABS_PRIORITY_COSIGNER'] || ''
+envVars['GET_ORDERS_CACHE_TTL_MS'] = process.env['GET_ORDERS_CACHE_TTL_MS'] || ''
 
 new APIStack(app, `${SERVICE_NAME}Stack`, {
   provisionedConcurrency: process.env.PROVISION_CONCURRENCY ? parseInt(process.env.PROVISION_CONCURRENCY) : 0,

lib/handlers/get-limit-orders/index.ts

@@ -8,6 +8,7 @@ import { ONE_DAY_IN_SECONDS } from '../../util/constants'
 
 import { log } from '../../Logging'
 import { LimitOrdersRepository } from '../../repositories/limit-orders-repository'
+import { DynamoQuoteMetadataRepository } from '../../repositories/quote-metadata-repository'
 import { OrderDispatcher } from '../../services/OrderDispatcher'
 import { OffChainRelayOrderValidator } from '../../util/OffChainRelayOrderValidator'
 import { OffChainUniswapXOrderValidator } from '../../util/OffChainUniswapXOrderValidator'
@@ -18,9 +19,11 @@ import { GetOrdersHandler } from '../get-orders/handler'
 import { OnChainValidatorMap } from '../OnChainValidatorMap'
 import { getMaxOpenOrders } from '../post-order/injector'
 import { GetLimitOrdersInjector } from './injector'
-import { DynamoQuoteMetadataRepository } from '../../repositories/quote-metadata-repository'
+import { getLimitOrdersQueryCache } from './query-cache'
 
-const repo = LimitOrdersRepository.create(new DynamoDB.DocumentClient())
+// This Lambda only reads, so its repositories share the sub-second query cache. Write
+// paths build them without it -- see query-cache.ts.
+const repo = LimitOrdersRepository.create(new DynamoDB.DocumentClient(), getLimitOrdersQueryCache)
 const quoteMetadataRepository = DynamoQuoteMetadataRepository.create(new DynamoDB.DocumentClient())
 const orderValidator = new OffChainUniswapXOrderValidator(() => new Date().getTime() / 1000, ONE_DAY_IN_SECONDS)
 const onChainValidatorMap = new OnChainValidatorMap<OrderValidator>()
@@ -44,7 +47,7 @@ const relayOrderService = new RelayOrderService(
   relayOrderValidator,
   relayOrderValidatorMap,
   EventWatcherMap.createRelayEventWatcherMap(),
-  RelayOrderRepository.create(new DynamoDB.DocumentClient()),
+  RelayOrderRepository.create(new DynamoDB.DocumentClient(), getLimitOrdersQueryCache),
   log,
   getMaxOpenOrders,
   new FillEventLogger(FILL_EVENT_LOOKBACK_BLOCKS_ON, AnalyticsService.create())

lib/handlers/get-limit-orders/injector.ts

@@ -6,6 +6,7 @@ import { LimitOrdersRepository } from '../../repositories/limit-orders-repositor
 import { ApiInjector, ApiRInj } from '../base/index'
 import { GetOrdersQueryParams, RawGetOrdersQueryParams } from '../get-orders/schema'
 import { ContainerInjected, getSharedRequestInjected } from '../shared/get'
+import { getLimitOrdersQueryCache } from './query-cache'
 
 export interface RequestInjected extends ApiRInj {
   limit: number
@@ -21,7 +22,7 @@ export class GetLimitOrdersInjector extends ApiInjector<
 > {
   public async buildContainerInjected(): Promise<ContainerInjected> {
     return {
-      dbInterface: LimitOrdersRepository.create(new DynamoDB.DocumentClient()),
+      dbInterface: LimitOrdersRepository.create(new DynamoDB.DocumentClient(), getLimitOrdersQueryCache),
     }
   }
 

lib/handlers/get-limit-orders/query-cache.ts

@@ -0,0 +1,9 @@
+import { OrdersQueryCache, QueryCache, queryCacheTtlFromEnv } from '../../repositories/QueryCache'
+
+// GET /limit-orders is polled by fillers the same way GET /orders is, against the same
+// kind of hot single-status GSI partition on the LimitOrders table. Its own instance
+// (rather than sharing get-orders') keeps hit/miss metrics attributable per endpoint.
+export const getLimitOrdersQueryCache: OrdersQueryCache = new QueryCache(
+  queryCacheTtlFromEnv(),
+  'GetLimitOrdersQueryCache'
+)

lib/handlers/get-orders/index.ts

@@ -13,17 +13,20 @@ import { ONE_DAY_IN_SECONDS } from '../../util/constants'
 
 import { log } from '../../Logging'
 import { LimitOrdersRepository } from '../../repositories/limit-orders-repository'
+import { DynamoQuoteMetadataRepository } from '../../repositories/quote-metadata-repository'
 import { OffChainRelayOrderValidator } from '../../util/OffChainRelayOrderValidator'
 import { OffChainUniswapXOrderValidator } from '../../util/OffChainUniswapXOrderValidator'
 import { FillEventLogger } from '../check-order-status/fill-event-logger'
 import { FILL_EVENT_LOOKBACK_BLOCKS_ON } from '../check-order-status/util'
 import { EventWatcherMap } from '../EventWatcherMap'
 import { OnChainValidatorMap } from '../OnChainValidatorMap'
 import { getMaxOpenOrders } from '../post-order/injector'
-import { DynamoQuoteMetadataRepository } from '../../repositories/quote-metadata-repository'
+import { getOrdersQueryCache } from './query-cache'
 
-const repo = DutchOrdersRepository.create(new DynamoDB.DocumentClient())
-const limitRepo = LimitOrdersRepository.create(new DynamoDB.DocumentClient())
+// This Lambda only reads, so its repositories share the sub-second query cache. Write
+// paths build them without it -- see query-cache.ts.
+const repo = DutchOrdersRepository.create(new DynamoDB.DocumentClient(), getOrdersQueryCache)
+const limitRepo = LimitOrdersRepository.create(new DynamoDB.DocumentClient(), getOrdersQueryCache)
 const quoteMetadataRepository = DynamoQuoteMetadataRepository.create(new DynamoDB.DocumentClient())
 const orderValidator = new OffChainUniswapXOrderValidator(() => new Date().getTime() / 1000, ONE_DAY_IN_SECONDS)
 const onChainValidatorMap = new OnChainValidatorMap<OrderValidator>()
@@ -48,7 +51,7 @@ const relayOrderService = new RelayOrderService(
   relayOrderValidator,
   relayOrderValidatorMap,
   EventWatcherMap.createRelayEventWatcherMap(),
-  RelayOrderRepository.create(new DynamoDB.DocumentClient()),
+  RelayOrderRepository.create(new DynamoDB.DocumentClient(), getOrdersQueryCache),
   log,
   getMaxOpenOrders,
   new FillEventLogger(FILL_EVENT_LOOKBACK_BLOCKS_ON, AnalyticsService.create())

lib/handlers/get-orders/injector.ts

@@ -7,6 +7,7 @@ import { BaseOrdersRepository } from '../../repositories/base'
 import { DutchOrdersRepository } from '../../repositories/dutch-orders-repository'
 import { ApiInjector, ApiRInj } from '../base/index'
 import { getSharedRequestInjected } from '../shared/get'
+import { getOrdersQueryCache } from './query-cache'
 import { GetOrdersQueryParams, RawGetOrdersQueryParams } from './schema'
 import { GetOrderTypeQueryParamEnum } from './schema/GetOrderTypeQueryParamEnum'
 
@@ -25,7 +26,7 @@ export interface ContainerInjected {
 export class GetOrdersInjector extends ApiInjector<ContainerInjected, RequestInjected, void, RawGetOrdersQueryParams> {
   public async buildContainerInjected(): Promise<ContainerInjected> {
     return {
-      dbInterface: DutchOrdersRepository.create(new DynamoDB.DocumentClient()),
+      dbInterface: DutchOrdersRepository.create(new DynamoDB.DocumentClient(), getOrdersQueryCache),
     }
   }
 

lib/handlers/get-orders/query-cache.ts

@@ -0,0 +1,11 @@
+import { OrdersQueryCache, QueryCache, queryCacheTtlFromEnv } from '../../repositories/QueryCache'
+
+// Sub-second cache over the list-query path. Fillers poll the same query shapes
+// continuously, so repeats inside this window are collapsed into a single read against
+// the hot GSI partition.
+//
+// Opt-in, not a repository default: background jobs (the unimind cron, the reaper) write
+// orders and immediately re-read them expecting fresh data. Only this read-only endpoint
+// passes it in. Shared across the repositories this endpoint builds so they pool hits --
+// the table name is part of every key, so Orders/LimitOrders/RelayOrders never collide.
+export const getOrdersQueryCache: OrdersQueryCache = new QueryCache(queryCacheTtlFromEnv(), 'GetOrdersQueryCache')

lib/repositories/QueryCache.ts

@@ -0,0 +1,103 @@
+import { OrderEntityType, QueryResult } from './base'
+
+type CacheEntry<T> = {
+  expiresAt: number
+  value: T
+}
+
+const DEFAULT_QUERY_CACHE_TTL_MS = 250
+
+// Set GET_ORDERS_CACHE_TTL_MS=0 to disable the query cache without a code change.
+export function queryCacheTtlFromEnv(): number {
+  const raw = process.env.GET_ORDERS_CACHE_TTL_MS
+  if (!raw) {
+    return DEFAULT_QUERY_CACHE_TTL_MS
+  }
+  const configured = Number(raw)
+  return Number.isFinite(configured) && configured >= 0 ? configured : DEFAULT_QUERY_CACHE_TTL_MS
+}
+
+/**
+ * Process-local TTL cache for read-only DynamoDB query results.
+ *
+ * Fillers poll the same chainId/orderStatus combinations continuously, so within a few
+ * hundred milliseconds many requests ask for a byte-identical page. Those queries all
+ * land on a single GSI partition key (e.g. `1_open`), which DynamoDB caps at ~3000 RCU/s
+ * regardless of the table's billing mode. Serving repeats from memory collapses them into
+ * one read and keeps the partition below that ceiling.
+ *
+ * Query keys embed caller-supplied values (swapper, pair, cursor), so the map is bounded:
+ * an unbounded one would be a memory leak in a long-lived execution environment.
+ */
+export class QueryCache<T> {
+  private readonly store = new Map<string, CacheEntry<T>>()
+
+  // metricPrefix names the hit/miss metrics emitted by repositories using this cache
+  // (e.g. 'GetOrdersQueryCache' -> GetOrdersQueryCacheHit/Miss). Each endpoint owns its
+  // cache instance, so its traffic stays distinguishable on dashboards.
+  constructor(
+    private readonly ttlMs: number,
+    public readonly metricPrefix: string,
+    private readonly maxEntries = 1000
+  ) {}
+
+  public get enabled(): boolean {
+    return this.ttlMs > 0
+  }
+
+  public get(key: string, now: number): T | undefined {
+    if (!this.enabled) {
+      return undefined
+    }
+    const entry = this.store.get(key)
+    if (!entry) {
+      return undefined
+    }
+    if (entry.expiresAt <= now) {
+      this.store.delete(key)
+      return undefined
+    }
+    return entry.value
+  }
+
+  public set(key: string, value: T, now: number): void {
+    if (!this.enabled) {
+      return
+    }
+    this.evict(now)
+    // Re-inserting an existing key keeps its original position in a Map, which would break
+    // the insertion-order-equals-expiry-order invariant evict() relies on. Delete first.
+    this.store.delete(key)
+    this.store.set(key, { expiresAt: now + this.ttlMs, value })
+  }
+
+  public get size(): number {
+    return this.store.size
+  }
+
+  public clear(): void {
+    this.store.clear()
+  }
+
+  /**
+   * Every entry gets the same TTL, so insertion order is also expiry order: we can stop
+   * sweeping at the first live entry, and drop from the front when over capacity.
+   */
+  private evict(now: number): void {
+    for (const [key, entry] of this.store) {
+      if (entry.expiresAt > now) {
+        break
+      }
+      this.store.delete(key)
+    }
+    while (this.store.size >= this.maxEntries) {
+      const oldest = this.store.keys().next()
+      if (oldest.done) {
+        break
+      }
+      this.store.delete(oldest.value)
+    }
+  }
+}
+
+export type OrdersQueryCache = QueryCache<QueryResult<OrderEntityType>>

lib/repositories/RelayOrderRepository.ts

@@ -7,10 +7,11 @@ import { RelayOrderEntity } from '../entities'
 import { BaseOrdersRepository, MODEL_NAME } from './base'
 import { GenericOrdersRepository } from './generic-orders-repository'
 import { OffchainOrderIndexMapper } from './IndexMappers/OffchainOrderIndexMapper'
+import { OrdersQueryCache } from './QueryCache'
 import { getTableIndices, TABLE_NAMES } from './util'
 
 export class RelayOrderRepository extends GenericOrdersRepository<string, string, null, RelayOrderEntity> {
-  static create(documentClient: DocumentClient): BaseOrdersRepository<RelayOrderEntity> {
+  static create(documentClient: DocumentClient, queryCache?: OrdersQueryCache): BaseOrdersRepository<RelayOrderEntity> {
     const log = Logger.createLogger({
       name: 'RelayOrdersRepository',
       serializers: Logger.stdSerializers,
@@ -76,7 +77,8 @@ export class RelayOrderRepository extends GenericOrdersRepository<string, string
       relayOrderEntity,
       nonceEntity,
       log,
-      new OffchainOrderIndexMapper<RelayOrderEntity>()
+      new OffchainOrderIndexMapper<RelayOrderEntity>(),
+      queryCache
     )
   }
 }