cache sigv4 compute

[luigi617]

Summary

• Cache the SigV4 signing key (kDate → kRegion → kService → kSigning) using PeriodicSweepCache so that requests with the same credentials, region, service, and date skip the 4-HMAC key derivation chain entirely

Motivation

SigV4 signing key derivation performs 4 sequential HMAC-SHA256 operations. For high-concurrency workloads (hundreds of in-flight requests to the same service), these redundant derivations become a measurable CPU cost. Caching the result eliminates this overhead for the common case where the signing parameters are stable across requests.

What changed

SigV4SignatureCalculator.signingKey() now uses a PeriodicSweepCache<SigningKeyCacheKey, ByteArray> to cache derived signing keys. The cache key is a data class of (secretAccessKey, region, service, date). On a cache hit, the previously computed key is returned immediately. Entries expire after 24 hours and are cleaned by periodic sweep.

Design alternatives considered

Approach	Pros	Cons	Why not chosen
Single volatile slot	Lock-free reads, ~5ns hit cost, highest throughput in single-account case	Thrashes on multi-account (degrades to no-cache)	Does not handle multi-account/multi-region workloads
ConcurrentHashMap	O(1) lookup, no coroutine overhead	Not available in Kotlin common (JVM-only); unbounded growth without manual eviction	Requires expect/actual, breaks multiplatform
Fixed-size array (N slots, FIFO)	Lock-free reads, bounded memory, handles N accounts	O(N) linear scan; LRU tracking adds complexity; not reusable	More code to maintain for marginal benefit over existing utility
PeriodicSweepCache (chosen)	Multi-slot, bounded by expiration, multiplatform, already battle-tested for SigV4a	Mutex serialization on every access; runBlocking overhead	—

We chose PeriodicSweepCache because:

1. It already exists in the codebase and is proven by SigV4a usage
2. It handles multi-account without thrashing — each credential+region+service+date gets its own slot
3. 24-hour expiration ensures stale day-entries are automatically cleaned
4. Matches the spirit of the Java SDK v2 approach (multi-slot, day-based invalidation) (source)

The Mutex overhead is acceptable because the critical section is tiny (a HashMap lookup + clock check) — contention is negligible compared to the 4-HMAC chain it replaces.

Benchmark Results

All benchmarks run on m7i.2xlarge instances with the following configuration:

• Operations per batch: 100,000
• Warmup batches: 3
• Measurement batches: 5

E2E Throughput (S3)

7 base instances, 4 target instances (2 outlier instances excluded from target due to host-level network throttling).

S3 Upload

Concurrency	Base (Gbps)	Target (Gbps)	Change	Base CPU%	Target CPU%	Base Mem (MB)	Target Mem (MB)
16	3.82 ± 0	3.78 ± 0	-1.0%	27.7	28.6	696.8	705.5
32	6.98 ± 0.17	7.06 ± 0.33	+1.1%	58.1	56.6	739.1	672.6
64	9.95 ± 0.71	10.20 ± 0.68	+2.5%	92.9	91.1	942.2	936.3
128	10.28 ± 0.57	10.49 ± 0.45	+2.0%	93.7	93.5	947.0	935.1

S3 Download

Concurrency	Base (Gbps)	Target (Gbps)	Change	Base CPU%	Target CPU%	Base Mem (MB)	Target Mem (MB)
16	5.72 ± 0.17	5.85 ± 0	+2.3%	52.8	50.0	973.0	978.8
32	9.08 ± 0.37	9.35 ± 0.59	+3.0%	83.6	80.3	1089.0	1219.9
64	9.82 ± 0.50	10.09 ± 0.58	+2.7%	83.8	80.9	1282.4	1230.9
128	8.96 ± 0.22	9.30 ± 0.24	+3.8%	87.2	85.9	1314.5	1349.0

Throughput observations:

• At high concurrency (64–128 connections), the optimization yields +2–4% throughput improvement for both uploads and downloads by reducing CPU spent on signing key derivation
• CPU usage is slightly lower in target, consistent with eliminating redundant HMAC work
• Memory usage remains comparable across both variants — no regression

E2E Latency (DynamoDB)

21 instances per variant for statistical significance.

DynamoDB PutItem

Metric	Base (ms)	Target (ms)	Change	Base CPU%	Target CPU%	Base Mem (MB)	Target Mem (MB)
Average	3.43 ± 0.70	3.19 ± 0.48	-7.0%	1.2	1.0	67.9	67.9
P50	3.28 ± 0.72	3.04 ± 0.46	-7.3%	—	—	—	—
P90	3.89 ± 0.75	3.59 ± 0.53	-7.7%	—	—	—	—
P99	6.13 ± 1.41	5.52 ± 1.24	-9.9%	—	—	—	—

DynamoDB GetItem

Metric	Base (ms)	Target (ms)	Change	Base CPU%	Target CPU%	Base Mem (MB)	Target Mem (MB)
Average	2.33 ± 0.64	2.06 ± 0.48	-11.6%	1.7	1.5	68.1	68.2
P50	2.15 ± 0.63	1.87 ± 0.45	-13.0%	—	—	—	—
P90	2.73 ± 0.80	2.41 ± 0.60	-11.7%	—	—	—	—
P99	5.03 ± 1.59	4.66 ± 1.31	-7.3%	—	—	—	—

Latency observations:

• DynamoDB PutItem shows consistent improvement across all percentiles (-7.0% to -9.9%) — the 4-HMAC derivation saving is directly visible in write latency
• DynamoDB GetItem shows strong improvement across all percentiles (-11.6% to -13.0% at P50) — even for fast reads, the signing overhead was a significant fraction of total latency
• Target cross-instance standard deviation is lower (0.48 vs 0.70 for PutItem), indicating more consistent performance
• No CPU or memory regression — resource usage is effectively identical

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[ashishdhingra]

Looks good comparing it with existing SigV4aSignatureCalculator.

[ianbotsf]

Correctness: We should not block on a coroutine in a non-suspend method. How we address this depends on how critical the cache is to the final result.

Can we withstand cache misses? If so, Mutex.tryLock is a non-suspend method that will acquire the lock if it's not already locked. The slight downside is that if it is already locked, it returns false immediately. That might mean sometimes we bypass the cache if it's busy and recalculate a signature. If we decide to go this route, I'd suggest adding a non-suspend tryGet method to ExpiringKeyedCache/PeriodicSweepCache.

If we cannot withstand cache misses then I think we need a different lock mechanism. kotlinx.atomicfu provides some lock primitives that spin rather than suspend. If we decide to go this route we should create a new interface/implementation of the cache.

[luigi617]

Cache misses is fine so I think Mutex.tryLock is the way to go.

How SigV4a is different in behavior to SigV4? What is the reason of having the block in SigV4a?

[ianbotsf]

Suggestion: This combination of data don't just represent a cache key, they represent the signing key regardless of caching. I suggest renaming it to SigningKey and using it as the argument to deriveKey instead of the four separate arguments. I also think you could add a secondary constructor that takes AwsSigningConfig and extracts the necessary fields (or alternately, an AwsSigningConfig.toSigningKey() extension method.

[ianbotsf]

Nit: This assertNotSame is a subset of the assertEquals and is thus redundant.

[ianbotsf]

Nit: Don't use assertEquals with boolean literals. Favor assertTrue and assertFalse.

[ianbotsf]

Question: A 24 hour sweep period combined with short-term credentials, multiservice usage pattern, and multiple accounts could stack up to quite a lot of entries. Should we be concerned about memory usage?

[luigi617]

You're right, I forgot about short-term credentials. Java SDK is using a FIFO cache with 300 entries. Do you have any suggestion on this?

The short-term credentials can last from few minutes to several hours. Computing SigV4 is cheap, the only problem is when it is in a high-concurrency environment then it becomes expensive, so probably using 1 hour like SigV4a?