perf: faster model resolution, JSON decoding, and request snapshotting (#413)

* perf: faster model resolution, JSON decoding, and request snapshotting

Reduce per-request CPU and allocations on the gateway hot path. Changes are
behavior-preserving for all valid input; the one intentional difference is
documented and tested.

Model resolution (O(1)):
- Add a lazy provider-selector index to the registry (qualifiedByName /
  qualifiedByType), invalidated at the existing single cache-invalidation point.
- Route qualified-selector resolution through it via an optional
  qualifiedSelectorResolver interface, with the catalog scan kept as a fallback
  for non-indexed lookups and raw slash-shaped model IDs.
- Resolution is now O(1) and constant in catalog size (was O(N), copying the
  full catalog several times per request): ~31us/164KB -> ~0.8us/0.3KB at 300
  models. Deduplicated the redundant name/type scans in resolveQualifiedSelector.

JSON decoding (goccy/go-json):
- Migrate internal/ + cmd/ from encoding/json to github.com/goccy/go-json
  (drop-in; package is named json). gjson is unchanged.
- ~3.8x faster realistic chat-body decode with fewer allocations.
- goccy is slightly more lenient than encoding/json on a couple of malformed
  inputs (leading-zero numbers; malformed values in skipped passthrough fields).
  Accepted under the gateway's accept-generously principle and pinned by
  TestDecoderLeniencyIsBounded.
- Drop the redundant gjson.ValidBytes walk in extractUnknownJSONFields (callers
  already validate via the preceding Unmarshal).

Request snapshot allocations:
- Add NewRequestSnapshotWithOwnedMaps so ingress capture owns the freshly built
  route/query/trace maps and body, cloning only the live header map.
- Add HeadersView (zero-copy) and route read-only callers to it.
- Remove the now-superseded NewRequestSnapshotWithOwnedBody constructor.

Perf harness:
- Make the gateway hot-path benchmark exercise the real Router + populated
  catalog (it previously bypassed routing, giving false confidence) and add a
  guard case for it. Add a resolution micro-benchmark.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(perf): tighten hot-path allocation guard thresholds

CI (linux/amd64) and local (darwin/arm64) produce identical allocation counts
and near-identical byte counts, confirming these are deterministic. Tighten the
ceilings from "intentionally generous" to ~10% over the measured baseline so the
guard catches smaller regressions while still absorbing Go/dependency drift:

  hot_path:          125 -> 120 allocs            (baseline 113)
  routed:            160 -> 150 allocs, 18->16 KB (baseline 137 / ~14.7 KB)
  responses_stream:  310 -> 222 allocs, 25->22 KB (baseline 202 / ~19.6 KB)
  shared_observers:  unchanged (already tight, no headroom to trim)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(providers): address PR review on selector index and benchmarks

- registry: trim publicName/ProviderType when building the qualified-selector
  index and skip empty keys, matching the trimmed lookup inputs and the previous
  catalog scan (which compared trimmed fields on both sides). Prevents the O(1)
  fast path from missing when provider metadata carries whitespace padding.
- resolve_bench_test: build exactly totalModels (round-robin across providers)
  instead of providersN*(n/6); the models=50/1000 cases previously benchmarked
  48/996 models due to integer truncation. Add benchSelector helper.
- request_snapshot_test: extend the owned-maps test to assert route/query/trace
  maps are owned (not cloned) while headers are still defensively cloned.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: SantiagoDePolonia

cmd/gomodel/health.go

@@ -2,7 +2,6 @@ package main
 
 import (
 	"context"
-	"encoding/json"
 	"fmt"
 	"io"
 	"net"
@@ -11,6 +10,8 @@ import (
 	"strings"
 	"time"
 
+	"github.com/goccy/go-json"
+
 	"gomodel/config"
 )
 

cmd/recordapi/main.go

@@ -9,7 +9,6 @@ package main
 
 import (
 	"bytes"
-	"encoding/json"
 	"flag"
 	"fmt"
 	"io"
@@ -18,6 +17,8 @@ import (
 	"path/filepath"
 	"strings"
 	"time"
+
+	"github.com/goccy/go-json"
 )
 
 const oracleDefaultModel = "openai.gpt-oss-120b"

go.mod

@@ -10,6 +10,7 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/bedrockruntime v1.53.5
 	github.com/cespare/xxhash/v2 v2.3.0
 	github.com/coder/websocket v1.8.15
+	github.com/goccy/go-json v0.10.6
 	github.com/google/uuid v1.6.0
 	github.com/jackc/pgx/v5 v5.10.0
 	github.com/joho/godotenv v1.5.1

go.sum

@@ -83,6 +83,8 @@ github.com/go-openapi/testify/enable/yaml/v2 v2.4.0 h1:7SgOMTvJkM8yWrQlU8Jm18VeD
 github.com/go-openapi/testify/enable/yaml/v2 v2.4.0/go.mod h1:14iV8jyyQlinc9StD7w1xVPW3CO3q1Gj04Jy//Kw4VM=
 github.com/go-openapi/testify/v2 v2.4.0 h1:8nsPrHVCWkQ4p8h1EsRVymA2XABB4OT40gcvAu+voFM=
 github.com/go-openapi/testify/v2 v2.4.0/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16pjwMkYkP4Ywn54=
+github.com/goccy/go-json v0.10.6 h1:p8HrPJzOakx/mn/bQtjgNjdTcN+/S6FcG2CTtQOrHVU=
+github.com/goccy/go-json v0.10.6/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
 github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
 github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
 github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=

internal/admin/handler_guardrails.go

@@ -1,11 +1,12 @@
 package admin
 
 import (
-	"encoding/json"
 	"errors"
 	"net/http"
 	"strings"
 
+	"github.com/goccy/go-json"
+
 	"github.com/labstack/echo/v5"
 
 	"gomodel/internal/core"

internal/admin/handler_live.go

@@ -1,13 +1,14 @@
 package admin
 
 import (
-	"encoding/json"
 	"fmt"
 	"net/http"
 	"strconv"
 	"strings"
 	"time"
 
+	"github.com/goccy/go-json"
+
 	"github.com/labstack/echo/v5"
 
 	"gomodel/internal/core"

internal/aliases/batch_preparer.go

@@ -2,10 +2,11 @@ package aliases
 
 import (
 	"context"
-	"encoding/json"
 	"fmt"
 	"strings"
 
+	"github.com/goccy/go-json"
+
 	"gomodel/internal/core"
 )
 

internal/anthropicapi/request.go

@@ -2,11 +2,12 @@ package anthropicapi
 
 import (
 	"bytes"
-	"encoding/json"
 	"fmt"
 	"io"
 	"strings"
 
+	"github.com/goccy/go-json"
+
 	"gomodel/internal/core"
 )
 

internal/anthropicapi/response.go

@@ -2,9 +2,10 @@ package anthropicapi
 
 import (
 	"bytes"
-	"encoding/json"
 	"strings"
 
+	"github.com/goccy/go-json"
+
 	"gomodel/internal/core"
 )
 

internal/anthropicapi/stream.go

@@ -3,9 +3,10 @@ package anthropicapi
 import (
 	"bufio"
 	"bytes"
-	"encoding/json"
 	"io"
 
+	"github.com/goccy/go-json"
+
 	"gomodel/internal/streaming"
 )