Skip to content

Commit 02af9fc

Browse files
authored
feat: add Hilt client and wire with FX (#35)
I [forked the versity gateway](https://github.com/fil-forge/versitygw) to add support for externally derived SigV4 signing keys, since Ingot never sees the secret key when it uses Hilt. This PR uses the fork, and adds an IAM service that calls Hilt to authorize requests, and authroizes locally with cached keys/delegations if possible. There's a bunch of churn here due to the imports being re-written to our fork, but also because the versitygw Ingot depends on was not the most recent version. I also moved `hiltclient` into a `hilt` package that now houses both `client` and `iam` sub-packages - sorry. I'm interested in whether the versity gateway change seems like a reasonable PR we could submit upstream? Depends on: * fil-forge/versitygw#1 * #35 * fil-forge/piri#32
1 parent 1c3c2dd commit 02af9fc

103 files changed

Lines changed: 4850 additions & 1869 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CLAUDE.md

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,10 @@ ingot depends only on these — it must **never** import `fil-forge/sprue` or
5252
`commands/{blob,content,http,assert,ucan,index,provider,access}`, `blobindex`
5353
(sharded-dag-index), `ucan` (ProofStore), `ucan/retrieval`, `didmailto`, `receipt`.
5454
- **`indexing-service/pkg/{client,types}`** — indexer query client.
55-
- **`versity/versitygw`** — the S3 REST front end (we implement `backend.Backend`).
55+
- **`fil-forge/versitygw`** — our fork of versity/versitygw, the S3 REST front
56+
end (we implement `backend.Backend`). The fork adds externally derived SigV4
57+
signing keys (`auth.Account.SigningKey`, `middlewares.RequestIAMService`) for
58+
the Hilt flow.
5659
- Plumbing: `go-cid`, `go-block-format`, `whyrusleeping/cbor-gen` (**not**
5760
go-ipld-prime), `multiformats/*`, `pgx/v5`, `goose/v3`, `spf13/{cobra,viper}`,
5861
`uber-go/fx`, `zap`.
@@ -96,10 +99,21 @@ Internal:
9699
- **`forgeclient/`** — carried-from-guppy edge client: `/blob/add`,
97100
`/ucan/conclude`, `/index/add`, `/provider/add`, `/access/delegate`, and the
98101
`/access` login flow.
102+
- **`iam/`** — the Hilt (auth-service) integration over the external
103+
`github.com/fil-forge/hilt/pkg/client` (RFC: forge-s3-tenant-management):
104+
versitygw `IAMService`/`RequestIAMService` authorizing each non-root
105+
request via `/s3/request/authorize` (derived SigV4 key), plus
106+
`DelegationCache` — a TTL cache (go-cache) of Hilt-issued delegations
107+
(authorize re-delegations + `/s3/bucket/info` chains) that the network
108+
read tier consumes as its per-space `/content/retrieve` proof store.
109+
**Forge mode requires Hilt** (`auth_service_url`/`auth_service_did`): the
110+
postgres registry forwards bucket create/delete/list to it, recovering
111+
the signed S3 request from the method's ctx.
99112
- **`tokenstore/`** — carried-from-guppy delegation store (`tokens.cbor`).
100113
- **`bucket/`** — per-object model: `manifest.go` (`ObjectManifest`, `Body`),
101114
`chunker.go` (`BodyCodec`/`FixedChunker`), `cbor_gen.go`.
102-
- **`mst/`** — the forked MST (only dep: go-cid).
115+
- **`mst/`** — the forked MST (deps: go-cid, blockstore, ucantone/did — trees
116+
carry their bucket's space for network-backed reads).
103117
- **`inmem/`**`MemStore` (Registry+Meta), `NopBaseReader`, `NopUploader`; backs
104118
standalone mode (slated for removal).
105119
- **`cars/`**, **`migrations/`**, **`internal/ucanexec/`**, **`gen/`**,
@@ -141,7 +155,9 @@ Viper/yaml-bindable. Key fields: `Enabled`, `Addr`, `DataDir`, `Region`,
141155
`DataPlane`/`CatalogPlane` `{SealBytes, SealAge, Ship, Retain}` overrides,
142156
`IndexerEndpoint`/`IndexerDID`, `ReadCacheBytes` (0 → 256 MiB, <0 → off),
143157
`UploadServiceURL`/`UploadServiceDID`/`UploadReceiptsURL`, `TokenStoreDir`
144-
(→ `DataDir`). `Config.ServerConfig()` is the single mapping site. The daemon's
158+
(→ `DataDir`), `HiltURL`/`HiltDID`/`HiltProofs` (tenant-management service;
159+
proofs = file path or string-encoded UCAN container, optional).
160+
`Config.ServerConfig()` is the single mapping site. The daemon's
145161
`DaemonConfig` (cmd/config.go) embeds `Config` + `Mode`/`PostgresDSN`/`Identity`.
146162

147163
## Testing

Dockerfile

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,7 @@
2424
# /usr/bin/ingot -- serve --config /etc/ingot/config.yaml
2525
# then attach your client on the mapped 2345 (smelt's compose.debug.yml pattern).
2626

27-
# ---- build base: runs natively on the builder, cross-compiles to target ----
28-
FROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS build
27+
FROM --platform=$BUILDPLATFORM golang:1.26-bookworm AS build
2928
ARG TARGETOS=linux
3029
ARG TARGETARCH
3130
WORKDIR /src

blockstore/cache.go

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import (
66
"io"
77
"sync"
88

9+
"github.com/fil-forge/ucantone/did"
910
block "github.com/ipfs/go-block-format"
1011
"github.com/ipfs/go-cid"
1112
mh "github.com/multiformats/go-multihash"
@@ -61,21 +62,23 @@ var (
6162
// Body blobs are large and streamed; caching one whole would defeat streaming
6263
// (and a single blob can exceed the whole budget). Small catalog blocks still
6364
// cache through GetBlock.
64-
func (c *Cached) OpenBlob(ctx context.Context, digest mh.Multihash) (io.ReadCloser, error) {
65+
func (c *Cached) OpenBlob(ctx context.Context, space did.DID, digest mh.Multihash) (io.ReadCloser, error) {
6566
if br, ok := c.base.(BlobReader); ok {
66-
return br.OpenBlob(ctx, digest)
67+
return br.OpenBlob(ctx, space, digest)
6768
}
6869
return nil, ErrNotFound
6970
}
7071

7172
// GetBlock returns the cached block if present, otherwise fetches it from the
72-
// base reader and caches it (subject to the byte budget).
73-
func (c *Cached) GetBlock(ctx context.Context, k cid.Cid) (block.Block, error) {
73+
// base reader and caches it (subject to the byte budget). The LRU is keyed by
74+
// CID only — blocks are content-addressed, so a hit is valid whatever space
75+
// originally fetched it.
76+
func (c *Cached) GetBlock(ctx context.Context, space did.DID, k cid.Cid) (block.Block, error) {
7477
key := k.KeyString()
7578
if blk, ok := c.get(key); ok {
7679
return blk, nil
7780
}
78-
blk, err := c.base.GetBlock(ctx, k)
81+
blk, err := c.base.GetBlock(ctx, space, k)
7982
if err != nil {
8083
return nil, err
8184
}

blockstore/cache_test.go

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ package blockstore
22

33
import (
44
"context"
5+
"github.com/fil-forge/ucantone/did"
56
"testing"
67

78
block "github.com/ipfs/go-block-format"
@@ -18,7 +19,7 @@ type countingReader struct {
1819
calls map[string]int
1920
}
2021

21-
func (r *countingReader) GetBlock(_ context.Context, c cid.Cid) (block.Block, error) {
22+
func (r *countingReader) GetBlock(_ context.Context, _ did.DID, c cid.Cid) (block.Block, error) {
2223
if r.calls == nil {
2324
r.calls = map[string]int{}
2425
}
@@ -50,7 +51,7 @@ func TestCached_Hit(t *testing.T) {
5051
k := mkcid(7)
5152

5253
for i := 0; i < 3; i++ {
53-
if _, err := c.GetBlock(context.Background(), k); err != nil {
54+
if _, err := c.GetBlock(context.Background(), did.Undef, k); err != nil {
5455
t.Fatalf("get: %v", err)
5556
}
5657
}
@@ -66,20 +67,20 @@ func TestCached_EvictsByBytes(t *testing.T) {
6667

6768
first, second, third := mkcid(1), mkcid(2), mkcid(3)
6869
for _, k := range []cid.Cid{first, second, third} {
69-
if _, err := c.GetBlock(context.Background(), k); err != nil {
70+
if _, err := c.GetBlock(context.Background(), did.Undef, k); err != nil {
7071
t.Fatalf("get: %v", err)
7172
}
7273
}
7374

7475
// first was evicted -> refetched (2 base hits).
75-
if _, err := c.GetBlock(context.Background(), first); err != nil {
76+
if _, err := c.GetBlock(context.Background(), did.Undef, first); err != nil {
7677
t.Fatalf("get: %v", err)
7778
}
7879
if got := base.calls[first.KeyString()]; got != 2 {
7980
t.Fatalf("expected first block evicted and refetched (2 base hits), got %d", got)
8081
}
8182
// third is the most-recently-used and still cached (1 base hit).
82-
if _, err := c.GetBlock(context.Background(), third); err != nil {
83+
if _, err := c.GetBlock(context.Background(), did.Undef, third); err != nil {
8384
t.Fatalf("get: %v", err)
8485
}
8586
if got := base.calls[third.KeyString()]; got != 1 {

0 commit comments

Comments
 (0)