Skip to content

Commit 479c0cd

Browse files
alanshawalanshaw
authored
feat: initial management API implementation (#1) (#2) (#3) (#4) (#5) (#6) (#8) (#9) (#10) (#11) (#12)
This PR introduces a new `pkg/store` layer with both in-memory and Postgres-backed implementations for core entities (provider, tenant, bucket, access key, delegation), plus embedded goose migrations and test utilities to run Postgres-backed store tests via testcontainers. **Changes:** - Add store interfaces + in-memory and Postgres implementations for provider/tenant/bucket/accesskey/delegation. - Add embedded goose migrations and a migrations runner to initialize Postgres schema. - Add test utilities and cross-implementation tests (memory + Postgres) for each store. --- Adds a new Go client for interacting with the upload service via UCAN RPC invocations, along with structured logging helpers to make invocation arguments/metadata easier to inspect in logs. **Changes:** - Introduce `pkg/client.UploadClient` with `RegisterCustomer` (`/customer/add`) and `ProvisionSpace` (`/provider/add`) operations. - Add `pkg/lib/zapucan` helpers for logging UCAN invocations and decoding CBOR-encoded IPLD maps into zap fields. - Bump `github.com/fil-forge/libforge` dependency version. --- Adds initial scaffolding for running hilt as a service: a Cobra-based `hilt serve` command that loads config via Viper and starts an Echo HTTP server wired together with Uber Fx. **Changes:** - Introduce Fx modules for config surfacing, zap logging, and Echo server lifecycle management. - Add Viper-based config loading with defaults, env var support, and pflag binding. - Add `cmd/main.go` CLI entrypoint plus basic Makefile/.gitignore and new dependencies (Echo, Cobra, Viper, Fx). --- Adds initial Tenant + Access Key management API scaffolding and wires store backends (memory/postgres) into the Fx app based on configuration, including new config types/flags and Postgres migration hooks. **Changes:** - Introduces API route stubs and request/response types for tenant + access key endpoints, and auto-registers routes onto the Echo server via an Fx group. - Adds storage backend selection (memory vs postgres) in `AppModule`, plus Fx modules to provide the corresponding store implementations. - Expands configuration to include `storage.*` and `storage.postgres.*` settings and flag bindings; standardizes PG unique-violation handling using `pgerrcode`. --- Adds a configurable “partner key” authentication mechanism to gate access to the Tenant API routes, wiring the key through config/flags and enforcing it via Echo middleware. **Changes:** - Introduces `AuthConfig` (`auth.partner_key`) and exposes it through the fx config module. - Wraps all Tenant API routes in partner-key bearer auth middleware while keeping `/` and `/health` open. - Adds new Echo middleware + unit tests for partner-key bearer authentication. --- Adds a KMS-agnostic `vault.Vault` interface plus in-memory and HashiCorp Vault (KV v2) implementations, and wires vault selection/config into the fx app and CLI so Hilt can persist private key material in a configurable backend. **Changes:** - Introduce `pkg/vault.Vault` interface with `ErrNotFound`, plus an in-memory implementation and a HashiCorp Vault (KV v2) implementation (including AppRole auth helper). - Add config schema + viper defaults + CLI flags for selecting/configuring the vault backend, and wire backend selection into the fx app graph. - Add tests for the vault interface contract and HashiCorp AppRole login using a testcontainers Vault dev container. --- Implements tenant provisioning via `PUT /tenants/{tenantId}` by introducing an external tenant identifier, wiring a did:plc directory client, generating/storing tenant keys in Vault, and persisting tenant records. **Changes:** - Added `external_id` to tenant storage model and introduced `GetByExternalID` for idempotent provisioning. - Added PLC directory configuration + fx wiring to provide a `plc.DirectoryClient`. - Implemented `ProvisionTenant` API handler with accompanying tests (memory stores + httptest PLC server). --- Implements the remaining tenant management endpoints by wiring real store-backed handlers and adding the store capabilities required to support tenant deletion cascades (buckets, access keys, delegations, vault secrets, and did:plc deactivation). **Changes:** - Implement `GET /tenants/:tenantId`, `POST /tenants/:tenantId/status`, and `DELETE /tenants/:tenantId` handlers (with did:plc deactivation + cascade cleanup). - Extend tenant/bucket/access-key stores with `Delete` and “list by tenant” capabilities, including pagination for buckets. - Add/extend unit tests for the new store methods and HTTP handlers; bump dependencies to newer `ucantone` and `dag-json-gen`. --- This PR implements the Management API access-key endpoints, including key material storage in Vault and issuing tenant→access-key UCAN delegations based on requested S3 permissions. It also extends the store layer to support access-key expiry and efficient bucket lookups by ID for response rendering. **Changes:** - Implement Create/List/Get/Delete access-key handlers, including Vault secret storage and delegation issuance/revocation. - Add `expires_at` support for access keys across schema, store implementations, and tests. - Extend bucket listing to support filtering by explicit bucket IDs via new `bucket.ListOption`/`bucket.ListConfig`. --- Adds initial scaffolding for Hilt’s UCAN-based RPC surface (used by Ingot for S3 tenant management), including an identity-backed UCAN server mounted on the existing Echo HTTP server. **Changes:** - Introduces UCAN RPC handler stubs and wires them into an `fx` module that builds a `ucantone` HTTP server. - Adds a configurable service identity (PEM-backed or ephemeral) and exposes a public `did:web` DID document endpoint. - Extends configuration/CLI flags to support identity settings.
1 parent 684c688 commit 479c0cd

84 files changed

Lines changed: 7330 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/go-check.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
name: Go Checks
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches: ["main"]
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: read
11+
12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
14+
cancel-in-progress: true
15+
16+
jobs:
17+
go-check:
18+
uses: ipdxco/unified-github-workflows/.github/workflows/go-check.yml@v1.0

‎.github/workflows/go-test-config.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"skip32bit": true,
3+
"skipOSes": ["windows"]
4+
}

‎.github/workflows/go-test.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: Go Test
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches: ['main']
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: read
11+
12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
14+
cancel-in-progress: true
15+
16+
jobs:
17+
go-test:
18+
uses: ipdxco/unified-github-workflows/.github/workflows/go-test.yml@v1.0
19+
with:
20+
go-versions: '["this"]'
21+
secrets:
22+
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

‎.github/workflows/publish-ghcr.yml‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
name: Container
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
# Cancel in-progress runs on new pushes to same ref
10+
concurrency:
11+
group: container-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
env:
15+
REGISTRY: ghcr.io
16+
IMAGE_NAME: ${{ github.repository }}
17+
18+
jobs:
19+
# Build both dev and prod on main push
20+
publish-main:
21+
name: Publish ${{ matrix.target }}
22+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
23+
runs-on: ubuntu-latest
24+
permissions:
25+
contents: read
26+
packages: write
27+
strategy:
28+
matrix:
29+
include:
30+
- target: prod
31+
suffix: ""
32+
- target: dev
33+
suffix: "-dev"
34+
steps:
35+
- name: Checkout
36+
uses: actions/checkout@v4
37+
38+
- name: Set up QEMU
39+
uses: docker/setup-qemu-action@v3
40+
41+
- name: Set up Docker Buildx
42+
uses: docker/setup-buildx-action@v3
43+
44+
- name: Log in to GHCR
45+
uses: docker/login-action@v3
46+
with:
47+
registry: ${{ env.REGISTRY }}
48+
username: ${{ github.actor }}
49+
password: ${{ secrets.GITHUB_TOKEN }}
50+
51+
- name: Extract metadata
52+
id: meta
53+
uses: docker/metadata-action@v5
54+
with:
55+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
56+
tags: |
57+
type=raw,value=main${{ matrix.suffix }}
58+
type=sha,suffix=${{ matrix.suffix }}
59+
60+
- name: Build and push
61+
uses: docker/build-push-action@v6
62+
with:
63+
context: .
64+
target: ${{ matrix.target }}
65+
platforms: linux/amd64,linux/arm64
66+
push: true
67+
tags: ${{ steps.meta.outputs.tags }}
68+
labels: ${{ steps.meta.outputs.labels }}
69+
cache-from: type=gha,scope=${{ matrix.target }}
70+
cache-to: type=gha,scope=${{ matrix.target }},mode=max
71+
72+
# Build check for PRs (no push) - single platform for speed
73+
build-check:
74+
name: Build Check (${{ matrix.target }})
75+
if: github.event_name == 'pull_request'
76+
runs-on: ubuntu-latest
77+
permissions:
78+
contents: read
79+
strategy:
80+
matrix:
81+
target: [dev, prod]
82+
steps:
83+
- name: Checkout
84+
uses: actions/checkout@v4
85+
86+
- name: Set up Docker Buildx
87+
uses: docker/setup-buildx-action@v3
88+
89+
- name: Build ${{ matrix.target }}
90+
uses: docker/build-push-action@v6
91+
with:
92+
context: .
93+
target: ${{ matrix.target }}
94+
platforms: linux/amd64
95+
push: false
96+
cache-from: type=gha,scope=${{ matrix.target }}

‎.github/workflows/release-binaries.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
name: Binaries Releaser
2+
on:
3+
release:
4+
types: [ published ]
5+
workflow_dispatch:
6+
workflow_run:
7+
workflows: [Releaser]
8+
types: [completed]
9+
10+
permissions:
11+
contents: write
12+
packages: write
13+
14+
jobs:
15+
bin-releaser:
16+
name: Release Binaries
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v4
21+
with:
22+
fetch-depth: 0
23+
- name: Set up Go
24+
uses: actions/setup-go@v5
25+
with:
26+
go-version: "1.25.x"
27+
- name: Set up QEMU
28+
uses: docker/setup-qemu-action@v3
29+
- name: Set up Docker Buildx
30+
uses: docker/setup-buildx-action@v3
31+
- name: Log in to the Container registry
32+
uses: docker/login-action@v3
33+
with:
34+
registry: ghcr.io
35+
username: ${{ github.actor }}
36+
password: ${{ secrets.GITHUB_TOKEN }}
37+
- name: Release Binaries and Containers
38+
uses: goreleaser/goreleaser-action@v6
39+
with:
40+
distribution: goreleaser
41+
version: latest
42+
args: release --clean
43+
env:
44+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/release-check.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
name: Release Checker
2+
3+
on:
4+
pull_request_target:
5+
paths: [ 'version.json' ]
6+
types: [ opened, synchronize, reopened, labeled, unlabeled ]
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: write
11+
pull-requests: write
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
jobs:
18+
release-check:
19+
uses: ipdxco/unified-github-workflows/.github/workflows/release-check.yml@v1.0

‎.github/workflows/releaser.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: Releaser
2+
3+
on:
4+
push:
5+
paths: [ 'version.json' ]
6+
workflow_dispatch:
7+
8+
permissions:
9+
contents: write
10+
11+
concurrency:
12+
group: ${{ github.workflow }}-${{ github.sha }}
13+
cancel-in-progress: true
14+
15+
jobs:
16+
releaser:
17+
uses: ipdxco/unified-github-workflows/.github/workflows/releaser.yml@v1.0

‎.github/workflows/tagpush.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
name: Tag Push Checker
2+
3+
on:
4+
push:
5+
tags:
6+
- v*
7+
8+
permissions:
9+
contents: read
10+
issues: write
11+
12+
concurrency:
13+
group: ${{ github.workflow }}-${{ github.ref }}
14+
cancel-in-progress: true
15+
16+
jobs:
17+
releaser:
18+
uses: ipdxco/unified-github-workflows/.github/workflows/tagpush.yml@v1.0

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
hilt

‎Dockerfile‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Build stage - runs natively on build machine, cross-compiles to target
2+
FROM --platform=$BUILDPLATFORM golang:1.26-bookworm AS build
3+
ARG TARGETARCH
4+
ARG TARGETOS=linux
5+
WORKDIR /go/src/hilt
6+
COPY go.mod go.sum* ./
7+
RUN --mount=type=cache,target=/go/pkg/mod \
8+
go mod download || true
9+
COPY . .
10+
11+
# Production build - stripped binary
12+
FROM build AS build-prod
13+
RUN --mount=type=cache,target=/go/pkg/mod \
14+
--mount=type=cache,target=/root/.cache/go-build \
15+
CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
16+
go build -ldflags="-s -w" -o /hilt ./cmd/main.go
17+
18+
# Development build - debug-friendly binary
19+
FROM build AS build-dev
20+
RUN --mount=type=cache,target=/go/pkg/mod \
21+
--mount=type=cache,target=/root/.cache/go-build \
22+
CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
23+
go build -gcflags="all=-N -l" -o /hilt ./cmd/main.go
24+
RUN GOARCH=${TARGETARCH} go install github.com/go-delve/delve/cmd/dlv@latest && \
25+
cp /go/bin/linux_${TARGETARCH}/dlv /go/bin/dlv 2>/dev/null || cp /go/bin/dlv /go/bin/dlv 2>/dev/null || true
26+
27+
# Production target - minimal runtime
28+
FROM debian:bookworm-slim AS prod
29+
RUN apt-get update && apt-get install -y --no-install-recommends \
30+
ca-certificates \
31+
curl \
32+
&& rm -rf /var/lib/apt/lists/*
33+
COPY --from=build-prod /hilt /usr/bin/hilt
34+
EXPOSE 8080
35+
ENTRYPOINT ["/usr/bin/hilt", "serve"]
36+
37+
# Development target - includes debug tools
38+
FROM debian:bookworm-slim AS dev
39+
RUN apt-get update && apt-get install -y --no-install-recommends \
40+
ca-certificates \
41+
curl \
42+
bash-completion \
43+
less \
44+
vim-tiny \
45+
procps \
46+
htop \
47+
strace \
48+
iputils-ping \
49+
dnsutils \
50+
net-tools \
51+
tcpdump \
52+
jq \
53+
&& rm -rf /var/lib/apt/lists/*
54+
COPY --from=build-dev /hilt /usr/bin/hilt
55+
COPY --from=build-dev /go/bin/dlv /usr/bin/dlv
56+
EXPOSE 8080
57+
ENTRYPOINT ["/usr/bin/hilt", "serve"]

0 commit comments

Comments
 (0)