Skip to content

S3 on Forge #36

Description

@frrist

Track the prototype-to-production path for an S3-compatible API on top of the Forge stack. The S3 listener (codename ms3t) currently lives at pkg/ms3t/ in this repo. See pkg/ms3t/architectural.md for the prototype-level design notes, the choice points, and open questions.

Why sprue, not a separate repo? A pragmatic prototype-stage call: keep ms3t in-process with sprue so it inherits sprue's identities + clients with zero auth coordination. Up for re-evaluation once we settle a tenant model.

In scope for this epic

Status snapshot

# Title Status
#37 Define S3 API subset for the spike proposed
#38 Bootstrap S3 facade in flight (#34)
#39 Bucket mutation data storage proposal proposed
#40 ms3t in smelt + S3 smoke tests proposed
#41 ms3t telemetry + Grafana dashboards partial (#35, storacha/smelt#23)
#42 Staging deployment + production infra (spike) proposed
#43 Tenant management API (spike) proposed

Explicitly deferred

  • Sharding. Assume all data received is <256 MB. The sharding proposal in flight is fil-one/RFC#2; revisit once it lands. Adjacent prior art: storacha/RFC#65 (Filepack archive format), storacha/RFC#66 (Virtual DAG in Sharded DAG Index), and shard.rfc in this repo.
  • Multipart upload, aws-chunked body decoding, AWS Sig V4 auth, pre-signed URLs, server-side encryption, x-amz-meta-* headers, versioning, object lock — these are FilOne hard requirements but are out of scope until Spike: tenant management API #43's spike proposes a sequencing.
  • GC of unreferenced bodies, multi-instance / sticky-bucket routing, direct-passthrough to piri — see pkg/ms3t/architectural.md#future-directions.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Fields

No fields configured for Initiative.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions