S3-compatible virtual sandbox alongside the R2 one?

[davidmyriel]

Hi, I'd like to write a generic S3 adapter for our users (I work at Tigris data).

@flue/sdk/cloudflare#getVirtualSandbox(R2) is great if you're on Cloudflare.
However, there's no equivalent for Node, GitHub Actions, Workers without an R2 binding, or buckets that aren't R2.
The virtual sandbox falls back to in-memory and the agent forgets everything between calls.

I'd like to add @flue/sdk/s3#getS3Sandbox. It is the same BashFactory shape as the existing one, generic over any S3-compatible endpoint (AWS, Tigris, MinIO, R2's S3 API). @aws-sdk/client-s3 as an optional peer dep, the way wrangler is today. Lazy-imported, so projects that don't use it pay nothing.

Happy to share the implementation if it'd help you decide.

Dave

[tunnckoCore]

Good idea for sure!

It's the reason i lean more to Cloudflare instead of the other options.

[ketankhairnar]

@davidmyriel — sketching this out as a fellow contributor in case it helps the maintainers shape the work.

One-line proposal. Mirror the existing getVirtualSandbox shape (factory return, lazy peer dep, MountableFs(InMemoryFs) mounted at /workspace) with a pure object-store storage model — no sidecar SQLite, no external metadata DB.

The three substantive choices

Choice	What	Why
Storage	Pure object-store + key conventions + CopyObject for mv/cp	Zero new infra; bucket is the source of truth; works on every S3-compatible backend in scope (AWS, Tigris, MinIO, R2-S3-API, GCS-XML).
Metadata	Per-object headers (x-amz-meta-flue-type, flue-mode) + in-process LRU caches (stat 60 s, list 30 s, negative 5 s)	Stateless. No schema, no migrations, no recovery procedure. Process restart is free.
Client surface	User passes their own S3Client instance	Mirrors getVirtualSandbox(bucket: R2Bucket) — instance, not config. flue stays out of credentials and out of provider-config drift.

Other details (peer-dep wiring, package layout, symlink encoding, empty-dir markers, conditional-write strategy, multipart thresholds) all follow from these three.

API sketch

export interface S3SandboxOptions {
  bucket: string;
  prefix?: string;                    // session isolation; default 'default'
  metadataMode?: 'auto' | 'sidecar';  // 'auto' = headers (v1); 'sidecar' = ENOTSUP in v1
  maxEntries?: number;                // readdir cap, default 10_000
  statTtlMs?: number;                 // default 60_000
  listTtlMs?: number;                 // default 30_000
  multipartThreshold?: number;        // default 5 * 1024 * 1024
}

export async function getS3Sandbox(): Promise<BashFactory>;
export async function getS3Sandbox(client: S3Client, options: S3SandboxOptions): Promise<BashFactory>;

@aws-sdk/client-s3 lives in peerDependenciesMeta.optional, lazy-imported, so users who never touch the s3 path pay nothing — same pattern as wrangler today.

Four PRs to land it

1. S3FileSystem adapter against the just-bash FS contract (read/write/stat/lstat/mkdir/readdir/rm/cp/mv/symlink/readlink/realpath). Mocked-S3 unit tests.
2. Caching + symlink resolver (depth cap 40 = POSIX SYMLOOP_MAX).
3. getS3Sandbox entry, package wiring (./s3 export, peer dep), example agent.
4. MinIO via testcontainers in CI; parity test against the existing CF path; connectors/sandbox--s3.md + docs/connect-s3.md worked examples.

Trade-offs being accepted (would go in the connector doc)

• mv is CopyObject + DeleteObject — server-side copy, no re-upload, but not atomic.
• link() is CopyObject — no shared inode, no refcount. Same compromise s3fs-fuse makes.
• appendFile is GET → concat → PUT — not atomic.
• Sub-second mtime is lost (S3 LastModified is second-resolution).
• Concurrent writers across processes are last-writer-wins in v1. Conditional writes (If-Match) are the v2 unlock — S3 supports them since Aug 2024 and R2/Tigris/MinIO match.

Real open questions for maintainers

1. link() semantics. CopyObject with documented divergence, or throw ENOTSUP? Both are defensible.
2. Empty-args overload parity with the CF module — should @flue/sdk/s3 also return an InMemoryFs-backed factory when called with no arguments, or only the bucket-backed shape?
3. Bundle target verification. The issue mentions Workers without an R2 binding. AWS SDK v3 has had a Workers-friendly fetch handler since 3.582; needs a Phase 3 verification step against a tsdown-built bundle.

Out of v1, deliberately

Sidecar SQLite, external SQL metadata, cross-bucket mv, customer-managed encryption. The metadataMode: 'sidecar' option name is reserved so v2 stays API-compatible.

Offer

David — if the shape lands well with maintainers, happy to take Phase 1 as a contract-locking PR for you to build on, or hand the whole thing to you. Either way works.

[FredKSchott]

+1! Can this just be a generic just-bash + S3 adapter? We're intentionally trying to treat the sandbox setup as "outside" of Flue, so living inside the Flue package doesn't make sense. Over time we'll probably make this even more explicit.

You could publish this as an npm package, and than anyone using just-bash or Flue could use it.

I was considering suggesting this as a connector for flue add s3, but that wouldn't be right fit because not all users will be using just-bash.

[ketankhairnar]

@FredKSchott - that framing makes sense. Reading my earlier sketch back through that lens, the S3FileSystem adapter and the cache/symlink-resolver layer are pure just-bash concerns; the only Flue-shaped piece was the getS3Sandbox entrypoint, and that collapses to a one-liner over the adapter once it exists.

@davidmyriel - proposed reshape: publish just-bash-fs-s3 (or whatever name suits) as a standalone npm package. Surface stays the same as my earlier sketch — user passes their own S3Client, headers-as-metadata, CopyObject for mv/cp, in-process LRU caches, lazy peer-dep on @aws-sdk/client-s3. Flue users wire it up themselves: init({ sandbox: () => new Bash({ fs: s3Fs(...) }) }) or via a small factory the package itself exports.

If it'd help with discoverability, Flue could optionally ship a connectors/sandbox--s3.md recipe pointing at the published package once it exists, but that's downstream of you publishing, and only if Fred wants it in-tree (his comment leans toward keeping it out).

Happy to write a contract-locking starter PR against just-bash to anchor the FS shape if that helps you avoid moving targets; let me know.

[astrobot-houston]

Possibly related contribution from @Xe — opened #171 which appears to be addressing this.

Their summary: Adds a new connector specification document for @tigrisdata/agent-shell, which provides sandboxed command execution and file I/O backed by Tigris object storage. This enables agents to operate on dedicated buckets, shared long-lived buckets, or per-invocation bucket forks for isolation.

Implementation: Xe/flue@Xe/sandbox-agent-shell (diff)

This comment was posted automatically when #171 was redirected. The implementation is preserved on the branch above so it can inform the work here.