Skip to content

add external storage endpoints#8299

Draft
Light2Dark wants to merge 16 commits intomainfrom
sham/storage-endpoints
Draft

add external storage endpoints#8299
Light2Dark wants to merge 16 commits intomainfrom
sham/storage-endpoints

Conversation

@Light2Dark
Copy link
Contributor

@Light2Dark Light2Dark commented Feb 13, 2026

📝 Summary

  • Add storage list and download endpoints (/api/storage/list_entries, /api/storage/download) that let the frontend browse and download files from external storage backends (obstore, fsspec)
  • Downloads go through VirtualFile.create_and_register(), a new helper that creates a virtual file URL served by the existing /@file/ route
  • Track storage namespaces in SessionView so they're pruned when variables go out of scope and replayed on reconnect

🔍 Description of Changes

📋 Checklist

  • I have read the contributor guidelines.
  • For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
  • Tests have been added for the changes made.
  • Documentation has been updated where applicable, including docstrings for API changes.
  • Pull request title is a good summary of the changes - it will be used in the release notes.

@vercel
Copy link

vercel bot commented Feb 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Feb 13, 2026 3:07pm

Request Review

@Light2Dark Light2Dark changed the title Sham/storage endpoints external storage endpoints Feb 13, 2026
@github-actions github-actions bot added the bash-focus Area to focus on during release bug bash label Feb 13, 2026
@Light2Dark Light2Dark added enhancement New feature or request preview Experimental or preview-only feature labels Feb 13, 2026
(e.g., using content-disposition headers).
"""
file_bytes = await self.download(path)
filename = path.rsplit("/", 1)[-1] or "download"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could the path by ""?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't fail, downloads the root. Added a test case

return

try:
entries = backend.list_entries(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this be async?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! missed this


name: ClassVar[str] = "storage-download-ready"
request_id: RequestId
url: Optional[str] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be better to just pass the bytes and a mimetype, and construct a data-url on the frontend. have you thought through the tradeoffs? the pro i see is that you don't need a dependency on the VFile system

Copy link
Contributor Author

@Light2Dark Light2Dark Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, my reasoning was we don't want to pass back a large payload through the websocket channel, similar to downloading table data 🤔. Let the frontend/browser handle downloading it. I don't think it's a strong reason..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok sounds good. It might fail since not tied to a cell but I can't remember

app_router.include_router(sql_router, prefix="/api/sql", name="sql")
app_router.include_router(
external_storage_router,
prefix="/api/external-storage",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thoughts on just /api/storage? just a cleaner route and maybe lets use move the filesystem there as well

@Light2Dark Light2Dark changed the title external storage endpoints add external storage endpoints Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bash-focus Area to focus on during release bug bash enhancement New feature or request preview Experimental or preview-only feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants