Skip to content

feat: add HuggingFace Buckets API support#22

Draft
assafvayner wants to merge 11 commits intomainfrom
worktree-buckets
Draft

feat: add HuggingFace Buckets API support#22
assafvayner wants to merge 11 commits intomainfrom
worktree-buckets

Conversation

@assafvayner
Copy link
Copy Markdown
Collaborator

Summary

  • Add HFBucket handle type (parallel to HFRepository) for bucket-scoped operations
  • Add bucket lifecycle methods on HFClient: create_bucket, delete_bucket, list_buckets, move_bucket
  • Add bucket-scoped methods on HFBucket: info, list_tree, get_paths_info, get_file_metadata, batch, upload_files, download_files, delete_files
  • Add HFBucketSync blocking wrapper with all sync equivalents
  • Add hfrs buckets CLI commands matching the Python hf buckets interface: create, list/ls, info, delete, remove/rm, move, cp
  • Add error handling improvements: BucketNotFound, Forbidden, Conflict, RateLimited variants
  • Refactor xet token URL handling to support both repo and bucket token endpoints

Design

Buckets are xet-backed mutable object storage with no git history. The implementation mirrors the Python huggingface_hub library's bucket API (minus sync_bucket).

  • Design spec: docs/superpowers/specs/2026-04-09-buckets-design.md
  • Implementation plan: docs/superpowers/plans/2026-04-09-buckets-implementation.md

New files

File Purpose
src/bucket.rs HFBucket handle (owner, name, client ref)
src/types/buckets.rs BucketInfo, BucketUrl, BucketTreeEntry, BucketFileMetadata
src/types/bucket_params.rs Param structs with TypedBuilder
src/api/buckets.rs All bucket API methods + sync wrappers
src/bin/hfrs/commands/buckets/ CLI: create, list, info, delete, remove, move, cp

Out of scope (future PRs)

  • sync_bucket — bidirectional sync with plan/apply
  • copy_files — full cross-repo/bucket copy API
  • Caching (buckets have no revisions)

Test plan

  • Unit tests pass (cargo test -p huggingface-hub)
  • Clippy clean (cargo clippy --all-features -D warnings)
  • CLI builds and hfrs buckets --help shows all subcommands
  • Manual test: create, list, upload, download, delete bucket with valid HF_TOKEN

Design for adding HuggingFace Buckets API to the Rust SDK,
covering HFBucket handle, API methods, xet integration,
error handling improvements, blocking API, and CLI commands
matching the Python hf buckets interface.
13-task plan covering error handling, types, HFBucket handle,
xet refactoring, API methods, blocking wrapper, and CLI commands.
…ants

Extend HFError with dedicated variants for HTTP 403, 409, and 429 status
codes, plus BucketNotFound for bucket-specific 404s. Update check_response
to map these status codes to the new variants instead of the generic Http
error. Remove now-redundant inner status code matching in format_hf_error.
Add impl HFBucket with info, list_tree, get_paths_info,
get_file_metadata, batch, delete_files, upload_files,
and download_files methods. Xet integration uses
bucket-specific token URLs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant