feat(rust): Initialize Rust CLI foundation #6915
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Set up project structure mirroring Go CLI architecture - Add dependencies with libsodium-sys-stable for all crypto operations - Implement core crypto module with Argon2, ChaCha20-Poly1305, and Blake2b - Create data models for accounts, files, and collections - Set up Clap-based CLI framework with account, export, and version commands - Add error handling with thiserror - Configure for static linking to create standalone binaries This establishes the foundation for converting the Ente CLI from Go to Rust, with a focus on maintaining compatibility with existing libsodium-based crypto. Co-Authored-By: Claude <[email protected]>
Typical Claude Code commit message: feat: implement user authentication - Added login endpoint - Implemented JWT tokens - Created middleware Created with Claude Code: https://claude.ai/code # <-- The promotional link Co-authored-by: Claude <[email protected]> # <-- The co-author line This memory is to ask claude to keep the co-author line but remove the self promo from the commit message it creates.
- Fix code formatting with cargo fmt - Remove unnecessary type casts - Use range contains instead of manual comparison - Prefix unused variables with underscore - Remove unused imports - Add allow(dead_code) for development phase Co-Authored-By: Claude <[email protected]>
For the current session cc was able to use that instruction to figure out all the linters etc to run. If that doesn't work in future sessions, we can use a longer instruction, something like what it itself suggested
## CI/CD Requirements
- Must pass `cargo fmt --check`
- Must pass `cargo clippy --all-targets --all-features`
- Must pass `RUSTFLAGS="-D warnings" cargo build`
- Fix all formatting before committing
- Address all clippy warnings
- Use `#![allow(dead_code)]` during development for unused code
## Code Quality
- Run `cargo fmt` before committing
- Fix clippy warnings: remove unnecessary casts, use idiomatic Rust
- Prefix unused variables with underscore
- Remove unused imports
- Replace libc::c_char with std::ffi::c_char for password parameter - Remove unnecessary libc dependency - Use standard library FFI types (available since Rust 1.64) This fixes the CI build error where libsodium expects *const c_char for the password parameter in crypto_pwhash. Co-Authored-By: Claude <[email protected]>
- Replace sled with SQLite for better reliability and tooling - Create schema with tables for accounts, secrets, collections, files, and sync state - Implement account storage with multi-account support - Add configuration and sync state management - Support for storing encrypted credentials separately - Add indices for performance optimization SQLite provides ACID transactions, better debugging tools, and a proven track record for reliability with user data. Co-Authored-By: Claude <[email protected]>
- Document current implementation status - Detail API client implementation steps - List all remaining components with specifications - Include testing strategy and migration notes - Provide file structure reference for navigation - Add implementation guidelines and environment variables This plan enables any developer to understand the project state and continue the conversion work from the current point. Co-Authored-By: Claude <[email protected]>
Add detailed development commands, architecture overview, and module descriptions to help future Claude instances understand the codebase structure and development workflow. Co-Authored-By: Claude <[email protected]>
Add complete API client implementation with authentication, file operations, and collection management. This enables the Rust CLI to interact with Ente servers for photo backup and sync operations. Key features: - Multi-account token management with secure storage - SRP authentication flow matching Go implementation - Retry logic with exponential backoff for network resilience - Full API coverage: auth, collections, files, trash, user details - Request/response models for all API endpoints - Separate download client for large file transfers - Smart CDN routing for production file downloads The implementation follows the conversion plan and maintains compatibility with the existing Go CLI API patterns. Co-Authored-By: Claude <[email protected]>
Add comprehensive account management functionality with secure SRP authentication. This enables users to add, list, update, and manage multiple Ente accounts for photos, locker, and auth apps. Key features: - Complete account add flow with SRP authentication - Two-factor authentication support (TOTP) - Secure key decryption and storage - Multi-account support with per-app configuration - Account list and update commands - Export directory management - Interactive CLI prompts with dialoguer The implementation integrates with the API client for authentication and securely stores account credentials in SQLite. Co-Authored-By: Claude <[email protected]>
Fixed issues preventing successful authentication: - Corrected Argon2 memory limit handling (API sends bytes, not KB) - Replaced Blake2b with crypto_kdf_derive_from_key for login subkey derivation - Fixed serde field names to match API expectations (srpUserID, sessionID) - Added non-interactive mode for CLI testing - Added support for ENTE_ENDPOINT environment variable The implementation now matches the web client's key derivation exactly, enabling successful authentication with both local and production servers. Co-Authored-By: Claude <[email protected]>
- Use std::ffi::c_char for libsodium FFI context parameter cast - Fix all clippy warnings to pass CI with RUSTFLAGS="-D warnings" - Update CLAUDE.md with FFI casting guidance for future development This ensures the code passes all CI checks including the stricter clippy settings used in GitHub Actions. Co-Authored-By: Claude <[email protected]>
…lation - Add endpoint field to accounts database table with default to production API - Update Account model to include endpoint field - Add --endpoint flag to account add command only - Remove ENTE_ENDPOINT environment variable support - Update account list to display endpoints in readable format - Each account now maintains its own endpoint, preventing confusion between test and production environments Co-Authored-By: Claude <[email protected]>
- Fix token encoding to use base64 URL with padding (matching Go implementation) - Add export command that iterates through collections and fetches files - Update API models to handle actual server response field names (ownerID vs ownerId) - Fix file download URLs for local/dev environments - Implement proper directory structure creation (YYYY/MM-Month format) - Add collection attributes and public URL models for complete API compatibility - Successfully exports encrypted files from both local and production endpoints The export command now: - Fetches all collections for an account - Iterates through each collection to get files - Downloads encrypted files and saves them to the export directory - Skips already downloaded files to support incremental exports Note: Files are still encrypted; decryption will be implemented in a future commit. Co-Authored-By: Claude <[email protected]>
- Applied cargo fmt to ensure consistent formatting - Fixed all clippy warnings (uninlined_format_args) - Code now passes all CI checks with RUSTFLAGS="-D warnings" Co-Authored-By: Claude <[email protected]>
Add explicit pre-commit commands that must be run before every commit to ensure CI passes. These commands simulate the CI environment locally. Co-Authored-By: Claude <[email protected]>
- Fix lifetime elision warnings in storage/mod.rs by adding explicit lifetimes - Collapse nested if-let statements in export.rs using let-chains - Code now passes: cargo clippy --all-targets --all-features Co-Authored-By: Claude <[email protected]>
Ensure clippy commands use --all-targets --all-features flags to match the CI environment exactly. This prevents CI failures from warnings that weren't caught locally. Co-Authored-By: Claude <[email protected]>
- Add ChaCha20-Poly1305 decryption for downloaded files - Decrypt file keys using master key - Extract nonce from encrypted file data - Add basic filename generation with extension detection - Comment out sync modules temporarily due to model mismatches The export command now properly decrypts files instead of saving them encrypted. Next steps: extract original filenames from decrypted metadata. Co-Authored-By: Claude <[email protected]>
- Create metadata module with FileMetadata struct - Decrypt file metadata to extract original filename - Use original filename in export path generation - Add proper file type detection from metadata - Implement filename sanitization for filesystem safety Files are now exported with their original names instead of generic IDs. Co-Authored-By: Claude <[email protected]>
- Add streaming cipher module using libsodium's secretstream API - Update file and metadata decryption to use streaming XChaCha20-Poly1305 - Fix decryption issues - files now properly decrypt - Successfully tested with real account - exports working for smaller files The export now correctly decrypts files using the same streaming cipher as the Go implementation. Large files may need chunked decryption support. Co-Authored-By: Claude <[email protected]>
- Implement chunked streaming decryption matching Go's 4MB buffer size - Update file decryption to use decrypt_file_data instead of decrypt_stream - Successfully tested with 33MB RAW image file - All test files now decrypt correctly Large files are now properly handled with chunked decryption, preventing memory issues and matching the Go implementation's behavior. Co-Authored-By: Claude <[email protected]>
- Show progress for each exported file with count - Improve export summary with emojis and better formatting - Add contextual success messages based on export results - Make export output more user-friendly The export now provides clear feedback during the process and a helpful summary at the end. Co-Authored-By: Claude <[email protected]>
- Mark streaming XChaCha20-Poly1305 implementation as complete - Document successful export functionality with all decryption working - Update testing status with successful real account exports - Add recent achievements section highlighting key milestones - Update feature parity progress checklist - Document what components are complete vs remaining The export functionality is now fully working with proper decryption of collections, files, and metadata. Updated PR description as well. Co-Authored-By: Claude <[email protected]>
Security improvements: - Remove all debug logs that output tokens, keys, or credentials - Remove email addresses from debug output - Remove encrypted keys and nonces from logs - Remove specific account references from documentation - Add security guidelines to CLAUDE.md No sensitive information (PII, credentials, tokens) should be logged even in debug mode. Updated guidelines to prevent future occurrences. Co-Authored-By: Claude <[email protected]>
- Add new `sync` command to fetch collections and file metadata - Change config directory from ~/.ente/ to ~/.config/ente-cli/ to avoid conflicts with Go CLI - Fix sync engine to use correct API endpoints (/collections/v2/diff instead of /diff) - Implement per-collection file syncing matching Go CLI behavior - Fix foreign key constraints in database schema - Add metadata-only and full sync options - Store database path in Storage struct for creating new instances - Successfully tested with real account: syncs 5 files and exports correctly The sync command now properly fetches all collections and files from the API, storing them in SQLite for offline access and incremental sync support. Co-Authored-By: Claude <[email protected]>
- Fixed formatting issues in sync/engine.rs - Added #[allow(dead_code)] for unused storage field in DownloadManager - Replaced manual clamp with .clamp() method Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
- Handle non-interactive mode in account add command - Fix cross-filesystem file move issue by using copy+delete instead of rename - Successfully tested downloading files from local server Co-Authored-By: Claude <[email protected]>
- Add configurable retry with exponential backoff for API calls - Handle 429 and 5xx errors with automatic retries - Add export filters for albums, shared, and hidden collections - Fix formatting and clippy warnings to pass CI checks Co-Authored-By: Claude <[email protected]>
Check both public magic metadata (for edited names) and regular metadata when determining file names during export and sync, matching Go CLI behavior Co-Authored-By: Claude <[email protected]>
…stamps Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
- Hash-based file deduplication prevents duplicate exports - Live photo extraction from ZIP archives - Update conversion status documenting feature completion - Make commit guidelines prominent in CLAUDE.md - Remove redundant commit format section Co-Authored-By: Claude <[email protected]>
Align with Go CLI by integrating sync into export workflow. Update CLAUDE.md to prevent default template usage in commits. Co-Authored-By: Claude <[email protected]>
Switch from date-based (YYYY/MM-Month) to album-based directory structure to ensure compatibility with Go CLI. Files now export to AlbumName/ folders with "Uncategorized" for files without albums. Co-Authored-By: Claude <[email protected]>
Export album and file metadata to .meta folders within each album directory. Enables incremental sync and compatibility with Go CLI exports. Co-Authored-By: Claude <[email protected]>
Fixed export to properly organize files into album folders by: - Fetching files from all collections using /collections/v2/diff endpoint - Decrypting encrypted collection names to get actual album names - Using decrypted album names for folder organization Files now export to proper album folders instead of all going to "Uncategorized". Tested and verified with local data. Co-Authored-By: Claude <[email protected]>
Changed metadata export to match Go CLI's timestamp format. Timestamps now serialize as ISO 8601 strings with timezone offset (e.g., "2025-07-23T19:48:06.098+05:30") instead of Unix microseconds. Also fixed clippy warnings to ensure CI compliance. Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
- Add rename detection by tracking files via ID in metadata - Remove old files (including live photo MOV components) when renamed - Copy live photo MOV components during hash deduplication - Preserve file deduplication optimization while handling renames correctly This ensures that when a file is renamed in Ente, the old file is removed and replaced with the renamed version, matching the Go CLI's behavior. Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Rust CLI achieves feature parity with Go CLI for photos app core functionality
Changes
Remaining