Skip to content

appsjuragan/rust-file-backend

BranchesTags

Repository files navigation

🚀 Rust File Backend (RFB)

Rust React License: MIT Version

Rust File Backend (RFB) is a high-performance, enterprise-grade file management system combining the memory safety and speed of Rust with a modern React frontend. Built for cost-efficiency through content-addressable storage (deduplication) and scalability via parallel multipart uploads.

💎 Core Features

⚡ Blazing Performance

  • Built on Axum web framework and Tokio async runtime
  • Handles thousands of concurrent requests with minimal overhead
  • 10× less memory usage compared to Node.js/Python equivalents

💰 Intelligent Storage Deduplication

  • SHA-256 Content Hashing eliminates duplicate storage
  • Instant "uploads" for previously stored content
  • Drastically reduced storage costs across all users

🛡️ Multi-Layered Security

  • ClamAV Integration: Real-time virus scanning
  • Magic Byte Verification: File type validation beyond extensions
  • Path Traversal Protection: Aggressive filename sanitization
  • JWT Authentication: Secure token-based access control
  • CAPTCHA Registration: Bot-resistant account creation
  • S3-Compatible Storage: Encrypted transport layer

🧩 Resilient Parallel Uploads

  • Custom chunked upload engine with parallel workers
  • Exponential backoff retry mechanism
  • Multi-GB file support on unstable connections
  • Configurable chunk sizes (default: 10MB)

🔗 File Sharing System

  • Public Share Links: Time-limited, token-based sharing with unique URLs
  • Password Protection: Argon2-hashed passwords for sensitive shares
  • Granular Permissions: view (inline preview) or download (attachment) modes
  • Folder Sharing: Share entire folders with browsable file listings
  • Access Logging: Track views, downloads, and password attempts with IP/User-Agent
  • Public Share Page: Beautiful, responsive frontend for recipients
  • Media Preview: Inline image, video, audio, and PDF preview on shared links

🖼️ Automatic Thumbnail Generation

  • WebP Format: Optimized thumbnails (256px) for minimal bandwidth
  • Multi-Format Support: Images, PDFs (via pdftocairo), and Videos (via ffmpeg)
  • Encrypted File Detection: Skips password-protected PDFs gracefully
  • Dedicated Worker: Separate thumbnail-worker process for asynchronous generation
  • Lazy Loading: Frontend loads thumbnails asynchronously with smooth animations

📋 Advanced File Operations

  • Copy/Paste: Recursive folder duplication with deduplication
  • Bulk Actions: Move, delete, and copy multiple items
  • Archive Preview: Inspect ZIP, 7z, RAR, TAR without extraction
  • Download Tickets: Time-limited shareable links
  • PDF Preview: Inline document viewing
  • Favorites: Star/unstar files and folders for quick access

🔗 File Sharing

  • Share Links: Public or user-specific sharing with configurable permissions
  • Password Protection: Argon2id-hashed passwords for secure access
  • Expiration Control: Configurable expiry up to 1 year
  • View/Download Modes: Inline display or attachment download
  • Access Logging: Track views, downloads, and password attempts
  • Public Share Page: Standalone viewer with media preview

🔍 Advanced Search & Filtering

  • Full-Text Search: Real-time filename search with debouncing
  • Fuzzy/Similarity Search: Find files even with typos
  • Regex & Wildcard: Power-user search patterns
  • Tag Filtering: Filter by user-defined tags
  • Category Filtering: Filter by auto-detected file categories
  • Date Range: Filter files by creation date
  • Size Range: Filter files by size boundaries
  • Favorites Filter: Show only starred items

🏗️ Architecture

System Overview

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│   React     │─────▶│  Axum API    │─────▶│  PostgreSQL  │
│  Frontend   │      │  (Rust)      │      │  Database    │
└─────────────┘      └──────────────┘      └─────────────┘
                            │
                            ├─────▶ S3/RustFS (File Storage)
                            └─────▶ ClamAV (Virus Scanning)

┌──────────────────────────────────────────────────────────┐
│  Background Workers (separate processes)                 │
│  ├── Worker: Virus scanning, cleanup, facts updates      │
│  └── Thumbnail Worker: WebP thumbnail generation         │
└──────────────────────────────────────────────────────────┘

Backend (api/)

Technology Stack:

  • Language: Rust 2024 Edition
  • Web Framework: Axum 0.7
  • ORM: SeaORM (PostgreSQL & SQLite)
  • Storage: AWS SDK for Rust (S3-compatible)
  • Security: JWT, Argon2, ClamAV, CAPTCHA
  • Runtime: Tokio async

Key Modules:

  • api/handlers/ — HTTP request handlers (auth, files, upload, captcha, users, settings, shares, health)
  • services/ — Business logic (file, upload, metadata, scanner, audit, facts, share, thumbnail, worker)
  • entities/ — Database models (SeaORM)
  • infrastructure/ — Storage, database, scanner adapters
  • utils/ — Validation, auth, encryption helpers

Features:

  • Chunked multipart uploads with resume capability
  • Content-based deduplication (SHA-256)
  • Background virus scanning queue
  • Metadata extraction (EXIF, ID3, PDF, Office)
  • Recursive folder operations
  • Download ticket generation
  • File sharing with password protection and access logs
  • WebP thumbnail generation (images, PDFs, videos)
  • Favorites and folder tree navigation

Frontend (web/)

Technology Stack:

  • Framework: React 18 + TypeScript
  • Build Tool: Vite 5
  • Runtime: Bun
  • Styling: Tailwind CSS + Glassmorphism
  • Icons: Lucide React
  • Tables: TanStack Table v8
  • State: React Context + Hooks

Key Components:

  • features/dashboard/ — Main file manager interface
  • features/auth/ — Login, register, OIDC
  • features/share/ — Public share page (password gate, media preview, folder browsing)
  • lib/ — Reusable file manager library
  • services/ — API client (upload, file operations, sharing)
  • components/ — Modals, toasts, context menus, sidebar with shares

Features:

  • Drag-and-drop file upload
  • Real-time upload progress with parallel chunks
  • Copy/Cut/Paste with keyboard shortcuts
  • Bulk selection and operations
  • File preview modals (images, PDF, archives)
  • Archive content inspection
  • Responsive grid/list views
  • Thumbnail previews with lazy loading
  • Share management (create, revoke, view logs)
  • Public share page with media viewer
  • Sidebar with active shares navigation
  • Favorites toggle and filtering

🚀 Quick Start

Prerequisites

  • Rust 1.84+
  • Bun 1.1+
  • PostgreSQL 14+ (or SQLite for development)
  • RustFS, MinIO, or AWS S3
  • Redis (optional, for caching)
  • ClamAV (optional, for scanning)
  • pdftocairo (optional, for PDF thumbnails)
  • ffmpeg (optional, for video thumbnails)

Local Development

  1. Clone Repository
git clone https://github.com/appsjuragan/rust-file-backend.git
cd rust-file-backend
  1. Backend Setup
cd api
cp ../.env.sample .env
# Edit .env with your database and S3 credentials

# Run migrations
cargo run --bin rust-file-backend -- --mode migrate

# Start API server
cargo run --bin rust-file-backend -- --mode api

# Start background worker (separate terminal)
cargo run --bin rust-file-backend -- --mode worker

# Start thumbnail worker (separate terminal)
cargo run --bin rust-file-backend -- --mode thumbnail-worker
  1. Frontend Setup
cd web
cp .env.example .env
# Edit .env to set VITE_API_URL=http://localhost:3000

bun install
bun run dev
  1. Access Application

Windows Quick Start

./run.bat

🐳 Docker Deployment

Build Images

# Backend
docker build -t rfb-api:latest ./api

# Frontend
docker build --build-arg VITE_API_URL=https://your-api-domain.com -t rfb-web:latest ./web

Pre-built Images (GHCR)

Official images are available on GitHub Container Registry:

# Pull Backend (v9)
docker pull ghcr.io/appsjuragan/rust-file-backend-api:v9

# Pull Frontend (v9)
docker pull ghcr.io/appsjuragan/rust-file-backend-web:v9

Production Compose Stack

The compose stack includes:

  • API server — Axum HTTP service
  • Background worker — Virus scanning, cleanup, facts updates
  • Thumbnail worker — WebP thumbnail generation (images, PDFs, videos)
  • PostgreSQL — Primary database
  • RustFS — S3-compatible object storage
  • Web — Nginx-served React frontend
docker compose up -d

📡 API Reference

Authentication

  • POST /register — Create new user (CAPTCHA-protected)
  • POST /login — Authenticate and receive JWT
  • POST /captcha — Generate CAPTCHA challenge
  • GET /auth/oidc/login — OIDC authentication flow
  • GET /auth/oidc/callback — OIDC callback handler

File Operations

  • POST /upload — Single file upload
  • POST /files/upload/init — Initialize chunked upload
  • GET /files/upload/sessions — List pending upload sessions
  • PUT /files/upload/:id/chunk/:num — Upload chunk
  • POST /files/upload/:id/complete — Finalize upload
  • DELETE /files/upload/:id — Abort chunked upload
  • GET /files — List files (with pagination, search, filters)
  • GET /files/:id — Download file
  • POST /files/:id/ticket — Generate download ticket
  • GET /download/:ticket — Download via ticket
  • DELETE /files/:id — Delete file/folder
  • PUT /files/:id/rename — Rename or move item
  • POST /files/:id/favorite — Toggle favorite status
  • GET /files/:id/thumbnail — Get WebP thumbnail

Bulk Operations

  • POST /files/bulk-delete — Delete multiple items
  • POST /files/bulk-move — Move multiple items
  • POST /files/bulk-copy — Copy multiple items (with recursion)

Folders

  • POST /folders — Create new folder
  • GET /folders/tree — Get full folder tree for navigation
  • GET /files/:id/path — Get folder breadcrumb path

Sharing

  • POST /shares — Create a share link (public/user, password, permissions)
  • GET /shares — List user's shares (optionally filter by file)
  • DELETE /shares/:id — Revoke a share link
  • GET /shares/:id/logs — Get share access logs

Public Share (No Auth Required)

  • GET /share/:token — Get shared item info (filename, type, permissions)
  • POST /share/:token/verify — Verify share password
  • GET /share/:token/download — Download shared file (with optional file_id for folder items)
  • GET /share/:token/list — List shared folder contents

Advanced

  • POST /pre-check — Check if file exists (deduplication)
  • POST /files/link — Link existing storage file
  • GET /files/:id/zip-contents — Preview archive contents

User & Settings

  • GET /users/me — Get user profile
  • PUT /users/me — Update profile
  • GET /users/avatar/:user_id — Get public avatar image
  • POST /users/me/avatar — Upload personal avatar
  • GET /users/me/facts — Get storage statistics
  • GET /settings — Get user preferences
  • PUT /settings — Update preferences

System

  • GET /health — Health check (DB, storage, version)
  • GET /system/validation-rules — Get file validation rules

Full API documentation available at /swagger-ui endpoint.

📦 Postman Collection

Import api/postman_collection.json for ready-to-use API requests with:

  • Pre-configured authentication
  • Example payloads for all endpoints including sharing
  • Environment variables
  • Postman test scripts for auto-setting tokens

🧪 Testing

Backend Tests

cd api
cargo test

Frontend Tests

cd web
bun test

Security Scanning

# Dependency audit
cargo audit

🧹 Code Quality

Formatting

# Backend
cd api && cargo fmt

# Frontend
cd web && bun run format

Linting

# Backend
cd api && cargo clippy -- -D warnings

# Frontend
cd web && bun run lint

All code follows:

  • Rust 2024 edition conventions
  • ESLint + Prettier for TypeScript
  • No unused imports or dead code

🔧 Configuration

Backend Environment Variables

DATABASE_URL=postgresql://user:pass@localhost/rfb
JWT_SECRET=your-secret-key
# OIDC (optional)
OIDC_ISSUER_URL=https://accounts.google.com
OIDC_CLIENT_ID=your-client-id
OIDC_CLIENT_SECRET=your-client-secret
OIDC_REDIRECT_URL=http://localhost:3000/auth/oidc/callback
OIDC_SKIP_DISCOVERY=false
MINIO_ENDPOINT=http://localhost:9000
MINIO_BUCKET=file-storage
MINIO_ACCESS_KEY=rustfsadmin
MINIO_SECRET_KEY=rustfsadmin
MINIO_REGION=us-east-1
CHUNK_SIZE=10485760
MAX_FILE_SIZE=1073741824
CLAMAV_HOST=localhost
CLAMAV_PORT=3310
ENABLE_VIRUS_SCAN=true
ALLOWED_ORIGINS=http://localhost:3000,http://localhost:5173

See .env.sample in the project root for a complete reference.

Frontend Environment Variables

VITE_API_URL=http://localhost:3000
VITE_CHUNK_SIZE=10485760

📊 Performance Benchmarks

  • Upload Speed: 500MB/s on local network
  • Concurrent Users: 10,000+ simultaneous connections
  • Memory Usage: ~50MB base (API server)
  • Deduplication Savings: Up to 80% storage reduction
  • Chunk Upload Parallelism: 4 workers default

🤝 Contributing

  1. Fork the repository
  2. Create feature branch (git checkout -b feature/amazing-feature)
  3. Commit changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing-feature)
  5. Open Pull Request

📜 License

Licensed under the MIT License. See LICENSE file for details.

Created with ❤️ by the AppsJuragan team.

🔗 Links

🎯 Roadmap

  • WebDAV support
  • Real-time collaboration
  • File versioning
  • Advanced search with filters
  • Mobile app (Capacitor)
  • End-to-end encryption option
  • File sharing with public links
  • Thumbnail generation
  • Favorites system

About

High-performance, thread-safe Rust file backend with S3 streaming, deduplication, and JWT auth. Optimized for 50k+ concurrent connections.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 6

v1.1.0: Refactored Backend and Modularized Frontend Latest
Feb 25, 2026
+ 5 releases

Packages

 
 
 

Contributors

Languages