fix documents

Signed-off-by: scepter914 <scepter914@gmail.com>

Author: scepter914

.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+  "cSpell.words": [
+    "mlcellar"
+  ]
+}

README.md

@@ -3,13 +3,24 @@
 <img src="./docs/asset/logo.png" width="250" alt="ml-cellar logo">
 
 `ml-cellar` provides a CLI to support model registry, storing ML models like a wine cellar and enabling minimal MLOps.
-By using GitLFS, `ml-cellar` offers essential MLOps functions including Artifact Store, Model Registry, and Serving.
+By using Git LFS, `ml-cellar` offers essential MLOps functions including Artifact Store, Model Registry, and Serving.
 
 If MLOps at BigTech companies can be compared to a large-scale winery, `ml-cellar` functions like a wine cellar in a small brewery.
 While AI projects should ideally adopt software like MLflow for MLOps, many projects and organizations cannot afford the development resources that BigTech companies have.
 As a result, existing MLOps software has primarily targeted companies that can allocate significant development costs to MLOps (like BigTech).
 `ml-cellar` makes "compromises" on MLOps by focusing only on the essential functions, enabling easy adoption of minimal MLOps.
 
+- Feature
+  - [ ] [Custom Transfer Agent](docs/docs_custom_transfer_agent.md)
+
+## GitHub Git LFS usage & billing notice
+
+This repository uses Git LFS (Large File Storage) to manage large files.
+Please note that **GitHub Git LFS has storage and bandwidth limits**.
+If the free quota included in your GitHub plan is exceeded, **you may incur additional charges**, or LFS uploads and downloads may be restricted depending on your billing settings.
+Before cloning or pushing large files, check your GitHub plan and the repository’s current Git LFS usage.
+If you need to handle many large files in your model registry, consider using a [Custom Transfer Agent](docs/docs_custom_transfer_agent.md) to replace the storage backend with AWS S3 or a similar service.
+
 ## Contribution
 
 If you want to contribute this project, please see the documents.
@@ -23,19 +34,21 @@ If you want to see example of the usage of `ml-cellar` as minimum MLOps, please
 
 ### 1. Install
 
-- Install Rust
-  - See [the official document](https://doc.rust-lang.org/cargo/getting-started/installation.html).
+- Install GitLFS
+  - See [the official document](https://github.com/git-lfs/git-lfs/wiki/Installation)
 
 ```sh
-curl https://sh.rustup.rs -sSf | sh
+curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
+sudo apt-get update
+sudo apt-get install git-lfs
+git lfs install
 ```
 
-- Install dependency
+- Install Rust
+  - See [the official document](https://doc.rust-lang.org/cargo/getting-started/installation.html).
 
 ```sh
-sudo apt install git-lfs awscli
-pip install aws2-wrap
-cargo install lfs-dal
+curl https://sh.rustup.rs -sSf | sh
 ```
 
 - Install `ml-cellar`
@@ -68,39 +81,29 @@ cd {your_ml_registry}
 ml-cellar init
 ```
 
-- Edit `.ml_cellar.toml`
+- Edit `.gitattributes` if you want to add for GitLFS
 
-```toml
-[ml_cellar]
-custom_transfer_agent = false
-```
-
-### 3. (Option) Setup with AWS S3
+```txt
+# --- Log ---
+*.log   filter=lfs diff=lfs merge=lfs -text
 
-Git LFS on GitHub has about 1 GB limit.
-If you need to handle files larger than 1 GB in the repository, I recommend integrating with AWS S3.
+# --- Your data ---
+*.db   filter=lfs diff=lfs merge=lfs -text
+```
 
-- Edit `.ml_cellar.toml` and change to `custom_transfer_agent = true`.
+- Edit `.mlcellar.toml`
 
 ```toml
 [ml_cellar]
-custom_transfer_agent = true
-```
+use_custom_transfer_agent = false
 
-- Edit `.lfsdalconfig` if you use AWS S3.
+[aws]
 
-```txt
-[lfs-dal]
-scheme = s3
-bucket = your_bucket
-region = your_region
 ```
 
-- Run setup command
+### 3. (Option) Setup with AWS S3
 
-```sh
-ml-cellar setup
-```
+If you need to handle many large files in your model registry, consider using a [Custom Transfer Agent](docs/docs_custom_transfer_agent.md) to replace the storage backend with AWS S3 or a similar service.
 
 ### 4. Start project
 

docs/docs_custom_transfer_agent.md

@@ -0,0 +1,50 @@
+
+# `ml-cellar` with Custom Transfer Agent
+## Purpose
+### Limitation in Github Git LFS
+
+According to [the GitHub documentation](https://docs.github.com/en/billing/concepts/product-billing/git-lfs
+), Git LFS on GitHub has a storage quota limit of 10 GB for Free and Pro accounts, as well as bandwidth limits.
+It also has limitations such as not being able to choose the server region.
+Using a Custom Transfer Agent, you can operate these on AWS S3 and are free from these limitations.
+(You can address the storage limitation by paying GitHub for additional capacity.)
+
+### Compare fee between GitHub Git LFS and AWS S3
+
+## Get started
+### Install
+
+- Install dependency
+
+```sh
+sudo apt install git-lfs awscli
+pip install aws2-wrap
+cargo install lfs-dal
+```
+
+### Setup
+
+- Edit `.mlcellar.toml` and change to `custom_transfer_agent = true`.
+
+```toml
+[ml_cellar]
+use_custom_transfer_agent = true
+
+[aws]
+profile = "{your_profile}"
+```
+
+- Edit `.lfsdalconfig` if you use AWS S3.
+
+```txt
+[lfs-dal]
+scheme = s3
+bucket = your_bucket
+region = your_region
+```
+
+- Run setup command
+
+```sh
+ml-cellar setup
+```