Docker-based Environment Setup & Define requirements.txt

[lilacs2039]

What to achive

• Define a requirements.txt file to clearly specify the necessary packages.
• Enable to set up the learning environment using Docker.
• Manage learning environments based on different RWKV versions.

Usage

1. Copy .env.sample to .env.
   • Edit COMPOSE_PROFILES in .env if you want to choose a different RWKV version or if another docker-compose service is added.
2. Start docker-compose: docker compose up
3. Access to JupyterLab: jump to http://localhost:8888/lab
4. Start training (using JupyterLab console tab)
   • This command worked successfully (at v4neo)

   python train.py --load_model "" --wandb "" --proj_dir "out" \
   --data_file "../data/enwik8" --data_type "utf-8" --vocab_size 0 \
   --ctx_len 512 --epoch_steps 5000 --epoch_count 500 --epoch_begin 0 --epoch_save 5 \
   --micro_bsz 12 --n_layer 6 --n_embd 512 --pre_ffn 0 --head_qk 0 \
   --lr_init 8e-4 --lr_final 1e-5 --warmup_steps 0 --beta1 0.9 --beta2 0.99 --adam_eps 1e-8 \
   --accelerator gpu --devices 1 --precision bf16 --strategy ddp_find_unused_parameters_false --grad_cp 0