Fix Dockerfile, make it more convenient and efficient#603
Conversation
Rather than relying on things outside the project dir, have a data dir with all the caches, model files etc, and exclude them from git and the docker context with the .ignore files. Also move the install step to last and put it in its own step, so it doesn't have to rebuild all the slow conda stuff every time there's a change to a file in the project.
|
Thanks for fixing the Dockerfile! Was looking for this because of the env variable issues. RUN conda create --name tortoise python=3.9 numba inflect \
&& conda activate tortoise \
&& conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia \
&& conda install transformers=4.29.2 \
&& conda clean --all \
&& echo "conda activate tortoise; cd /app" >> "${HOME}/.bashrc"Because the commands are all stringed together in a single shell list, I can't really tell which command is going interactive, but it's not create (I checked), so it must be one of the installs or clean. RUN conda create -y --name tortoise python=3.9 numba inflect \
&& conda activate tortoise \
&& conda install -y pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia \
&& conda install -y transformers=4.29.2 \
&& conda clean -y --all \
&& echo "conda activate tortoise; cd /app" >> "${HOME}/.bashrc" |
|
Also, I had to install the NVIDIA container toolkit to be able tor run the image with the --gpus flag, maybe this could be added to the Readme https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html |
|
@bitplane is this working properly ? |
docker_data, which is excluded via.gitignoreand.dockerignorecondaorapt updateevery time there's a source code changerequirements.txtand dopip install -rbefore adding the code, so we only download from pypi when there's a change to the python deps.pip, so no junk is created that needs to be cleaned upconda activateandcd apppart in root's.bashrcso it's ready to go when you log in*.pycfiles andbuilddirs from the Docker build context, in case the user has been running things outside docker