Fine tuning detection fails due to absolute path (+ requirements)

[VLucet]

Search before asking

• I have searched the Pytorch-Wildlife issues and found no similar bug report.

Bug

I tried the new fine-tuning feature but ran into 3 issues:

1. Absolute path in the data config yaml:
   

   CameraTraps/PW_FT_detection/data/data_example.yaml

   Line 1 in 7efc18d

   path: /home/v-druizlopez/ssdprivate/CameraTraps/PW_FT_detection/data/data_example # Fix to relative path

2. Conda choked on setuptools requirements

INFO: pip is looking at multiple versions of pytorchwildlife to determine which version is compatible with other requirements. This could take a while.

The conflict is caused by:
    The user requested setuptools==75.6.0
    pytorchwildlife 1.1.1 depends on setuptools==59.5.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

3. Default ultralytics datasets download path was set as ...CameraTraps/PW_FT_detection/datasets, requiring me to edit nano ~/.config/Ultralytics/settings.json

FileNotFoundError: 
Dataset 'data/data_example.yaml' images not found ⚠️, missing path '/home/vlucet/Documents/WILDLab/camtrap-rof/repos/CameraTraps/PW_FT_detection/datasets/data/data_example/images/val'
Note dataset download directory is '/home/vlucet/Documents/WILDLab/camtrap-rof/repos/CameraTraps/PW_FT_detection/datasets'. You can update this in '/home/vlucet/.config/Ultralytics/settings.json'

My PR adresses (1) and (2) by changing the path and removing the setuptools req. I'm not sure about (3): maybe the simplest is to have an empty "datasets" dir in PW_FT_detection?

Environment

# packages in environment at /home/vlucet/miniconda3/envs/PW_Finetuning_Detection:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
absl-py                   2.1.0                    pypi_0    pypi
aiofiles                  23.2.1                   pypi_0    pypi
annotated-types           0.7.0                    pypi_0    pypi
antlr4-python3-runtime    4.9.3                    pypi_0    pypi
anyio                     4.6.0                    pypi_0    pypi
appdirs                   1.4.4                    pypi_0    pypi
asttokens                 2.4.1                    pypi_0    pypi
attrs                     24.2.0                   pypi_0    pypi
bzip2                     1.0.8                h4bc722e_7    conda-forge
ca-certificates           2024.8.30            hbcca054_0    conda-forge
certifi                   2024.8.30                pypi_0    pypi
cffi                      1.17.1                   pypi_0    pypi
chardet                   5.2.0                    pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
contourpy                 1.3.0                    pypi_0    pypi
crowsetta                 5.1.0                    pypi_0    pypi
cycler                    0.12.1                   pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
exceptiongroup            1.2.2                    pypi_0    pypi
executing                 2.1.0                    pypi_0    pypi
fastapi                   0.115.0                  pypi_0    pypi
ffmpy                     0.4.0                    pypi_0    pypi
filelock                  3.16.1                   pypi_0    pypi
fire                      0.6.0                    pypi_0    pypi
fonttools                 4.54.0                   pypi_0    pypi
fsspec                    2024.9.0                 pypi_0    pypi
gradio                    4.44.0                   pypi_0    pypi
gradio-client             1.3.0                    pypi_0    pypi
grpcio                    1.66.1                   pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
httpcore                  1.0.5                    pypi_0    pypi
httpx                     0.27.2                   pypi_0    pypi
huggingface-hub           0.25.1                   pypi_0    pypi
idna                      3.10                     pypi_0    pypi
importlib-resources       6.4.5                    pypi_0    pypi
ipython                   8.27.0                   pypi_0    pypi
jedi                      0.19.1                   pypi_0    pypi
jinja2                    3.1.4                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
kiwisolver                1.4.7                    pypi_0    pypi
ld_impl_linux-64          2.43                 h712a8e2_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.1.0               h77fa898_1    conda-forge
libgcc-ng                 14.1.0               h69a702a_1    conda-forge
libgomp                   14.1.0               h77fa898_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libsqlite                 3.46.1               hadc24fc_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
markdown                  3.7                      pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
matplotlib                3.9.2                    pypi_0    pypi
matplotlib-inline         0.1.7                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
multimethod               1.12                     pypi_0    pypi
munch                     4.0.0                    pypi_0    pypi
mypy-extensions           1.0.0                    pypi_0    pypi
ncurses                   6.5                  he02047a_1    conda-forge
networkx                  3.3                      pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
nvidia-cublas-cu12        12.1.3.1                 pypi_0    pypi
nvidia-cuda-cupti-cu12    12.1.105                 pypi_0    pypi
nvidia-cuda-nvrtc-cu12    12.1.105                 pypi_0    pypi
nvidia-cuda-runtime-cu12  12.1.105                 pypi_0    pypi
nvidia-cudnn-cu12         9.1.0.70                 pypi_0    pypi
nvidia-cufft-cu12         11.0.2.54                pypi_0    pypi
nvidia-curand-cu12        10.3.2.106               pypi_0    pypi
nvidia-cusolver-cu12      11.4.5.107               pypi_0    pypi
nvidia-cusparse-cu12      12.1.0.106               pypi_0    pypi
nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
nvidia-nvjitlink-cu12     12.6.68                  pypi_0    pypi
nvidia-nvtx-cu12          12.1.105                 pypi_0    pypi
omegaconf                 2.3.0                    pypi_0    pypi
opencv-python             4.10.0.84                pypi_0    pypi
opencv-python-headless    4.10.0.84                pypi_0    pypi
openssl                   3.3.2                hb9d3cd8_0    conda-forge
orjson                    3.10.7                   pypi_0    pypi
packaging                 24.1                     pypi_0    pypi
pandas                    2.2.3                    pypi_0    pypi
pandera                   0.21.0                   pypi_0    pypi
parso                     0.8.4                    pypi_0    pypi
pexpect                   4.9.0                    pypi_0    pypi
pillow                    10.4.0                   pypi_0    pypi
pip                       24.2               pyh8b19718_1    conda-forge
prompt-toolkit            3.0.47                   pypi_0    pypi
protobuf                  3.20.1                   pypi_0    pypi
psutil                    6.0.0                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.3                    pypi_0    pypi
py-cpuinfo                9.0.0                    pypi_0    pypi
pycparser                 2.22                     pypi_0    pypi
pydantic                  2.9.2                    pypi_0    pypi
pydantic-core             2.23.4                   pypi_0    pypi
pydub                     0.25.1                   pypi_0    pypi
pygments                  2.18.0                   pypi_0    pypi
pyparsing                 3.1.4                    pypi_0    pypi
python                    3.10.15         h4a871b0_0_cpython    conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
python-multipart          0.0.10                   pypi_0    pypi
pytorchwildlife           1.1.1                    pypi_0    pypi
pytz                      2024.2                   pypi_0    pypi
pyyaml                    6.0.2                    pypi_0    pypi
readline                  8.2                  h8228510_1    conda-forge
requests                  2.32.3                   pypi_0    pypi
rich                      13.8.1                   pypi_0    pypi
ruff                      0.6.7                    pypi_0    pypi
scikit-learn              1.6.0                    pypi_0    pypi
scipy                     1.14.1                   pypi_0    pypi
seaborn                   0.13.2                   pypi_0    pypi
semantic-version          2.10.0                   pypi_0    pypi
setuptools                59.5.0                   pypi_0    pypi
shellingham               1.5.4                    pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
soundfile                 0.12.1                   pypi_0    pypi
stack-data                0.6.3                    pypi_0    pypi
starlette                 0.38.6                   pypi_0    pypi
supervision               0.23.0                   pypi_0    pypi
sympy                     1.13.3                   pypi_0    pypi
tensorboard               2.17.1                   pypi_0    pypi
tensorboard-data-server   0.7.2                    pypi_0    pypi
termcolor                 2.4.0                    pypi_0    pypi
thop                      0.1.1-2209072238          pypi_0    pypi
threadpoolctl             3.5.0                    pypi_0    pypi
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomlkit                   0.12.0                   pypi_0    pypi
torch                     2.4.1                    pypi_0    pypi
torchaudio                2.4.1                    pypi_0    pypi
torchvision               0.19.1                   pypi_0    pypi
tqdm                      4.66.5                   pypi_0    pypi
traitlets                 5.14.3                   pypi_0    pypi
triton                    3.0.0                    pypi_0    pypi
typeguard                 4.4.1                    pypi_0    pypi
typer                     0.12.5                   pypi_0    pypi
typing-extensions         4.12.2                   pypi_0    pypi
typing-inspect            0.9.0                    pypi_0    pypi
tzdata                    2024.2                   pypi_0    pypi
ultralytics               8.2.100                  pypi_0    pypi
ultralytics-thop          2.0.8                    pypi_0    pypi
ultralytics-yolov5        0.1.1                    pypi_0    pypi
urllib3                   2.2.3                    pypi_0    pypi
uvicorn                   0.30.6                   pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
websockets                12.0                     pypi_0    pypi
werkzeug                  3.0.4                    pypi_0    pypi
wget                      3.2                      pypi_0    pypi
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.17.0                   pypi_0    pypi
xz                        5.2.6                h166bdaf_0    conda-forge

Minimal Reproducible Example

NA

Additional

NA

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[danielaruizl1]

Hi @VLucet thanks for bringing this up!

Issues (1) and (3) are indeed related. The reason we initially used the absolute path was to prevent the error (3) that occurs when the path is changed to relative. As you correctly noted, we prefer not to edit ~/.config/Ultralytics/settings.json directly, so we decided to implement a verification process in data_example.py to check if the path is relative. If it is, the script converts it to an absolute path and then updates the YAML file accordingly (Check commit 17df72e).

Regarding issue (2) we specify the version of setuptools due to a known vulnerability in versions up to 69.1.1, which allows for remote code execution via the package_index module's download functions. This vulnerability is one of the reasons we updated pytorchwildlife to Python 3.10. In contrast, pytorchwildlife 1.1.1 used Python 3.8 and required setuptools==59.5.0. Therefore, we addressed the issue you mentioned by deleting the version of pytorchwildlife in the environment.yaml.

Please let us know if you have any further suggestions. We highly appreciate your efforts in helping us improve the repository!

[VLucet]

This is perfect, thanks for the fixes!