I think there is big problem when install ludwig, even I try many times with many python version

[karamata]

Describe the bug
I get error when trying install ludwig from source. Even I try many python version still the same.

I also try install numpy first then re-try, but still same error

To Reproduce

1. create new environment with python 3.11.10

2. clone source

3. pip install -e .

4. See error

File "/tmp/pip-build-env-_l7j0ec5/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 303, in get_finalized_command
          cmd_obj.ensure_finalized()
        File "/tmp/pip-build-env-_l7j0ec5/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
          self.finalize_options()
        File "<string>", line 25, in finalize_options
      AttributeError: 'dict' object has no attribute '__NUMPY_SETUP__'
      [end of output]

• OS: Ubuntu 24.04
• Python 3.11.10
• Ludwig from source

[mhabedank]

Hi @karamata,
we are aware of this. There have been many breaking changes in many different dependencies over the last view month. We are on it ( #4041 ). It's hard to tell when we are done but we aim to make Ludwig compatible to Python 3.10, 3.11 and 3.12.
Currently it should run with 3.9. Have you tried it?

[karamata]

Hi @karamata, we are aware of this. There have been many breaking changes in many different dependencies over the last view month. We are on it ( #4041 ). It's hard to tell when we are done but we aim to make Ludwig compatible to Python 3.10, 3.11 and 3.12. Currently it should run with 3.9. Have you tried it?

Also try from python 3.8 -> 3.11, the same error

[C-L-STARK]

Hi @karamata, we are aware of this. There have been many breaking changes in many different dependencies over the last view month. We are on it ( #4041 ). It's hard to tell when we are done but we aim to make Ludwig compatible to Python 3.10, 3.11 and 3.12. Currently it should run with 3.9. Have you tried it?

from python 3.9 3.10 3.11 3.12
tried for all same issues but not work for me.

when to update ?

[C-L-STARK]

@karamata do u fixed it now?

[gyyxs88]

You can try to fix the issue by referring to the dependency libraries I provided. I have successfully run Ludwig in my environment using these dependencies, which also resolved the torchtext error. However, it can only run on the CPU, as GPU computation is not supported for now, and I haven't found a solution to this issue yet.

System: Windows 11
Version: Python 3.9
Dependency Management: Poetry

The dependency libraries are as follows:
python = "3.9.13"
cython = ">=0.25"
h5py = ">=2.6,<3.0.0 || >3.0.0"
numpy = "1.19.5"
pandas = ">=1.0,<1.1.5 || >1.1.5,<2.2.0"
scipy = ">=0.18"
tabulate = ">=0.7"
scikit-learn = "^1.5.2"
tqdm = "^4.67.1"
setuptools = "^75.6.0"
torchtext = "0.18.0"
pydantic = "<2.0"
transformers = ">=4.42.3"
tifffile = "2023.7.10"
imagecodecs = "^2024.9.22"
tokenizers = ">=0.15"
spacy = "2.3.9"
pyyaml = "5.1"
absl-py = "^2.1.0"
kaggle = "^1.6.17"
requests = "^2.32.3"
fsspec = {version = "<=2023.10.0", extras = ["http"]}
dataclasses-json = "^0.6.7"
jsonschema = "4.5.0"
marshmallow = "^3.23.1"
marshmallow-jsonschema = "^0.13.0"
marshmallow-dataclass = "8.5.4"
tensorboard = "^2.18.0"
nltk = "^3.9.1"
torchmetrics = "0.11.0"
torchinfo = "^1.8.0"
filelock = "^3.16.1"
psutil = "^6.1.0"
protobuf = "^5.29.1"
py-cpuinfo = "9.0.0"
gpustat = "^1.1.1"
rich = ">=12.4.4,<12.5.0"
packaging = "^24.2"
retry = "^0.9.2"
sacremoses = "^0.1.1"
sentencepiece = "^0.2.0"
getdaft = "0.1.20"
bitsandbytes = "<0.41.0"
xlwt = "^1.3.0"
xlrd = "2.0.1"
xlsxwriter = "1.4.3"
openpyxl = "3.0.7"
pyxlsb = "1.0.8"
pyarrow = "<15.0.0"
lxml = "^5.3.0"
html5lib = "^1.1"
datasets = "2.19.0"
urllib3 = "<2"
ludwig = "^0.10.4"
torchvision = "0.18.0"
torchaudio = "2.3.0"

It should be noted that when using Poetry to install torchtext, it will automatically install torch 2.3.0. If you are using other dependency management tools, you may need to install torch 2.3.0 separately. Based on preliminary testing, updated versions of torch and GPU-supported versions are unable to run, resulting in a bitsandbytes error: "CUDA Setup failed despite GPU being available." For more details, refer to the issues section on the bitsandbytes GitHub repository. A fix is currently awaited.

[C-L-STARK]

official update was in progress.