Compatibility with 5090, pytorch 2.8.0 nightly?

[RodriMora]

Hi!

Is there compatibility with pytorch 2.8.0 nightlies? they are required to work with the Nvidia 5090.

I tried running:

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

In the second step of the docs:

2. Install dependencies
sudo apt install libtbb-dev libssl-dev libcurl4-openssl-dev libaio1 libaio-dev libfmt-dev libgflags-dev zlib1g-dev patchelf
pip3 install packaging ninja cpufeature numpy openai
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126 #changed this one

But got this error:

 x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 /home/ubuntuai/ktransformers_torch28/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/binding.o /home/ubuntuai/ktransformers_torch28/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/gptq_marlin/gptq_marlin.o /home/ubuntuai/ktransformers_torch28/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/gptq_marlin/gptq_marlin_repack.o -L/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/torch/lib -L/usr/local/cuda-12.8/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-311/vLLMMarlin.cpython-311-x86_64-linux-gnu.so
  CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ubuntuai/ktransformers_torch28/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/ubuntuai/ktransformers_torch28/.venv/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=1']
  CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ubuntuai/ktransformers_torch28/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/ubuntuai/ktransformers_torch28/.venv/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=1', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4.post1+cu128torch28avx2']
  build_temp: /home/ubuntuai/ktransformers_torch28/csrc/balance_serve/build
  Traceback (most recent call last):
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 412, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 478, in <module>
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "<string>", line 246, in run
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/wheel/_bdist_wheel.py", line 387, in run
      self.run_command("build")
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
      self.build_extensions()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1052, in build_extensions
      build_ext.build_extensions(self)
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
      self._build_extensions_serial()
    File "/home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
      self.build_extension(ext)
    File "<string>", line 406, in build_extension
    File "/usr/lib/python3.11/subprocess.py", line 571, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['cmake', '/home/ubuntuai/ktransformers_torch28/csrc/balance_serve', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ubuntuai/ktransformers_torch28/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/ubuntuai/ktransformers_torch28/.venv/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=1', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4.post1+cu128torch28avx2']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Building wheel for ktransformers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/ubuntuai/ktransformers_torch28/.venv/bin/python3.11 /home/ubuntuai/ktransformers_torch28/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmp3i6sz8pi
  cwd: /home/ubuntuai/ktransformers_torch28
  Building wheel for ktransformers (pyproject.toml) ... error
  ERROR: Failed building wheel for ktransformers
Failed to build ktransformers
ERROR: Could not build wheels for ktransformers, which is required to install pyproject.toml-based projects