Tensorflow GPU points to wrong package

[JimothyJohn]

Problem

Working on a Tensorflow 1.15 project and noticed that the builder is pulling from "tensorflow_gpu" instead of "tensorflow-gpu", leading to less versions being available. Relevant output:

#15 19.35 Collecting tensorflow_gpu==1.15.0
#15 19.53   Could not find a version that satisfies the requirement tensorflow_gpu==1.15.0 (from versions: 0.12.1, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.9.0, 1.10.0, 1.10.1, 1.11.0, 1.12.0, 1.12.2, 1.12.3, 1.13.1, 1.13.2, 1.14.0, 2.0.0a0, 2.0.0b0, 2.0.0b1)
#15 19.90 No matching distribution found for tensorflow_gpu==1.15.0

cog.yaml configuration section:

  python_packages:
    - "tensorflow==1.15.0"

Suggestion

Change the "_" to "-" in pkg/config/tf_compatability_matrix.json and pkg/config/config_test.go

[bfirsh]

Thanks for the report, @JimothyJohn! That matrix file is automatically generated, so it is the importer that creates that we'll need to fix I think.

[JimothyJohn]

After looking more into this it seems that the problem lies with the Ubuntu release and the corresponding default python installation. I noticed the CUDA and CuDNN version are correct, but it's pulling Ubuntu 16.04 which delivers with Python 2.7 and no Python 3. For example:

  python_packages:
    - "tensorflow==2.4.0"

leads to

=> => transferring context: 15.68kB                                          0.0s
 => [stage-0 1/9] FROM docker.io/nvidia/cuda:11.0-cudnn8-devel-ubuntu16.04 

and the same error above. Maybe need to add an Ubuntu item to the compatibility matrix and cog.yaml?

I understand this is a pretty major feature request.

[bfirsh]

We install Python separately with pyenv, so in theory it should be using Python 3.

If you want to see the dockerfile that is being generated, you can run cog debug dockerfile.

[zeke]

@allcontributors please add @JimothyJohn for bugs 🪲

[allcontributors]

@zeke

I've put up a pull request to add @JimothyJohn! 🎉