Installation problem with keras3 and backend tensorflow under Windows 11

[ULK-KU]

Hi,

my installation of keras3 using GPU support and the torch backend failed. Before adding an installation issue on this Github page concerning this probem I tried to install the basic default installation steps as indicated below:

First I tried to resolve the installation issue by running the following in a fresh R session

# install the development version of packages, in case the
# issue is already fixed but not on CRAN yet.
install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
if (is.null(reticulate::virtualenv_starter()))
  reticulate::install_python()
keras3::install_keras()

Up to this point there was no error.

Then I tested tensorflow

tensorflow::as_tensor("Hello World")

This failed but following the recommendation on this page I issued

tensorflow::install_tensorflow() 

After this (again after terminating R) the function call

tensorflow::as_tensor("Hello World")

succeeded. I saw tf.Tensor(b'Hello World', shape=(), dtype=string). So Tensorflow has been successfully installed.

But the first few lines of the keras3 MNIST example on https://keras3.posit.co/articles/examples/vision/mnist_convnet.html failed:

> library(keras3)
> num_classes <- 10
> input_shape <- c(28, 28, 1)
> c(c(x_train, y_train), c(x_test, y_test)) %<-% dataset_mnist()
Fehler: ModuleNotFoundError: No module named 'keras.ops'
Run `reticulate::py_last_error()` for details.

Therefore I gathered some diagnostic info after the RStudio - Session - Terminate R:

reticulate::py_config()
python:         C:/Users/ulk/AppData/Local/Programs/Python/Python39/python.exe
libpython:      C:/Users/ulk/AppData/Local/Programs/Python/Python39/python39.dll
pythonhome:     C:/Users/ulk/AppData/Local/Programs/Python/Python39
version:        3.9.12 (tags/v3.9.12:b28265d, Mar 23 2022, 23:52:46) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/ulk/AppData/Local/Programs/Python/Python39/Lib/site-packages/numpy
numpy_version:  1.26.4
keras:          C:\Users\ulk\AppData\Local\Programs\Python\Python39\lib\site-packages\keras

NOTE: Python version was forced by use_python() function
> tensorflow::tf_config()
TensorFlow v2.10.1 (C:\Users\ulk\AppData\Local\Programs\Python\Python39\lib\site-packages\keras)
Python v3.9 (C:/Users/ulk/AppData/Local/Programs/Python/Python39/python.exe)
> reticulate::import("tensorflow")
Module(tensorflow)
> reticulate::py_last_error()
NULL
> sessionInfo()
R version 4.5.1 (2025-06-13 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)

Matrix products: default
LAPACK version 3.12.1

locale:
  [1] LC_COLLATE=German_Germany.utf8  LC_CTYPE=German_Germany.utf8    LC_MONETARY=German_Germany.utf8
[4] LC_NUMERIC=C                    LC_TIME=German_Germany.utf8    

time zone: Europe/Berlin
tzcode source: internal

attached base packages:
  [1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
  [1] keras3_1.4.0.9000

loaded via a namespace (and not attached):
  [1] base64enc_0.1-3        codetools_0.2-20       Matrix_1.7-3           lattice_0.22-7        
[5] magrittr_2.0.3         reticulate_1.42.0      glue_1.8.0             png_0.1-8             
[9] generics_0.1.4         lifecycle_1.0.4        cli_3.6.5              tensorflow_2.16.0.9000
[13] vctrs_0.6.5            grid_4.5.1             zeallot_0.2.0          rsconnect_1.5.0       
[17] tfruns_1.5.3           compiler_4.5.1         rstudioapi_0.17.1      tools_4.5.1           
[21] whisker_0.4.1          pillar_1.11.0          dotty_0.1.0            Rcpp_1.1.0            
[25] rlang_1.1.6            jsonlite_2.0.0        
> 

How to resolve this installation problem?

[t-kalinowski]

Can you please provide the output from running these commands?

install.packages("keras3")
reticulate:::rm_all_reticulate_state(TRUE)

Restart the R session. If in RStudio:

rstudioapi::restartSession()

Then:

library(keras3)

op_convert_to_tensor("Hello World")

[ULK-KU]

Thank you. The output is:

R version 4.5.1 (2025-06-13 ucrt) -- "Great Square Root"
Copyright (C) 2025 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64

R ist freie Software und kommt OHNE JEGLICHE GARANTIE.
Sie sind eingeladen, es unter bestimmten Bedingungen weiter zu verbreiten.
Tippen Sie 'license()' or 'licence()' für Details dazu.

R ist ein Gemeinschaftsprojekt mit vielen Beitragenden.
Tippen Sie 'contributors()' für mehr Information und 'citation()',
um zu erfahren, wie R oder R packages in Publikationen zitiert werden können.

Tippen Sie 'demo()' für einige Demos, 'help()' für on-line Hilfe, oder
'help.start()' für eine HTML Browserschnittstelle zur Hilfe.
Tippen Sie 'q()', um R zu verlassen.

> install.packages("keras3")
Installiere Paket nach ‘D:/_Dateien_von_ulk/Dokumente/R/win-library/4.5’
(da ‘lib’ nicht spezifiziert)
versuche URL 'https://cran.rstudio.com/bin/windows/contrib/4.5/keras3_1.4.0.zip'
Content type 'application/zip' length 13150806 bytes (12.5 MB)
downloaded 12.5 MB

Paket ‘keras3’ erfolgreich ausgepackt und MD5 Summen abgeglichen

Die heruntergeladenen Binärpakete sind in 
C:\Users\ulk\AppData\Local\Temp\RtmpwH04jI\downloaded_packages
> reticulate:::rm_all_reticulate_state(TRUE)
Files removed: 467 (969.2 MB)
> rstudioapi::restartSession()
Restarting R session...
> library(keras3)
> 
> op_convert_to_tensor("Hello World")
Fehler: ModuleNotFoundError: No module named 'keras.ops'
Run `reticulate::py_last_error()` for details.
>

reticulate::py_last_error() result in NULL. Diagnostics coincide with results given in the the first post.

[t-kalinowski]

Thank you, what is the output of reticulate::py_config() after the above sequence?

Does setting Sys.setenv(RETICULATE_PYTHON='managed') at the start of the R session change the outcome?

# fresh R session
Sys.setenv("RETICULATE_PYTHON"="managed")
library(keras3)

try(op_convert_to_tensor("Hello World"))
print(reticulate::py_config())

[ULK-KU]

Thank you very much. The input and output are given in the next chunk

R version 4.5.1 (2025-06-13 ucrt) -- "Great Square Root"
Copyright (C) 2025 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64

R ist freie Software und kommt OHNE JEGLICHE GARANTIE.
Sie sind eingeladen, es unter bestimmten Bedingungen weiter zu verbreiten.
Tippen Sie 'license()' or 'licence()' für Details dazu.

R ist ein Gemeinschaftsprojekt mit vielen Beitragenden.
Tippen Sie 'contributors()' für mehr Information und 'citation()',
um zu erfahren, wie R oder R packages in Publikationen zitiert werden können.

Tippen Sie 'demo()' für einige Demos, 'help()' für on-line Hilfe, oder
'help.start()' für eine HTML Browserschnittstelle zur Hilfe.
Tippen Sie 'q()', um R zu verlassen.

> print(reticulate::py_config())
python:         C:/Users/ulk/AppData/Local/Programs/Python/Python39/python.exe
libpython:      C:/Users/ulk/AppData/Local/Programs/Python/Python39/python39.dll
pythonhome:     C:/Users/ulk/AppData/Local/Programs/Python/Python39
version:        3.9.12 (tags/v3.9.12:b28265d, Mar 23 2022, 23:52:46) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/ulk/AppData/Local/Programs/Python/Python39/Lib/site-packages/numpy
numpy_version:  1.26.4

NOTE: Python version was forced by use_python() function
> # fresh R session
> Sys.setenv("RETICULATE_PYTHON"="managed")
> library(keras3)
Fehler: Laden von Paket oder Namensraum für ‘keras3’: fehlgeschlagen
 .onLoad in loadNamespace() für 'keras3' fehlgeschlagen, Details:
  Aufruf: py_module_import(module, convert = convert)
  Fehler: ModuleNotFoundError: No module named 'keras.ops'
Run `reticulate::py_last_error()` for details.

> reticulate::py_last_error()

── Python Exception Message ─────────────────────────────────────────────────────────────────────────────────
Traceback (most recent call last):
  File "D:\_Dateien_von_ulk\Dokumente\R\win-library\4.5\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "D:\_Dateien_von_ulk\Dokumente\R\win-library\4.5\reticulate\python\rpytools\loader.py", line 96, in _run_hook
    module = hook()
  File "D:\_Dateien_von_ulk\Dokumente\R\win-library\4.5\reticulate\python\rpytools\loader.py", line 120, in _hook
    return _find_and_load(name, import_)
ModuleNotFoundError: No module named 'keras.ops'

── R Traceback ──────────────────────────────────────────────────────────────────────────────────────────────
     ▆
  1. └─base::library(keras3)
  2.   ├─base::tryCatch(...)
  3.   │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
  4.   │   └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
  5.   │     └─base (local) doTryCatch(return(expr), name, parentenv, handler)
  6.   └─base::loadNamespace(package, lib.loc)
  7.     └─base (local) runHook(".onLoad", env, package.lib, package)
  8.       ├─base::tryCatch(fun(libname, pkgname), error = identity)
  9.       │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 10.       │   └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
 11.       │     └─base (local) doTryCatch(return(expr), name, parentenv, handler)
 12.       └─keras3 (local) fun(libname, pkgname)
 13.         └─reticulate::py_register_load_hook(...)
 14.           └─keras3 (local) hook()
 15.             └─reticulate::import("keras.ops", convert = FALSE)
 16.               └─reticulate:::py_module_import(module, convert = convert)
See `reticulate::py_last_error()$r_trace$full_call` for more details.

Now the error occurs immediately after library(keras3). In a fresh R Session without Sys.setenv("RETICULATE_PYTHON"="managed") the call library(keras3) behaves silently, but the error desribed in the intial post above still occurs.

[t-kalinowski]

NOTE: Python version was forced by use_python() function

Where is the use_python() call coming from? Is it in your .Rprofile?

It looks like you ran those commands in an R session where reticulate had already initialized (the wrong) Python.

Can you please run these commands in a fresh R session?

pak::pak("rstudio/reticulate")
reticulate:::rm_all_reticulate_state(TRUE)

Then in a new fresh R session:

rstudioapi::restartSession()

Sys.setenv("RETICULATE_PYTHON"="managed")
library(keras3)

try(op_convert_to_tensor("Hello World"))
print(reticulate::py_config())

[ULK-KU]

Thank you very much.

• The system-wide file C:\Program Files\R\R-4.5.1\etc\Rprofile.site does not contain a use_python() statement.
• My local subdirectory does not contain a Rprofile.
• My project directory"D:/_Dateien_von_ulk/Clouds/OneDrive - ku.de/1--Misc-R/tf+keras+torch--R/install-R-tf+keras3+torch" also does not contain a Rprofile file.

The result of

pak::pak("rstudio/reticulate")
reticulate:::rm_all_reticulate_state(TRUE)

is given below:

R version 4.5.1 (2025-06-13 ucrt) -- "Great Square Root"
Copyright (C) 2025 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64

R ist freie Software und kommt OHNE JEGLICHE GARANTIE.
Sie sind eingeladen, es unter bestimmten Bedingungen weiter zu verbreiten.
Tippen Sie 'license()' or 'licence()' für Details dazu.

R ist ein Gemeinschaftsprojekt mit vielen Beitragenden.
Tippen Sie 'contributors()' für mehr Information und 'citation()',
um zu erfahren, wie R oder R packages in Publikationen zitiert werden können.

Tippen Sie 'demo()' für einige Demos, 'help()' für on-line Hilfe, oder
'help.start()' für eine HTML Browserschnittstelle zur Hilfe.
Tippen Sie 'q()', um R zu verlassen.

> pak::pak("rstudio/reticulate")
✔ Loading metadata database ... done                           
                                                                            
ℹ No downloads are needed
✔ 1 pkg + 11 deps: kept 12 [6.7s]
> 
> reticulate:::rm_all_reticulate_state(TRUE)
No cache found at: C:\Users\ulk\AppData\Local\uv\cache
WARNING: No matching packages
Files removed: 0 (0 bytes)
> 

After a new start of R the following commands were issued:

rstudioapi::restartSession()

Sys.setenv("RETICULATE_PYTHON"="managed")
library(keras3)

try(op_convert_to_tensor("Hello World"))
print(reticulate::py_config())

The result follows:

R version 4.5.1 (2025-06-13 ucrt) -- "Great Square Root"
Copyright (C) 2025 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64

R ist freie Software und kommt OHNE JEGLICHE GARANTIE.
Sie sind eingeladen, es unter bestimmten Bedingungen weiter zu verbreiten.
Tippen Sie 'license()' or 'licence()' für Details dazu.

R ist ein Gemeinschaftsprojekt mit vielen Beitragenden.
Tippen Sie 'contributors()' für mehr Information und 'citation()',
um zu erfahren, wie R oder R packages in Publikationen zitiert werden können.

Tippen Sie 'demo()' für einige Demos, 'help()' für on-line Hilfe, oder
'help.start()' für eine HTML Browserschnittstelle zur Hilfe.
Tippen Sie 'q()', um R zu verlassen.

> rstudioapi::restartSession()
> 
Restarting R session...
> Sys.setenv("RETICULATE_PYTHON"="managed")
> library(keras3)
Warnmeldung:
The request to `use_python("C:/Users/ulk/AppData/Local/Programs/Python/Python39/python.exe")` will be ignored because the environment variable RETICULATE_PYTHON is set to "managed" 

> try(op_convert_to_tensor("Hello World"))
Downloading cpython-3.11.13-windows-x86_64-none (download) (24.3MiB)
 Downloading cpython-3.11.13-windows-x86_64-none (download)
Downloading pygments (1.2MiB)
Downloading tensorboard (5.2MiB)
Downloading numpy (15.1MiB)
Downloading grpcio (4.1MiB)
Downloading h5py (2.7MiB)
Downloading pillow (6.7MiB)
Downloading tensorflow (358.5MiB)
Downloading setuptools (1.1MiB)
Downloading pandas (10.8MiB)
Downloading jedi (1.5MiB)
Downloading libclang (25.2MiB)
Downloading keras (1.3MiB)
Downloading tensorflow-io-gcs-filesystem (1.4MiB)
Downloading scipy (36.8MiB)
 Downloading tensorflow-io-gcs-filesystem
 Downloading pygments
 Downloading h5py
 Downloading setuptools
 Downloading keras
 Downloading grpcio
 Downloading tensorboard
 Downloading pillow
 Downloading jedi
 Downloading pandas
 Downloading numpy
 Downloading libclang
 Downloading scipy
 Downloading tensorflow
Installed 60 packages in 26.04s
tf.Tensor(b'Hello World', shape=(), dtype=string)
> 
> print(reticulate::py_config())
python:         C:/Users/ulk/AppData/Local/R/cache/R/reticulate/uv/cache/archive-v0/UE5n9nqQg89EB2j06ndWm/Scripts/python.exe
libpython:      C:/Users/ulk/AppData/Local/R/cache/R/reticulate/uv/python/cpython-3.11.13-windows-x86_64-none/python311.dll
pythonhome:     C:/Users/ulk/AppData/Local/R/cache/R/reticulate/uv/cache/archive-v0/UE5n9nqQg89EB2j06ndWm
virtualenv:     C:/Users/ulk/AppData/Local/R/cache/R/reticulate/uv/cache/archive-v0/UE5n9nqQg89EB2j06ndWm/Scripts/activate_this.py
version:        3.11.13 (main, Jul 11 2025, 22:36:59) [MSC v.1944 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/ulk/AppData/Local/R/cache/R/reticulate/uv/cache/archive-v0/UE5n9nqQg89EB2j06ndWm/Lib/site-packages/numpy
numpy_version:  1.26.4
keras:          C:\Users\ulk\AppData\Local\R\cache\R\RETICU~1\uv\cache\ARCHIV~1\UE5N9N~1\Lib\site-packages\keras

NOTE: Python version was forced by py_require()

Now I restarted R again in a fresh session and started the MNIST example given in https://keras3.posit.co/articles/examples/vision/mnist_convnet.html.

So it is important to use a managed Python via

Sys.setenv("RETICULATE_PYTHON"="managed")
library(keras3)

as a replacement of the system version of Python (in my case 3.9.12)

Thomasz, thank you very much for your help!!! The problem with use_backend("torch", gpu=TRUE) is still present but this is caused by an old NVIDIA driver version (cuda_11.8.0_522.06_windows.exe & cudnn-windows-x86_64-8.6.0.163_cuda11-archive). Unfortunately this driver cannot be replaced due to other requirements.