🚸 Basic UX of tracking an Rmd needs to be improved

[falexwolf]

So long that that we get a warning because lamindb is worried that the source code might have changed since running ln$finish() and then we get this warning.

[falexwolf]

The second time I run this, it's alright:

But then I still don't have a report

Frustrating. 😅

[falexwolf]

Ok, now I ran laminr::lamin_save() and this worked:

Problem is: I had already a half-baked html lying around, so the report that got uploaded doesn't actually match the latest state of the Rmd

[falexwolf]

So, I enable this button now:

And then I hit "Save" so that knitting starts, and I get this error:

Surprising. 😅

So I do what it tells me to do and I get something incomprehensible:

> reticulate::py_last_error()

── Python Exception Message ──────────────────────────────────────────────────────────────────────────────────────────────
TypeError: 'Transform' object is not subscriptable

── R Traceback ───────────────────────────────────────────────────────────────────────────────────────────────────────────
     ▆
  1. └─ln$Artifact("./test_file.txt", description = "test file")$save()
  2.   ├─laminr:::py_to_r_nonull(...)
  3.   │ └─reticulate::py_to_r(x)
  4.   │   ├─reticulate::is_py_object(x <- py_to_r_cpp(x))
  5.   │   └─reticulate:::py_to_r_cpp(x)
  6.   └─private$.py_object[["save"]](upload = upload, ...)
  7.     ├─reticulate::py_to_r(result)
  8.     └─laminr:::py_to_r.lamindb.models.artifact.Artifact(result)
  9.       └─laminr:::wrap_python(...)
 10.         ├─base::try(suppress_future_warning(obj[[.name]]), silent = TRUE)
 11.         │ └─base::tryCatch(...)
 12.         │   └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 13.         │     └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
 14.         │       └─base (local) doTryCatch(return(expr), name, parentenv, handler)
 15.         ├─laminr:::suppress_future_warning(obj[[.name]])
 16.         │ ├─base::with(...)
 17.         │ ├─reticulate:::with.python.builtin.object(...)
 18.         │ │ ├─base::tryCatch(...)
 19.         │ │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 20.         │ │ └─base::force(expr)
 21.         │ └─base::eval(expr)
 22.         ├─obj[[.name]]
 23.         └─reticulate:::`[[.python.builtin.object`(obj, .name)
 24.           └─reticulate:::py_get_attr_or_item(x, name, FALSE)
 25.             └─reticulate:::py_maybe_convert(object, py_has_convert(x))
 26.               ├─reticulate::py_to_r(x)
 27.               └─laminr:::py_to_r.lamindb.models.transform.Transform(x)
 28.                 └─laminr:::wrap_python(...)
 29.                   ├─base::try(suppress_future_warning(obj[[.name]]), silent = TRUE)
 30.                   │ └─base::tryCatch(...)
 31.                   │   └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 32.                   │     └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
 33.                   │       └─base (local) doTryCatch(return(expr), name, parentenv, handler)
 34.                   ├─laminr:::suppress_future_warning(obj[[.name]])
 35.                   │ ├─base::with(...)
 36.                   │ ├─reticulate:::with.python.builtin.object(...)
 37.                   │ │ ├─base::tryCatch(...)
 38.                   │ │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
 39.                   │ │ └─base::force(expr)
 40.                   │ └─base::eval(expr)
 41.                   ├─obj[[.name]]
 42.                   └─reticulate:::`[[.python.builtin.object`(obj, .name)
 43.                     └─reticulate:::py_get_attr_or_item(x, name, FALSE)
 44.                       └─reticulate::py_get_item(x, name)
See `reticulate::py_last_error()$r_trace$full_call` for more details.

[falexwolf]

All the same when I click on the "Knit" button...

So, instead of using "Knit", I just manually re-run all cells and this works.

But now I still don't have my up-to-date html.

[falexwolf]

On knitting I keep getting the same error.

I've tried 10 different things to get RStudio to show me the traceback so that I can debug this, but no chance...

Code

---

title: "my-r-notebook.rmd"
output: html_document
date: "2025-07-06"

knitr::opts_chunk$set(
error = TRUE,
collapse = FALSE,
comment = "",
R.options = list(width = 200, traceback.max.lines = Inf)
)

ln <- laminr::import_module("lamindb")  # instantiate the central object of the API

tryCatch({
  ln$track("wRbyzq26gFVN")
}, error = function(e) {
  cat("R error:", conditionMessage(e), "\n")
  cat("Full call stack:\n")
  calls <- sys.calls()
  for(i in seq_along(calls)) {
    cat(sprintf("%d. %s\n", i, paste(deparse(calls[[i]], width.cutoff = 500), collapse = " ")))
  }
  cat("Python error details:\n")
  reticulate::py_last_error()
  stop(e)
})

ln$Artifact("./test_file.txt", description="test file")$save()

ln$finish()

Giving up on this for now.

[falexwolf]

So, now I try to run without passing the stem uid (against the recommendation in the logging) and I get this error:

I restart the R session and still the same thing:

It might be that somehow during knitting or otherwise it started to no longer recognize my initial config for the reticulate environment but instead made its own env

[falexwolf]

And this was it. If I set the conda env correctly at the top of the Rmd, it works, up to character rendering in the logging

But when I then try to save it outside of the notebook it again doesn't work because it made its own env again

So, what I have to do is go on the CLI with my conda env set and it's good up to the improper character rending that was also an issue in RStudio up there

[falexwolf]

I've also taken a look at the PAK file and it's looong

1500 lines for an environment that has almost nothing installed

Every single package has 40 unnecessary lines:

    {
      "ref": "htmltools",
      "package": "htmltools",
      "version": "0.5.8.1",
      "type": "standard",
      "direct": true,
      "binary": true,
      "dependencies": ["base64enc", "digest", "fastmap", "rlang"],
      "vignettes": false,
      "needscompilation": true,
      "metadata": {
        "RemoteType": "standard",
        "RemotePkgRef": "htmltools",
        "RemoteRef": "htmltools",
        "RemoteRepos": "https://cran.rstudio.com/",
        "RemotePkgPlatform": "aarch64-apple-darwin20",
        "RemoteSha": "0.5.8.1"
      },
      "sources": ["https://cran.rstudio.com//bin/macosx/big-sur-arm64/contrib/4.4/htmltools_0.5.8.1.tgz"],
      "target": "bin/macosx/big-sur-arm64/contrib/4.4/htmltools_0.5.8.1.tgz",
      "platform": "aarch64-apple-darwin20",
      "rversion": "4.4",
      "directpkg": true,
      "license": "GPL (>= 2)",
      "sha256": "\n        1132d64f4f7abfb888a19622433b0054c679b016cdd3a7993bea2373ed52cd1a",
      "dep_types": ["Depends", "Imports", "LinkingTo"],
      "params": [],
      "install_args": "",
      "repotype": "cranlike"
    }

What we want instead, I think, is something like the requirements.txt that only lists versions and packages.

[lazappi]

The error about the uid length seems to be coming from Python. Are they supposed to be 12 or 16 characters?

When you click the "Knit" button the document is rendered in a separate session which is why you need to set the {reticulate} environment inside the document. I'm not sure why you are seeing the "JWT not set" error in the first place though. That looks like maybe it can't find a config file but I don't know why that would be happening.

I've also taken a look at the PAK file and it's looong

We went with the {pak} lockfile because you wanted something that could be used to re-install the environment. We could output a file with packages and versions instead but then you wouldn't be able to install everything with a single command.