Skip to content

Implement local CPython kernel support, with pip as the environment manager #87

@intentionally-left-nil

Description

@intentionally-left-nil

Background

Currently, runt supports pyodide as the kernel piece of the runtime agent. This works well in many cases, however pyodide has distinct limitations (e.g. around threading, sockets) that cause some packages to be incompatible with it. We therefore additionally need to support using regular, CPython-based kernels running in some environments.

The act of creating and discovering compute is going to be an expansive feature that we build out over time. For now, the trick is to keep our focus small, while still thinking ahead to discoverable, heterogeneous compute in the future.

With that, here's the initial CPython support.

Acceptance criteria

  • As a user, I can run a command to start the runtime agent, where the agent and the python component are running in the same place.
  • The python environment has packages controlled by pip only (for now).
  • The environment can either be persistent, created and managed by the user OR
  • Ephemeral - created, managed, and destroyed automatically by the runtime agent
  • Package management (which packages are available in an environment) is handled up-front. From the runtime agent's perspective, the environment is static while the runtime agent is running. We'll cover upgrading or removing packages at a later time
  • The user can configure the following pieces: The path of the python executable from which to make a venv, the location of the venv, whether this is a permanent environment (created and managed outside the runtime agent) vs an ephemeral one, and the package manager used to control the environment (although for this ticket the only package manager they can choose is pip).
  • If the user doesn't specify these options via the command line or environment variable, then: the python in $path is chosen, the default path is ~/.anode/envs/<notebook_id>_, the type is ephemeral, and the package manager is pip

Sub-issues

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions