Skip to content

Commit 24c72a2

Browse files
thisisjofrankthisisjofrank
authored
Python snapshot (#2920)
1 parent cf3dd87 commit 24c72a2

File tree

4 files changed

+227
-3
lines changed

4 files changed

+227
-3
lines changed

‎examples/_data.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -393,6 +393,11 @@ export const sidebar = [
393393
href: "/examples/snapshots_tutorial/",
394394
type: "example",
395395
},
396+
{
397+
title: "Boot a Python environment with snapshots",
398+
href: "/examples/snapshot_python_tutorial/",
399+
type: "tutorial",
400+
},
396401
],
397402
},
398403
{

‎examples/tutorials/snapshot_python.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
---
2+
title: "Boot a Python environment with snapshots"
3+
description: "Create a sandbox, preload Python + scientific packages, snapshot it, and boot zero-setup sandboxes that run a Mandelbrot explorer."
4+
url: /examples/snapshot_python_tutorial/
5+
---
6+
7+
In this tutorial we will use the Deno Sandbox SDK to create a bootable volume,
8+
set up a development environment on that volume and snapshot it for future use.
9+
This workflow is ideal for languages like Python where environment setup can be
10+
slow, but the resulting filesystem can be reused across many sandboxes.
11+
12+
To use the Deno Sandbox SDK, you will need a Deno Deploy account and an access
13+
token. You can [sign up for free](https://console.deno.com/deploy) and create an
14+
access token in the dashboard under **Sandboxes** → **Overview** → **+ Create
15+
Token**.
16+
17+
## Create a basic Deno application and install the @deno/sandbox SDK
18+
19+
First, create a new directory for this project and initialize a Deno
20+
application:
21+
22+
```sh
23+
deno init my-snapshot-project
24+
cd my-snapshot-project
25+
deno add jsr:@deno/sandbox
26+
```
27+
28+
## 1. Bake Python into a bootable volume
29+
30+
Create a new file `setup_python.ts`. This script will create a new volume, boot
31+
it, install Python and some popular libraries, then snapshot the volume for
32+
future use. The idea is to do the slow setup work once, then reuse the snapshot
33+
to boot new sandboxes in seconds.
34+
35+
In this file we will import the Sandbox SDK, create a client and set up a
36+
function to initialize a volume with Python:
37+
38+
```ts title="setup_python.ts"
39+
import { SandboxClient } from "@deno/sandbox";
40+
const client = new SandboxClient();
41+
42+
async function initSandbox() {
43+
// ... we'll fill this in next
44+
}
45+
```
46+
47+
Inside that function, we'll create a new volume with 10GB of space, give it a
48+
slug for easy reference and specify the region:
49+
50+
```ts title="setup_python.ts (cont.)"
51+
const volume = await client.volumes.create({
52+
region: "ord",
53+
slug: "fun-with-python",
54+
capacity: "10GB",
55+
});
56+
```
57+
58+
Next we boot a sandbox with that volume as its root filesystem. The
59+
`await using` pattern ensures that the sandbox is automatically cleaned up when
60+
we're done with it:
61+
62+
```ts title="setup_python.ts (cont.)"
63+
await using sandbox = await client.sandboxes.create({
64+
region: "ord",
65+
root: volume.slug,
66+
});
67+
```
68+
69+
We install Python and some common scientific libraries using `apt` and `pip`.
70+
Note the use of `--break-system-packages` with pip -- since this sandbox owns
71+
the whole system, we can safely bypass pip's usual protections against messing
72+
with system files:
73+
74+
```ts title="setup_python.ts (cont.)"
75+
await sandbox.sh`sudo apt-get update -qq`;
76+
await sandbox
77+
.sh`sudo apt-get install -y python3 python3-pip python3-venv python3-dev build-essential`;
78+
79+
await sandbox.sh`sudo pip3 install --break-system-packages \
80+
requests \
81+
httpx \
82+
numpy \
83+
pandas \
84+
python-dotenv`;
85+
86+
console.log("Verifying Python installation...");
87+
88+
await sandbox.sh`python3 --version`;
89+
await sandbox.sh`pip3 --version`;
90+
```
91+
92+
The final step of this function is to return the `volume.id` so that we can pass
93+
it to the snapshot step:
94+
95+
```ts title="setup_python.ts (cont.)"
96+
return volume.id;
97+
```
98+
99+
(Remember to close the function with a `}` after this return.)
100+
101+
Next we call this function and snapshot the resulting volume. This will create a
102+
new snapshot with the slug `fun-with-python-snapshot` that we can boot from
103+
later:
104+
105+
```ts title="setup_python.ts (cont.)"
106+
const volumeId = await initSandbox();
107+
108+
console.log("Snapshotting the volume...");
109+
110+
const snapshot = await client.volumes.snapshot(volumeId, {
111+
slug: "fun-with-python-snapshot",
112+
});
113+
114+
console.log("Created Python snapshot " + snapshot.id);
115+
```
116+
117+
Run the script with network and environment access so it can reach the Sandbox
118+
API:
119+
120+
```sh
121+
deno run -N -E setup_python.ts
122+
```
123+
124+
Snapshots are read-only. Any sandbox that mounts this slug now sees a filesystem
125+
with Python and the listed packages ready to go.
126+
127+
## Boot directly from the snapshot
128+
129+
Create a new file called `use_python.ts`. This script will boot a new sandbox
130+
directly from the snapshot we created in the previous step, without running any
131+
setup commands. This demonstrates how you can reuse a snapshot to skip setup and
132+
get straight to running your code.
133+
134+
We start by importing the SDK and creating a client, just like before:
135+
136+
```ts title="use_python.ts"
137+
import { Client } from "@deno/sandbox";
138+
139+
const client = new Client();
140+
```
141+
142+
Then we create a new sandbox, but this time we specify the snapshot slug in the
143+
`root` field instead of a volume:
144+
145+
```ts title="use_python.ts (cont.)"
146+
await using sandbox = await client.sandboxes.create({
147+
region: "ord",
148+
root: "fun-with-python-snapshot",
149+
port: 8000,
150+
timeout: "30m",
151+
});
152+
```
153+
154+
We've given this sandbox a 30-minute timeout and exposed port 8000, which our
155+
Python app will use later.
156+
157+
For now, we'll put in a placeholder string, which we'll replace with our actual
158+
Python app code in a moment, and set it up to be written into the sandbox
159+
filesystem at `/tmp/app.py`:
160+
161+
```ts title="use_python.ts (cont.)"
162+
const appCode = `# Python app code goes here`;
163+
164+
await sandbox.fs.writeTextFile("/tmp/app.py", appCode);
165+
```
166+
167+
Finally we spawn the Python app in the background and print the public URL where
168+
it's reachable. The `await p.output()` line keeps the script running so the
169+
sandbox doesn't immediately shut down:
170+
171+
```ts title="use_python.ts (cont.)"
172+
const p = await sandbox.sh`python3 /tmp/app.py`.spawn();
173+
174+
console.log("\nMandelbrot Explorer running at", sandbox.url);
175+
176+
await p.output();
177+
```
178+
179+
Now let's fill in `appCode` with a simple Python HTTP server that renders a
180+
colorful ASCII Mandelbrot fractal using numpy. This is just an example to
181+
showcase the scientific libraries we installed in the snapshot -- you can
182+
replace it with any Python code you like!
183+
184+
Grab the python code from
185+
[the github repo](https://github.com/denoland/tutorial-with-snapshot/blob/7b8e5331ab22968a7fc52dc84e1613072c7494d1/use_python.ts#L18-L131)
186+
and paste it into the `appCode` string in `use_python.ts`.
187+
188+
Now we're ready to run this script:
189+
190+
```sh
191+
$ deno run -A use_python.ts
192+
```
193+
194+
Open the logged URL in your browser and start zooming; every tile is powered by
195+
the numpy installation you baked into the snapshot earlier!
196+
197+
## Iterate, branch, or clean up
198+
199+
- Need to tweak the environment? Create a writable fork from the snapshot, boot
200+
it, change packages, then snapshot again:
201+
202+
```ts
203+
const fork = await client.volumes.create({
204+
region: "ord",
205+
slug: "python-sandbox-fork",
206+
capacity: "10GB",
207+
from: "fun-with-python-snapshot",
208+
});
209+
```
210+
211+
- Done with a snapshot? Remove it to free space:
212+
213+
```ts
214+
await client.snapshots.delete("fun-with-python-snapshot");
215+
```
216+
217+
🦕 With this workflow you can hand teammates, AI agents or CI jobs a slug that
218+
boots a fully stocked development environment in seconds. Install once,
219+
snapshot, and skip setup forever.

‎examples/tutorials/snapshots.md‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ inherit the same environment without running installers again.
1515

1616
We will:
1717

18-
1. Start from the `builtin:debian-13` base image.
18+
1. Start a shapshot
1919
2. Install Node.js and some global tooling exactly once.
2020
3. Snapshot the prepared volume into `my-toolchain-snapshot`.
2121
4. Boot new sandboxes from that snapshot and verify the tools are ready the
@@ -50,8 +50,7 @@ Create a new volume based on the `builtin:debian-13` image:
5050
const volume = await client.volumes.create({
5151
region: "ord",
5252
slug: "my-toolchain",
53-
capacity: "10GiB",
54-
from: "builtin:debian-13",
53+
capacity: "10GB",
5554
});
5655

5756
console.log(`Bootable volume ready: ${volume.slug}`);

‎runtime/reference/std/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ include usage examples.
6464
- [@std/ulid](./ulid/) – Generation of Universally Unique Lexicographically Sortable Identifiers (ULIDs)
6565
- [@std/uuid](./uuid/) – Generators and validators for UUIDs
6666
- [@std/webgpu](./webgpu/) – UNSTABLE: Utilities for working with the Web GPU API
67+
- [@std/xml](./xml/) – XML parsing and serialization for Deno.
6768
- [@std/yaml](./yaml/) – Parsing and serializing of YAML files
6869
<!-- packages:end -->
6970

0 commit comments

Comments
 (0)