|
| 1 | +--- |
| 2 | +description: How do I use volumes in Windmill to persist files across job runs? |
| 3 | +--- |
| 4 | + |
| 5 | +import DocCard from '@site/src/components/DocCard'; |
| 6 | + |
| 7 | +# Volumes |
| 8 | + |
| 9 | +Volumes provide persistent file storage that can be attached to [scripts](../../getting_started/0_scripts_quickstart/index.mdx) via code annotations. Files written to a volume during a job run are synced back to [object storage](../38_object_storage_in_windmill/index.mdx) and restored on the next run, enabling stateful workflows like caching, model storage, or agent memory. |
| 10 | + |
| 11 | +Volumes are an [Enterprise](/pricing) feature. On the Community Edition, volumes are limited to 20 volumes per workspace and 50 MB per file. |
| 12 | + |
| 13 | +## Setup |
| 14 | + |
| 15 | +Volumes require two things to be configured in your workspace: |
| 16 | + |
| 17 | +1. A [workspace object storage](../38_object_storage_in_windmill/index.mdx#workspace-object-storage) (S3, Azure Blob, GCS, or filesystem). |
| 18 | +2. A **volume storage** selection in the workspace settings under **Storage > Volume storage**. This can be set to the primary storage or any configured secondary storage. If volume storage is not selected, scripts with volume annotations will fail with an error. |
| 19 | + |
| 20 | +Volume data is stored under the `volumes/<workspace_id>/<volume_name>/` prefix in the selected storage. |
| 21 | + |
| 22 | +## Declaring volumes |
| 23 | + |
| 24 | +Volumes are declared with comment annotations at the top of your script. The syntax is: |
| 25 | + |
| 26 | +``` |
| 27 | +<comment_prefix> volume: <name> <mount_path> |
| 28 | +``` |
| 29 | + |
| 30 | +Where: |
| 31 | +- `<name>` is the volume identifier. Must be 2–255 characters, start and end with an alphanumeric character, and contain only `a-z`, `A-Z`, `0-9`, `.`, `_`, or `-`. |
| 32 | +- `<mount_path>` is the path where the volume will be available during execution. With [nsjail sandboxing](../../advanced/security_isolation/index.mdx), this can be an absolute path (restricted to `/tmp/`, `/mnt/`, `/opt/`, `/home/`, `/data/` prefixes) or a relative path. Without sandboxing, only relative paths are supported (e.g. `.claude`, `data/models`) — they are resolved relative to the job's working directory via a symlink. |
| 33 | + |
| 34 | +A script can mount up to 10 volumes. Duplicate volume names or mount paths within the same script are not allowed. |
| 35 | + |
| 36 | +### TypeScript / JavaScript |
| 37 | + |
| 38 | +```ts |
| 39 | +// volume: mydata data |
| 40 | +export async function main() { |
| 41 | + // Read and write files in ./data (relative to job working directory) |
| 42 | + const fs = await import("fs"); |
| 43 | + fs.writeFileSync("data/result.json", JSON.stringify({ ok: true })); |
| 44 | +} |
| 45 | +``` |
| 46 | + |
| 47 | +### Python |
| 48 | + |
| 49 | +```python |
| 50 | +# volume: mydata data |
| 51 | +def main(): |
| 52 | + with open("data/result.json", "w") as f: |
| 53 | + f.write('{"ok": true}') |
| 54 | +``` |
| 55 | + |
| 56 | +### With sandbox (absolute paths) |
| 57 | + |
| 58 | +When using the [`sandbox` annotation](../../advanced/security_isolation/index.mdx#sandbox-annotation), you can use absolute mount paths: |
| 59 | + |
| 60 | +```ts |
| 61 | +// sandbox |
| 62 | +// volume: mydata /tmp/data |
| 63 | +export async function main() { |
| 64 | + const fs = await import("fs"); |
| 65 | + fs.writeFileSync("/tmp/data/result.json", JSON.stringify({ ok: true })); |
| 66 | +} |
| 67 | +``` |
| 68 | + |
| 69 | +### Multiple volumes |
| 70 | + |
| 71 | +You can mount multiple volumes on the same script (up to 10): |
| 72 | + |
| 73 | +```python |
| 74 | +# sandbox |
| 75 | +# volume: training-data /tmp/data |
| 76 | +# volume: model-weights /tmp/models |
| 77 | +def main(): |
| 78 | + # /tmp/data has your training data |
| 79 | + # /tmp/models has your model weights |
| 80 | + pass |
| 81 | +``` |
| 82 | + |
| 83 | +## Dynamic volume names |
| 84 | + |
| 85 | +Volume names support interpolation with `$workspace` and `$args[...]` placeholders, resolved at runtime: |
| 86 | + |
| 87 | +| Placeholder | Resolves to | |
| 88 | +|---|---| |
| 89 | +| `$workspace` | Current workspace ID | |
| 90 | +| `$args[param]` | Value of job input `param` | |
| 91 | +| `$args[param.nested]` | Nested value from a JSON input | |
| 92 | + |
| 93 | +For example: |
| 94 | + |
| 95 | +```python |
| 96 | +# volume: $workspace-cache cache |
| 97 | +# volume: data-$args[env] data |
| 98 | +def main(env: str): |
| 99 | + pass |
| 100 | +``` |
| 101 | + |
| 102 | +When called with `env="prod"` in workspace `acme`, mounts volumes `acme-cache` and `data-prod`. |
| 103 | + |
| 104 | +## How it works |
| 105 | + |
| 106 | +1. **Before execution** — Windmill acquires an exclusive lease on the volume, then downloads the volume's files from object storage into a local directory using a per-worker LRU cache (up to 10 GB). Files are compared by size and MD5 hash to skip unchanged files. |
| 107 | +2. **During execution** — The volume directory is bind-mounted (read-write) into the job's [nsjail](../../advanced/security_isolation/index.mdx) sandbox, or symlinked into the job directory when running without sandboxing. |
| 108 | +3. **After execution** — If the volume is writable, changed and new files are synced back to object storage. Deleted files are removed from the remote. Volume metadata (size, file count) is updated. The lease is released. |
| 109 | + |
| 110 | +Symlinks inside a volume are preserved: they are serialized to a metadata file in object storage and restored on download. |
| 111 | + |
| 112 | +### Exclusive leasing |
| 113 | + |
| 114 | +Only one job can use a given volume at a time. Windmill acquires an exclusive lease (60-second TTL, auto-renewed every 10 seconds) on each volume before downloading. If the volume is already leased by another worker, the job waits for the lease to be released. |
| 115 | + |
| 116 | +### Agent workers |
| 117 | + |
| 118 | +Volumes work with [agent workers](../28_agent_workers/index.mdx). Agent workers interact with volumes through an HTTP proxy API on the Windmill server — the same lease, download, sync-back, and permission logic applies. |
| 119 | + |
| 120 | +## Permissions |
| 121 | + |
| 122 | +Volumes support fine-grained permissions through Windmill's standard sharing model. You can share a volume with specific users or groups and set read-only or read-write access. |
| 123 | + |
| 124 | +- **Owner** — the user who created the volume (or whose job first created it). Has full read-write access and can delete the volume. |
| 125 | +- **Read-only** — the job can use the volume but changes are not synced back to object storage after execution. |
| 126 | +- **Read-write** — the job can both read from and write to the volume. |
| 127 | +- **No permission** — if a volume has permissions set and the job's user has no matching entry, the job will fail. |
| 128 | + |
| 129 | +Volumes with no permissions set (empty `extra_perms`) are accessible to all workspace users. |
| 130 | + |
| 131 | +Admins always have full access to all volumes. |
| 132 | + |
| 133 | +Permissions can be managed from the Volumes drawer in the [Assets](../52_assets/index.mdx) page using the share button. |
| 134 | + |
| 135 | +## Managing volumes |
| 136 | + |
| 137 | +### Creating volumes |
| 138 | + |
| 139 | +Volumes can be created in two ways: |
| 140 | + |
| 141 | +- **Automatically** — when a job with a volume annotation runs and the volume doesn't exist yet, it is created with the job's user as owner. |
| 142 | +- **Manually** — from the Volumes drawer in the [Assets](../52_assets/index.mdx) page using the "New volume" button. |
| 143 | + |
| 144 | +### Exploring and deleting |
| 145 | + |
| 146 | +From the Volumes drawer you can: |
| 147 | + |
| 148 | +- View metadata: file count, total size, owner, last used timestamp. |
| 149 | +- Explore files in the volume using the S3 file browser. |
| 150 | +- Share the volume and manage permissions. |
| 151 | +- Delete the volume (owner, users with write permission, or admins). Deleting removes both the database metadata and all files from object storage. A volume that is currently leased by a running job cannot be deleted. |
| 152 | + |
| 153 | +<!-- TODO: add screenshot of the Volumes drawer from the Assets page showing a list of volumes with their metadata --> |
| 154 | + |
| 155 | +## Volumes and sandboxing |
| 156 | + |
| 157 | +Volumes work with both sandboxed ([nsjail](../../advanced/security_isolation/index.mdx)) and non-sandboxed execution, but the mount path behavior differs: |
| 158 | + |
| 159 | +- **With nsjail (sandbox)** — The volume directory is bind-mounted read-write at the declared mount path inside the sandbox. Both absolute (restricted to `/tmp/`, `/mnt/`, `/opt/`, `/home/`, `/data/`) and relative paths work. Relative paths are resolved relative to the nsjail working directory (`/tmp/bun` for Bun/TypeScript, `/tmp/deno` for Deno, `/tmp` for Python and others). |
| 160 | +- **Without nsjail** — Only relative mount paths are supported. Windmill creates a symlink from the job directory to the volume's local directory. |
| 161 | + |
| 162 | +You can combine volumes with the [`sandbox` annotation](../../advanced/security_isolation/index.mdx#sandbox-annotation) to enable per-script sandboxing, which is required if you need absolute mount paths: |
| 163 | + |
| 164 | +```ts |
| 165 | +// sandbox |
| 166 | +// volume: agent-memory .claude |
| 167 | +export async function main() { |
| 168 | + // Sandboxed execution with persistent .claude directory |
| 169 | +} |
| 170 | +``` |
| 171 | + |
| 172 | +<DocCard |
| 173 | + title="AI sandbox" |
| 174 | + description="Run AI coding agents with sandboxing and persistent volumes" |
| 175 | + href="/docs/core_concepts/ai_sandbox" |
| 176 | +/> |
| 177 | + |
| 178 | +<DocCard |
| 179 | + title="Security and process isolation" |
| 180 | + description="Sandbox annotation and nsjail configuration" |
| 181 | + href="/docs/advanced/security_isolation" |
| 182 | +/> |
0 commit comments