Skip to content

Commit 22096d6

Browse files
committed
[minor-release] docs: add batch downloads documentation to README and wrap collapsible sections
1 parent a0f1f1c commit 22096d6

1 file changed

Lines changed: 75 additions & 5 deletions

File tree

README.md

Lines changed: 75 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ The primary downloaders and their supported aliases are as follows:
2525
| `ytdlp` | `yt-dlp`, `youtube-dl`, `ytdl` | Wraps the `yt-dlp` binary for sites Danzo doesn't natively support (YouTube, etc.) |
2626
| `resume` | - | Resume downloads from saved interrupted job state |
2727
| `clean` | - | Clear local cache for interrupted/incomplete downloads |
28+
| `batch` | - | Download multiple resources of different types in batch from a file or stdin |
2829

2930

3031
Following are examples to get started with various flags:
@@ -57,6 +58,12 @@ Following are examples to get started with various flags:
5758
danzo ytdlp "https://www.youtube.com/watch?v=VizjMEe0agI" -o marigold.mp4
5859
danzo ytdlp "https://vimeo.com/173855964" # (default yt-dlp output template)
5960
```
61+
- Download multiple files in batch (mixed types in parallel)
62+
```bash
63+
danzo batch downloads.txt # (plaintext batch file)
64+
danzo batch jobs.yaml --workers 4 # (YAML batch file with 4 workers)
65+
cat urls.txt | danzo batch # (pipe urls directly from stdin)
66+
```
6067

6168
## Installation
6269

@@ -123,10 +130,11 @@ Follow these links to quickly jump to the relevant provider:
123130
- [AWS S3 Downloads](#aws-s3-downloads)
124131
- [GitHub Release Downloads](#github-release-downloads)
125132
- [yt-dlp Downloads](#yt-dlp-downloads)
133+
- [Batch Downloads](#batch-downloads)
126134

127135
### HTTP(S) Downloads
128136

129-
Unfold to read
137+
<details><summary>Unfold to read</summary>
130138

131139
The output filename will be inferred from the URL and Danzo will use 8 connection threads and 1 worker by default. You can also specify an output filename manually like:
132140

@@ -167,12 +175,13 @@ danzo clean
167175
```
168176

169177
> ✦ Failed chunks are automatically retried up to 5 times before failing the entire file. Additionally, Danzo automatically runs a clean for a download event once it is successful.
178+
</details>
170179
171180

172181

173182
### M3U8 Stream Downloads
174183

175-
Unfold to read
184+
<details><summary>Unfold to read</summary>
176185

177186
Danzo supports downloading streamed content from M3U8 manifests. This is commonly used for video streaming services, live broadcasts, and VOD content.
178187

@@ -196,12 +205,13 @@ danzo hls "https://rumble.com/v893ud-something.html" -e rumble
196205
danzo hls "https://www.dailymotion.com/video/a999aas" -e dailymotion
197206
danzo hls "https://dai.ly/a999aas" -e dailymotion
198207
```
208+
</details>
199209

200210

201211

202212
### AWS S3 Downloads
203213

204-
Unfold to read
214+
<details><summary>Unfold to read</summary>
205215

206216
There are 2 ways of downloading objects from S3:
207217

@@ -226,12 +236,13 @@ AWS session profiles are used to allow for flexibility and ease of access. As a
226236
> ⚠︎ For successful authentication, Danzo needs to use a profile that is configured for the same region as the S3 bucket.
227237
228238
> ✎ For S3 downloads, the `connections` flag determines how many objects will be downloaded in parallel if downloading a folder.
239+
</details>
229240
230241

231242

232243
### GitHub Release Downloads
233244

234-
Unfold to read
245+
<details><summary>Unfold to read</summary>
235246

236247
It is often a task to download GitHub project releases because it requires figuring out the exact name of the asset file based on the OS and architecture of the machine. Danzo simplifies this process and only requires you to provide the owner and the project name. It uses that to automatically identify the correct latest release for its host's architecture and OS.
237248

@@ -247,12 +258,13 @@ If the user selection process needs to be manually kicked off, use Danzo like so
247258
```bash
248259
danzo ghrelease "owner/repo" --manual
249260
```
261+
</details>
250262

251263

252264

253265
### yt-dlp Downloads
254266

255-
Unfold to read
267+
<details><summary>Unfold to read</summary>
256268

257269
For sites Danzo doesn't natively support (YouTube, Vimeo with audio, etc.), the `ytdlp` command wraps the `yt-dlp` binary and streams its progress into the Danzo TUI so it looks and behaves like every other Danzo job.
258270

@@ -276,6 +288,64 @@ The wrapper:
276288
- If the chosen output path already exists, falls back to `name-(1).ext` (same behavior as `http` / `git-clone`).
277289

278290
> ✎ This is intentionally a thin wrapper - any flags beyond `--output/-o` should be configured on the `yt-dlp` side (e.g., via its `--config-location`).
291+
</details>
292+
293+
### Batch Downloads
294+
295+
<details><summary>Unfold to read</summary>
296+
297+
Danzo supports batch downloading multiple files from a plaintext file, a YAML/JSON configuration, or directly piped from standard input (`stdin`). This allows running mixed job types (HTTP, HLS stream, GitHub Releases, S3, yt-dlp, Torrents) in parallel.
298+
299+
#### Prefix Mapping
300+
URLs can be prefixed with `prefix::` to explicitly set the download provider:
301+
- `http::` -> HTTP download
302+
- `hls::` / `livestream::` / `live-stream::` / `m3u8::` -> HLS live stream video download
303+
- `ghr::` / `github-release::` / `ghrelease::` -> GitHub release download
304+
- `s3::` -> AWS S3 download
305+
- `ytdlp::` / `yt-dlp::` / `youtube-dl::` -> yt-dlp download
306+
- `torrent::` -> BitTorrent / Magnet link download
307+
308+
If no prefix is present, standard HTTP is used as a fallback (with auto-detection for `s3://`, `magnet:`, `.m3u8`, etc.).
309+
310+
#### Plain-Text Format
311+
Each line represents a job with the format `[PREFIX::]URL [OUTPUT_PATH]`. Whitespace splits the URL and optional output path. Output paths with spaces can be wrapped in double quotes.
312+
313+
Example plaintext file `downloads.txt`:
314+
```text
315+
# Mixed batch download list
316+
ytdlp::https://www.youtube.com/watch?v=VizjMEe0agI "youtube video.mp4"
317+
https://example.com/file.zip ./downloads/my-file.zip
318+
s3::s3://mybucket/dataset/
319+
```
320+
321+
Execute via:
322+
```bash
323+
danzo batch downloads.txt --workers 3
324+
```
325+
326+
#### YAML Configuration
327+
Allows setting specific connection counts or custom options per job:
328+
```yaml
329+
- url: "ytdlp::https://www.youtube.com/watch?v=VizjMEe0agI"
330+
output: "marigold.mp4"
331+
- url: "https://example.com/largefile.zip"
332+
output: "archive.zip"
333+
connections: 32
334+
- url: "s3::s3://mybucket/dataset/"
335+
profile: "prod-profile"
336+
```
337+
338+
Execute via:
339+
```bash
340+
danzo batch jobs.yaml --workers 4
341+
```
342+
343+
#### Stdin Piping
344+
Piping links from other commands directly:
345+
```bash
346+
cat list.txt | danzo batch --workers 2
347+
```
348+
</details>
279349

280350

281351

0 commit comments

Comments
 (0)