Releases: jjjake/internetarchive
Releases · jjjake/internetarchive
Release list
Version 5.10.1
Version 5.10.0
Features and Improvements
- Added
ia download --rangefor partial (byte-range) downloads. It requires
--stdoutand is repeatable, taking[FILE:]START-ENDvalues: a bare
range binds to the named file (vary the range or the file, not both at once),
orFILE:START-ENDbinds each range to its own file. Ranges may be given as
START-END, open-endedSTART-, suffix-N(the lastNbytes), or
bytes=..., and a single value may carry several comma-separated ranges
(0-9,50-99), fetched in order. Segments are streamed back-to-back with no separator, so e.g.
WARC records selected via a CDX index's compressed offset/length can be piped
straight tozcat. Useful for partial fetches of private items (configured
credentials are used).Item.download(),File.download(), and the
top-levelinternetarchive.download()gained aheadersargument, and
Item.download()arange_jobsargument; passing aRangeheader is
treated as an intentional partial fetch and disables resume and full-file
checksum validation. An unsatisfiable range (HTTP416) fails fast with a
clear message instead of being retried; a range covering the whole file
returns the full contents (HTTP200). If any segment fails,
ia downloadexits non-zero, so a downstream pipe consumer can tell the
output is incomplete. - Added
ia tasks --follow-task-log <task_id>to follow a task log live
as the task runs (tail -fstyle), stopping automatically when the task
finishes. Combine with-p lines=-Nto seed the lastNlines first
(Tasks APIlinessemantics, as with--get-task-log); any other
-pparams are forwarded to the Tasks API. A new
ArchiveSession.follow_task_log()method exposes the same behavior to the
library.
Bugfixes
- Fixed
File.download(stdout=True)consulting the local filesystem: a
same-named local file could cause the stream to be skipped (length/date or
checksum match) or trigger the auto-resume code path, which seeks the output
and fails on a pipe. A stdout download now ignores any on-disk file. - Fixed a retried
stdoutdownload falling back to writing a local disk file
instead of the pipe (leaving the pipe empty). Astdoutdownload now always
writes tostdout, even across retries. - Fixed auto-resume corrupting a file when a resumed transfer was itself retried:
the internalRangeheader was not recomputed for the retry, so it no longer
matched the (grown) local file and the seek offset, re-fetching already-written
bytes. The resumeRangeis now recomputed from the current file size on
every attempt. - Fixed
ia tasks --parametercrashing when combined with
--get-task-log. Parameters such aslinesare now merged into the
task log request's query string, allowingia tasks -G <task_id> -p lines=100to fetch a truncated log.get_task_log()gained aparams
argument;paramsandrequest_kwargsare now keyword-only and kept
distinct, so request kwargs (e.g.timeout,headers) are no longer
serialized into the URL as query parameters
(#764 <https://github.com/jjjake/internetarchive/pull/764>_).
Version 5.9.0
Features and Improvements
- Behavior change:
ia download,Item.download(),
File.download(), and the top-levelinternetarchive.download()
now sendcnt=0by default, so downloads no longer count toward
archive.org view counts. Pass--count-views(CLI) or
count_views=True(library) to opt back in; the parameter is
omitted from the request entirely in that case. ia download,ia delete, andia listnow accept--glob
and--excludeflags multiple times in addition to the existing
pipe-separated form.--glob "*.mp4" --glob "*.xml"is now
equivalent to--glob "*.mp4|*.xml", and the two forms can be
mixed.Item.get_files()also pipe-splits list-element patterns
forglob_patternandexclude_pattern, so callers can pass
mixed forms likeglob_pattern=['*.mp4|*.xml', '*.jpg']
(#769 <https://github.com/jjjake/internetarchive/pull/769>_).
Bugfixes
- Fixed
write_reviewsending POST data as form-encoded instead of JSON.
Now sends the request body as a JSON-encoded dict withContent-Type: application/jsonvia the requestsjsonparameter
(#753 <https://github.com/jjjake/internetarchive/pull/753>_). - Restored
append_listdeduplication that was lost in a prior refactor,
preventing duplicate values from being appended to multi-value metadata
fields (#767 <https://github.com/jjjake/internetarchive/pull/767>_). - Fixed
ia metadata --removefailing to match semicolon-delimited
subjects with leading whitespace (e.g."foo; bar; baz"split into
[" bar", " baz"]). Values are now stripped after splitting
(#768 <https://github.com/jjjake/internetarchive/pull/768>_).
Documentation
- Corrected default
ia.iniconfig path in docs, thanks to
@bkjoh <https://github.com/bkjoh>_
(#766 <https://github.com/jjjake/internetarchive/pull/766>_). - Fixed Installation page rendering bugs
(#770 <https://github.com/jjjake/internetarchive/pull/770>_). - Fixed Sphinx render warnings in the
get_configdocstring. - Linked the CLI Quick Start and Upload warning to the canonical
Collections docs
(#772 <https://github.com/jjjake/internetarchive/pull/772>_).
Version 5.8.0
Features and Improvements
- Rewrote
PostDataActionin CLI to accept both JSON strings andkey:valueformat.
Previously only JSON was accepted; nowkey:valuepairs are also supported. - Added "can be specified multiple times" hints to repeatable CLI flags.
- Standardized CLI help text capitalization across all subcommands.
Bugfixes
- Fixed CLI argument parsing bug where
nargs="+"on options like--header,
--metadata, and--formatwould greedily consume subsequent positional
arguments (e.g. the identifier). Changed tonargs=1with repeated flags. - Fixed mutable
default={}bug in custom argparse actions (PostDataAction,
QueryStringAction,MetadataAction,FlattenListAction) that caused state
leakage when parsers were reused across multiple invocations. - Fixed
RecursionErrorcaused bysocket.connectbeing monkey-patched on every
ArchiveSessioninstantiation. Each new session would wrap the already-patched
function, causing infinite mutual recursion after ~980 sessions. The patch is now
applied once at module import time
(#746 <https://github.com/jjjake/internetarchive/issues/746>_).
v5.7.2
+++++++++
Features and Improvements
-
Added support for appending a custom suffix to the User-Agent string.
The default User-Agent (including access key) is always sent to ensure
proper request tracking.Example: With
user_agent_suffix = MyApp/1.0, the full User-Agent becomes:
internetarchive/5.7.2 (Darwin x86_64; N; en; ACCESS_KEY) Python/3.9.0 MyApp/1.0- CLI:
ia --user-agent-suffix "MyApp/1.0" <command> - Config file:
user_agent_suffix = MyApp/1.0in[general]section - Python API:
get_session(config={'general': {'user_agent_suffix': 'MyApp/1.0'}})
- CLI:
Bugfixes
- Fixed bug where metadata append was not working correctly when source metadata field was a list, and failing with obscure messages in some cases.
- Fixed inverted logic for
--download-historyflag inia downloadwhere passing the flag would ignore history files instead of downloading them (#735 <https://github.com/jjjake/internetarchive/issues/735>_).
Version 5.7.1
Bugfixes
- Explicitly set Content-Type header in account API requests.
This header wasn't being set in some cases and caused some requests to fail.
Version 5.7.0
Bugfixes
- Fixed critical bug in
ia delete --globandia delete --formatwhere all files were being deleted regardless
of the glob pattern. This bug was introduced in version v5.4.1 (2025-07-16). - More metadata insert bugfixes and refactoring.
Version 5.6.1
Bugfixes
- Fixed bug where metadata insert was clobbering and indexed modify writes were inserting.
Version 5.6.0
Bugfixes
- Fixed bug in
ia downloadwhere identifier was needlessly being validated and blocking some items from being downloaded. - Fixed regression in
ia download --stdoutwhere directories were being created.
Features and Improvements
- Added support for the Retry-After HTTP header to improve handling of rate-limited API responses.
- Added support for configuring IA-S3 keys via IA_ACCESS_KEY_ID and IA_SECRET_ACCESS_KEY environment variables.
- Added
headersparameter tointernetarchive.files.File.download()for adding custom headers to download requests. - Improved Windows filename sanitization.
Version 5.5.1
Security
- Fixed a critical directory traversal vulnerability in File.download(). All users are urged to upgrade immediately. This prevents malicious filenames from writing files outside the target directory, a risk especially critical for Windows users.
- Added automatic filename sanitization with platform-specific rules.
- Added path resolution checks to block directory traversal attacks.
- Introduced warnings when filenames are sanitized to maintain user awareness.
Please see the security advisory for more details.
Bugfixes
- Fixed bug in JSON parsing for ia upload --file-metadata ....