Skip to content

Releases: jjjake/internetarchive

Version 5.10.1

Choose a tag to compare

@jjjake jjjake released this 29 Jun 22:43
5277f9f

Features and Improvements

  • Changed the ia tasks --follow-task-log short option from -F to -f
    (to match tail -f). -F was only present in 5.10.0 and is no longer
    accepted.

Version 5.10.0

Choose a tag to compare

@jjjake jjjake released this 29 Jun 21:56
71379c5

Features and Improvements

  • Added ia download --range for partial (byte-range) downloads. It requires
    --stdout and is repeatable, taking [FILE:]START-END values: a bare
    range binds to the named file (vary the range or the file, not both at once),
    or FILE:START-END binds each range to its own file. Ranges may be given as
    START-END, open-ended START-, suffix -N (the last N bytes), or
    bytes=..., and a single value may carry several comma-separated ranges
    (0-9,50-99), fetched in order. Segments are streamed back-to-back with no separator, so e.g.
    WARC records selected via a CDX index's compressed offset/length can be piped
    straight to zcat. Useful for partial fetches of private items (configured
    credentials are used). Item.download(), File.download(), and the
    top-level internetarchive.download() gained a headers argument, and
    Item.download() a range_jobs argument; passing a Range header is
    treated as an intentional partial fetch and disables resume and full-file
    checksum validation. An unsatisfiable range (HTTP 416) fails fast with a
    clear message instead of being retried; a range covering the whole file
    returns the full contents (HTTP 200). If any segment fails,
    ia download exits non-zero, so a downstream pipe consumer can tell the
    output is incomplete.
  • Added ia tasks --follow-task-log <task_id> to follow a task log live
    as the task runs (tail -f style), stopping automatically when the task
    finishes. Combine with -p lines=-N to seed the last N lines first
    (Tasks API lines semantics, as with --get-task-log); any other
    -p params are forwarded to the Tasks API. A new
    ArchiveSession.follow_task_log() method exposes the same behavior to the
    library.

Bugfixes

  • Fixed File.download(stdout=True) consulting the local filesystem: a
    same-named local file could cause the stream to be skipped (length/date or
    checksum match) or trigger the auto-resume code path, which seeks the output
    and fails on a pipe. A stdout download now ignores any on-disk file.
  • Fixed a retried stdout download falling back to writing a local disk file
    instead of the pipe (leaving the pipe empty). A stdout download now always
    writes to stdout, even across retries.
  • Fixed auto-resume corrupting a file when a resumed transfer was itself retried:
    the internal Range header was not recomputed for the retry, so it no longer
    matched the (grown) local file and the seek offset, re-fetching already-written
    bytes. The resume Range is now recomputed from the current file size on
    every attempt.
  • Fixed ia tasks --parameter crashing when combined with
    --get-task-log. Parameters such as lines are now merged into the
    task log request's query string, allowing ia tasks -G <task_id> -p lines=100 to fetch a truncated log. get_task_log() gained a params
    argument; params and request_kwargs are now keyword-only and kept
    distinct, so request kwargs (e.g. timeout, headers) are no longer
    serialized into the URL as query parameters
    (#764 <https://github.com/jjjake/internetarchive/pull/764>_).

Version 5.9.0

Choose a tag to compare

@jjjake jjjake released this 28 May 18:54

Features and Improvements

  • Behavior change: ia download, Item.download(),
    File.download(), and the top-level internetarchive.download()
    now send cnt=0 by default, so downloads no longer count toward
    archive.org view counts. Pass --count-views (CLI) or
    count_views=True (library) to opt back in; the parameter is
    omitted from the request entirely in that case.
  • ia download, ia delete, and ia list now accept --glob
    and --exclude flags multiple times in addition to the existing
    pipe-separated form. --glob "*.mp4" --glob "*.xml" is now
    equivalent to --glob "*.mp4|*.xml", and the two forms can be
    mixed. Item.get_files() also pipe-splits list-element patterns
    for glob_pattern and exclude_pattern, so callers can pass
    mixed forms like glob_pattern=['*.mp4|*.xml', '*.jpg']
    (#769 <https://github.com/jjjake/internetarchive/pull/769>_).

Bugfixes

  • Fixed write_review sending POST data as form-encoded instead of JSON.
    Now sends the request body as a JSON-encoded dict with Content-Type: application/json via the requests json parameter
    (#753 <https://github.com/jjjake/internetarchive/pull/753>_).
  • Restored append_list deduplication that was lost in a prior refactor,
    preventing duplicate values from being appended to multi-value metadata
    fields (#767 <https://github.com/jjjake/internetarchive/pull/767>_).
  • Fixed ia metadata --remove failing to match semicolon-delimited
    subjects with leading whitespace (e.g. "foo; bar; baz" split into
    [" bar", " baz"]). Values are now stripped after splitting
    (#768 <https://github.com/jjjake/internetarchive/pull/768>_).

Documentation

  • Corrected default ia.ini config path in docs, thanks to
    @bkjoh <https://github.com/bkjoh>_
    (#766 <https://github.com/jjjake/internetarchive/pull/766>_).
  • Fixed Installation page rendering bugs
    (#770 <https://github.com/jjjake/internetarchive/pull/770>_).
  • Fixed Sphinx render warnings in the get_config docstring.
  • Linked the CLI Quick Start and Upload warning to the canonical
    Collections docs
    (#772 <https://github.com/jjjake/internetarchive/pull/772>_).

Version 5.8.0

Choose a tag to compare

@jjjake jjjake released this 18 Feb 22:54
cfc55fa

Features and Improvements

  • Rewrote PostDataAction in CLI to accept both JSON strings and key:value format.
    Previously only JSON was accepted; now key:value pairs are also supported.
  • Added "can be specified multiple times" hints to repeatable CLI flags.
  • Standardized CLI help text capitalization across all subcommands.

Bugfixes

  • Fixed CLI argument parsing bug where nargs="+" on options like --header,
    --metadata, and --format would greedily consume subsequent positional
    arguments (e.g. the identifier). Changed to nargs=1 with repeated flags.
  • Fixed mutable default={} bug in custom argparse actions (PostDataAction,
    QueryStringAction, MetadataAction, FlattenListAction) that caused state
    leakage when parsers were reused across multiple invocations.
  • Fixed RecursionError caused by socket.connect being monkey-patched on every
    ArchiveSession instantiation. Each new session would wrap the already-patched
    function, causing infinite mutual recursion after ~980 sessions. The patch is now
    applied once at module import time
    (#746 <https://github.com/jjjake/internetarchive/issues/746>_).

v5.7.2

Choose a tag to compare

@jjjake jjjake released this 29 Jan 21:28

+++++++++

Features and Improvements

  • Added support for appending a custom suffix to the User-Agent string.
    The default User-Agent (including access key) is always sent to ensure
    proper request tracking.

    Example: With user_agent_suffix = MyApp/1.0, the full User-Agent becomes:
    internetarchive/5.7.2 (Darwin x86_64; N; en; ACCESS_KEY) Python/3.9.0 MyApp/1.0

    • CLI: ia --user-agent-suffix "MyApp/1.0" <command>
    • Config file: user_agent_suffix = MyApp/1.0 in [general] section
    • Python API: get_session(config={'general': {'user_agent_suffix': 'MyApp/1.0'}})

Bugfixes

  • Fixed bug where metadata append was not working correctly when source metadata field was a list, and failing with obscure messages in some cases.
  • Fixed inverted logic for --download-history flag in ia download where passing the flag would ignore history files instead of downloading them (#735 <https://github.com/jjjake/internetarchive/issues/735>_).

Version 5.7.1

Choose a tag to compare

@jjjake jjjake released this 29 Oct 16:49

Bugfixes

  • Explicitly set Content-Type header in account API requests.
    This header wasn't being set in some cases and caused some requests to fail.

Version 5.7.0

Choose a tag to compare

@jjjake jjjake released this 17 Oct 17:50

Bugfixes

  • Fixed critical bug in ia delete --glob and ia delete --format where all files were being deleted regardless
    of the glob pattern. This bug was introduced in version v5.4.1 (2025-07-16).
  • More metadata insert bugfixes and refactoring.

Version 5.6.1

Choose a tag to compare

@jjjake jjjake released this 14 Oct 16:51

Bugfixes

  • Fixed bug where metadata insert was clobbering and indexed modify writes were inserting.

Version 5.6.0

Choose a tag to compare

@jjjake jjjake released this 10 Oct 17:49

Bugfixes

  • Fixed bug in ia download where identifier was needlessly being validated and blocking some items from being downloaded.
  • Fixed regression in ia download --stdout where directories were being created.

Features and Improvements

  • Added support for the Retry-After HTTP header to improve handling of rate-limited API responses.
  • Added support for configuring IA-S3 keys via IA_ACCESS_KEY_ID and IA_SECRET_ACCESS_KEY environment variables.
  • Added headers parameter to internetarchive.files.File.download() for adding custom headers to download requests.
  • Improved Windows filename sanitization.

Version 5.5.1

Choose a tag to compare

@jjjake jjjake released this 05 Sep 18:34

Security

  • Fixed a critical directory traversal vulnerability in File.download(). All users are urged to upgrade immediately. This prevents malicious filenames from writing files outside the target directory, a risk especially critical for Windows users.
  • Added automatic filename sanitization with platform-specific rules.
  • Added path resolution checks to block directory traversal attacks.
  • Introduced warnings when filenames are sanitized to maintain user awareness.

Please see the security advisory for more details.

Bugfixes

  • Fixed bug in JSON parsing for ia upload --file-metadata ....