Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **Targeted single-file upload connection-fault retry opt-out.** `retryUploadConnectionFaults` defaults to `true` on client and per-upload options; per-call values take precedence. Set it to `false` to surface single-file POST network faults and pre-response-header timeouts while retaining server/auth/upload-URL retries and backoff. A replay can create duplicate versions retained by Object Lock. Multipart parts still replace the same part number and retain their existing retry policy. Success-body failures and body-read timeouts remain independently controlled by `retryResponseBodyFailures` (default `false`).

### Changed

- **BREAKING:** `UrlGuard` now requires HTTPS for allowed hosts. Plain-HTTP `apiUrl`, `downloadUrl`, or `s3ApiUrl` values are rejected. Supply an HTTPS URL, or use a custom transport to bypass the guard.
Expand Down
53 changes: 53 additions & 0 deletions MIGRATION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,58 @@
# Migration guide

## Single-file upload connection-fault retry opt-out (unreleased)

Upload POST connection faults retry by default with a fresh upload URL. If a
single-file POST reached B2 before the connection dropped, replay can create
duplicate versions. Object Lock retention or legal hold can prevent deleting
those duplicates. Multipart retries replace the same part number instead.

Previously, `retry: { maxRetries: 0 }` stopped connection-fault retries only by
also disabling the existing server, auth-token, and upload-URL retries. Use
`retryUploadConnectionFaults: false` to disable just the single-file POST
connection-fault retry while keeping those recovery paths and backoff:

```ts
import { B2Client, BufferSource } from '@backblaze-labs/b2-sdk'

async function uploadWithConnectionFaultOptOut(): Promise<void> {
const client = new B2Client({
applicationKeyId: 'your-application-key-id',
applicationKey: 'your-application-key',
retryUploadConnectionFaults: false,
})
await client.authorize()
const bucket = await client.getBucket('your-existing-bucket')
if (bucket === null) throw new Error('Bucket not found')
await bucket.upload({
fileName: 'data.bin',
source: new BufferSource(new Uint8Array([1, 2, 3])),
})
}

uploadWithConnectionFaultOptOut().catch(console.error)
```

Replace the credentials and bucket name before running. You can also set the
option directly on `Bucket.upload()` or `B2Object.upload()`. A per-call value
overrides the client default, including `true` on a client configured with
`false`. Omitting the option retains the current default-on behavior.

The option covers upload-callback network faults, including wrapped
`TypeError`/`DOMException` errors and `requestTimeoutMs` timeouts **before
response headers arrive**. Upload-URL acquisition failures keep retrying under
their existing policy. Multipart uploads and write streams ignore the new
setting and retain their existing retry policy.

Failures while reading a successful response body, including body-read
timeouts, still follow `retryResponseBodyFailures`. Its default is `false`;
setting it to `true` can replay a stored single-file POST even when
`retryUploadConnectionFaults` is `false`. This option is not an exactly-once
guarantee. Keeping both options `false` avoids these two automatic single-file
replay paths without disabling server/auth/upload-URL retries.

## v4 response-shape migrations

This guide covers the breaking changes introduced while aligning the SDK's
TypeScript surface with Backblaze B2's **v4** native API (the `v0.4.0` line,
tracked in [#193](https://github.com/backblaze-labs/b2-sdk-typescript/issues/193)).
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,11 @@ Transient upload failures are retried with a fresh B2 upload URL, matching B2's

Use `onUploadRetry` to log or count retry attempts, compare returned file IDs and SHA-1 values when reconciling uploads, and configure lifecycle or version-retention rules for buckets where duplicate versions must be cleaned up automatically. Retry diagnostics are event-only: the SDK does not log, count, or persist them unless this listener records them. Payload re-POSTs and fresh-URL fetches spend one upload retry budget and are bounded by `retry.maxRetries + 1` attempts per file or part; aggregate retries scale with multipart transfer concurrency. Upload 429 throttling backs off on the same upload URL instead of fetching a new one.

Single-file and multipart part upload POST connection failures retry by default with a fresh upload URL, as B2 recommends for connection resets and socket timeouts. Set `retry.maxRetries: 0` to disable these retries. A repeated single-file POST can create a duplicate file version if the first POST succeeded before the connection failed, especially in versioned or Object Lock buckets; re-uploading the same multipart part number replaces that part. Unreadable success response bodies remain opt-in with `retryResponseBodyFailures: true` for both upload modes; that option re-posts the payload with a fresh URL, bounded by `retry.maxRetries`. Any uploaded `fileRetention` or `legalHold` applies to each duplicate file version and can prevent deletion until the retention policy expires or the hold is cleared.
Single-file and multipart part upload POST connection failures retry by default with a fresh upload URL, as B2 recommends for connection resets and socket timeouts. A repeated single-file POST can create a duplicate file version if the first POST succeeded before the connection failed, especially in versioned or Object Lock buckets. Any uploaded `fileRetention` or `legalHold` applies to each duplicate and can prevent deletion until retention expires or the hold is cleared. Re-uploading the same multipart part number replaces that part.

Set `retryUploadConnectionFaults: false` on `B2Client`, `Bucket.upload()`, or `B2Object.upload()` to stop automatic single-file POST replay after a connection fault, including a timeout before response headers arrive. Per-call values override the client default; the default remains `true`. The opt-out preserves 503, bad-auth-token, bad-upload-URL, upload-URL acquisition retries, and backoff. Multipart parts and write streams ignore this setting and retain their existing retry policy. `retry.maxRetries: 0` still sets the entire retry budget to zero rather than excluding just this class.

Unreadable success response bodies, including body-read timeouts, are controlled independently by `retryResponseBodyFailures` (default `false`). Setting it to `true` permits replay even when `retryUploadConnectionFaults` is `false`, bounded by `retry.maxRetries`. The connection-fault opt-out is not an exactly-once guarantee. See [MIGRATION.md](./MIGRATION.md) for configuration examples and the response-header/body boundary.

Large-file part retries are coordinated per part, not by a shared circuit breaker. During a B2 pod or main-API incident, concurrent parts can independently back off with jitter, fetch fresh part URLs, and retry in parallel; aggregate fresh-URL traffic scales with multipart concurrency and `retry.maxRetries`. Multipart workers abort sibling work promptly after a fatal part failure, and best-effort cleanup is bounded so aborting a stream does not hang indefinitely. Tune `concurrency`, `retry`, and `onUploadRetry` for operators that need outage counters or stricter load shedding. Each HTTP attempt has a 15 minute deadline by default; for streamed download bodies the timeout is idle/no-progress and resets after each chunk. Set `retry.requestTimeoutMs` higher for very slow upload links or to `0` to rely only on your own `AbortSignal`.

Expand Down Expand Up @@ -648,7 +652,7 @@ The SDK automatically retries transient errors with exponential backoff:

- **401 expired_auth_token**: re-authorizes and retries
- **408, 429, transient 5xx (500, 502, 503, 504)**: exponential backoff with jitter, respects `Retry-After` header
- **Network errors**: retried with backoff
- **Network errors**: retried with backoff; high-level single-file upload POST connection faults can be excluded with `retryUploadConnectionFaults: false`
- **Permanent errors** (403 cap_exceeded, 404 not_found, etc.): thrown immediately

Configure retry behavior:
Expand Down
20 changes: 18 additions & 2 deletions src/bucket.ts
Original file line number Diff line number Diff line change
Expand Up @@ -416,20 +416,28 @@ export class Bucket {
readonly info: BucketInfo
readonly #client: B2Client
private readonly uploadRetryOptions: RetryOptions
readonly #retryUploadConnectionFaults: boolean

/**
* @param client - The parent B2Client instance.
* @param info - The bucket metadata from the API.
* @param uploadRetryOptions - Resolved client upload retry defaults.
* @param retryUploadConnectionFaults - Client default for single-file POST connection retries.
*
* @internal
*/
constructor(client: B2Client, info: BucketInfo, uploadRetryOptions: RetryOptions) {
constructor(
client: B2Client,
info: BucketInfo,
uploadRetryOptions: RetryOptions,
retryUploadConnectionFaults = true,
) {
this.#client = client
this.info = info
this.id = info.bucketId
this.name = info.bucketName
this.uploadRetryOptions = uploadRetryOptions
this.#retryUploadConnectionFaults = retryUploadConnectionFaults
}

/**
Expand All @@ -439,7 +447,13 @@ export class Bucket {
* @returns A B2Object handle bound to this bucket and file name.
*/
file(fileName: string): B2Object {
return new B2Object(this.#client, this, fileName, this.uploadRetryOptions)
return new B2Object(
this.#client,
this,
fileName,
this.uploadRetryOptions,
this.#retryUploadConnectionFaults,
)
}

/**
Expand Down Expand Up @@ -480,6 +494,8 @@ export class Bucket {
...smallOptions,
bucketId: this.id,
retry: uploadRetryOptions,
retryUploadConnectionFaults:
options.retryUploadConnectionFaults ?? this.#retryUploadConnectionFaults,
})
}

Expand Down
30 changes: 28 additions & 2 deletions src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import type {
ListKeysResponse,
} from './types/key.ts'
import type { ReplicationConfiguration } from './types/replication.ts'
import { resolveRetryUploadConnectionFaults } from './upload/retry.ts'
import { throwIfSignalAborted } from './util/abort.ts'
import { DEFAULT_PAGE_SIZE } from './util/defaults.ts'
import { type PaginatorOptions, paginateItems } from './util/paginator.ts'
Expand Down Expand Up @@ -67,6 +68,19 @@ export interface B2ClientOptions {
readonly transport?: HttpTransport
/** Override retry behavior (max retries, backoff, and per-attempt timeout). */
readonly retry?: Partial<RetryOptions>
/**
* Whether high-level single-file upload POST connection faults retry.
* Defaults to true; a per-upload value overrides this client setting.
* Includes network errors, wrapped `TypeError`/`DOMException` failures, and
* timeouts before response headers arrive.
* Replaying a stored POST can create duplicate versions retained by Object Lock.
* Set false to surface these faults while retaining server/auth/URL retries
* and backoff. Upload-URL acquisition failures retain their retry policy.
* Multipart parts and write streams ignore this setting.
* Independent of `retryResponseBodyFailures`: success-body read failures,
* including body-read timeouts, follow that option and can still replay when true.
*/
readonly retryUploadConnectionFaults?: boolean
/** Custom user-agent string prepended to the SDK default. */
readonly userAgent?: string
/**
Expand Down Expand Up @@ -149,6 +163,7 @@ export class B2Client {
private readonly realmUrl: string
private readonly userAllowedSuffixes: readonly string[] | undefined
private readonly resolvedUploadRetryOptions: RetryOptions
readonly #retryUploadConnectionFaults: boolean
private readonly reauthCoalescer: ReauthCoalescer<string>

/**
Expand All @@ -163,6 +178,9 @@ export class B2Client {
bindAccountInfoAuthContext(this.#accountInfo, this.realmUrl, this.applicationKeyId)
this.userAllowedSuffixes = options.allowedHostSuffixes
this.resolvedUploadRetryOptions = { ...DEFAULT_RETRY_OPTIONS, ...options.retry }
this.#retryUploadConnectionFaults = resolveRetryUploadConnectionFaults(
options.retryUploadConnectionFaults,
)
this.reauthCoalescer = new ReauthCoalescer((signal) => this.doReauthorize(signal))

let baseTransport: HttpTransport
Expand Down Expand Up @@ -335,7 +353,12 @@ export class B2Client {
this.#accountInfo.getAuthToken(),
request,
)
return new Bucket(this, info, this.resolvedUploadRetryOptions)
return new Bucket(
this,
info,
this.resolvedUploadRetryOptions,
this.#retryUploadConnectionFaults,
)
}

/**
Expand All @@ -360,7 +383,10 @@ export class B2Client {
...options,
},
)
return resp.buckets.map((info) => new Bucket(this, info, this.resolvedUploadRetryOptions))
return resp.buckets.map(
(info) =>
new Bucket(this, info, this.resolvedUploadRetryOptions, this.#retryUploadConnectionFaults),
)
}

/**
Expand Down
6 changes: 6 additions & 0 deletions src/object.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,12 +136,14 @@ export class B2Object {
readonly #client: B2Client
private readonly bucket: Bucket
private readonly uploadRetryOptions: RetryOptions
readonly #retryUploadConnectionFaults: boolean

/**
* @param client - The parent B2Client instance.
* @param bucket - The parent Bucket this object belongs to.
* @param fileName - The file path within the bucket.
* @param uploadRetryOptions - Resolved client upload retry defaults.
* @param retryUploadConnectionFaults - Client default for single-file POST connection retries.
*
* @internal
*/
Expand All @@ -150,11 +152,13 @@ export class B2Object {
bucket: Bucket,
fileName: string,
uploadRetryOptions: RetryOptions,
retryUploadConnectionFaults = true,
) {
this.#client = client
this.bucket = bucket
this.fileName = fileName
this.uploadRetryOptions = uploadRetryOptions
this.#retryUploadConnectionFaults = retryUploadConnectionFaults
}

/**
Expand Down Expand Up @@ -198,6 +202,8 @@ export class B2Object {
bucketId: this.bucket.id,
fileName: this.fileName,
retry: uploadRetryOptions,
retryUploadConnectionFaults:
options.retryUploadConnectionFaults ?? this.#retryUploadConnectionFaults,
})
}

Expand Down
Loading
Loading