Skip to content

Feature request: conditions API#175

Closed
alsenz wants to merge 5 commits intothanos-io:mainfrom
alsenz:feat_conditions_API_129
Closed

Feature request: conditions API#175
alsenz wants to merge 5 commits intothanos-io:mainfrom
alsenz:feat_conditions_API_129

Conversation

@alsenz
Copy link
Copy Markdown

@alsenz alsenz commented Apr 13, 2025

Resolves #129

Adds support for conditional object uploads.

Extends ObjectAttributes with optional ObjectVersion (if supported), and Upload(...) with three new ObjectUploadOption, depending on provider support.

  1. IfMatch: the object is written only if the existing object matches the provided version
  2. IfNotMatch: the object is written only if the existing object does not match the provided version
  3. IfNotExists: the object is written only if no object already exists for the provided key

Supports two forms of ObjectVersion:

  1. Generational versions (i.e. increasing integers)
  2. Etags

Not all providers support conditional write. The following providers are supported:

  1. Filesystem: uses extended filesystem attributes. Only supported if the host system supports extended attributes.
  2. InMem
  3. GCS
  4. Azure
  5. S3: IfNotMatch is not yet supported (as it is not supported by AWS).

Clients can check conditional API by calling SupportedObjectUploadOptions on the Bucket interface.

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

API changes

  1. Added 3 new ObjectUploadOptions: IfMatch, IfNotMatch and IfNotExists
    1.1 Made ObjectUploadOption typed, repeating the existing IterOption pattern.
    1.2 Added three new parameter modifiers- WithIfMatch, WithIfNotExists and WithIfNotMatch, and WithContentType to support the previous un-typed ObjectUploadOption
  2. Added optional ObjectVersion field to ObjectAttributes
  3. Added acceptance tests for IfMatch, IfNotMatch, IfNotExists and for Attributes API
  4. Added Bucket provider implementations for:
    1.1 filesystem: this now uses Extended Attributes, if supported by the host system
    1.2 inmem
    1.3 GCS
    1.4 Azure
    1.5 S3, excepting IfNoneMatch which is not fully supported (by AWS) yet
    1.6 Wrappers

Backwards compatibility

API changes are additive and should be backwards compatible.

  • The Upload API changes use a trailing variadic parameter so should be backwards compatible

  • The ObjectAttributes add new field so should be backwards compatible

  • The SupportedObjectUploadOptions interface method potentially breaks existing Bucket implementations. Library users who implement their own providers will need to implement this method. Full or stub implementations are provided for all provider implementations in the libary.

Dependencies

  • Adds dependency on github.com/pkg/xattr, which is BSD-2-clause licensed.

Verification

Added acceptance tests for

  • IfMatch
  • IfNotMatch
  • IfExists
  • Versions on Attributes

Did not run integration tests on other object store providers, as I do not have access to test environments, but these should be unchanged.

@alsenz alsenz changed the title Feature request: conditions API #129 Feature request: conditions API Apr 13, 2025
@alsenz alsenz force-pushed the feat_conditions_API_129 branch from 9e361e0 to 6362f65 Compare April 20, 2025 09:42
alsenz added 3 commits April 20, 2025 10:47
…, inmem providers

Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
@alsenz alsenz force-pushed the feat_conditions_API_129 branch from 6362f65 to 7d699e8 Compare April 20, 2025 09:47
alsenz added 2 commits April 21, 2025 14:38
Signed-off-by: Tom Plowman <7210552+alsenz@users.noreply.github.com>
@alsenz alsenz closed this Apr 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: conditions API

1 participant