|
| 1 | +# S3 Object Tagging in Ingot |
| 2 | + |
| 3 | +Implementation spec for S3 object tagging: the three per-version tagging methods, tag |
| 4 | +stamping at version creation, the tag-count echo, and the conformance surface. Tagging is the |
| 5 | +second tenant of the per-key version-state tree that |
| 6 | +[`s3-object-lock.md`](./s3-object-lock.md) §3 and §4.1 specify, and this document rides that |
| 7 | +design: the `Tags` field already exists on `VersionState`, reserved for exactly this feature, |
| 8 | +so tagging adds handlers and conformance rows with no format change. |
| 9 | + |
| 10 | +--- |
| 11 | + |
| 12 | +## 1. How tagging works |
| 13 | + |
| 14 | +A tag set is per-version mutable state: up to ten key/value pairs attached to one object |
| 15 | +version, readable and replaceable at any time through `GetObjectTagging`, |
| 16 | +`PutObjectTagging`, and `DeleteObjectTagging`, each taking an optional `versionId`. A version |
| 17 | +can also be born tagged: `PutObject`, `CopyObject`, and `CreateMultipartUpload` accept an |
| 18 | +`x-amz-tagging` header (a URL-encoded query string), and `CopyObject` chooses between |
| 19 | +inheriting the source version's tags and taking the header via `x-amz-tagging-directive`. |
| 20 | +`GET` and `HEAD` report only the count (`x-amz-tagging-count`); reading the tags themselves |
| 21 | +takes the dedicated API. |
| 22 | + |
| 23 | +Tagging differs from lock in three ways that shape the implementation. It has **no bucket |
| 24 | +gate**: tags work on any bucket, unversioned included, so the check order is the lock |
| 25 | +order minus the lock-enabled step. Its **absent state is a success**: a version with no tags |
| 26 | +answers an empty tag set (200), where lock answers a 400 sentinel. And it has a **true unset |
| 27 | +operation**: `DeleteObjectTagging` clears the set, which is what the state tree's |
| 28 | +empty-block elision rule (`s3-object-lock.md` §4.1 rule 3) exists for — lock never |
| 29 | +exercises it, tagging does. Because unversioned buckets carry tags, the write-rule discard |
| 30 | +paths (null replacement, null eviction) now genuinely encounter state entries; the §9 |
| 31 | +cleanup rule of the lock design already covers them, so no delete path changes. |
| 32 | + |
| 33 | +versitygw's controller owns validation and rendering: `PutObjectTagging` bodies are parsed |
| 34 | +and validated there (`utils.ParseTagging`: at most ten tags, key length 128, value length |
| 35 | +256, the duplicate-key and invalid-character checks), and the backend receives a clean |
| 36 | +`map[string]string`. The `x-amz-tagging` header is the backend's to parse, through the |
| 37 | +shared `backend.ParseObjectTags` helper (URL decoding plus the key/value length checks), |
| 38 | +before any bytes are ingested. |
| 39 | + |
| 40 | +--- |
| 41 | + |
| 42 | +## 2. The backend contract |
| 43 | + |
| 44 | +The three methods (versitygw `backend/backend.go:90`): |
| 45 | + |
| 46 | +```go |
| 47 | +GetObjectTagging(ctx, bucket, object, versionId string) (map[string]string, error) |
| 48 | +PutObjectTagging(ctx, bucket, object, versionId string, tags map[string]string) error |
| 49 | +DeleteObjectTagging(ctx, bucket, object, versionId string) error |
| 50 | +``` |
| 51 | + |
| 52 | +Creation-time inputs: `Tagging *string` on `s3response.PutObjectInput` and |
| 53 | +`CreateMultipartUploadInput`, plus `Tagging` and `TaggingDirective` on `CopyObjectInput`. |
| 54 | +The controller defaults the directive to `COPY` when the header is absent, so the backend |
| 55 | +only ever sees `COPY` or `REPLACE`. Outputs: `TagCount *int32` on `HeadObjectOutput` and |
| 56 | +`GetObjectOutput`, set only when the resolved version carries at least one tag. |
| 57 | + |
| 58 | +The check order is `s3-object-lock.md` §6 without its lock-enabled step: |
| 59 | + |
| 60 | +1. `registry.Get` misses: `ErrNoSuchBucket`. |
| 61 | +2. The key is absent from the top MST: `ErrNoSuchKey`. |
| 62 | +3. `classifyVersionID` rejects the token: `InvalidArgument` (`InvalidArgVersionId`). |
| 63 | +4. The named version resolves per the versioning grammar; a miss is `GetNoSuchVersionErr`. |
| 64 | +5. The resolved version is a delete marker: `ErrMethodNotAllowed` (all three methods; pinned |
| 65 | + by `Versioning_PutGetDeleteObjectTagging_delete_marker`). |
| 66 | + |
| 67 | +Past the order, absence is never an error: `GetObjectTagging` on a version without tags |
| 68 | +returns the empty map (the controller renders an empty `<TagSet/>`, pinned by |
| 69 | +`GetObjectTagging_unset_tags`), and `DeleteObjectTagging` on a version without tags is an |
| 70 | +idempotent success. |
| 71 | + |
| 72 | +--- |
| 73 | + |
| 74 | +## 3. Storage |
| 75 | + |
| 76 | +`VersionState.Tags` (`s3-object-lock.md` §4.1) holds the set, and every rule written there |
| 77 | +applies unchanged: |
| 78 | + |
| 79 | +- **Merge**: `PutObjectTagging` and `DeleteObjectTagging` own `Tags` and carry `Retention` |
| 80 | + and `LegalHold` verbatim; the lock methods carry `Tags` in return. A replaced set is |
| 81 | + last-write-wins on the one field. |
| 82 | +- **Elision**: an empty replacement (`PutObjectTagging` with an empty `TagSet`, or |
| 83 | + `DeleteObjectTagging`) stores nil, and a state block left with every field absent is |
| 84 | + removed from the tree; the emptied tree drops `State` off the leaf. |
| 85 | +- **Upgrade**: a manifest-arm key takes its leaf on the first tag write, the same |
| 86 | + first-state-write amendment to versioning invariant 6 the lock design made. This is now |
| 87 | + reachable on unversioned buckets. |
| 88 | +- **Cleanup**: a commit that permanently removes a version removes its state entry, on every |
| 89 | + removal path. Tags on unversioned buckets make the discard paths live: a null-replacing |
| 90 | + PUT discards the old null version and prunes its tag entry in the same commit. |
| 91 | + |
| 92 | +Reads resolve exactly as lock reads do: the resolver already holds the leaf, a nil `State` |
| 93 | +answers without another fetch, and one tree seek plus one block fetch serves a tagged key. |
| 94 | + |
| 95 | +--- |
| 96 | + |
| 97 | +## 4. Version-creating writes |
| 98 | + |
| 99 | +- **PutObject** parses `input.Tagging` through `backend.ParseObjectTags` before ingest, so an |
| 100 | + invalid header fails fast and uploads nothing. The parsed set joins the version's initial |
| 101 | + `VersionState` beside any lock headers and stamps in the same commit |
| 102 | + (`s3-object-lock.md` §7): the version and its tags land in one root swap. |
| 103 | +- **CopyObject** with directive `COPY` gives the destination version the source *version's* |
| 104 | + tags (the resolved source, current or version-scoped); with `REPLACE` it parses the |
| 105 | + request's own header. Lock state is never inherited on copy; tags are, exactly when the |
| 106 | + directive says so. |
| 107 | +- **CreateMultipartUpload** parses the header for validation at create, carries the raw |
| 108 | + string on the session row, and `CompleteMultipartUpload` stamps the parsed set onto the |
| 109 | + version it commits, exactly as a single-shot PUT would have. |
| 110 | + |
| 111 | +--- |
| 112 | + |
| 113 | +## 5. Reads |
| 114 | + |
| 115 | +`HeadObject` and `GetObject` gain `TagCount`: after `resolveVersion`, the same single |
| 116 | +state-block fetch that fills the lock echo fields counts the tags, and the field is set only |
| 117 | +when the count is nonzero. Keys without state, and versions without tags, cost and report |
| 118 | +nothing. The lock echo stays gated on the bucket's lock configuration; the tag count is not |
| 119 | +(tagging has no gate), so the state fetch now happens whenever the resolved leaf carries a |
| 120 | +state tree. |
| 121 | + |
| 122 | +--- |
| 123 | + |
| 124 | +## 6. Conformance and testing |
| 125 | + |
| 126 | +The tagging groups run under the plain conf: their buckets are neither lock-enabled nor |
| 127 | +versioned, so the plain teardown applies. The versioned tagging behaviors are the |
| 128 | +`Versioning_*` rows, which run in the existing versioned Versioning category. |
| 129 | + |
| 130 | +**New categories** (`itest/versity_tagging_test.go`): `TestPutObjectTagging` |
| 131 | +(non_existing_object, long_tags, duplicate_keys, tag_count_limit, invalid_tags, success), |
| 132 | +`TestGetObjectTagging` (non_existing_object, unset_tags, invalid_parent, success), and |
| 133 | +`TestDeleteObjectTagging` (non_existing_object, success_status, success, |
| 134 | +expected_bucket_owner). |
| 135 | + |
| 136 | +**Versioning-group additions**: the eight tagging rows |
| 137 | +(`Versioning_{Put,Get,Delete}ObjectTagging_invalid_versionId` and |
| 138 | +`_non_existing_object_version`, `Versioning_PutGetDeleteObjectTagging_delete_marker`, |
| 139 | +`Versioning_PutGetDeleteObjectTagging_success`). |
| 140 | +`Versioning_AccessControl_object_tagging_policy` stays excluded with the AccessControl |
| 141 | +family (admin-API users). |
| 142 | + |
| 143 | +**Promotions**: `PutObject_tagging`, `CreateMultipartUpload_with_tagging`, |
| 144 | +`CopyObject_should_copy_tagging`, `CopyObject_should_replace_tagging`, and |
| 145 | +`GetObject_success` (its ETag assertions passed all along; it XFailed on the `TagCount` |
| 146 | +echo). |
| 147 | + |
| 148 | +**Unit tests** (inmem harness): put/get/delete round-trip including the empty-set answers; |
| 149 | +tagging on an unversioned bucket (the gate-free order, the manifest-arm upgrade, and the |
| 150 | +null-replacement discard pruning the old version's tags); the versionId grammar and marker |
| 151 | +sentinels; the merge carrying lock fields across tag writes and tags across lock writes; |
| 152 | +header stamping on PUT, both copy directives, and the MPU carry; the `TagCount` echo. |
| 153 | + |
| 154 | +--- |
| 155 | + |
| 156 | +## 7. Out of scope |
| 157 | + |
| 158 | +Bucket tagging (`Put/Get/DeleteBucketTagging`: bucket-level documents, registry-shaped like |
| 159 | +the lock configuration), tag-based authorization (policy conditions on tags), and the |
| 160 | +`Versioning_AccessControl_object_tagging_policy` case (admin-API users). The versioning |
| 161 | +design's out-of-scope list drops object tagging and points here, as does the lock design's. |
| 162 | + |
| 163 | +--- |
| 164 | + |
| 165 | +## 8. Implementation map |
| 166 | + |
| 167 | +| Where | Change | |
| 168 | +|---|---| |
| 169 | +| `s3frontend/objecttag.go` (new) | the three methods over the §2 check order; the tag half of the state-block echo | |
| 170 | +| `s3frontend/objectlock.go` | the state-target resolution and state-write helpers take a require-lock flag (tagging passes false); the echo helper adds the tag count and fetches whenever the leaf carries state | |
| 171 | +| `s3frontend/object.go`, `copy.go`, `multipart.go` | header parse via `backend.ParseObjectTags` (§4), copy-directive handling, session carry, `TagCount` on Head/Get | |
| 172 | +| `migrations/sql/` (new) | `multipart_sessions.tagging` (the raw header string) | |
| 173 | +| `registry/stores.go`, `stores_postgres.go`, `inmem/store.go` | `MultipartSession.Tagging` | |
| 174 | +| `itest/versity_tagging_test.go` (new), `versity_{object,multipart,versioning}_test.go` | new categories, the versioning rows, promotions (§6) | |
| 175 | +| `docs/s3-versioning.md`, `docs/s3-object-lock.md` | out-of-scope lists point here | |
0 commit comments