wip

Author: BlaineEXE

keps/sig-storage/1979-object-storage-support/README.md

@@ -303,7 +303,7 @@ The fundamental key to this design is the bi-directional "pointer" between Bucke
 1. Admin allows User to use BucketClass
 2. User creates BucketClaim that uses BucketClass
 3. COSI controller observes BucketClaim
-   1. Controller applies `objectstorage.k8s.io/cleanup` finalizer to the BucketClaim
+   1. Controller applies `objectstorage.k8s.io/bucketclaim-protection` finalizer to the BucketClaim
    2. Controller looks up corresponding Bucket
    3. If Bucket does not exist, Controller creates intermediate Bucket resource with these details:
       1. Bucket.name is `bc-`+`<BucketClaim.UID>` (safe if multiple controllers active)
@@ -313,7 +313,7 @@ The fundamental key to this design is the bi-directional "pointer" between Bucke
    5. Controller waits for the intermediate Bucket to be reconciled by COSI sidecar
 4. COSI Sidecar detects intermediate Bucket resource
    1. If the Bucket's driver matches the sidecar's driver, continue
-   2. Sidecar applies `objectstorage.k8s.io/cleanup` finalizer to intermediate Bucket
+   2. Sidecar applies `objectstorage.k8s.io/bucket-protection` finalizer to intermediate Bucket
    3. Sidecar calls the COSI driver via gRPC to provision the OSP bucket
    4. If OSP returns provision fail, COSI sidecar reports error to Bucket status and retries w/ backoff
 5. When OSP returns provision success, COSI sidecar updates Bucket status `ReadyToUse` to true
@@ -339,7 +339,7 @@ In early COSI feedback and in other object storage self-service frameworks, user
    3. Admin must ensure that the Bucket binds only to a specific BucketClaim by specifying the BucketClaim parent reference by namespace and name
 2. COSI sidecar detects the Bucket resource
    1. If the Bucket's driver matches the sidecar's driver, continue
-   2. Sidecar applies `objectstorage.k8s.io/cleanup` finalizer to Bucket
+   2. Sidecar applies `objectstorage.k8s.io/bucket-protection` finalizer to Bucket
    3. Sidecar calls the COSI driver via gRPC call to check that the existing OSP bucket exists
    4. Sidecar exits with retry backoff if existing bucket is nonexistent
    5. When Bucket prep is successful, COSI sidecar updates Bucket status `ReadyToUse` to true
@@ -348,7 +348,7 @@ In early COSI feedback and in other object storage self-service frameworks, user
 4. COSI controller observes BucketClaim
    1. Controller validates BucketClaim fields
    2. Controller looks up corresponding Bucket - if DNE, retry with backoff (or when Bucket is created)
-   3. Controller applies `objectstorage.k8s.io/cleanup` finalizer to BucketClaim
+   3. Controller applies `objectstorage.k8s.io/bucketclaim-protection` finalizer to BucketClaim
    4. If BucketClaim reference set by admin on Bucket doesn't match, error out
    5. Apply Full BucketClaim reference info (with UID) to Bucket spec (Bucket is now bound to claim)
    6. Set BucketClaim status to point to Bucket (claim is now bound to Bucket)
@@ -381,18 +381,18 @@ When a BucketClaim with Bucket reclaim policy `Reclaim` that is deleted, the Buc
       1. If Bucket doesn't exist, go to **CLEANUP** (Bucket was deleted or never existed)
       2. If Bucket-BucketClaim binding is not valid, error out
       3. Apply `objectstorage.k8.io/bucketclaim-being-deleted` annotation to Bucket
-         (tells controller that it's safe to proceed with Bucket deprovisioning)
+         (tells Sidecar that it's safe to proceed with Bucket deprovisioning)
       4. If Bucket deletion policy is `Delete`, add deletion timestamp to Bucket
       5. If `Retain`, nothing more for Controller to do
-      6. **CLEANUP**: Controller removes BucketClaim `objectstorage.k8s.io/cleanup` finalizer
+      6. **CLEANUP**: Controller removes BucketClaim `objectstorage.k8s.io/bucketclaim-protection` finalizer
 3. COSI Sidecar detects Bucket update
    1. If the Bucket's driver matches the sidecar's driver, continue
    2. If `objectstorage.k8.io/bucketclaim-being-deleted` annotation, continue
    3. If reclaim policy is `Delete`
       1. If Bucket has nil deletion timestamp, exit (do not deprovision without deletion timestamp)
       2. Sidecar calls the COSI driver via gRPC to de-provision the OSP bucket
       3. If OSP returns provision fail, Sidecar reports error to Bucket status and retries gRPC call
-      4. When OSP returns provision success, remove Bucket `objectstorage.k8s.io/cleanup` finalizer
+      4. When OSP returns provision success, remove Bucket `objectstorage.k8s.io/bucket-protection` finalizer
    4. If deletion policy is `Retain`, nothing more to do
 
 COSI Sidecar should not have Bucket delete permissions.
@@ -416,7 +416,7 @@ If a BucketClaim is in deleting state, no new BucketAccesses can be created for
 4. COSI Controller detects the BucketAccess resource
    1. Controller looks up corresponding BucketClaim
    2. If BucketClaim is being deleted, error without retry
-   3. Controller sets `objectstorage.k8s.io/cleanup` finalizer on BucketAccess
+   3. Controller sets `objectstorage.k8s.io/bucketaccess-protection` finalizer on BucketAccess
    4. Controller sets `objectstorage.k8s.io/has-bucketaccess-references` annotation on corresponding BucketClaim
       (block claim from being deleted until access is deleted)
    5. If BucketClaim not ready, exit with retry
@@ -427,15 +427,15 @@ If a BucketClaim is in deleting state, no new BucketAccesses can be created for
 5. COSI Sidecar detects the BucketAccess resource
    1. BucketAccess status now shows corresponding Bucket name and BucketAccess info, so sidecar can provision
    2. If the BucketAccess's driver matches the sidecar's driver, continue
-   3. Sidecar applies `objectstorage.k8s.io/cleanup` finalizer to the BucketAccess if needed
+   3. Sidecar applies `objectstorage.k8s.io/bucketaccess-protection` finalizer to the BucketAccess if needed
    4. Sidecar looks up the Bucket to get necessary info
    5. If Bucket has `objectstorage.k8.io/bucketclaim-being-deleted` annotation or deletion timestamp, error without retry
       (this indicates the claim is being deleted, possibly race condition missed in Controller)
    6. Sidecar calls the COSI driver via gRPC to generate unique access credentials for the Bucket
       1. For `IAM` auth, driver is responsible for setting up the ServiceAccount at this step
    7. If OSP returns provision fail, Sidecar reports error to BucketAccess status and retries gRPC call
    8. When OSP returns provision success, COSI sidecar:
-      1. Applies `objectstorage.k8s.io/cleanup` finalizer to the Secret
+      1. Applies `objectstorage.k8s.io/bucketaccess-protection` finalizer to the Secret
       2. Updates the BucketAccess Secret with all info needed to access the OSP bucket
       3. Updates BucketAccess status `ReadyToUse` to true
 
@@ -449,7 +449,7 @@ COSI does not set up or manage mounting BucketAccess information to Pods consumi
 2. COSI Controller detects BucketAccess resource's deletion timestamp
    1. Initially, Controller does nothing, waiting for Sidecar to set `objectstorage.k8s.io/sidecar-cleanup-finished` annotation
 3. COSI Sidecar detects BucketAccess resource's deletion timestamp
-   1. Sidecar removes `objectstorage.k8s.io/cleanup` finalizer from the BucketAccess Secret
+   1. Sidecar removes `objectstorage.k8s.io/bucketaccess-protection` finalizer from the BucketAccess Secret
    2. Sidecar deletes the BucketAccess Secret (should happen before OSP access is removed via gRPC)
    3. Sidecar calls the COSI driver via gRPC to revoke the associated access credentials
    4. If OSP returns de-provision fail, COSI sidecar reports error to BucketAccess status and retries gRPC call
@@ -458,7 +458,7 @@ COSI does not set up or manage mounting BucketAccess information to Pods consumi
       1. Sets `objectstorage.k8s.io/sidecar-cleanup-finished` annotation on BucketAccess
 4. Controller detects BucketAccess resource update, with deletion timestamp
    1. Controller removes `objectstorage.k8s.io/has-bucketaccess-references` from BucketClaim if this is the last BucketAccess against the BucketClaim (this allows BucketClaim to start deletion, if applicable)
-   2. Controller removes `objectstorage.k8s.io/cleanup` from BucketAccess
+   2. Controller removes `objectstorage.k8s.io/bucketaccess-protection` from BucketAccess
 
 #### Attaching Bucket Information to Pods
 
@@ -484,10 +484,12 @@ This section describes the current design for sharing buckets with other namespa
 Annotations:
 - `objectstorage.k8s.io/bucketclaim-being-deleted`: applied to a Bucket when the Controller detects that the Bucket's bound BucketClaim is being deleted
 - `objectstorage.k8s.io/has-bucketaccess-references`: applied to a BucketClaim when the Controller detects that one or more BucketAccesses reference the claim
-- `objectstorage.k8s.io/sidecar-cleanup-finished`: applied to a BucketAccess when the Sidecar has finished cleaning up, allowing the Controller to begin its cleanup operations
+- `objectstorage.k8s.io/sidecar-cleanup-finished`: applied to a BucketAccess when the Sidecar has finished cleaning up, allowing the Controller to begin its final cleanup operations
 
 Finalizers:
-- `objectstorage.k8s.io/cleanup`: prevents a COSI-managed resource from being deleted until COSI has cleaned up underlying resources and/or OSP data
+- `objectstorage.k8s.io/bucketclaim-protection`: applied to BucketClaims to prevent them from being deleted until COSI has cleaned up intermediate and underlying resources
+- `objectstorage.k8s.io/bucket-protection`: applied to Buckets to prevent them from being deleted until COSI has cleaned up underlying resources
+- `objectstorage.k8s.io/bucketaccess-protection`: applied to BucketAccesses and BucketAccess Secrets to prevent them from being deleted until COSI has cleaned up underlying resources
 
 #### Bucket
 
@@ -511,7 +513,7 @@ Bucket {
     // +required
     DeletionPolicy DeletionPolicy
 
-    // Name of the BucketClaim that resulted in the creation of this Bucket.
+    // References the BucketClaim that resulted in the creation of this Bucket.
     // For statically-provisioned buckets, set the namespace and name of the BucketClaim that is
     // allowed to bind to this Bucket.
     BucketClaim corev1.ObjectReference