S3UTILS-226 [squash] complete usage description

Add context of the script, its principle, and the main logs output

Author: jonathan-gramain

cleanupMpuOrphans.js

@@ -41,7 +41,32 @@ const LOG_PROGRESS_INTERVAL = (
 const USAGE = `
 cleanupMpuOrphans.js
 
-This script cleans up orphaned multipart upload (MPU) data from S3 buckets.
+This script cleans up orphaned multipart upload (MPU) data and
+metadata from S3 buckets.
+
+Context:
+    There can be S3 metadata keys from internal MPU shadow buckets,
+    possibly associated with orphaned RING objects, that are left
+    behind in some rare cases of duplicate and concurrent "complete
+    MPU" or "abort MPU" requests on the same MPU object. They take up
+    storage space and may cause listing slowdowns, until they are
+    manually removed.
+
+    Technically, orphaned parts are defined as part metadata which do
+    not have a corresponding overview key with the same upload ID
+    (overview keys are present for incomplete, but visible, MPUs).
+
+Principle:
+    This script takes care of cleaning up those orphaned parts data and
+    metadata keys with a two-phase process for each target bucket:
+
+    - Discovery phase: builds a map of orphaned MPU upload IDs present
+      in the bucket
+
+    - Cleanup phase: scans the bucket with a versioned listing,
+      matches any completed MPU with their orphaned counterpart to
+      detect used sproxyd keys, and only deletes the unused ones along
+      with the orphaned metadata
 
 Usage:
     node cleanupMpuOrphans.js
@@ -59,6 +84,31 @@ Optional environment variables:
     TRACE: set to 1 to trace every request to bucketd and sproxyd
     LISTING_PAGE_SIZE: number of keys to list per listing request (default ${DEFAULT_LISTING_PAGE_SIZE})
     LOG_PROGRESS_INTERVAL: interval in seconds between progress update log lines (default ${DEFAULT_LOG_PROGRESS_INTERVAL})
+
+Logs:
+    Logs are output to stdout in JSON format using the standard
+    werelogs formatting. The main log messages are:
+
+    starting discovery phase: scanning MPU shadow bucket
+        logged at the beginning of the discovery phase for a given bucket
+    orphaned MPU found
+        logged during discovery phase for each orphaned MPU discovered
+    discovery phase complete
+        logged when the discovery phase has completed for a given bucket
+    starting cleanup phase: scanning bucket
+        logged at the beginning of the cleanup phase for a given bucket
+    deleted orphaned sproxyd key
+        [VERBOSE ONLY] logged for each orphaned sproxyd key deleted
+    not deleting sproxyd key used by completed MPU
+        [VERBOSE ONLY] logged for each orphaned sproxyd key still used, NOT deleted
+    deleted orphaned part metadata
+        [VERBOSE ONLY] logged for each orphaned part metadata key deleted
+    cleanup phase complete
+        logged when the cleanup phase has completed for a given bucket
+    completed MPU orphan cleanup
+        logged when cleanup is complete for all target buckets/RAFT sessions
+    progress update
+        logged every LOG_PROGRESS_INTERVAL with progress stats
 `;
 
 const log = new werelogs.Logger('s3utils:cleanupMpuOrphans');