feat: add ETH Archive Service diagram

Author: pulsargranular

auth.md

@@ -122,48 +122,55 @@ sequenceDiagram
 
 ## Option C (Proposal Swen)
 
-### Initial thoughts
+### Initial thoughts and running environment
 
 **Ingestor**
 
 - can run anywhere
-- therefore, it cannot contain any secret
-- needs to talk to SciCat and Archiver Service API
-- needs to be authorised with a user token
+  - as a central service
+  - as a desktop application
+- it cannot contain any secret (e.g. service account for SciCat)
+- app must be authorised by the user
 
 **User**
 
-- does not want to log in all the time
-- is only interested in starting a job
+- does not want to log in all the time (single sign-on preferred)
+- is only interested in _starting_ a job, e.g.
   - archive data
   - unarchive data
-- his authentication token can time out
+- the user's authentication token can time out
+- the job itself needs to be able to report when its done, independently of the user
 
 **SciCat**
 
 - only accepts authenticated requests
+  - currently only SciCat tokens
+  - Service Accounts
 - issues its own SciCat tokens (JWT with HS256 algorithm, aka «self signed»)
   - after a user has successfully logged in to Keycloak
-- currently only accepts SciCat tokens
-- so it acts as an authority instance
+- acts as an authority instance
 - offers a self-made mechanism to check if a SciCat token is valid
 
-**Archiver Service**
+**ETHZ Archiver Service**
 
 - only accepts authenticated requests
-- issues Keycloak service tokens (JWT with RS256, public key signed)
-- currently only accepts JWT tokens issued and signed by Keycloak
+  - JWT RS256 tokens issued and signed by Keycloak
+  - JWT HS256 tokens issued by SciCat
+- can issue a Ingester token for a valid SciCat token
+- has service account for SciCat
+- must be able to report the status of a dataset any time to SciCat
 
-### What needs to be done
+### Use-case: User archives data (MinIO S3 Storage)
 
-- Archiver Service
-  - needs to be able to accept SciCat 🔑 as well (see ScopeMArchiver#146)
-  - needs to be able to create valid SciCat 🔑 (variant A)
-- SciCat
-  - can exchange Ingestor JWT 🔑 for Ingestor SciCat 🔑 (variant B)
-  - accepts all JWT 🔑 issued and signed by Keycloak (variants C+D)
+**Notes**: 
 
-**Note**: the diagram below does not yet include any authorisation information. It only includes authentication. In future we would like to use JWT 🔑 that contain authorisation information, e.g. tokens for every dataset upload.
+- the diagram below does not include any authorisation information, only authentication.
+- In future we would like to use JWT 🔑 that contain authorisation information, e.g. tokens for every dataset upload.
+- ETH uses MinIO S3 instead of Globus
+- the ETHZ Archiver Service has service accounts:
+   - for S3 to get pre-signed URLs for data upload
+   - for SciCat to report when the archiving of a dataset has finished
+- PSI needs a similar service to ensure reporting back to SciCat 
 
 
 ```mermaid
@@ -172,46 +179,43 @@ sequenceDiagram
   participant B as Browser
   participant S as Scicat Backend
   participant I as Ingestor Service
-  participant G as Storage (Globus)
   participant A as ETHZ Archiver Service
+  participant M as MinIO S3 Storage
   participant K as Keycloak
 
-  B -) S: Access
+  Note over B, K: Authorise User
+  B -) S: request access
   S -) K: User Login (redirect)
-  K --) S: User JWT🔑
-  S ->> S: Exchange User JWT 🔑 for SciCat 🔑
-  S --) B: User SciCat 🔑
-  B --) I: User SciCat 🔑
-  I --) A: User SciCat 🔑
-  alt tbd. ScopeMArchiver issue 146
-    A -) S: verify + request User info (User SciCat 🔑)
-    S -) A: OK + User info
-  end
-  A -) K: request Ingestor JWT 🔑 (user/pw)
-  K --) A: Ingestor JWT 🔑
-  A --) I: Ingestor JWT 🔑
-  I ->> I: store refresh-token of Ingestor JWT 🔑
-  I -) A: request S3 credentials (Ingestor JWT 🔑)
-  A --) I: S3 credentials
-  I ->> I: upload to S3 ⏳
-  I -) A: report S3 upload finished (Ingestor JWT 🔑)
-
-  alt Variant A: Archiver can create Ingestor SciCat 🔑
-    A -) S: request Ingestor SciCat 🔑 (secret/Basic Auth)
-    S --) A: Ingestor SciCat 🔑
-    A --) I: Ingestor SciCat 🔑
-    I -) S: report dataset upload finished (Ingestor SciCat 🔑)
-  else Variant B: SciCat exchanges Ingestor JWT 🔑 for SciCat 🔑
-    I -) S: request Ingestor SciCat 🔑 (Ingestor JWT 🔑)
-    S ->> S: Exchange Ingestor JWT 🔑 for SciCat 🔑
-    S --) I: Ingestor SciCat 🔑
-    I -) S: report dataset upload finished (Ingestor SciCat 🔑)
-  else Variant C: SciCat accepts Ingestor JWT 🔑
-    I -) S: report archiving finished (Ingestor JWT 🔑)
-  else Variant D: Archiver report directly back to SciCat
-    A ->> A: store Ingestor JWT 🔑
-    A ->> A: wait until upload is finished ⏳
-    A -) S: report archiving finshed (Ingestor JWT 🔑)
+  K -) S: User JWT🔑
+  S ->> S: Exchange User JWT 🔑 —> User SciCat 🔑
+  S -) B: User SciCat 🔑
+
+  Note over B, K: ETH Archiver Service<br/>Authorise Ingestor
+  B -) I: provide User SciCat 🔑 via Cookie
+  I -) A: request /token<br/>User SciCat 🔑
+  A -) S: verify + request User info (User SciCat 🔑)
+  S -) A: OK + User info 📜
+  A -) K: request Ingestor JWT 🔑 (Keycloak Service Account 🔑)
+  K -) A: Ingestor JWT 🔑 + refresh 🔑
+  A -) I: Ingestor JWT 🔑 + refresh 🔑
+
+  Note over I, K: Get presigned S3 URLs for upload
+  I -) A: request S3 URLs (Ingestor JWT 🔑)
+  A -) M: request S3 URLs (MinIO 🔑)
+  M -) A: S3 URLs 🔑
+  A -) I: S3 URLs 🔑
+
+  Note over I, K: Upload data (refresh tokens)
+  I -) M: upload data (S3 URLs 🔑) ⏳
+  loop renew Ingestor JWT 🔑 if needed
+    I -) K: request Ingestor JWT (refresh 🔑)
+    K -) I: new Ingestor JWT 🔑 + refresh 🔑
   end
 
+  Note over S, M: Report upload finished
+
+  I -) A: report data upload to MinIO finished (Ingestor JWT 🔑)
+  A -) M: finish upload workflow
+  A -) S: report archiving finished (Service Account 🔑)
+
 ```