identity service: refactoring, documentation, and testing... #1289 (#1290)

Signed-off-by: Angelo De Caro <adc@zurich.ibm.com>

Author: adecaro

Makefile

@@ -41,7 +41,7 @@ download-fabric:
 unit-tests:
 	@go test -cover $(shell go list ./... | grep -v '/integration/')
 	cd integration/nwo/; go test -cover ./...
-	cd token/services/identity/storage/kvs/hashicorp/; go test -cover ./...
+	cd token/services/storage/db/kvs/hashicorp/; go test -cover ./...
 
 .PHONY: unit-tests-race
 unit-tests-race:
@@ -97,7 +97,7 @@ integration-tests-dvp-dlog:
 tidy:
 	@go mod tidy
 	cd tools; go mod tidy
-	cd token/services/identity/storage/kvs/hashicorp; go mod tidy
+	cd token/services/storage/db/kvs/hashicorp; go mod tidy
 
 .PHONY: clean
 clean:

docs/driverapi.md

@@ -0,0 +1,18 @@
+# FabToken
+
+FabToken is a straightforward implementation of the Driver API.
+It prioritizes simplicity over privacy, storing all token transaction details openly on the ledger for anyone with access to view ownership and activity.
+FabToken exclusively supports long-term identities based on a standard X.509 certificate scheme,
+which reveals the owner's enrollment ID in plain text.
+Tokens are directly represented on the ledger as JSON-formatted data based on the `token.Token` structure.
+The `Owner` field of this structure stores the identity information.
+The `Identity Service` handles the encoding/decoding of this field.
+
+## Security
+
+`FabToken` does not provide privacy for tokens or identities.
+The validator guarantees the following:
+*   **Issuer Authorization**: Only issuers listed in the public parameters can issue tokens.
+*   **Auditor Authorization**: Only auditors listed in the public parameters can audit transactions.
+*   **Owner Authorization**: Only legitimate owners can spend their tokens.
+*   **Value Preservation**: In a transfer, the sum of inputs matches the sum of outputs.

docs/drivers/dlogwogh.md

@@ -0,0 +1,187 @@
+# Identity Service
+
+The **Identity Service** (`token/services/identity`) provides a unified interface for managing identities, signatures, and verification within the Fabric Token SDK. 
+It abstracts the underlying cryptographic details, allowing the SDK to support multiple identity types (e.g., X.509, Idemix) and different storage backends seamlessly.
+
+This service is a fundamental component used by token drivers and other services (like the Token Transaction (TTX) service) to handle:
+*   **Signatures**: Generating and verifying signatures for transactions.
+*   **Identity Resolution**: resolving long-term identities to ephemeral identities and vice-versa.
+*   **Auditability**: Managing audit information to reveal the enrollment ID behind an anonymous identity (if allowed).
+*   **Role Management**: Handling identities for different roles (Issuer, Auditor, Owner, Certifier).
+
+## Architecture
+
+The Identity Service is designed to implement the **Driver API** interfaces defined in `token/driver/wallet.go`. 
+This ensures that the token management system can interact with any identity implementation through a standard set of methods.
+
+### Conceptual Metaphor
+
+To better understand the components, imagine a **Corporate Security Department**:
+
+*   **Identity Service (The Department)**: The entire division responsible for security and access.
+*   **Wallet Service (The Keymaster)**: Maintains a registry of all employees (Wallets) and the badges (Keys) they possess. It knows who has access to what.
+*   **Identity Provider (The Passport Office)**: Verifies identities. It checks if a badge is authentic and extracts information from it (Project ID, Department ID).
+*   **Roles (Job Titles)**: Groups of employees with similar access rights (e.g., "Auditors" have special keys to view records, "Issuers" have keys to create assets).
+*   **Key Managers (The Badge Makers)**: The specific machines or protocols used to create the badges (e.g., one machine makes standard plastic cards (X.509), another makes high-tech smart cards (Idemix)).
+
+### Component Mapping
+
+Here is how the service components map to the Driver API:
+
+| Component              | Implements Driver Interface | Description                                           |
+|:-----------------------|:----------------------------|:------------------------------------------------------|
+| `identity.Provider`    | `driver.IdentityProvider`   | Core identity management & verification.              |
+| `wallet.Service`       | `driver.WalletService`      | Registry for all wallets (Owner, Issuer, etc.).       |
+| `role.LongTermOwnerWallet`     | `driver.OwnerWallet`        | Long Term Identity-based  Onwer wallet functionality. |
+| `role.AnonymousOwnerWallet`     | `driver.OwnerWallet`        | Anonymous Identity-based  Onwer wallet functionality. |
+| `role.IssuerWallet`    | `driver.IssuerWallet`       | Issuer wallet functionality.                          |
+| `role.AuditorWallet`   | `driver.AuditorWallet`      | Auditor wallet functionality.                         |
+| `role.CertifierWallet` | `driver.CertifierWallet`    | Certifier wallet functionality.                       |
+
+### Component Interaction Diagram
+
+```mermaid
+classDiagram
+    direction TB
+    %% Driver Interfaces
+    class IdentityProvider {
+        <<interface>>
+        +GetSigner()
+        +GetAuditInfo()
+        +IsMe()
+    }
+    class WalletService {
+        <<interface>>
+        +OwnerWallet()
+        +IssuerWallet()
+        +RegisterRecipientIdentity()
+    }
+    
+    %% Concrete Implementations
+    class identity.Provider {
+        -Storage
+        -Deserializers
+        -SignerCache
+    }
+    class wallet.Service {
+        -RoleRegistry
+        -IdentityProvider
+        -OwnerWallet
+        -IssuerWallet
+        -AuditorWallet
+        -CertifierWallet
+    }
+    class role.Role {
+        -LocalMembership 
+        +GetIdentityInfo()
+    }
+    class membership.KeyManagerProvider {
+        <<interface>>
+        +Get() KeyManager
+    }
+
+    identity.Provider ..|> IdentityProvider : Implements
+    wallet.Service ..|> WalletService : Implements
+    wallet.Service --> identity.Provider : Uses
+    wallet.Service --> role.Role : Uses (via RoleRegistry)
+    role.Role --> membership.KeyManagerProvider : Uses (via LocalMembership)
+
+    note for identity.Provider "Handles low-level crypto\nand identity verification"
+    note for wallet.Service "High-level management\nof wallets and roles"
+```
+
+### LocalMembership
+
+The `LocalMembership` component (`token/services/identity/membership`) plays a pivotal role in managing local identities for a specific role (e.g., Owner, Issuer).
+
+*   **Binding**: Each instance is bound to a list of **Key Managers**.
+*   **Identity Wrapping**: When a Key Manager generates an identity (based on the configuration), `LocalMembership` automatically wraps it using `WrapWithType`. 
+    This ensures that the generated identity carries the correct type information required by the system (as defined in `token/services/identity/typed.go`).
+*   **Role Implementation**: `LocalMembership` serves as the foundational implementation for `role.Role`. 
+    When you interact with a Role to resolve an identity or sign a transaction, you are effectively delegating to the underlying `LocalMembership`.
+
+### Example: Wiring Services
+
+The following example demonstrates how these services are instantiated and wired together, as seen in the ZKATDLog driver:
+
+```go
+func (d *Base) NewWalletService(...) (*wallet.Service, error) {
+    // 1. Create Identity Provider
+    identityProvider := identity.NewProvider(...)
+
+    // 2. Initialize Membership Role Factory
+    roleFactory := membership.NewRoleFactory(...)
+
+    // 3. Configure Key Managers (e.g. Idemix and X.509 for Owner role)
+    // we have one key manager to handle fabtoken tokens and one for each idemix issuer public key in the public parameters
+    kmps := make([]membership.KeyManagerProvider, 0)
+    // ... add Idemix Key Manager Providers ...
+    kmps = append(kmps, x509.NewKeyManagerProvider(...))
+
+    // 4. Create and Register Roles
+    roles := role.NewRoles()
+    
+    // Owner Role (with anonymous identities)
+    ownerRole, err := roleFactory.NewRole(identity.OwnerRole, true, nil, kmps...)
+    roles.Register(identity.OwnerRole, ownerRole)
+    
+    // Issuer Role (no anonymous identities)
+    issuerRole, err := roleFactory.NewRole(identity.IssuerRole, false, pp.Issuers(), x509.NewKeyManagerProvider(...))
+    roles.Register(identity.IssuerRole, issuerRole)
+    
+    // ... Register Auditor and Certifier roles ...
+
+    // 5. Create Wallet Service with the registered roles
+    return wallet.NewService(
+        logger,
+        identityProvider,
+        deserializer,
+        // Convert the roles registry into the format expected by the wallet service
+        wallet.Convert(roles.Registries(...)),
+    ), nil
+}
+```
+
+## Identity Types
+
+The Identity Service leverages a wrapper called **TypedIdentity** to support various identity schemes uniformly. 
+This allows the SDK to be extensible and capable of handling different cryptographic requirements.
+
+### TypedIdentity
+
+`TypedIdentity` (defined in `token/services/identity/typed.go`) acts as a generic container. 
+It wraps the raw identity bytes with a type label, enabling the system to verify deserializers and process signatures correctly without hardcoding implementation details.
+*   **Structure**: Contains a `Type` (string) and the `Identity` (raw bytes).
+
+### Default Key Managers
+
+The identity service includes two primary implementations for concrete identities:
+
+#### 1. X.509
+Standard PKIX identities.
+*   **Transparency**: Verification reveals the identity of the signer.
+*   **Usage**: Ideal for infrastructure components (nodes, services) or scenarios where anonymity is not required.
+*   **Implementation**: `token/services/identity/x509`.
+
+#### 2. Idemix (Identity Mixer)
+Advanced identity encryption based on Zero-Knowledge Proofs (ZKP).
+*   **Anonymity**: Users can prove they hold a valid credential without revealing their actual identity.
+*   **Unlinkability**: Different transactions from the same user appear uncorrelated.
+*   **Auditability**: Includes "audit info" facilitating regulatory compliance by allowing authorized auditors to reveal the identity.
+*   **Implementation**: `token/services/identity/idemix`.
+
+### Other Identity Types
+
+The architecture supports specialized identity types for complex use cases:
+
+#### Multisig
+Located in `token/services/identity/multisig`.
+*   **Concept**: An identity that wraps multiple sub-identities.
+*   **Usage**: Useful for requiring multiple signatures or representing a group of parties.
+*   **Auditability**: Aggregates audit information for all underlying identities.
+
+#### HTLC (Hashed Time Lock Contract)
+Located in `token/services/identity/interop/htlc`.
+*   **Concept**: A script-based identity used primarily for interoperability mechanisms like atomic swaps.
+*   **Structure**: Encapsulates a **Sender** identity, a **Recipient** identity, hash lock information, and a timeout.
+*   **Behavior**: Validation involves satisfying the script conditions (e.g., providing the hash preimage).

docs/drivers/fabtoken.md

@@ -0,0 +1,37 @@
+## The Network Service
+
+The `network` service, located under `token/services/network`, provides other services with a consistent and predictable interface to the backend (e.g., Fabric).
+Internally, the network service mirrors the structure of the Token API, consisting of a `Provider` of network instances and the `Network` instances themselves.
+
+The network service architecture is depicted below:
+
+![network_service.png](../imgs/network_service.png)
+
+The Fabric-based network implementation utilizes the Fabric Smart Client for configuration and operations, including chaincode queries and transaction broadcasting.
+
+During bootstrap, the Token SDK processes the TMS defined in the configuration.
+For each TMS, the network provider retrieves the network instance corresponding to the `network` and `channel` specified in the TMS ID.
+Failure to retrieve the network instance results in a bootstrap failure.
+Upon success, the `Connect` function on the `Network` instance is invoked with the target namespace.
+This function establishes a connection to the backend, enabling the Token SDK to receive updates on public parameters and transaction finality.
+
+When `Connect` is called, the Fabric network implementation establishes two `Fabric Delivery` streams to receive committed blocks:
+- One stream is used to analyze transactions that update the public parameters.
+  More specifically, for each transaction in a block, the parser checks if the RW set contains a write whose key is the `setup key`.
+  The setup key is set as `\x00seU+0000`. If such a key is found in a valid transaction, then the listener added upon calling `Connect` does the following:
+    - It invokes the `Update` function of the TMS provider passing the TMS ID and the byte representation of the new public parameters.
+      The function works as follows: if a TMS instance with the passed ID does not exist, it creates one.
+      If a TMS with that ID already exists, then:
+        - A new instance of TMS is created with the new public parameters.
+        - If the previous step succeeds, then the `Done` function on the old TMS instance is invoked to release all allocated resources.
+    - If the above step succeeds, then the public parameters are appended to the `PublicParameters` table.
+- The other stream is dedicated to transaction finality. Services can add listeners to the `Network` instance to listen for the finality of specific transactions.
+  The `ttx` service and the `audit` service add a listener when a transaction has reached the point of being ready to be submitted to the ordering service.
+  (For more information, look at the sections dedicated to these services). Both services use the same listener.
+  This listener performs the following actions upon notification of the finality of a transaction:
+    - If the transaction's status is valid, then the token request's hash contained in the transaction is matched against the hash of the token request stored in the database.
+      If they match, then the `Tokens` table is updated by inserting the new tokens and marking the spent tokens as deleted.
+      The corresponding token request in the `Requests` table is marked as `Valid` with a change of the status field.
+    - If the transaction's status is invalid, then the corresponding token request in the `Requests` table is marked as `Invalid` or `Deleted`.
+      In all other cases, an error is returned.
+