Add support for ICSF hardware keys in API ML

docs/appendix/zowe-yaml-configuration.md

@@ -147,15 +147,15 @@ In the `zowe.yaml`, you can define default values which can be overridden in mor
 In Zowe YAML configuration, the certificate definition shares the same format which can be used in several configuration entries. For example, `zowe.certificate`, `components.<component>.certificate`, and `haInstances.<ha-instance>.components.<component>.certificate`. The certificate definition may include the following entries:
 
 - **keystore.type**  
- Specifies the type of the keystore. If you are using keystore, this value usually should be `PKCS12`. If you are using keyring, this value should be `JCERACFKS`.
+ Specifies the type of the keystore. If you are using keystore, this value usually should be `PKCS12`. If you are using z/OS keyring, this value should be one of `JCEHYBRIDRACFKS`, `JCECCARACFKS`, `JCERACFKS`.
 - **keystore.file**  
  Specifies the path of the keystore file. If you are using keyring, this should look like `safkeyring://<keyring-owner>/<keyring-name>`. For example, `safkeyring://ZWESVUSR/ZoweKeyring`.
 - **keystore.password**  
  Specifies the password of the keystore.
 - **keystore.alias**  
  Represents the alias name of the certificate stored in keystore. If you are using keyring, this is the certificate label connected to the keyring.
 - **truststore.type**  
- Specifies the type of the truststore file. If you are using keystore, this value usually should be `PKCS12`. If you are using keyring, this value should be `JCERACFKS`.
+ Specifies the type of the truststore file. If you are using keystore, this value usually should be `PKCS12`. If you are using z/OS keyring, this value should be one of `JCEHYBRIDRACFKS`, `JCECCARACFKS`, `JCERACFKS`.
 - **truststore.file**  
  Specifies the path to the truststore file. If you are using keyring, this should look like `safkeyring://<keyring-owner>/<keyring-name>`, and usually will be the same value of `keystore.file`.
 - **truststore.password**  
@@ -194,13 +194,14 @@ The high-level configuration `zowe` supports these definitions:
 - **zowe.externalDomains**  
  Specifies a list of external domains to be used by the Zowe instance. This configuration is an array of domain name strings.
  In Sysplex deployment, this value is the DVIPA domain name defined in Sysplex Distributor. For example,
-   
+
    ```yaml
    zowe:
     externalDomains:
     - external.my-company.com
     - additional-dvipa-domain.my-company.com
    ```
+
  In Kubernetes deployment, this value is the domain name you will use to access your Zowe running in a Kubernetes cluster.
 - **zowe.externalPort**  
  Specifies the port that is to be exposed to external Zowe users. By default, this value is set based on Zowe APIML Gateway port.
@@ -218,7 +219,7 @@ The high-level configuration `zowe` supports these definitions:
     environments:
       MY_NEW_ENV: value-of-my-env
    ```
- 
+
  :::note
  Variables defined here are global to all Zowe components, on all HA instances.
 

docs/extend/extend-apiml/api-mediation-icsf-keyring.md

@@ -0,0 +1,60 @@
+# Zowe API Mediation Layer - ICSF Hardware Keyring
+
+Zowe version 3.5.0 introduces support in the API Mediation Layer for ICSF-backed private keys.
+Previously this was supported only via AT-TLS with limitations
+
+Depending on configuration, the API ML may use the private key for signing tokens.
+
+## Configuring the z/OS system
+
+Enabling Zowe API Mediation Layer to use ICSF keyrings requires changes to the server user authorization and Java security policy.
+
+### Server user permissions
+
+The Zowe server user must be granted access to specific `CSFSERV` class resources in order to interact with ICSF.
+Ensure that the user has `READ` access to the following profiles in the `CSFSERV` class:
+
+```plaintext
+CSFIQF
+CSFOWH
+CSFRNG
+CSFRNGL
+CSFPKG
+CSFDSG
+CSFDSV
+CSFPKX
+CSFPKI
+CSFEDH
+```
+
+These permissions are necessary for key generation, encryption/decryption, signing of JWT tokens and other cryptographic operations performed via ICSF.
+
+### Java configuration
+
+Using it requires changes to the Java installation:
+
+:::tip:::
+
+Zowe bundles an updated version of the java security policy file. This can be enabled by setting:
+
+```yaml
+zowe:
+  environments:
+    XXX_XXX_XXX: true
+```
+
+::::::
+
+## Configuring Zowe to Use ICSF Keyrings
+
+Update the `zowe.certificate` section in your `zowe.yaml` configuration file as follows:
+
+* Set `zowe.certificate.keystore.type` to `JCEHYBRIDRACFKS`
+
+* Set `zowe.certificate.truststore.type` to `JCEHYBRIDRACFKS`
+
+Make sure `zowe.certificate.trustore.name` and `zowe.certificate.keystore.name` has protocol `safkeyring://` or `safkeyringhybridjce`
+
+## Troubleshooting
+
+For more information about troubleshooting, check [troubleshooting](../../troubleshoot/troubleshoot-zos-certificate.md).

docs/extend/extend-apiml/onboard-plain-java-enabler.md

@@ -727,7 +727,7 @@ The following example shows enabler configuration with keyrings.
 
 **Example:**
 
-```
+```yaml
 ssl:
     keyAlias: localhost
     keyPassword: password

docs/troubleshoot/troubleshoot-zos-certificate.md

@@ -19,10 +19,11 @@ As an API Mediation Layer user, you may encounter problems when configuring cert
 **Symptoms**
 
 Some Zowe Desktop applications do not work when Zowe creates a PKCS12 keystore. A message may appear in the log such as the following:
+
 * ZWES1060W Failed to init TLS environment, rc=1(Handle is not valid)
 * ZWES1065E Failed to configure https server. Check agent https setting.
 
-These messages indicate that ZSS cannot read the generated keystore. As such, parts of Zowe are not functional. 
+These messages indicate that ZSS cannot read the generated keystore. As such, parts of Zowe are not functional.
 
 **Solutions**
 
@@ -68,7 +69,8 @@ The certificate appears to be correct, but the Gateway and the Discovery Service
 The password is only used for USS PKCS12 certificate files. The keyring is protected by SAF permissions. Note that in some configurations, Zowe does not work if the password value is empty in the keyring configuration. We recommended that you assign a value to `password` as shown in the following example:
 
 **Example:**
-```
+
+```yaml
 certificate:
     keystore:
       type: JCERACFKS
@@ -106,9 +108,9 @@ You used an external certificate and Single Sign-On to deploy Zowe. When you log
 **Solution:**
 
 This issue might occur when you use a Zowe version of 1.12.0 or later. To resolve the issue, you can download your external root certificate and intermediate certificates in PEM format. Then, add the following parameter in the `zowe.yaml` file.
- 
+
 ```environments.ZWED_node_https_certificateAuthorities: "/path/to/zowe/keystore/local_ca/localca.cer-ebcdic","/path/to/carootcert.pem","/path/to/caintermediatecert.pem"```
- 
+
 Recycle your Zowe server. You should be able to log in to the Zowe Desktop successfully now.
 
 ## Browser unable to connect due to a CIPHER error
@@ -334,6 +336,7 @@ API ML components do not start properly because they fail to load the JCERACFKS
 The keyring, however, is configured correctly and the STC user can access it. 
 
 **Examples:**
+
 ```
 2023-06-27 13:07:45.138 ..35m<ZWEACS1:main:67502789>..0;39m APIMTST ..31mERROR..0;39m ..36m(o.a.t.u.n.SSLUtilBase)..0;39m Failed to load keystore type .JCERACFKS. with path .safkeyring://ZWESVUSR/ZOWERING. due to .JCERACFKS not found.
 java.security.KeyStoreException: JCERACFKS not found                                                                                                            
@@ -352,7 +355,8 @@ JCERACFKS KeyStore not available
 
 In Java 11 releases before 11.0.17.0, the `IBMZSecurity` security provider is not enabled by default. Locate the `java.security` configuration file in the `$JAVA_HOME/conf/security` USS directory 
 and open the file for editing. Modify the list of security providers and insert `IBMZSecurity` on second position. The list of enabled security providers should resemble the following series:
-```
+
+```properties
 security.provider.1=OpenJCEPlus
 security.provider.2=IBMZSecurity
 security.provider.3=SUN
@@ -368,4 +372,53 @@ security.provider.12=JdkLDAP
 security.provider.13=JdkSASL
 security.provider.14=SunPKCS11
 ```
+
 For more information see the steps in [Enabling the IBMZSecurity provider](https://www.ibm.com/docs/en/semeru-runtime-ce-z/11?topic=guide-ibmzsecurity#ibmzsecurity__enabling_z_provider__title__1).
+
+## Failed to load JCECCARACFKS keyring when using ICSF under ACF2
+
+The issue below may occur only under ACF2, due to differences in how ACF2 handles digital certificate
+and key resource access compared to TSS and RACF.
+
+**Symptom**
+
+The Zowe log contains the following ERROR message:  
+`com.ibm.crypto.ibmjcehybrid.provider.IBMJCEHybridException: Failover exhausted, all registered providers attempted and failed.`
+
+**Explanation**
+
+This error occurs when the `Java Cryptography Extension` (JCE) provider cannot access the `RSA` key dataset defined for Zowe.
+In ACF2, access to RSA key resources is controlled through the CSK resource class,
+which governs permissions to use or read private keys stored in the security database.
+If the Zowe started task ID lacks the appropriate `READ` access to the RSA key resource, all configured JCE providers
+fail to initialize, resulting in this failover error.
+
+**Solution**
+
+Grant the Zowe started task user ID `READ` access to the `ZOWERSAKEY` resource and rebuild the CSK class:
+
+`SET RESOURCE(CSK)
+RECKEY ZOWERSAKEY ADD(USER(ZWESVUSR) SERVICE(READ) ALLOW)
+F ACF2,REBUILD(CSK)`
+
+**Symptom**
+
+The Zowe log contains the following error message:  
+`IBMJCEHybridException: Errors encountered loading keyring. Keyring could not be loaded as a JCECCARACFKS or JCERACFKS keystore.
+Exception#0 java.io.IOException: R_datalib (IRRSDL00) error: not RACF authorized to use the requested service (8, 8, 8)`
+
+**Explanation**
+
+This error indicates that the Zowe started task user ID (`ZWESVUSR`) does not have the required authorization to
+access the R_datalib callable service (`IRRSDL00`).
+In ACF2, this corresponds to the FACILITY class resource `DIGTCERT.LISTRING`.
+Without this access, Zowe cannot load the required certificates or keyrings for establishing secure TLS connections.
+
+**Solution**
+
+Grant the started task user the necessary read access to the `DIGTCERT.LISTRING` resource and rebuild the `FACILITY` class:
+
+`SET RESOURCE(FAC)
+RECKEY IRR ADD(DIGTCERT.LISTRING USER(ZWESVUSR) SERVICE(READ) ALLOW)
+F ACF2,REBUILD(FAC)`
+