bug(fabric): chaincode installation fails

[kostiantyn-bonebriukh]

Describe the bug
Chaincode installation fails when you're running a playbook from step 3 of this README.md
https://github.com/hyperledger/bevel/blob/v1.1.0/platforms/hyperledger-fabric/configuration/README.md#execution

To Reproduce
Steps to reproduce the behavior:

1. Follow https://github.com/hyperledger/bevel/blob/v1.1.0/platforms/hyperledger-fabric/configuration/README.md#execution
2. When you reach step 3, it won't finish successfully.
3. In the failing pod see the error:

Getting Orderer TLS certificates from Vault.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    60  100    60    0     0   1261      0 --:--:-- --:--:-- --:--:--  1304
Vault read API call response: {"errors":["1 error occurred:\n\t* permission denied\n\n"]}
Error: Failed to read Vault secret.
Error Details: 1 error occurred:
	* permission denied

Expected behavior
No issues when you follow the README.md.

Screenshots
No screenshot.

Environment (please complete the following information):

• Bevel version: https://github.com/hyperledger/bevel/blob/v1.1.0
• Runtime environment: ghcr.io/hyperledger/bevel-build:latest. All commands are executed from the docker image

Additional context
Adding information mentioned in Discord.
https://discord.com/channels/905194001349627914/1291821929807872030/1293882579392401448

<@383004601542115342>
On both 1.1.0 and develop branches, the chaincode setup doesn't work.

TLS and stuff is published to Vault for peers and orderes using this pattern
https://github.com/hyperledger/bevel/blob/develop/platforms/hyperledger-fabric/configuration/roles/helm_component/templates/value_peer.tpl#L32

        secretEngine: {{ vault.secret_path | default("secretsv2") }}
        secretPrefix: "data/{{ network.env.type }}{{ name }}"

And this is Chaincode Vault key generation pattern
https://github.com/hyperledger/bevel/blob/develop/platforms/hyperledger-fabric/configuration/roles/helm_component/templates/install_chaincode_job.tpl#L39

      orderersecretprefix: {{ vault.secret_path | default('secretsv2') }}/data/{{ org.name | lower }}/peerOrganizations/{{ namespace }}/orderer

Of course
{{ network.env.type }}{{ name }} is not the same as {{ org.name | lower }}.

{{ name }} is the org.name .

And the rest of the secret path is also invalid. There is no
peerOrganizations/{{ namespace }}/orderer in Vault which is expected by the Chaincode setup Bevel playbook.

Could you advise how chaincodes could be set up?

Alvaro Picazo — Yesterday at 10:36 AM
Feel free to create an Issue and we can work on it!

[michaelkhalsa]

In case it is of help, I ran into this issue recently after tearing down and redeploying an eks cluster using a separate vault, and with no proxy (single cluster, single org, 3 orderers, 2 peers, single chain code).

When I deploy again on the new cluster, in the network.yaml file, I first change the
network.env.type to a new value
and also update the path for gitops flux
This prevents issues with local cached values, previously flux entries, etc., so I get a clean deploy.

The playbook creates a ACL vault policy based on the org name and saves it to the vault, for example:
bevel-vault-mgmt-ca-myorgname-net-policy

If you run the playbook again for a clean deployment to a new cluster, without 'first' editing this policy in the vault to point to the new path within the secretsvs kv engine, then you will get a permission denied error when attempting to read/write from the vault. The playbook looks to see if the policy is present, but does not rewrite it if present.

As an example (for me), when I tore down the old cluster, and created a new cluster with network.env.type: "helfdev3"

path "secretsv2/data/hlfdev3subskribo/" { capabilities = ["list", "read", "create", "update"] } path "secretsv2/data/hlfdevsubskribo/credentials/" { capabilities = ["list", "read", "create", "update"] }

I changed hlfdev2 --> hlfdev3. (which maps to network.env.type in the platform.yaml file)
If you have multiple orgs and a single cluster, then do this for each org policy.

---

By the way, after editing a few other areas of the playbook to get a successfull run, it still does not present a working blockchain. All the certs are created, pods deployed, genesis block created with the correct channel name and peers/orderers and saved to vault / flux. Except there is no join-channels job, no chaincode deployed, and no actual channel that shows on a peer chaincode list command on the peer cli. It is as if that entire section is missing from the network-deploy playbook.

Of course, I cannot connect to it from blockchain explorer in that state as well (after copying over crypto materials from the vault).

Also after finishing the non-operator section as entered the site.yaml,with no errors, it then jumps into the operator fabric version of the network deploy script, but skipping every single task in it.

Since there is no logic path in the site.yaml file, I assume that somewhere the network.env.type variable is changed by one of the tasks (still trying to trace where).

I am deploying from a Mac (apple silicon), and had to use a few hacks, such as running in sudo (would not work otherwise), setting
-e "install_arch=arm64"
-e "bin_install_dir=/Users/michaelkhalsa/hlf/bevel/bin"

And modified the check/setup role for Wait for certs to be created in vault, to use a command instead of a shell, as otherwise it would never complete (as trying to run 'vault' from target, instead of controller.