Fix problems with failing `prepare-dependencies.sh` script

[michalinacienciala]

Background

We are using keep-network/keep-core dependency in several projects:

• solidity-contracts
• token-dashboard
• coverage-pools

The keep-network/keep-core package contains a KEEP-related contract called TokenStaking which is the same name that we use for staking contract for T token. We wanted to deploy both contracts in the same deployments, so we hit a contract name conflict (see issue wighawag/hardhat-deploy#241). That's why we created a prepare-dependencies.sh script that gets executed as a last step of installation of threshold-network/solidity-contracts project and renames the keep-core's TokenStaking artifact to KeepTokenStaking.

The issue(s)

The script is used as a postinstall script and uses the INIT_CWD variable as a parameter:

solidity-contracts/package.json

Line 29 in f468a7d

"postinstall": "./scripts/prepare-dependencies.sh $INIT_CWD",

The INIT_CWD isn't set by us, it's a variable created by Yarn that should hold the full path of the localization where yarn run/yarn install was executed from. Unfortunately, sometimes it does not get configured correctly (it happens seemingly randomly) and causes the failures of the script.

Another issue is that in the script we're copying the files from the following location:

solidity-contracts/scripts/prepare-dependencies.sh

Line 20 in f468a7d

SOURCE_DIR="$ROOT_DIR/node_modules/@keep-network/keep-core/artifacts"

This location sometimes may not be available - this is due to the fact that Yarn flattens the structure of dependencies. For example, when threshold-network/solidity-contracts is a dependency of some project, Yarn may not include @keep-network/keep-core in threshold-network/solidity-contracts's node_modules if @keep-network/keep-core is already included in node_modules of another dependency.

Examples:

https://github.com/keep-network/tbtc-v2/actions/runs/4969553945/jobs/8912960954:

error /home/runner/work/tbtc-v2/tbtc-v2/solidity/node_modules/@threshold-network/solidity-contracts: Command failed.
Exit code: 1
Command: ./scripts/prepare-dependencies.sh $INIT_CWD
Arguments: 
Directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity/node_modules/@threshold-network/solidity-contracts
Output:
Preparing dependencies artifacts
Root directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity
Source directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity/node_modules/@keep-network/keep-core/artifacts
Destination directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core
mv: cannot stat '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TokenStaking.json': No such file or directory

https://github.com/keep-network/tbtc-v2/actions/runs/4969553945/jobs/8892795133

error /home/runner/work/tbtc-v2/tbtc-v2/solidity/node_modules/@keep-network/ecdsa/node_modules/@keep-network/random-beacon/node_modules/@threshold-network/solidity-contracts: Command failed.
Exit code: 1
Command: ./scripts/prepare-dependencies.sh $INIT_CWD
Arguments: 
Directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity/node_modules/@keep-network/ecdsa/node_modules/@keep-network/random-beacon/node_modules/@threshold-network/solidity-contracts
Output:
Preparing dependencies artifacts
Root directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity
Source directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity/node_modules/@keep-network/keep-core/artifacts
Destination directory: /home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestArrayUtils.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestCurveRewards.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestModUtils.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestSimpleBeneficiary.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestSimpleReceiver.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestSimpleStakerRewards.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TestToken.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TokenDistributor.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TokenGeyser.json': File exists
cp: cannot create regular file '/home/runner/work/tbtc-v2/tbtc-v2/solidity/external/npm/@keep-network/keep-core/artifacts/TokenGrant.json': File exists

https://github.com/keep-network/keep-core/actions/runs/4969527266/jobs/8892744375:

error /home/runner/work/keep-core/keep-core/solidity/ecdsa/node_modules/@keep-network/random-beacon/node_modules/@threshold-network/solidity-contracts: Command failed.
Exit code: 1
Command: ./scripts/prepare-dependencies.sh $INIT_CWD
Arguments: 
Directory: /home/runner/work/keep-core/keep-core/solidity/ecdsa/node_modules/@keep-network/random-beacon/node_modules/@threshold-network/solidity-contracts
Output:
Preparing dependencies artifacts
Root directory: /home/runner/work/keep-core/keep-core/solidity/ecdsa
Source directory: /home/runner/work/keep-core/keep-core/solidity/ecdsa/node_modules/@keep-network/keep-core/artifacts
Destination directory: /home/runner/work/keep-core/keep-core/solidity/ecdsa/external/npm/@keep-network/keep-core
cp: cannot create directory '/home/runner/work/keep-core/keep-core/solidity/ecdsa/external/npm/@keep-network/keep-core/artifacts': File exists

Solutions

There are several approaches we could take to tackle this:

1. Try to fix each problem directly (change the script to not rely on the INIT_CWD variable, figure out how to work with flattened node_modules structure)
2. Try to get rid of the problem with the name conflict by publishing new NPM packages with keep-core contracts that would be the exact copies of the current ones that are in use, with the only difference being the name of the TokenStaking contract (it could be renamed to KeepTokenStaking). This way we would no longer have a name conflict and could get rid of the prepare-dependencies.sh script.
3. Deal with all those dependencies by just hardcoding ABI and addresses. v1 TokenStaking or KeepToken addresses are not going to change. This may be even quicker time-wise than figuring out what is happening with environment variables. And we'll make build times shorter. Double win.

[michalinacienciala]

The problem got fixed by @dimpar in #143.

I published new NPM -dev packages for the following projects:

@keep-network/random-beacon
@keep-network/ecdsa
@keep-network/coverage-pools

I also removed some no longer needed workarounds in the projects that use @threshold-network/solidity-contracts as a dependency:
threshold-network/token-dashboard#531
keep-network/tbtc-v2#629

We still may hit the issue occasionally when deploying on Mainnet or Goerli (to fix that we would have to redeploy @threshold-network/solidity-contracts NPM packages (and all the downstream projects) for those environments. But I think we can live with this, as we don't deploy on those environments often.

[michalinacienciala]

Another PRs related to the removal of prepare-dependencies.sh:
threshold-network/token-dashboard#533
keep-network/keep-core#3618
keep-network/coverage-pools#233
keep-network/tbtc-v2#631