cardano-testnet executable: take node config file as optional input #6103

smelc · 2025-02-05T14:45:53Z

Description

This PR makes it possible for callers of the cardano-testnet binary to pass a custom node configuration file, as well as custom genesis files. To do so, a new --node-config flag is added to the cardano-testnet executable:

Usage: cardano-testnet cardano [--num-pool-nodes COUNT | --node-config FILEPATH]
  ...

Available options:
  --num-pool-nodes COUNT   Number of pool nodes. Note this uses a default node
                           configuration for all nodes. (default: 1)
  --node-config FILEPATH   Path to the node's configuration file (which is
                           generated otherwise). This option also allows to pass
                           custom genesis files: if an era genesis file pointed
                           to in this file (fields "AlonzoGenesisFile",
                           "ShelleyGenesisFile", etc.) doesn't exist, it is
                           automatically generated. So point to an existing
                           genesis file to use your own file, or simply omit it
                           to have it generated by cardano-testnet. See
                           https://github.com/IntersectMBO/cardano-node/blob/master/cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/CommandLineExecutable.hs
                           for an example of usage.

As explained above, this flag also allows to pass custom genesis files.

How to trust this PR

Execute the new test

The new behavior is tested in the new test CommandLineExecutable.hs which you can execute locally as follows:

> cabal test cardano-testnet-test --test-options '-p "/cardano-testnet-executable/"'
Running 1 test suites...
Test suite cardano-testnet-test: RUNNING...
test/Spec.hs
  Spec
    CLI
      cardano-testnet-executable: OK (60.07s)
          ✓ cardano-testnet-executable passed 1 test.

All 1 tests passed (60.07s)
Test suite cardano-testnet-test: PASS
Test suite logged to:
/home/churlin/dev/cardano-node/dist-newstyle/build/x86_64-linux/ghc-8.10.7/cardano-testnet-10.0.0/t/cardano-testnet-test/noopt/test/cardano-testnet-10.0.0-cardano-tes

Note how the test finishes in 60 seconds, which is a duration chosen in CommandLineExecutable.hs, during which we check that nodes have been produced.

Execute the `cardano-testnet` binary directly

You can also run the cardano-testnet binary directly as follows (here using a custom node configuration file and a custom alonzo and conway genesis, but leaving the shelley genesis (the hardest to tweak) to be autogenerated:

> cat test-cardano-testnet.sh
#!/usr/bin/env bash
set -eux

TMP_DIR=$(mktemp -d)

for config_file in alonzo-genesis.json configuration.json conway-genesis.json
do
  cp cardano-testnet/test/cardano-testnet-test/files/input/executable/$config_file $TMP_DIR/.
done

cabal run cardano-testnet -- cardano --node-config $TMP_DIR/configuration.json --output-dir $TMP_DIR --testnet-magic 42
> ./test-cardano-testnet.sh
  ✗ <interactive> failed at src/Testnet/Property/Run.hs:89:7
    after 1 test.
    shrink path: 1:
  
    forAll0 =
      ━━━━ File: /tmp/nix-shell.iZ2TPc/tmp.ly3nO21w1e/current-stake-pools.json ━━━━
      [
          "pool1r8fuwkk3kkfekh6el0kzydrn009yqd89mrv4zpjq77wg6639ese"
      ]
    
    forAll1 =
      Reading file: /tmp/nix-shell.iZ2TPc/tmp.ly3nO21w1e/current-stake-pools.json
    
    forAll2 =
      ━━━━ command ━━━━
      /home/churlin/.local/state/cabal/store/ghc-8.10.7/cardano-cli-10.4.0.0-e-cardano-cli-3b9fee1097cc434bbca7740006a83745fcc229aa57d45fcca2712990862bc6b9/bin/cardano-cli latest query stake-pools --out-file /tmp/nix-shell.iZ2TPc/tmp.ly3nO21w1e/current-stake-pools.json
    
    forAll3 =
      /tmp/nix-shell.iZ2TPc/tmp.ly3nO21w1e/current-stake-pools.json
...
... lots of output
...
forAll75 =
      Reusing /tmp/nix-shell.iZ2TPc/tmp.ly3nO21w1e
  
    This failure can be reproduced by running:
    > recheckAt (Seed 1622863211725641548 12217359344083085813) "1:" <property>
  
Testnet is running.  Type CTRL-C to exit.

At this point the testnet is running and you can kill this process using CTRL-C, but it doesn't kill the cardano-node process. ⚠️ This will need to be fixed soon as proposed below. Also we will want to unplug ourselves from the test monad in this scenario and this is already tracked here: #6122

In the meantime, users can either do killall cardano-node to clean up or use a combination of pidof cardano-node and kill. I didn't want to do in this PR, as this one is quite big already.

Follow-up issues to create

@Jimbo4350> please tick them if you agree 🙂 I'll create them

Have pTestnetNodeOptions return a list of TestnetNodeOption and TestnetNodeOption be UserProvidedNodeOptions Filepath | SpoNodeOptions [Srting] | RelayNodeOptions [ String]. In other words merge the types TestnetNodeOptions and AutomaticNodeOption. This will generalize the behavior so that the user can create multiple nodes, no matter whether they are using custom options or using default behavior (Spo or Relay). This will require a bit of generalization in Cardano.hs but not much.
Check consistency of the value passed to cardanoTestnet: if the user passes a custom node configuration file and a custom genesis file (as a Haskell value right now), we should check that the genesis file pointed to by the node configuration file (if any) contains the same Haskell value as the custom genesis file.
Change cardanoTestnet to take Filepath for genesis files instead of Haskell values, maybe?
Check that the CLI flags changing Shelley genesis values are not used when passing a custom Shelley genesis file (TODOs are added in the code too about this).
Created already: cardano-testnet: runTestnet: don't tie it with H.Integration #6122

Checklist

Commit sequence broadly makes sense and commits have useful messages
CI passes.
New tests are added
Self-reviewed the diff

Jimbo4350

How do you plan to deal with conflicts in the node configuration yaml file? I think this is the wrong approach. We don't want to be forced to reconcile configuration file differences. All we should need the is the node configuration yaml file.

cardano-testnet/src/Parsers/Run.hs

smelc · 2025-02-12T10:13:51Z

For future reference, if we leave the call to cardano-cli debug check-node-configuration, we have the following failures with the following genesis files:

> cabal run cardano-testnet -- cardano --output-dir cl-output-dir --node-config cl-example/configuration.json --testnet-magic 44
  ✗ <interactive> failed at src/Testnet/Start/Cardano.hs:283:3
    after 1 test.
    shrink path: 1:
  
    forAll0 =
      Process exited with non-zero exit-code: 1
      ━━━━ stdout ━━━━
      Checking byron genesis file: cl-example/byron.genesis.spec.json
      
      ━━━━ stderr ━━━━
      Error reading node configuration at: cl-example/byron.genesis.spec.json: Incorrect schema for GenesisData.
       Error: expected field protocolConsts

It seems the code is requiring too much data here when deserializing the Byron genesis. This data is actually not required to start the node ~~(i.e. if we remove the check, the testnet starts fine). So it seems overkill to require this data when starting a testnet.~~

[edit] Oh well, it seems the node doesn't like it either actually:

✗ <interactive> failed at src/Testnet/Start/Cardano.hs:342:5
    after 1 test.
    shrink path: 1:
  
    forAll0 =
      Cardano node process did not start: CardanoProtocolInstantiationError (CardanoProtocolInstantiationErrorByron (GenesisReadError "/home/churlin/dev/cardano-node/cl-example/byron.genesis.spec.json" (GenesisDataSchemaError (SchemaError {seExpected = "field protocolConsts", seActual = Nothing}))))
      
      cardano-node: There was an error parsing the genesis file: "/home/churlin/dev/cardano-node/cl-example/byron.genesis.spec.json" Error: GenesisDataSchemaError (SchemaError {seExpected = "field protocolConsts", seActual = Nothing})

The node itself, when it starts, doesn't require hashes of genesis files. So it seems overkill to require it to start a testnet. See also #6103 (comment)

smelc · 2025-02-18T12:56:14Z

All we should need the is the node configuration yaml file.

@Jimbo4350> Updated version of this PR now does that 👍

… node configuration file (it was unused) This will be done differently in #6103

Jimbo4350

This is an improvement 👍 . Have a look at my comments.

Jimbo4350 · 2025-02-18T13:41:21Z

cardano-testnet/src/Parsers/Cardano.hs

+        <>  OA.showDefault
+        <>  OA.value 1)
+    , UserNodeOptions
+        <$> strOption ( long "node-config"


So this is an improvement however the resultant type should be [TestnetNodeOptions]. Users (or QA) may want to for example create forking conditions whereby nodes have different configurations. This will have to be done in another PR at some point.

Yes I agree, but as you say, this ought to be done later. This PR does a lot of changes already.

Jimbo4350 · 2025-02-18T13:49:46Z

cardano-testnet/src/Parsers/Run.hs

+                Nothing -> error $ "Genesis files not specified in node configuration file: " <> nodeInputConfigFile
+                Just x -> x
+            adjustedProtocolConfig =
+              -- Make all the files be relative to the location of the config file.


Jimbo4350 · 2025-02-18T13:52:06Z

cardano-testnet/src/Parsers/Run.hs

+          alonzoGenesisFile
+          conwayGenesisFile
+  where
+    getShelleyGenesises (NodeProtocolConfigurationCardano _byron shelley alonzo conway _hardForkConfig) =


We also want to confirm the validity of the byron genesis but that can be a separate PR in the future.

Jimbo4350 · 2025-02-18T13:55:50Z

cardano-testnet/src/Testnet/Start/Cardano.hs

+  where
+    nSpoNodes =
+      case cardanoNodes of
+        UserNodeOptions _ -> 1


Hardcoding this isn't great. This is related to my comment above regarding [TestnetNodeOptions]

Jimbo4350 · 2025-02-18T13:58:42Z

cardano-testnet/src/Testnet/Start/Cardano.hs

-    -- to run a testnet. "create-staked" is not a good way to do this especially because it
-    -- makes assumptions about where things should go and where genesis template files should be.
-    -- See all of the ad hoc file creation/renaming/dir creation etc below.
-    H.failMessage GHC.callStack "Specifying node configuration files per node not supported yet."


We need to note that we want to support configuration files per node. However it's not urgent to do this now.

cardano-testnet/src/Testnet/Start/Types.hs

Jimbo4350 · 2025-02-18T14:51:37Z

cardano-testnet/src/Testnet/Start/Types.hs

-cardanoNumRelays CardanoTestnetOptions{cardanoNodes} =
-  NumRelays . length $ filter isRelayNodeOptions cardanoNodes
+cardanoNumRelays CardanoTestnetOptions{cardanoNodes=_} =
+  undefined


Jimbo4350 · 2025-02-18T14:59:16Z

cardano-testnet/src/Testnet/Start/Cardano.hs

  => HasCallStack
  => CardanoTestnetOptions -- ^ The options to use
-  -> Conf
-  -> UserNodeConfig -- ^ The node configuration file to use. If omitted it's generated.


This is an indication that TestnetNodeOptions ought to be renamed to TestnetNodeConfiguration.
A type like the following would be more indicative:

data TestnetNodeConfiguration = UserSpecifiedConfiguration FilePath | DefaultConfiguration [AutomaticNodeOption]

However what we really want is:

data TestnetNodeConfiguration = UserSpecifiedConfiguration FilePath | DefaultSpoConfiguration [String] | DefaultRelayConfiguration [String]

This gives users the most flexibility in that they can spin up a testnet with a mixture of nodes. Haddocks are desperately needed to indicate that [String] types are command line executable modifiers.

…erve existing behavior too

Jimbo4350 · 2025-03-14T12:01:12Z

cardano-testnet/src/Parsers/Run.hs

+              adjustFilePaths (takeDirectory nodeInputConfigFile </>) protocolConfig
+            (shelley, alonzo, conway) = getShelleyGenesises adjustedProtocolConfig
+        -- The paths to the genesis files have to exist in the node configuration file,
+        -- but the files themselves don't have to exist. So if the user wants to pass


If the user wants to call cardanoTestnet and mix and match their genesis files then fine but this should not be possible at the executable level. If all the genesis files do not exist in the user provided configuration we fail hard. We do not provide them with our defaults. They only use our defaults if they provide no configuration.

but this should not be possible at the executable level.

Why not? I'm not sure there's much value in requiring all configuration files from the user.

Let's continue discussion here: #6137 (comment)

@Jimbo4350> design you prefer implemented here: #6148

Approve the one you want merged!

smelc · 2025-03-14T15:49:56Z

Closed in favor of #6148

smelc added the cardano-testnet label Feb 5, 2025

smelc changed the title ~~cardano-testnet: allow to take config files as input~~ cardano-testnet: allow to take node config and genesis files as input Feb 5, 2025

smelc linked an issue Feb 5, 2025 that may be closed by this pull request

[FEATURE] - cardano-testnet: Allow to pass custom node configuration file and custom genesis files #6069

Closed

17 tasks

Base automatically changed from smelc/cardano-testnet-support-custom-node-config-file to master February 5, 2025 16:14

smelc force-pushed the smelc/testnet-pass-config-files branch from 3894926 to 8e37423 Compare February 7, 2025 13:53

smelc marked this pull request as ready for review February 7, 2025 13:53

smelc requested a review from a team as a code owner February 7, 2025 13:53

Jimbo4350 requested changes Feb 7, 2025

View reviewed changes

cardano-testnet/src/Parsers/Run.hs Show resolved Hide resolved

smelc force-pushed the smelc/testnet-pass-config-files branch 2 times, most recently from 63c60a5 to 6ca79c5 Compare February 11, 2025 16:16

smelc added a commit that referenced this pull request Feb 12, 2025

cardano-testnet: remove check that is too conservative

d67ff61

The node itself, when it starts, doesn't require hashes of genesis files. So it seems overkill to require it to start a testnet. See also #6103 (comment)

smelc force-pushed the smelc/testnet-pass-config-files branch from 6ca79c5 to d67ff61 Compare February 12, 2025 10:38

smelc added a commit that referenced this pull request Feb 12, 2025

cardano-testnet: remove check that is too conservative

c78c34f

The node itself, when it starts, doesn't require hashes of genesis files. So it seems overkill to require it to start a testnet. See also #6103 (comment)

smelc force-pushed the smelc/testnet-pass-config-files branch from d67ff61 to c78c34f Compare February 12, 2025 10:39

smelc added a commit that referenced this pull request Feb 12, 2025

cardano-testnet: remove check that is too conservative

1f7c3d8

The node itself, when it starts, doesn't require hashes of genesis files. So it seems overkill to require it to start a testnet. See also #6103 (comment)

smelc force-pushed the smelc/testnet-pass-config-files branch from c78c34f to 1f7c3d8 Compare February 12, 2025 14:25

smelc added a commit that referenced this pull request Feb 12, 2025

cardano-testnet: remove check that is too conservative

6234a1a

The node itself, when it starts, doesn't require hashes of genesis files. So it seems overkill to require it to start a testnet. See also #6103 (comment)

smelc force-pushed the smelc/testnet-pass-config-files branch from 18c0d3e to 50849e1 Compare February 12, 2025 15:30

smelc added a commit that referenced this pull request Feb 12, 2025

cardano-testnet: remove check that is too conservative

38b11ab

The node itself, when it starts, doesn't require hashes of genesis files. So it seems overkill to require it to start a testnet. See also #6103 (comment)

smelc force-pushed the smelc/testnet-pass-config-files branch 3 times, most recently from 8117ffc to 3671f83 Compare February 18, 2025 12:14

smelc changed the title ~~cardano-testnet: allow to take node config and genesis files as input~~ cardano-testnet: allow to take node config file as input Feb 18, 2025

smelc added a commit that referenced this pull request Feb 18, 2025

cardano-testnet: remove possibility to programmatically pass a custom…

f95a273

… node configuration file (it was unused) This will be done differently in #6103

smelc mentioned this pull request Feb 18, 2025

cardano-testnet: remove possibility to programmatically pass a custom node configuration file (it was unused) #6118

Merged

3 tasks

smelc added a commit that referenced this pull request Feb 18, 2025

cardano-testnet: remove possibility to programmatically pass a custom…

0459379

… node configuration file (it was unused) This will be done differently in #6103

smelc force-pushed the smelc/testnet-pass-config-files branch from c25acc3 to 6dc7fb4 Compare February 18, 2025 14:52

Jimbo4350 reviewed Feb 18, 2025

View reviewed changes

smelc changed the title ~~cardano-testnet: allow to take node config file as input~~ cardano-testnet executable: take node config file as optional input Mar 11, 2025

smelc force-pushed the smelc/per-genesis-file-origin branch from ad8dbd6 to 70c0e00 Compare March 11, 2025 14:51

smelc force-pushed the smelc/testnet-pass-config-files branch from 1b6c518 to c4aba77 Compare March 11, 2025 15:04

smelc force-pushed the smelc/per-genesis-file-origin branch from 70c0e00 to 9cd0ae0 Compare March 11, 2025 15:07

smelc force-pushed the smelc/testnet-pass-config-files branch from c4aba77 to 4d424b6 Compare March 11, 2025 15:51

smelc force-pushed the smelc/per-genesis-file-origin branch from 9cd0ae0 to 72ea5dc Compare March 11, 2025 18:45

smelc force-pushed the smelc/testnet-pass-config-files branch from 4d424b6 to fa3a62e Compare March 12, 2025 13:37

Base automatically changed from smelc/per-genesis-file-origin to master March 12, 2025 16:15

smelc force-pushed the smelc/testnet-pass-config-files branch 8 times, most recently from 89d6e24 to d922fb3 Compare March 12, 2025 17:03

smelc marked this pull request as ready for review March 12, 2025 17:04

smelc force-pushed the smelc/testnet-pass-config-files branch from d922fb3 to 47f3da4 Compare March 13, 2025 13:32

smelc added 4 commits March 13, 2025 14:41

cardano-testnet: allow to take node configuration file as input, pres…

c0b1856

…erve existing behavior too

Adapt golden files

c8cdb96

Adapt cardano-testnet-test accordingly

6fd617b

Test cardano-testnet executable

2d20baf

smelc force-pushed the smelc/testnet-pass-config-files branch from 47f3da4 to 2d20baf Compare March 13, 2025 13:42

This was referenced Mar 13, 2025

[FEATURE] - cardano-testnet: Allow to pass custom node configuration file and custom genesis files #6069

Closed

Document the revamped upcoming cardano-testnet binary cardano-foundation/developer-portal#1426

Merged

Jimbo4350 requested changes Mar 14, 2025

View reviewed changes

Try call cardano-testnet directly

077159c

smelc mentioned this pull request Mar 14, 2025

cardano-testnet: allow to take node configuration file as input #6148

Merged

8 tasks

smelc closed this Mar 14, 2025

cardano-testnet executable: take node config file as optional input #6103

cardano-testnet executable: take node config file as optional input #6103

Uh oh!

Conversation

smelc commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How to trust this PR

Execute the new test

Execute the cardano-testnet binary directly

Follow-up issues to create

Checklist

Uh oh!

Jimbo4350 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

smelc commented Feb 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smelc commented Feb 18, 2025

Uh oh!

Jimbo4350 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jimbo4350 Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jimbo4350 Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smelc commented Mar 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

smelc commented Feb 5, 2025 •

edited

Loading

Execute the `cardano-testnet` binary directly

smelc commented Feb 12, 2025 •

edited

Loading

Jimbo4350 Feb 18, 2025 •

edited

Loading

Jimbo4350 Mar 14, 2025 •

edited

Loading