add a psql subcommand to cncluster script for convenient DB inspection #2672

mblaze-da · 2025-10-13T14:07:10Z

[static]

I've tested this by running the new subcommand against different databases in the cluster on a scratchnet.

sequencer-0-pg

scratchnetc % cncluster psql sv-1 global-domain-0-sequencer                                                          
INFO: Retrieving DB connection info from pod description...
INFO: Retrieving DB password from secret [sequencer-0-pg-secrets]...
INFO: Deploying debug pod on version 0.4.19
pod/splice-debug created
INFO: Waiting for debug pod to become ready
pod/splice-debug condition met
INFO: Opening terminal on debug pod
psql (16.10 (Ubuntu 16.10-0ubuntu0.24.04.1), server 14.19 (Debian 14.19-1.pgdg13+1))
Type "help" for help.

cantonnet=#

The optional search path parameter can be specified and it is set as shown in the following example:

sv-app postgres

scratchnetc % cncluster psql sv-1 sv-app test
INFO: Retrieving DB connection info from pod description...
INFO: Retrieving DB password from secret [postgres-secrets]...
INFO: Deploying debug pod on version 0.4.19
pod/splice-debug created
INFO: Waiting for debug pod to become ready
pod/splice-debug condition met
INFO: Opening terminal on debug pod
psql (16.10 (Ubuntu 16.10-0ubuntu0.24.04.1), server 14.19 (Debian 14.19-1.pgdg13+1))
Type "help" for help.

cantonnet=# show search_path;
 search_path 
-------------
 test
(1 row)

[static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

julientinguely-da

Really nice, thanks! Looks good overall, just a few minor comments/suggestions. Gonna try it out now :)

julientinguely-da · 2025-10-13T14:29:55Z

build-tools/cncluster

    effect: "NoSchedule"
 EOF

+  # shellcheck disable=SC2086


let's try to remove the shellcheck disable=SC2086 by adding the parentheses (like "$SHELL_COMMAND")

Then it would not work. We need this to expand.

julientinguely-da · 2025-10-13T14:32:25Z

build-tools/cncluster

    _update_cluster_config "$deployment_config" "operatorDeployment"
 }

+subcommand_whitelist[psql]=""


nit: could we add the ### on top to match with the other methods

julientinguely-da · 2025-10-13T14:36:05Z

build-tools/cncluster

+  local PASSWORD_REF
+  PASSWORD_REF=$(
+    kubectl get deployment \
+      --namespace sv-1 \


I think you forgot to parametrize the namespace here

julientinguely-da · 2025-10-13T14:38:46Z

build-tools/cncluster

+  _info "Retrieving DB connection info from pod description..."
+  local DB_INIT_COMMAND
+  DB_INIT_COMMAND=$(
+    kubectl describe pod \


suggestion: couldn't we use kubectl get pod instead of describe here, would seem less brittle?

I don't think it would make it much more reliable. We'd still need to sed the psql command out of a larger script in the "command" field. We'd just exchange a grep -i psql for a more complicated JSON path expression.

julientinguely-da · 2025-10-13T14:47:17Z

Also you can add fixes https://github.com/hyperledger-labs/splice/issues/2217 on top of the PR description such that it links the PR to the issue and automatically close it when merged.

[static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

build-tools/cncluster

martinflorian-da · 2025-10-13T16:28:41Z

build-tools/cncluster

+        shift 2
+        ;;
+      -d=*|--dbname=*)
+        local DB_NAME="${1#*=}"


OK so at least on the deployment I tested on (sv-app) this is wrong: we don't want the default db (cantonnet), but the one that the init container is creating sv_sv_1:

until errmsg=$(psql -h 10.60.65.197 -p 5432 --username=cnadmin --dbname=cantonnet -c 'create database sv_sv_1' 2\u003e\u00261); do\n if [[ $errmsg == *\"already exists\"* ]]; then\n echo \"Database sv_sv_1 already exists. Done.\"\n break\n fi\n\n echo \"trying to create postgres database sv_sv_1, last error: $errmsg\";\n sleep 2;\ndone\n"

(sorry for the horrible paste, but I hope the point comes across; create database sv_sv_1 is the interesting one)

martinflorian-da · 2025-10-13T16:32:23Z

build-tools/cncluster

+  subcmd_debug_shell " \
+    /bin/env
+      PGPASSWORD=$DB_PASSWORD \
+      ${SEARCH_PATH:+PGOPTIONS=--search_path=$SEARCH_PATH} \


I wonder if there is a nice way to infer the search path...

I often just guess/know it (is it perhaps always the same as database name for our deployments?).

But maybe there is some nice postgres way to just add all existing schemas to the search path or something...

Well, I tried to set the search path to include all schemas but I couldn't get it to work. Listing all schemas is simple:

PGPASSWORD=$DB_PASSWORD psql \ --host=$DB_HOST \ --port=$DB_PORT \ --username=$DB_USERNAME \ --dbname=$DB_NAME \ --tuples-only \ --pset format=unaligned \ --command "SELECT string_agg(schema_name, ',') FROM information_schema.schemata"

as well as setting the search path:

PGOPTIONS=--search_path=$(...)

What I couldn't figure out was how to delay the evaluation of the schema listing till pod runtime while at the same time keeping the session interactive. I could get either one or the other to work. Not both at the same time. I gave up and implemented a simple mapping that uses DB_NAME with exceptions for participant, sequencer and mediator, as we've discussed on Slack. TBH I don't want to invest much more time into improving this as I don't think it is that important.

Perhaps this could be achieved by modifying the pod configuration but I did not want to duplicate any code from the debug_shell subcommand.

I tested it with participant and sv-app and it seems to be working.

Sounds like a perfectly reasonable decision to me!

[static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

isegall-da · 2025-10-15T10:11:26Z

Should we add some commentary and examples in some README?

…sed on application name [static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

mblaze-da · 2025-10-16T00:38:52Z

@isegall-da there is a section about connecting to PG DBs with the help of cncluster in the internal repo that I could update. But this would be a separate PR, right?

martinflorian-da · 2025-10-16T08:34:28Z

But this would be a separate PR, right?

@mblaze-da Yep. Agree that it would be nice to do this as a follow-up.

martinflorian-da

I love it and will use it today; thanks a lot!

martinflorian-da · 2025-10-16T08:37:48Z

(tested the use-cases that I need, they work! :) )

add a psql subcommand to cncluster script for convenient DB inspection

031e074

[static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

mblaze-da requested review from julientinguely-da and martinflorian-da October 13, 2025 14:07

mblaze-da linked an issue Oct 13, 2025 that may be closed by this pull request

Add a cncluster psql #2217

Closed

julientinguely-da approved these changes Oct 13, 2025

View reviewed changes

fix a hardcoded namespace in cncluster psql

b3f8ac9

[static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

martinflorian-da reviewed Oct 13, 2025

View reviewed changes

build-tools/cncluster Outdated Show resolved Hide resolved

martinflorian-da reviewed Oct 13, 2025

View reviewed changes

correct the DB name resolution in the cncluster psql command

7a1651b

[static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

infer the schema name for search_path in cncluster psql subcommand ba…

be7010b

…sed on application name [static] Signed-off-by: Mateusz Błażejewski <mateusz.blazejewski@digitalasset.com>

martinflorian-da approved these changes Oct 16, 2025

View reviewed changes

mblaze-da merged commit 71d7adb into main Oct 16, 2025
40 checks passed

mblaze-da deleted the mblaze-da/cncluster-psql/2217 branch October 16, 2025 08:42

martinflorian-da mentioned this pull request Oct 17, 2025

Merge main into canton-3.4 #2746

Merged

add a psql subcommand to cncluster script for convenient DB inspection #2672

add a psql subcommand to cncluster script for convenient DB inspection #2672

Uh oh!

Conversation

mblaze-da commented Oct 13, 2025

Uh oh!

julientinguely-da left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

julientinguely-da commented Oct 13, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mblaze-da Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isegall-da commented Oct 15, 2025

Uh oh!

mblaze-da commented Oct 16, 2025

Uh oh!

martinflorian-da commented Oct 16, 2025

Uh oh!

martinflorian-da left a comment

Choose a reason for hiding this comment

Uh oh!

martinflorian-da commented Oct 16, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mblaze-da Oct 16, 2025 •

edited

Loading