Allow platform setting to have prefix/suffix around subshell

[MetRonnie]

This so that we can do e.g.

platform = $(rose config platform hpc_live)-bg

Currently we have to resort to

platform = $(echo "$(rose config platform hpc_live)-bg")

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• No dependency changes
• Tests are included
• Changelog entry included if this is a change that can affect users
• No docs PR applicable
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[oliver-sanders]

The change makes sense, however, platform subshell is really Cylc 7 hangover, [platforms] were intended to mitigate the need for this sort of thing.

It looks like this approach is using Rose to configure Cylc with the Rose config edited when hpc_live updates?

Could this be done the other way around, editing the Cylc config instead?

# global.cylc

[platform groups]
  [[hpc-live]]
    platforms = aaa
  [[hpc-live-bg]]
    platforms = aaa-bg

Or even pull in the value from an external source of truth:

#!Jinja2

{% import "os" as os %}

[platform groups]
  [[hpc-live]]
    platforms = {{ os.fdopen("hpc_live_file").read() }}
  [[hpc-live-bg]]
    platforms = {{ os.fdopen("hpc_live_file").read() }}-bg

[MetRonnie]

It looks like this approach is using Rose to configure Cylc with the Rose config edited when hpc_live updates?

Only using Rose for the simple parsing of the config file (~/.metomi/rose.conf) to pick out the hpc_live setting. Could use something like jq with a JSON config file, but jq is not generally installed. The config file itself has a few other settings used for switching HPCs, e.g.

[platform]
hpc_live = zone1
hpc_mirror = zone2
hpc_switch = zone3

Could this be done the other way around, editing the Cylc config instead?

This would not pick up changes to hpc_live without a reload, which would rather be avoided for an operational suite of dozens of workflows.

Brainstorming here, would it be worth implementing dynamically updatable platforms in some other way, as an 8.6.0 feature?

[MetRonnie]

Tentatively putting this against 8.5.x as it's equally a fix as it is a feature?

[oliver-sanders]

Brainstorming here, would it be worth implementing dynamically updatable platforms in some other way, as an 8.6.0 feature?

I think we need a better understanding of the use case and a quick think.

I suspect that the only reason we are defining the platform at all is to ensure that it is defined when the workflow is started up. Beyond this broadcasts take over? If so, then this doesn't need to be dynamic at all.

This would not pick up changes to hpc_live without a reload, which would rather be avoided for an operational suite of dozens of workflows.

Yes, we would need to run a command to do this:

$ cylc reload -g '*'

I thought we're using broadcasts for this anyway?

Lets discuss when we've gathered a bit more insight.

[oliver-sanders]

LGTM.

Maybe consider adding an example that uses a prefix/suffix into the config docs.

There is one, very minor flaw in that the regex will match something like $(foo)-$(bar) and attempt to run it leaving a cryptic error. But doesn't really matter.

[oliver-sanders]

[minor] can prevent this first group from being captured using ?::

Suggested change

	PLATFORM_REC_COMMAND = re.compile(r'(\$\()\s*(.*)\s*(\))')
	PLATFORM_REC_COMMAND = re.compile(r'(?:\$\()\s*(.*)\s*(\))')

[wxtim]

Seems fine. Will approve once I've seen matching Docs PR.

[MetRonnie]

Added 1 line of documentation in this PR

[wxtim]

Feel free to merge after tests pass