Skip to content

reload fail on xtrigger removal causing depleted task pool #6815

@dwsutherland

Description

@dwsutherland

Description

On removal of xtrigger and orphaned (active/incomplete) task, reload fails resulting in a partially populated pool:

2025-06-25T03:52:13Z INFO - Reloading task definitions.
2025-06-25T03:52:13Z WARNING - [20250625T0000Z/bar/01:failed] will not spawn children - task definition removed
2025-06-25T03:52:13Z INFO - [20250626T0000Z/foo:waiting(queued)] reloaded task definition
2025-06-25T03:52:13Z ERROR - Traceback (most recent call last):
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/scheduler_cli.py", line 696, in cylc_play
        asyncio.get_running_loop()
    RuntimeError: no running event loop
    During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/scheduler.py", line 1026, in process_command_queue
        n_warnings = await cmd.__anext__()
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/commands.py", line 420, in reload_workflow
        schd.pool.reload(config)
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/task_pool.py", line 1050, in reload
        self._reload_taskdefs()
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/task_pool.py", line 1142, in _reload_taskdefs
        self.create_data_store_elements(itask)
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/task_pool.py", line 252, in create_data_store_elements
        self.data_store_mgr.increment_graph_window(
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/data_store_mgr.py", line 811, in increment_graph_window
        self.generate_ghost_task(
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/data_store_mgr.py", line 1297, in generate_ghost_task
        self._process_internal_task_proxy(itask, tproxy)
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/data_store_mgr.py", line 1550, in _process_internal_task_proxy
        sig = self.schd.xtrigger_mgr.get_xtrig_ctx(
      File "/home/sutherlander/cylc/cylc-flow/cylc/flow/xtrigger_mgr.py", line 626, in get_xtrig_ctx
        ctx = deepcopy(self.xtriggers.functx_map[label])
    KeyError: 'wall_clock'
2025-06-25T03:52:13Z ERROR - Command "reload_workflow" failed. ID=41329dc8-0c35-49d3-8125-feb63f7ec2dc
    'wall_clock'

Reproducible Example

Version: 8.4.4.dev

Run the following workflow:
$ cylc vip <workflow>

[scheduling]
    initial cycle point = 20250625T00
    [[graph]]
        PT12H = """
foo[-PT12H] => foo => bar
@wall_clock => bar
"""
[runtime]
    [[foo]]
        script = sleep 5
    [[bar]]
        script = false

While a bar is failed, remove bar from the definition:

[scheduling]
    initial cycle point = 20250625T00
    [[graph]]
        PT12H = """
foo[-PT12H] => foo
"""
[runtime]
    [[foo]]
        script = sleep 5

reinstall/reload the workflow:
$ cylc vr <workflow>/run1

Expected Behaviour

Workflow reloaded successfully

Metadata

Metadata

Assignees

Labels

bugSomething is wrong :(

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions