Static validation of config files ignores non-default workflow(s)

[nick-ohhh]

Meta:

CircleCI CLI Version:

0.1.30888+v0.1.30888 (from nix) Linux and 0.1.30995 (homebrew) Mac

Operating System:

macOS and Linux

---

Current behavior:

version: 2.1

parameters:
  run-job:
    type: enum
    enum:
      [
        test,
        test2,
      ]
    default: test
jobs:
  a:
    docker:
      - image: cimg/base:2023.03
    steps:
      - run: echo "this is job A"

workflows:
  j:
    when:
      equal: [ test2, << pipeline.parameters.run-job >> ]
    jobs:
      - a:
          requires:
            - does_not_exist

circleci config validate says the config.yml is valid, and it's not.

The problem gets detected only when one tries to actually run the test2 workflow (which, can happen periodically in the background by scheduled triggers). Changing the default for run-job to be test2 and rerunning circleci config validate .circleci/test.yml correctly identifies the problem and fails with - Job 'a' requires 'does_not_exist', which is the name of 0 other jobs in workflow 'j'.

If one has many "non default" workflow parameters defined, they break with such missing dependencies and the periodically triggered workflows fail, often going unnoticed until someone happens to look at CCI UI and see that it failed there. Failure notifications, e.g. on Slack, also do not get sent because the pipeline fails to even start, so there are cases of such issues going unnoticed for days, with multiple tests just not running without realizing it.

Expected behavior:

circleci config validate correctly marks a config.yml file as invalid, if the default parameter differs from an from an expected workflow parameter that would otherwise fail.

When did this begin / Was this previously working?:

Unknown if this was previously working.