Skip to content

Auto-detect required builds from YAML net values in generate_workflows#4629

Draft
Copilot wants to merge 4 commits intodevelopfrom
copilot/fix-workflow-execs-building
Draft

Auto-detect required builds from YAML net values in generate_workflows#4629
Copilot wants to merge 4 commits intodevelopfrom
copilot/fix-workflow-execs-building

Conversation

Copy link
Contributor

Copilot AI commented Mar 6, 2026

Description

When using ./generate_workflows.sh -b -y "YAML_FILE", only the default builds (UFS, UPP, UFS_Utils, GFS-utils) were triggered. System-specific builds (GSI, GDASApp, etc.) were only added to the build when using -G/-E/-S/-C flags, leaving users with incomplete builds when specifying YAMLs directly via -y.

Changes

  • Auto-detect build flags from YAML net values: After the -G/-E/-S/-C handling blocks, if none of those flags were specified, each YAML in the list is scanned for its net value and appropriate build flags are added automatically:
    • net: gfsgfs gsi gdas (equivalent to -G)
    • net: gefsgefs (equivalent to -E)
    • net: sfssfs (equivalent to -S)
    • net: gcafsgcafs gdas (equivalent to -C)
  • Deduplication: Build flags are only added if not already present, so mixed YAML lists (e.g., multiple GFS YAMLs) don't produce duplicate flags.
  • Updated -b/-B help text to document the automatic system build selection behavior.
# Before: only default builds triggered (UFS, UPP, etc.)
./generate_workflows.sh -b -y "C96_atm3DVar_extended"

# After: GSI and GDASApp builds are also triggered automatically
./generate_workflows.sh -b -y "C96_atm3DVar_extended"
# equivalent to: ./generate_workflows.sh -b -G -y "C96_atm3DVar_extended"

Resolves #2837

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? NO
    • GFS
    • GEFS
    • SFS
    • GCAFS
  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

Logic verified by testing grep-based net: detection against existing YAML files in dev/ci/cases/pr/ for all four system types (gfs, gefs, sfs, gcafs) and mixed YAML lists.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary
Original prompt

This section details on the original issue you should resolve

<issue_title>Not all required execs are built when using generate_workflows and a single yaml</issue_title>
<issue_description>### What is wrong?

I recently ran into an issue where when I was using the ./generate_workflows -b -y "YAML_FILE" that only the forecast execs were built and not GSI/GDAS.

@ChristopherHill-NOAA also just ran into this issue running:

./generate_workflows.sh -b -y "C96_atm3DVar_extended"

What should have happened?

all required builds should be triggered even if running a single -y "YAML" option when using generate_workflows (or appropriate caveats should be added to -h option).

What machines are impacted?

All or N/A

What global-workflow hash are you using?

develop and dev/gfsv17

Steps to reproduce

./generate_workflows.sh -b -y "C96_atm3DVar_extended"

With appropriate other options.

Additional information

No response

Do you have a proposed solution?

No response</issue_description>

<agent_instructions>Modify the build flag feature of dev/workflow/generate_workflows.sh to check the net in the included YAMLs and select the appropriate builds. If gfs, use the builds for the -G flag. If gefs, use the builds for the -E flag. sfs maps to -S, and gcafs to -C.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@DavidHuber-NOAA This is a feature I've considered making, but it's not intended to be supported at this time. I'm going to rewrite this as a feature and hand it to copilot. @JessicaMeixner-NOAA Not an urgent issue by any means.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: DavidHuber-NOAA <69919478+DavidHuber-NOAA@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix issue with missing execs in workflow generation Auto-detect required builds from YAML net values in generate_workflows Mar 6, 2026
DavidHuber-NOAA and others added 2 commits March 20, 2026 14:55
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Not all required execs are built when using generate_workflows and a single yaml Wrong version number for prepobs in operational workflow

2 participants