Skip to content

Generate versioned __DOT__ entries for aus/eus/extended-eus RPM repos#216

Open
rhartman93 wants to merge 1 commit into
mainfrom
generate-versioned-rpm-repo-entries
Open

Generate versioned __DOT__ entries for aus/eus/extended-eus RPM repos#216
rhartman93 wants to merge 1 commit into
mainfrom
generate-versioned-rpm-repo-entries

Conversation

@rhartman93

@rhartman93 rhartman93 commented May 5, 2026

Copy link
Copy Markdown
Contributor

Problem: The upstream repository-to-cpe.json inconsistently includes versioned repo entries (e.g. reponame__9_DOT_2). Add a jq pre-processing step that reliably generates these by extracting version numbers from repo_relative_urls containing /aus/, /eus/, or /extended-eus/ paths.

Modify hack/render-known-rpm-repositories.sh to add a jq pre-processing step between curl and yq:

 1. Keep all existing keys from the JSON (including any __DOT__ entries already present)
 2. Generate new DOT entries for non-DOT repos whose repo_relative_urls contain /aus/, /eus/, or /extended-eus/ paths with version numbers
 3. The generated key format: {reponame}__{major}_DOT_{minor} (e.g. rhel-9-for-x86_64-appstream-aus-rpms__9_DOT_2)
 4. Merge generated entries into the JSON data, then pass to the existing yq pipeline unchanged

 The jq filter:
 - Iterates entries, skips keys already containing __\d+_DOT_
 - For remaining entries, checks if any repo_relative_urls match /aus/|/eus/|/extended-eus/
 - Extracts major.minor version from matching URLs via regex capture
 - Creates new keys {original_key}__{major}_DOT_{minor} with empty value objects
 - Merges these into the original .data object (deduped by key)

 File to modify

 - hack/render-known-rpm-repositories.sh — insert jq step between curl and yq

 Verification

 - Run the updated script against repos.json (as a local file instead of curl) and compare output to current data/known_rpm_repositories.yml
 - Confirm all existing DOT entries are preserved
 - Confirm new DOT entries appear for repos like rhel-9-for-x86_64-appstream-aus-rpms (should get __9_DOT_2, __9_DOT_4, __9_DOT_6)
 - Confirm non-aus/eus repos are unaffected

Generated with Claude Code (Opus 4.6)

The upstream repository-to-cpe.json inconsistently includes versioned
repo entries (e.g. reponame__9_DOT_2). Add a jq pre-processing step
that reliably generates these by extracting version numbers from
repo_relative_urls containing /aus/, /eus/, or /extended-eus/ paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rhartman93 rhartman93 requested a review from ralphbean as a code owner May 5, 2026 13:58
select(.value.repo_relative_urls | any(test("/aus/|/eus/|/extended-eus/"))) |
.key as $key |
(.value.repo_relative_urls[] |
select(test("/aus/|/eus/|/extended-eus/")) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also have e4s?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm likely going to drop this PR. It was discovered that the removal of these repos was an error on the SDEngine side. There's no agreed upon policy around when data should be removed at all if at all. So the data source has been restored and prodsec is discussing that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants