Skip to content
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/requirements-sync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Sync Requirements to GitHub Issues

on:
push:
branches: [main, public-main]
paths:
- 'requirements/requirements.json'
workflow_dispatch:

permissions:
contents: read
issues: write
Comment on lines +10 to +12
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Missing projects: write permission for GitHub Projects integration.

The sync-issues.sh script uses gh project commands (item-add, field-create, item-edit) which require the projects: write permission. Without it, project operations will silently fail when falling back to GITHUB_TOKEN.

Either add the permission or document that PROJECT_TOKEN secret must be configured with appropriate scopes:

🔧 Option 1: Add permission (if using fine-grained PAT or GitHub App)
 permissions:
   contents: read
   issues: write
+  projects: write
📝 Option 2: Document PROJECT_TOKEN requirement

Add a comment in the workflow or README explaining that PROJECT_TOKEN secret must be a PAT with repo and project scopes for full functionality, otherwise project sync will be skipped with warnings.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
permissions:
contents: read
issues: write
permissions:
contents: read
issues: write
projects: write
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/requirements-sync.yml around lines 10 - 12, The workflow
is missing the projects: write permission required by the gh project commands
used in sync-issues.sh (item-add, field-create, item-edit); update the
permissions block in the workflow to include "projects: write" so GITHUB_TOKEN
can perform project operations, and/or add a brief comment in the
workflow/README clarifying that if using PROJECT_TOKEN it must be a PAT with
repo and project scopes as a fallback for full functionality.


concurrency:
group: requirements-sync
cancel-in-progress: true

jobs:
sync:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Sync issues and project
env:
GH_TOKEN: ${{ secrets.PROJECT_TOKEN || secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: ./requirements/sync-issues.sh
13 changes: 12 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ WORKER_SCRIPT := scripts/worker.sh
add-worker-nodes worker-status approve-worker-csrs \
deploy-csr-approver delete-csr-approver deploy-dpucluster-csr-approver delete-dpucluster-csr-approver \
delete-dpf-hcp-provisioner-operator \
verify-deployment verify-workers verify-dpu-nodes verify-dpudeployment \
verify-deployment verify-workers verify-dpu-nodes verify-dpudeployment verify-requirements \
run-traffic-flow-tests tft-setup tft-cleanup tft-show-config tft-results aicli-list \
validate-env-files generate-env

Expand Down Expand Up @@ -272,6 +272,12 @@ verify-dpu-nodes:
verify-dpudeployment:
@$(VERIFY_SCRIPT) verify-dpudeployment

verify-requirements:
@echo "================================================================================"
@echo "Running Requirements Verification..."
@echo "================================================================================"
@./requirements/verify-requirements.sh $(VERIFY_REQ_ARGS)

validate-env-files:
@$(ENV_SCRIPT) validate-env-files

Expand Down Expand Up @@ -338,6 +344,11 @@ help:
@echo " verify-workers - Wait for worker nodes to be Ready in host cluster"
@echo " verify-dpu-nodes - Wait for DPU nodes to be Ready in DPUCluster"
@echo " verify-dpudeployment - Wait for DPUDeployment to be Ready"
@echo " verify-requirements - Run requirement tests and produce a report"
@echo " Use VERIFY_REQ_ARGS to pass options, e.g.:"
@echo " make verify-requirements VERIFY_REQ_ARGS='REQ-001 REQ-049'"
@echo " make verify-requirements VERIFY_REQ_ARGS='--output report.md'"
@echo " make verify-requirements VERIFY_REQ_ARGS='--json'"
@echo ""
@echo "Traffic Flow Tests:"
@echo " run-traffic-flow-tests - Run kubernetes-traffic-flow-tests for network validation"
Expand Down
72 changes: 72 additions & 0 deletions requirements/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Requirements Tracking

This directory contains tooling to track project requirements as GitHub Issues, managed via a GitHub Project board.

## How It Works

1. **`requirements.json`** is the source of truth. Each requirement declares its description, test details, labels, and (once synced) the corresponding GitHub issue number.
2. **`sync-issues.sh`** reads the JSON and creates or updates GitHub Issues and a GitHub Project.
3. **`update-results.sh`** takes `go test -json` output and posts pass/fail results as comments on the corresponding issues.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Documentation doesn't match script behavior.

Line 9 states update-results.sh "takes go test -json output," but the script actually runs test_implementation commands directly from requirements.json. It does not consume piped input.

📝 Suggested fix
-3. **`update-results.sh`** takes `go test -json` output and posts pass/fail results as comments on the corresponding issues.
+3. **`update-results.sh`** runs each requirement's `test_implementation` command and posts pass/fail results as comments on the corresponding issues.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
3. **`update-results.sh`** takes `go test -json` output and posts pass/fail results as comments on the corresponding issues.
3. **`update-results.sh`** runs each requirement's `test_implementation` command and posts pass/fail results as comments on the corresponding issues.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@requirements/README.md` at line 9, The README line about update-results.sh is
incorrect: the script does not consume piped go test -json input but instead
executes the test_implementation commands defined in requirements.json and posts
pass/fail comments; update the sentence to state that update-results.sh reads
requirements.json, runs each test_implementation command directly, captures
their exit/results, and posts comments (remove the claim about taking go test
-json from stdin).

4. **GitHub Actions** automate both steps — syncing on push and posting test results on a schedule.

## JSON Schema

Each entry in `requirements.json`:

| Field | Type | Description |
|--------------------|------------|--------------------------------------------------------------|
| `id` | string | Stable identifier, e.g. `REQ-001`. Never changes. |
| `requirement` | string | Brief description of the requirement. |
| `test_description` | string | What the test verifies / acceptance criteria. |
| `test_implementation` | string | Go test reference: `package/path::TestFunctionName`. |
| `labels` | string[] | GitHub labels to apply (created automatically if missing). |
| `issue_number` | int\|null | Populated by `sync-issues.sh`. Do not edit manually. |
Comment on lines +21 to +23
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Schema documentation references a field the scripts don't use.

The issue_number field is documented as "Populated by sync-issues.sh", but the scripts use REQ-XXX labels to find issues, not stored issue numbers. This field appears to be vestigial or planned for future use.

📝 Suggested fix

Either remove the field from the documentation:

 | `test_implementation` | string  | Go test reference: `package/path::TestFunctionName`.         |
 | `labels`           | string[]   | GitHub labels to apply (created automatically if missing).   |
-| `issue_number`     | int\|null  | Populated by `sync-issues.sh`. Do not edit manually.        |

Or clarify it's optional/unused:

-| `issue_number`     | int\|null  | Populated by `sync-issues.sh`. Do not edit manually.        |
+| `issue_number`     | int\|null  | (Optional) Stored issue number. Currently unused; scripts find issues by REQ-XXX label. |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@requirements/README.md` around lines 21 - 23, Update the schema docs to stop
claiming `issue_number` is populated by `sync-issues.sh`: either remove the
`issue_number` field entirely from the table, or mark it clearly as
optional/unused and note that `sync-issues.sh` identifies issues via REQ-XXX
labels (see `labels`) rather than storing numbers; ensure the table row
referencing `issue_number` and the sentence about `sync-issues.sh` are edited
accordingly so docs match actual script behavior.


## Local Usage

### Prerequisites

- [`gh`](https://cli.github.com/) CLI installed and authenticated (`gh auth login`)
- [`jq`](https://jqlang.github.io/jq/) installed
- `GITHUB_REPOSITORY` env var set (or let the script auto-detect from `gh`)

### Sync requirements to issues

```bash
# Dry run (no changes made)
./requirements/sync-issues.sh --dry-run

# Create/update issues and project
./requirements/sync-issues.sh
```

After creating new issues, the script updates `requirements.json` with issue numbers. Commit this change.

### Post test results

```bash
# Run tests and pipe to the results script
go test ./... -json 2>&1 | ./requirements/update-results.sh
```
Comment on lines +47 to +50
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Example usage is incorrect.

The example shows piping go test output to update-results.sh, but the script doesn't read from stdin—it executes test_implementation commands directly.

📝 Suggested fix
 ### Post test results

 ```bash
-# Run tests and pipe to the results script
-go test ./... -json 2>&1 | ./requirements/update-results.sh
+# Run requirement tests and post results to issues
+./requirements/update-results.sh
+
+# Or test specific requirements
+./requirements/update-results.sh REQ-001 REQ-002
+
+# Dry run (no GitHub updates)
+./requirements/update-results.sh --dry-run
</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against the current code and only fix it if needed.

In @requirements/README.md around lines 47 - 50, Replace the incorrect example
that pipes go test into the script with usage that invokes the
update-results.sh script directly; update the README example to show calling
./requirements/update-results.sh to run requirement tests and post results,
demonstrate calling it with specific requirement IDs (e.g., REQ-001 REQ-002),
and show the --dry-run option; ensure the reference to the script name
update-results.sh is used so readers run the script itself rather than piping go
test output into it.


</details>

<!-- fingerprinting:phantom:medusa:ocelot -->

<!-- This is an auto-generated comment by CodeRabbit -->


### Adding a new requirement

1. Add an entry to `requirements.json` with a new `id` and `"issue_number": null`.
2. Run `sync-issues.sh` (or push to main — the GitHub Action will handle it).
3. The script creates the issue and writes the number back to the JSON.

Comment on lines +52 to +57
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Adding a new requirement section is inconsistent with actual workflow.

The documentation says sync-issues.sh "writes the number back to the JSON," but the script uses labels to find issues and doesn't modify requirements.json.

📝 Suggested fix
 ### Adding a new requirement

-1. Add an entry to `requirements.json` with a new `id` and `"issue_number": null`.
+1. Add an entry to `requirements.json` with a new `id`, requirement text, labels, and optionally a `test_implementation`.
 2. Run `sync-issues.sh` (or push to main — the GitHub Action will handle it).
-3. The script creates the issue and writes the number back to the JSON.
+3. The script creates the issue with a `REQ-XXX` label matching the `id`. No JSON modifications are needed.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@requirements/README.md` around lines 52 - 57, The README's "Adding a new
requirement" section is incorrect: sync-issues.sh does not write the issue
number back to requirements.json; instead it uses labels to locate issues.
Update the text in requirements/README.md to reflect the actual workflow:
instruct contributors to add a requirement with "issue_number": null in
requirements.json, explain that sync-issues.sh (or the GitHub Action) will
create or find issues by label but will not modify requirements.json, and add a
note that maintainers must manually update the "issue_number" in
requirements.json (or run a dedicated script) if automatic writing is needed;
reference requirements.json and sync-issues.sh by name so reviewers can locate
the relevant files.

## GitHub Actions

### `requirements-sync.yml`

Runs automatically when `requirements/requirements.json` is pushed to `main`. Creates/updates issues, then commits the updated JSON back with `[skip ci]` to avoid loops.

### `requirements-results.yml`

Runs on weekdays at 06:00 UTC (and via manual dispatch). Runs `go test`, posts results to the matching issues as comments.

### Secrets

| Secret | Required | Purpose |
|-----------------|----------|------------------------------------------------------------|
| `PROJECT_TOKEN` | Optional | PAT with `project` scope for GitHub Projects v2 operations. Falls back to `GITHUB_TOKEN` (which works for issues but not projects). |
Loading