Skip to content

test: just debugging the tests#3634

Open
jrichter1 wants to merge 2 commits into
redhat-developer:mainfrom
jrichter1:run-the-stupid-tests
Open

test: just debugging the tests#3634
jrichter1 wants to merge 2 commits into
redhat-developer:mainfrom
jrichter1:run-the-stupid-tests

Conversation

@jrichter1

Copy link
Copy Markdown
Member

Hey, I just made a Pull Request!

do not merge, obviously

✔️ Checklist

  • A changeset describing the change and affected packages. (more info)
  • Added or Updated documentation
  • Tests for new functionality and regression tests for bug fixes
  • Screenshots attached (for UI changes)

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 30, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 8:11 AM UTC · Completed 8:20 AM UTC
Commit: f2a4ef9 · View workflow run →

@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.03%. Comparing base (f2a4ef9) to head (74f8ba8).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3634   +/-   ##
=======================================
  Coverage   54.03%   54.03%           
=======================================
  Files        2325     2325           
  Lines       89195    89195           
  Branches    24936    24924   -12     
=======================================
  Hits        48193    48193           
- Misses      40707    40708    +1     
+ Partials      295      294    -1     
Flag Coverage Δ *Carryforward flag
adoption-insights 83.70% <ø> (ø)
ai-integrations 67.95% <ø> (ø) Carriedforward from 7c9a102
app-defaults 69.79% <ø> (ø) Carriedforward from 7c9a102
augment 46.39% <ø> (ø) Carriedforward from 7c9a102
boost 74.35% <ø> (ø) Carriedforward from 7c9a102
bulk-import 72.46% <ø> (ø) Carriedforward from 7c9a102
cost-management 14.10% <ø> (ø) Carriedforward from 7c9a102
dcm 61.81% <ø> (ø) Carriedforward from 7c9a102
extensions 61.53% <ø> (ø)
global-floating-action-button 71.18% <ø> (ø) Carriedforward from 7c9a102
global-header 59.71% <ø> (ø)
homepage 49.84% <ø> (ø)
install-dynamic-plugins 56.77% <ø> (ø) Carriedforward from 7c9a102
konflux 91.49% <ø> (ø) Carriedforward from 7c9a102
lightspeed 68.50% <ø> (ø) Carriedforward from 7c9a102
mcp-integrations 85.46% <ø> (ø) Carriedforward from 7c9a102
orchestrator 37.11% <ø> (ø) Carriedforward from 7c9a102
quickstart 65.63% <ø> (ø) Carriedforward from 7c9a102
sandbox 79.56% <ø> (ø) Carriedforward from 7c9a102
scorecard 82.67% <ø> (ø)
theme 61.26% <ø> (ø) Carriedforward from 7c9a102
translations 7.25% <ø> (ø) Carriedforward from 7c9a102
x2a 78.68% <ø> (ø) Carriedforward from 7c9a102

*This pull request uses carry forward flags. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2a4ef9...74f8ba8. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review

Findings

Medium

  • [missing-authorization] — No linked issue authorizes this PR. The title and body indicate an ad-hoc debugging session with no traceable work item. Changes span five workspaces plus a dependency bump, none of which trace to any approved task.
    Remediation: Either link an authorizing issue or close this PR and open a properly scoped one once the debugging is complete.

  • [debugging artifacts] workspaces/adoption-insights/README.md — A stray 1 character is appended to README files across five workspaces (adoption-insights, extensions, global-header, homepage, scorecard). These are debugging artifacts with no functional or documentary purpose that would corrupt documentation and pollute git history if merged.
    Remediation: Remove all debugging artifacts (the appended 1) from all five README.md files before any merge.

  • [scope-creep] workspaces/global-header/package.json:54 — The PR bumps @playwright/test from 1.60.0 to 1.61.1 in the global-header workspace. This dependency upgrade is unrelated to "debugging tests" and is bundled alongside throwaway README mutations without any authorization or changelog justification.
    Remediation: Split the Playwright version bump into its own PR with a linked issue or renovate/dependabot ticket.

  • [version inconsistency] workspaces/global-header/package.json:54 — The root workspace package.json bumps @playwright/test to 1.61.1, but packages/app/package.json and packages/app-legacy/package.json within the same workspace still pin @playwright/test at 1.60.0. This version mismatch can lead to multiple Playwright versions being resolved.
    Remediation: Update @playwright/test to 1.61.1 in both sub-package package.json files and regenerate the yarn.lock.

  • [misleading-label] — The PR is not marked as a draft (draft: false) despite the body explicitly stating "do not merge, obviously." A non-draft PR signals to CI, reviewers, and merge queues that it is a legitimate candidate for review and merge.
    Remediation: Convert the PR to draft status to prevent accidental merge.


Labels: PR is explicitly not ready to merge; adding do-not-merge/work-in-progress to match stated intent.

Previous run

Review

Findings

Critical

  • [unauthorized change] — This PR has no linked issue and the author's own title ("test: just debugging the tests") and body ("do not merge, obviously") explicitly indicate this is a temporary debugging experiment not intended for merge. Despite this, the PR carries the ready-for-merge label while draft is false, creating a dangerous contradiction: the author says do not merge, but the label signals readiness. There is no authorization (issue, RFC, or ADR) for any of the changes in this PR.
    Remediation: Remove the ready-for-merge label immediately. Do not merge this PR. If the e2e test helper (waitForComponent) is genuinely needed, it should be submitted in a separate PR linked to an issue that describes the test reliability problem it solves, without the junk README changes.

Medium

  • [scope pollution] workspaces/adoption-insights/README.md — Five README files across five unrelated workspaces (adoption-insights, extensions, global-header, homepage, scorecard) have had the character 1 appended as junk content. These are meaningless changes that appear to be debug artifacts used to trigger CI pipelines across multiple workspace labels. They pollute the git history of five independent workspaces with no functional or documentation value.
    Remediation: Remove all five junk README modifications from this PR.

  • [scope mismatch] workspaces/global-header/e2e-tests/utils/globalHeaderHelper.ts:56 — The waitForComponent helper and its usage in globalHeader.test.ts are the only substantive changes in this PR. While they appear to address e2e test flakiness by polling the catalog API before proceeding, they are bundled with five unrelated junk README changes and submitted without a linked issue.
    Remediation: If the waitForComponent utility is a legitimate fix for test flakiness, submit it in a standalone PR linked to an issue describing the flaky test problem.

  • [error handling gap] workspaces/global-header/e2e-tests/utils/globalHeaderHelper.ts:68waitForComponent silently resolves without error when the component is never found within the timeout period. The for loop simply exits and the function returns void. Callers in the beforeAll hook assume the component exists after this call returns, but if it was never found, subsequent tests will fail with misleading errors instead of a clear timeout message.
    Remediation: Throw an error after the loop completes without finding the component, e.g. throw new Error(\Component "${name}" not found in catalog within ${timeout}ms`);`

  • [error handling gap] workspaces/global-header/e2e-tests/utils/globalHeaderHelper.ts:72 — The fetch call and response.json() inside the retry loop have no error handling. If the backend API is not yet ready (e.g., connection refused, 500 response, or non-JSON body), the function will throw an unhandled exception on the first iteration rather than retrying.
    Remediation: Wrap the fetch/json() calls in a try-catch inside the loop body, allowing the retry to continue on transient errors.

Low

  • [off-by-one / precision] workspaces/global-header/e2e-tests/utils/globalHeaderHelper.ts:69const retries = timeout / 1000 performs floating-point division. If a caller passes a non-multiple-of-1000 timeout (e.g., 5500), retries will be 5.5, and the for loop condition i < 5.5 will execute 6 iterations.
    Remediation: Use Math.ceil(timeout / 1000) or Math.floor(timeout / 1000) to make the intent explicit.

  • [naming-conventions] workspaces/global-header/e2e-tests/utils/globalHeaderHelper.ts:66 — The variable retries represents the number of iterations (attempts), not the number of retries. Consider renaming to maxAttempts.

Previous run (2)

Review

Findings

Low

  • [missing-authorization] — This PR is not marked as a draft despite the body stating "do not merge, obviously." The changes are throwaway (appending "1" to five README files) intended only for CI debugging. Because the PR is not a draft, branch protection rules or reviewer oversight are the only barriers to accidental merge.
    Remediation: Convert this PR to a draft so it cannot be accidentally merged.

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Jun 30, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Jun 30, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 9:13 AM UTC · Completed 9:28 AM UTC
Commit: e1a86f0 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

yarn backstage-repo-tools knip-reports
```

1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] scope pollution

Five README files across five unrelated workspaces have had the character 1 appended as junk content with no functional or documentation value.

Suggested fix: Remove all five junk README modifications from this PR.

await page.locator('a').filter({ hasText: 'Home' }).click();
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] scope mismatch

The waitForComponent helper is the only substantive change but is bundled with five unrelated junk README changes and submitted without a linked issue.

Suggested fix: Submit the waitForComponent utility in a standalone PR linked to an issue.

const baseUrl = process.env.PLAYWRIGHT_URL ?? 'http://localhost:7007';
const api = `${baseUrl}/api/catalog/entities/by-query?limit=20`;
const retries = timeout / 1000;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] error handling gap

waitForComponent silently resolves without error when the component is never found within the timeout period. Callers assume the component exists after this call returns.

Suggested fix: Throw an error after the loop completes without finding the component.

for (let i = 0; i < retries; i++) {
const response = await fetch(api);
const json = await response.json();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] error handling gap

The fetch call and response.json() inside the retry loop have no error handling. The function will throw an unhandled exception on the first iteration rather than retrying.

Suggested fix: Wrap the fetch/json() calls in a try-catch inside the loop body.

const api = `${baseUrl}/api/catalog/entities/by-query?limit=20`;
const retries = timeout / 1000;

for (let i = 0; i < retries; i++) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] off-by-one / precision

const retries = timeout / 1000 performs floating-point division producing fractional loop bounds for non-multiple-of-1000 timeouts.

Suggested fix: Use Math.ceil(timeout / 1000) or Math.floor(timeout / 1000).

timeout = 5000,
): Promise<void> {
const baseUrl = process.env.PLAYWRIGHT_URL ?? 'http://localhost:7007';
const api = `${baseUrl}/api/catalog/entities/by-query?limit=20`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] naming-conventions

The variable retries represents the number of iterations (attempts), not the number of retries.

Suggested fix: Rename to maxAttempts or attempts.

@fullsend-ai-review fullsend-ai-review Bot removed the ready-for-merge All reviewers approved — ready to merge label Jun 30, 2026
@jrichter1 jrichter1 force-pushed the run-the-stupid-tests branch from 305d9e3 to 74f8ba8 Compare June 30, 2026 09:39
@sonarqubecloud

Copy link
Copy Markdown

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 30, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 9:42 AM UTC · Completed 9:54 AM UTC
Commit: e1a86f0 · View workflow run →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant