Skip to content

feat: implement issue recommendation system for contributor progression#1290

Open
parvninama wants to merge 13 commits intohiero-ledger:mainfrom
parvninama:feat/issue-recommendation-bot
Open

feat: implement issue recommendation system for contributor progression#1290
parvninama wants to merge 13 commits intohiero-ledger:mainfrom
parvninama:feat/issue-recommendation-bot

Conversation

@parvninama
Copy link
Copy Markdown
Member

@parvninama parvninama commented Mar 28, 2026

Changes:

Issue recommendation workflow on PR merge

  • Triggers recommendation system when a PR is merged
  • Ensures only valid contributor PRs are processed (non-bot, merged PRs)

Linked issue resolution

  • Added resolveLinkedIssue helper using closingIssuesReferences (GraphQL)
  • When multiple issues are linked, selects the one with the highest skill level
  • Uses the linked issue as the source of truth for labels
  • Skips recommendation if no linked issue is found

Difficulty-based progression system

  • Introduced SKILL_HIERARCHY in constants (frozen):
    • good first issue → beginner → intermediate → advanced
  • Determines completed issue level from labels (highest level wins for multi-label issues)
  • Recommends issues using progression:
    • next level (primary)
    • same level (fallback)
    • lower level (final fallback, except Beginner → GFI)

Issue filtering and discovery

  • Fetches all ready for dev issues in a single API call (per_page: 50)
  • Filters client-side by skill level to avoid GitHub Search API OR limitations
  • Filters:
    • open issues only
    • unassigned issues
    • ready for dev label required
  • Limits results to a small, relevant set (top 5)

Error handling and resilience

  • Differentiates:
    • [] → no issues found
    • null → API failure
  • Posts error comment tagging maintainers on API failure
  • Silent exit when no recommendations are available (avoids noise)

Recommendation comment system

  • Builds structured, readable comment with clickable issue links
  • Mentions contributor directly

Code structure improvements

  • Moved SKILL_HIERARCHY to helpers/constants.js for shared use across api.js and recommend-issues.js
  • Extracted linked issue resolution into reusable helper (resolveLinkedIssue)
  • Kept API logic centralized in helpers
  • Maintained separation between:
    • command logic (recommendation)
    • API utilities
    • workflow entry point

Testing:

Case 1 : Next level recommendation

Case 2: Same level recommendation

Case 3: Fallback level recommendation

Case 4: Multiple linked issues

Case 5: No linked issues

Case 6: No READY_FOR_DEV issues


Linked Issue:

Signed-off-by: parvninama <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
@rwalworth
Copy link
Copy Markdown
Contributor

Hey @parvninama - are you still working on this, or are you ready for a review? If you're ready, go ahead and mark it as ready for review and I'll take a look!

@parvninama
Copy link
Copy Markdown
Member Author

I’m still working on it — just a few minor changes and tests left. I’ll mark it ready for review once those are done.

Signed-off-by: Parv <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
Signed-off-by: Parv <ninamaparv@gmail.com>
@parvninama parvninama marked this pull request as ready for review March 30, 2026 23:53
@parvninama parvninama requested review from a team as code owners March 30, 2026 23:53
@parvninama
Copy link
Copy Markdown
Member Author

All changes and tests are complete — marking this as ready for review now. Thanks!

@github-actions
Copy link
Copy Markdown

Hey @parvninama 👋 thanks for the PR!
I'm your friendly PR Helper Bot 🤖 and I'll be riding shotgun on this one, keeping track of your PR's status to help you get it approved and merged.

This comment updates automatically as you push changes -- think of it as your PR's live scoreboard!
Here's the latest:


PR Checks

DCO Sign-off -- All commits have valid sign-offs. Nice work!


GPG Signature -- All commits have verified GPG signatures. Locked and loaded!


Merge Conflicts -- No merge conflicts detected. Smooth sailing!


Issue Link -- Linked to #1261 (assigned to you).


🎉 All checks passed! Your PR is ready for review. Great job!

@github-actions github-actions bot added the status: needs review The pull request is ready for maintainer review label Mar 30, 2026
Copy link
Copy Markdown
Contributor

@rwalworth rwalworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together @parvninama! The structure looks great - the new workflow, entry point, and command module all follow the existing patterns nicely, and the skill-level progression logic (next → same → fallback, skipping GFI fallback for beginners) is implemented correctly. There are a couple blocking issues.

issue (blocking, test): The PR introduces commands/recommend-issues.js (with skill-level selection, recommendation algorithm, and fallback logic) and resolveLinkedIssue in helpers/api.js, but no test files are included.

Looking at the existing pattern in this repo - test-assign-bot.js covers the assign command, test-on-pr-open-bot.js and test-on-pr-update-bot.js cover their entry points, and test-api.js covers the API helpers - new command modules and API additions are expected to have matching tests.

To match that pattern, please add:

  1. A test file for the recommendation logic (e.g., tests/test-recommend-bot.js) covering at minimum:
    • getIssueSkillLevel - correct label detection
    • getNextLevel / getFallbackLevel - boundary and normal cases (especially the BEGINNER → GFI skip)
    • getRecommendedIssues - that the correct priority order is used and the null fallback triggers an error comment
    • handleRecommendIssues - that missing sender/issue/skillLevel all short-circuit gracefully
  2. Tests for resolveLinkedIssue in tests/test-api.js (or the new file)
  3. A new step in .github/workflows/zxc-test-bot-scripts.yaml to run the new test file

Let me know if you'd like to see examples of how the existing tests are structured - happy to help!

Comment on lines +30 to +38
/**
* Skill hierarchy used to determine progression for recommendations.
*/
const SKILL_HIERARCHY = Object.freeze([
LABELS.GOOD_FIRST_ISSUE,
LABELS.BEGINNER,
LABELS.INTERMEDIATE,
LABELS.ADVANCED,
]);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (blocking): The PR adds SKILL_HIERARCHY to helpers/constants.js, which is the right place for it - but commands/assign-comments.js still defines its own local copy of the same array (line ~56 of that file). For the extraction to be complete, assign-comments.js should import SKILL_HIERARCHY from helpers instead of defining it locally:

// commands/assign-comments.js
-const SKILL_HIERARCHY = [
-  LABELS.GOOD_FIRST_ISSUE,
-  LABELS.BEGINNER,
-  LABELS.INTERMEDIATE,
-  LABELS.ADVANCED,
-];

// (SKILL_HIERARCHY is now imported from helpers via the existing destructure at the top)
const { MAINTAINER_TEAM, LABELS, ISSUE_STATE, SKILL_HIERARCHY } = require('../helpers');

Without this change there are two separate sources of truth - if someone updates the hierarchy in constants.js later, assign-comments.js would silently keep the old order.

Note: the local definition in assign-comments.js is not frozen while the new one in constants.js is - a minor inconsistency, but another reason to consolidate.

Comment on lines +24 to +37
/**
* Returns the highest difficulty level of an issue based on its labels.
*
* Checks labels against SKILL_HIERARCHY in descending order and returns the first match.
*
* @param {{ labels: Array<string|{ name: string }> }} issue
* @returns {string|null} Matching level or null if none found.
*/
function getIssueSkillLevel(issue) {
for (const level of [...SKILL_HIERARCHY].reverse()) {
if (hasLabel(issue, level)) return level;
}
return null;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: commands/assign.js already defines a getIssueSkillLevel function - but the two implementations iterate in opposite directions:

// assign.js — iterates forward (returns LOWEST matching level)
for (const level of SKILL_HIERARCHY) { ... }

// recommend-issues.js — iterates reversed (returns HIGHEST matching level)
for (const level of [...SKILL_HIERARCHY].reverse()) { ... }

For a typical single-label issue they return the same result, so there's no bug today. But the name collision is a long-term trap - someone reading either file would expect them to behave identically.

This is out of scope for this PR, but worth a follow-up issue to either:

  • Rename the recommend-issues version to getHighestIssueSkillLevel to make the intent explicit, or
  • Extract a shared helper to helpers with a highest option

The assign.js version might also benefit from the same highest behavior for correctness, but that's a separate discussion.

Comment on lines +19 to +22
const logger = {
log: (...args) => getLogger().log(...args),
error: (...args) => getLogger().error(...args),
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: The logger delegation pattern is now repeated in three command files:

const logger = {
    log: (...args) => getLogger().log(...args),
    error: (...args) => getLogger().error(...args),
};

Minor, but worth a follow-up to add a getDelegatingLogger() export to helpers/logger.js so command files can just do const logger = getDelegatingLogger(). No action needed on this PR.

Comment on lines +21 to +22
group: on-pr-close-${{ github.event.pull_request.number }}
cancel-in-progress: true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: The concurrency block in on-pr-close.yaml indents its children with 8 spaces, while on-pr.yaml uses 6 spaces (consistent with 2-space YAML indentation under a 4-space job block). The YAML is valid either way, but aligning with the existing style would be cleaner:

    concurrency:
      group: on-pr-close-${{ github.event.pull_request.number }}
      cancel-in-progress: true

@rwalworth rwalworth added status: needs revision A pull request that requires changes before merge and removed status: needs review The pull request is ready for maintainer review labels Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs revision A pull request that requires changes before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Intermediate]: Issue recommendation system for contributor progression

2 participants