added new repo info tool with test#6
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new GitHub repository summarization tool (repo_info_tool) that fetches and summarizes repository information using the GitHub REST API. The tool extracts README content, installation instructions, CLI commands, API examples, and other metadata to help the agent verify tool candidates before making final recommendations.
Key Changes:
- Introduced
tool_repo_summarythat fetches curated repo content via GitHub API - Updated agent workflow to mandate repo verification (via
repo_info) for top-K candidates before final selection - Added comprehensive test script for the new repo summarization functionality
Reviewed Changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_repo_summary.py |
New smoke test for repo summarization with CLI arguments and assertions |
src/ai_agent/generator/prompts.py |
Updated agent instructions to mandate repo verification step for candidates |
src/ai_agent/agent/tools/repo_info_tool.py |
Complete rewrite: added GitHub API fetcher, markdown summary builder, and extractors |
src/ai_agent/agent/agent.py |
Updated repo_info tool to use new implementation with URL validation and error handling |
.env.dist |
Added GITHUB_TOKEN environment variable for GitHub API authentication |
| Minimal smoke test for the repo summarizer tool. | ||
|
|
||
| Usage (from your project root): | ||
| python tests/test_repo_summary.py \ |
There was a problem hiding this comment.
The usage example shows running with 'python tests/test_repo_summary.py' but the script uses argparse and should be run as a module or directly. Consider updating to show the correct invocation method consistent with the test execution pattern.
Suggested change
| python tests/test_repo_summary.py \ | |
| python -m tests.test_repo_summary \ |
| + "\n9. Accuracy scoring: task(40)+compat(30)+features(30); incorporate original formats & 2D/3D nature from metadata; penalize format conversions (−5) if heavy." | ||
| + "\n10. Never fabricate tool outputs; if run_example not executed do NOT reference execution results." | ||
| + "\n11. After ranking, call resolve_demo_link(name) for each tool you plan to return (up to 3). THEN include demo_link for those tools in final JSON choices. If a link is missing after resolution, omit demo_link for that tool. Never guess a URL." | ||
| + "\n4. Mandatory repo verification before final output: After search_tools (and optional rerank), take the top K ≤ 3 candidates you plan to return and you MUST call repo_info(url) once for each. Use the repo URL from the candidate payload (field name repo_url; fallback keys: github, url, homepage). If a candidate has no repo URL, drop it rather than guessing. Only after repo_info confirms alignment with the requested task should you call resolve_demo_link(name). Do not return any candidate that wasn’t verified by repo_info. Call `repo_info(url)` **only** with a GitHub repo URL or `owner/repo`. If a candidate lacks that, **drop it** (don’t pass papers, docs, or homepages)." |
There was a problem hiding this comment.
[nitpick] This instruction is extremely long (149 words) and contains repetitive guidance about dropping candidates without repo URLs. Consider breaking it into sub-points or simplifying the language to improve readability.
Suggested change
| + "\n4. Mandatory repo verification before final output: After search_tools (and optional rerank), take the top K ≤ 3 candidates you plan to return and you MUST call repo_info(url) once for each. Use the repo URL from the candidate payload (field name repo_url; fallback keys: github, url, homepage). If a candidate has no repo URL, drop it rather than guessing. Only after repo_info confirms alignment with the requested task should you call resolve_demo_link(name). Do not return any candidate that wasn’t verified by repo_info. Call `repo_info(url)` **only** with a GitHub repo URL or `owner/repo`. If a candidate lacks that, **drop it** (don’t pass papers, docs, or homepages)." | |
| + "\n4. Mandatory repo verification before final output:" | |
| + "\n a. After search_tools (and optional rerank), take the top K ≤ 3 candidates you plan to return." | |
| + "\n b. For each candidate, call repo_info(url) using the repo URL from the candidate payload (field name: repo_url; fallback keys: github, url, homepage)." | |
| + "\n c. If a candidate does not have a valid GitHub repo URL or owner/repo, drop it. Do NOT guess or use papers, docs, or homepages." | |
| + "\n d. Only after repo_info confirms alignment with the requested task should you call resolve_demo_link(name)." | |
| + "\n e. Do not return any candidate that was not verified by repo_info." |
| @@ -126,8 +146,7 @@ def run_agent(task: str, image_data_url: str | None = None, excluded: List[str] | |||
| short_meta = " ".join(x.strip() for x in image_meta.splitlines() if x.strip()) | |||
| hidden_meta += "\n(Image Metadata: " + short_meta[:500] + ("…" if len(short_meta) > 500 else "") + ")" | |||
| prompt = task + extra_context + hidden_meta | |||
There was a problem hiding this comment.
The tool call limit was increased from 6 to 10, but there's no comment explaining why this change was necessary. Given that repo_info can now be called up to 3 times per run, consider adding a brief comment explaining the rationale for this increase.
Suggested change
| prompt = task + extra_context + hidden_meta | |
| prompt = task + extra_context + hidden_meta | |
| # Increased tool_calls_limit from 6 to 10 because repo_info can now be called up to 3 times per run. | |
| # This ensures the agent has enough tool calls available for typical tasks. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.