Skip to content

Add: beginner-friendly file analyzer example#380

Open
itsroshanharry wants to merge 4 commits into
hud-evals:mainfrom
itsroshanharry:feature/add-file-analyzer-example
Open

Add: beginner-friendly file analyzer example#380
itsroshanharry wants to merge 4 commits into
hud-evals:mainfrom
itsroshanharry:feature/add-file-analyzer-example

Conversation

@itsroshanharry
Copy link
Copy Markdown

@itsroshanharry itsroshanharry commented Mar 22, 2026

Description

Add a beginner-friendly file analyzer example to help new users understand HUD SDK core concepts.

Problem

Currently, the examples jump from very basic (00_agent_env.py - just one tool call) to advanced production-ready agents (01_codex, 02_opencode). There's no intermediate example for beginners learning the SDK.

Solution

This example fills that gap by demonstrating:

  • Creating environments with custom tools (@env.tool())
  • Defining evaluation scenarios (@env.scenario())
  • Running agents with different models
  • Understanding automatic tool usage
  • Reward-based evaluation

Why This Is Valuable

  1. Fills a gap: Provides progression path from basic to advanced
  2. Beginner-friendly: Uses simple, relatable tools (file operations)
  3. Educational: Extensive comments and helpful output explaining what happened
  4. Complete: Shows full workflow from setup to evaluation
  5. Flexible: Includes command-line options for different models

Example Output

======================================================================
FILE ANALYZER AGENT
======================================================================

🤖 Model: gpt-4o-mini
📋 Task: Analyze README.md file
🔧 Tools: list_files, read_file, count_words

======================================================================

✅ Task completed!
📊 Reward: 1.0
🔢 Steps taken: 0

📝 Agent's response:
The **README.md** file exists, and here are the details:
- **Word Count:** 70 words
- **Lines:** 14
- **Characters:** 1000
- **Average Word Length:** 14.3 characters

Testing

  • Tested with gpt-4o-mini (default)
  • Tested with gpt-4o
  • Verified all tools work correctly
  • Confirmed evaluation scenario works
  • Checked code formatting with ruff format
  • Verified linting with ruff check
  • Added documentation to examples/README.md

Checklist

  • Code follows project style guidelines
  • Added documentation to examples/README.md
  • Tested the example works
  • Includes clear usage instructions
  • Has helpful comments and docstrings
  • Includes shebang and proper imports
  • Follows existing example patterns

Additional Context

This is my first contribution to the HUD SDK. I created this example while learning the framework and thought it would help other beginners understand the core concepts. The example is self-contained and doesn't modify any existing code.

I'm excited to contribute to the project and help make it more accessible to new users!


Note

Low Risk
Adds a new standalone example script and documentation without modifying library/runtime code; risk is limited to example usability (e.g., path handling and tool output).

Overview
Adds a new beginner example examples/05_file_analyzer_agent.py that defines simple file tools (list_files, read_file, count_words), an analyze-readme scenario with basic reward scoring, and a CLI to run the agent against different models.

Updates examples/README.md to document how to run the new example (including --model and --verbose) and position it as an intermediate learning step.

Written by Cursor Bugbot for commit c8923aa. This will update automatically on new commits. Configure here.

- New example demonstrating core HUD SDK concepts
- Shows environment creation with custom tools
- Demonstrates scenario-based evaluation
- Includes clear documentation and usage examples
- Fills gap between basic (00) and advanced (01/02) examples
- Perfect starting point for new users learning the SDK
Comment thread examples/05_file_analyzer_agent.py Outdated
Comment thread examples/05_file_analyzer_agent.py Outdated
- Fix average word length calculation to exclude whitespace
- Change 'Steps taken' to 'Tool calls' for accuracy
- Add comment explaining the calculation
Comment thread examples/05_file_analyzer_agent.py
Comment thread examples/05_file_analyzer_agent.py Outdated
- Add directory traversal protection to list_files and read_file tools
- Validate all file paths are within working directory before access
- Fix tool calls metric to count actual MCP calls from trace
- Prevent negative count when agent errors early

Addresses Bugbot security and accuracy concerns
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment thread examples/05_file_analyzer_agent.py Outdated
- Add guard to prevent AttributeError when agent returns no content
- Handle edge case where response is None before calling .lower()

Addresses Bugbot review comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants