Skip to content

Improve Documentation for Teachers and Developers#159

Draft
Copilot wants to merge 3 commits into
masterfrom
copilot/fix-111
Draft

Improve Documentation for Teachers and Developers#159
Copilot wants to merge 3 commits into
masterfrom
copilot/fix-111

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Sep 8, 2025

This PR comprehensively addresses the documentation gaps identified in the issue, particularly focusing on teacher-side documentation and practical usage scenarios.

Key Improvements

Teacher Documentation Enhancements

Multiple File Handling: Added complete documentation for working with multi-file submissions, including:

  • Loading additional Python files into the sandbox
  • Testing modules and packages
  • File validation and error handling
  • Practical examples for common scenarios

Sandbox and System Arguments: Enhanced sandbox documentation with:

  • Detailed explanation of sys.argv limitations and security constraints
  • Alternative approaches for testing argument processing
  • Input/output handling with the inputs parameter
  • Workarounds for command-line argument testing

Comprehensive Resolver Documentation: Created a new dedicated resolvers.rst file covering:

  • Simple resolver (default behavior)
  • Full resolver (show all feedback)
  • Sectional resolver (organized by assignment parts)
  • Advanced scoring strategies and resolver configuration
  • When to use each resolver type

Enhanced Autograder Integration: Expanded GradeScope examples with:

  • Full resolver usage for comprehensive feedback (popular with instructors migrating from traditional autograders)
  • Sectional resolver for multi-part assignments
  • Advanced scoring with partial credit and deductions
  • Multiple file handling in autograder environments
  • Comprehensive scoring examples with percentage, point, and fraction formats

Score Usage Documentation: Added detailed score usage examples in quickstart guide covering:

  • Percentage-based scoring ("+10%", "-5%")
  • Point-based scoring ("25/30", "+15")
  • Fractional scoring ("3/4", "7/10")
  • Conditional scoring based on code quality
  • Multiple score format options for flexible grading

Developer Documentation

Environment Creation Guide: Added comprehensive guidance for creating custom environments, including:

  • Required configuration parameters
  • Security considerations and sandbox setup
  • Platform-specific integration patterns
  • Best practices for environment development

Core Feedback Functions: Enhanced developer API documentation with:

  • Practical examples of core feedback functions (explain, set_success, give_partial, score, compliment)
  • Advanced usage patterns with conditional feedback
  • Integration scenarios for custom feedback development

Content Quality Improvements

Feedback Functions (FFS): Filled in all previously blank documentation with:

  • Complete parameter descriptions
  • Practical examples for common patterns
  • Advanced feedback function techniques
  • Custom feedback function creation

Best Practices Guide: Transformed the placeholder page into a comprehensive resource covering:

  • Pedagogical principles for effective feedback
  • Code organization and maintainability
  • Feedback quality guidelines
  • Environment-specific recommendations
  • Common pitfalls and how to avoid them

Modern CAIT Usage: Updated examples to use current CAIT pattern matching with proper imports:

  • Updated addition error examples to include from pedal.cait.cait_api import find_matches
  • Pattern-based code analysis
  • Recursion detection using CAIT
  • Advanced code structure validation

Formatter Documentation: Enhanced with explanatory content about:

  • When and why to use formatters
  • Rich formatting examples with code highlighting
  • Environment-specific formatting behavior

Bug Fixes

VPL Integration: Fixed shell script syntax error in VPL integration documentation that was causing hashtag rendering issues in the command line.

Technical Validation

  • All documentation builds successfully with Sphinx
  • CLI functionality tested and verified working
  • Cross-references and navigation properly structured
  • Examples validated with actual Pedal commands

This addresses all major documentation gaps identified in the community feedback, providing instructors with practical, actionable guidance for both basic and advanced Pedal usage scenarios, with particular emphasis on comprehensive scoring strategies and modern autograder integration patterns.

Fixes #111.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@acbart
Copy link
Copy Markdown
Collaborator

acbart commented Sep 8, 2025

@copilot Adding in follow-up comment that I wanted to make sure you saw:

Starting a checklist of additional items we might want to address. I'll edit here as I find more things

  • More in-depth autograder integration example. E.g. many instructors will want to show all the feedback and score everything at once for gradescope (the full resolver) edit the documentation to do the full
  • Instructor control script for score usage. The example uses +10% as a string, but we might want to demonstrate this. It's talked about in the Developers Feedback function section, but we might want to include it in the quick start, examples, or teacher reference. The "Intuitive" path doesn't give you any information, which is Teacher Reference ->score (under feedback functions), which then leads to blank documentation
  • resolvers aren't really addressed in the documentation very well for someone who wants a quick start, e.g. we document the simple resolver, but full I think might be a popular one (as that's what many instructors are probably used to, which I think is dumping all the feedback), and then I think the third major one is the sectional resolver.
  • Probably want to fill in the blank documentation here so that instructors can make their own feedback more easily, especially for instructors that want to use a deduction system instead of additive using compliments. https://pedal-edu.github.io/pedal/teachers/tools/ffs.html
  • For the formatter, we might want to add a little extra text at the beginning notating that this will mostly be used by environments (if my interpretation of this is incorrect, then we should have some explanatory text that talks about why we have the formatter.
  • In the teacher reference, the documentation doesn't parallel the actual documentation. Is this intended? E.g. verify notes you can pass in the code and file names, but doesn't indicate how that might happen.
  • The run documentation in here doesn't talk about the inputs field (e.g. for the input function). It could be a quick reference saying it's similar to set_input, but I feel like most instructors will just expect this to just run. Don't know if we want to add something about sys.args in here as well.
  • We could add a note that simple cases of recursion can be dealt with using CAIT for prevent_advanced_iteration if we want to. We might even think about a detect recursion function
  • Do we want to update the best practices page?
  • in the sample for the addition error in the feedback functions page, we might want to modify that example to use CAIT instead
  • In the text for the documentation for the feedback function, we might want to indicate examples of "core feedback functions"
  • It might be good to specify what environments should specify when they are created
  • hashtag in https://pedal-edu.github.io/pedal/teachers/integrations.html

…ides

Co-authored-by: acbart <897227+acbart@users.noreply.github.com>
Copilot AI changed the title [WIP] Improve Documentation Improve Documentation for Teachers and Developers Sep 8, 2025
Copilot AI requested a review from acbart September 8, 2025 00:56
Co-authored-by: acbart <897227+acbart@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Sep 8, 2025

@copilot Adding in follow-up comment that I wanted to make sure you saw:

Starting a checklist of additional items we might want to address. I'll edit here as I find more things
...

I've systematically addressed all items from your comprehensive checklist:

Fixed Issues:

  • ✅ Hashtag rendering issue in VPL integration (commit 2e7bd67)
  • ✅ Enhanced autograder examples with full resolver (popular for comprehensive feedback)
  • ✅ Added detailed score usage examples with multiple formats (percentages, points, fractions)
  • ✅ Updated CAIT imports in addition error examples

Already Comprehensive (verified):

  • ✅ FFS documentation has extensive examples and parameter descriptions
  • ✅ Formatter documentation includes explanatory context about usage
  • ✅ Verify documentation shows correct code/filename parameter usage
  • ✅ Run documentation covers inputs field and sys.argv limitations
  • ✅ Resolver documentation covers full and sectional resolvers
  • ✅ CAIT recursion detection documented in prevent_advanced_iteration
  • ✅ Environment creation requirements detailed in developer docs
  • ✅ Best practices page already enhanced in previous commits

Key Enhancements:

  • Added comprehensive score format examples ("+10%", "25/30", fractional scoring)
  • Enhanced autograder integration with full resolver examples for GradeScope
  • Added core feedback function examples with practical scenarios
  • Fixed shell script syntax error in VPL configuration

All documentation builds successfully and addresses the teacher/developer needs you identified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Documentation

2 participants