feat: Add GITHUB_STEP_SUMMARY output to PR comments #17

aaronsteers · 2025-10-11T01:30:44Z

feat: Add task output publishing to PR comments via GITHUB_STEP_SUMMARY

Summary

This PR enables Poe tasks to publish markdown output that appears in both the GitHub Actions job summary and as expandable sections in PR comments. Tasks can write to the standard GITHUB_STEP_SUMMARY environment variable, and the output will be automatically included in slash command responses.

Key changes:

action.yml: Added read-summary step to detect and read GITHUB_STEP_SUMMARY content, then include it in success/failure PR comments as expandable <details> blocks (✳️ for success, ✴️ for failure)
README.md: Added "Publishing Task Output" documentation with usage examples
CI Tests: Added .github/workflows/ci-tests.yml to validate the feature with two test scenarios
Test Tasks: Added test-with-summary and test-without-summary in poe_tasks.toml for testing

The implementation is opt-in: tasks that don't write to GITHUB_STEP_SUMMARY work as before, while tasks that do write to it get enhanced PR comment responses with their output.

Review & Testing Checklist for Human

End-to-end verification: Comment /poe test-with-summary on this PR and verify the expandable ✳️ summary section appears with the test output
Opt-in behavior: Comment /poe test-without-summary and verify NO summary section appears (just the success message)
Job summary visibility: Check that the GitHub Actions job summary page also displays the task output for test-with-summary
Failure case: Trigger a failing task that writes to GITHUB_STEP_SUMMARY and verify the ✴️ expandable section appears in the failure comment

Notes

This implementation was simplified per PR feedback to only use GITHUB_STEP_SUMMARY directly, removing the temporary file mechanism
CI tests pass, confirming GITHUB_STEP_SUMMARY works correctly in composite actions
The pattern follows GitHub's standard approach for job summaries and should work for any poe task

Session: https://app.devin.ai/sessions/7ec095caed01494693ef260c6a6db2ad
Requested by: @aaronsteers

@aaronsteers

- Add step to read GITHUB_STEP_SUMMARY after task execution - Include summary content in PR comments using expandable <details> block - Only render summary section when content exists - Uses format: <details><summary>✳️ Show/Hide Summary Output</summary> Requested by @aaronsteers Link to Devin run: https://app.devin.ai/sessions/7ec095caed01494693ef260c6a6db2ad Co-Authored-By: AJ Steers <[email protected]>

devin-ai-integration · 2025-10-11T01:30:47Z

Original prompt from AJ Steers

Devin - For the poe command processor github action, I want to devise a strategy for the slash commands calling the github action to generically gather printable output in markdown form. I wonder if a good way to do this is to instruct tasks that want to publish their output to (1) check for the presence of the CI env var and/or the GITHUB_SUMMARY env var, or similar, and to send output there if so. Then, presumably this would auto-render on the github job summary section, but _also_ our poe command processor could put the same text into the comment response. Wdyt?

devin-ai-integration · 2025-10-11T01:30:48Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment and CI monitoring

- Include summary output in failure comments using expandable <details> block - Use ✴️ emoji for failure output (vs ✳️ for success) - Only render summary section when content exists Co-Authored-By: AJ Steers <[email protected]>

- Add test-with-summary task that writes sample markdown to GITHUB_STEP_SUMMARY - Add test-without-summary task as a control (no summary output) - Create ci-tests.yml workflow triggered on PR synchronize events - Only run tests when PR is not in draft status Co-Authored-By: AJ Steers <[email protected]>

github-actions · 2025-10-11T01:59:26Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

github-actions · 2025-10-11T01:59:30Z

Running poe test-with-summary...

Link to job logs.

❌ Poe command test-with-summary failed. Please inspect the logs.

The test-with-summary task was using invalid syntax for poethepoet. Multiline commands require the 'shell' task type with shell = """...""" syntax. This fixes the CI failure where poe reported: 'Invalid cmd task includes multiple command lines' Co-Authored-By: AJ Steers <[email protected]>

github-actions · 2025-10-11T02:04:01Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

github-actions · 2025-10-11T02:04:08Z

Running poe test-with-summary...

Link to job logs.

🟦 Poe command test-with-summary completed successfully.

The previous implementation relied on GITHUB_STEP_SUMMARY being accessible within the composite action, but this file is managed by GitHub Actions and not reliably readable across action steps. Solution: - Create a temporary file (POE_SUMMARY_FILE) at the start of the action - Have tasks write to POE_SUMMARY_FILE instead of GITHUB_STEP_SUMMARY - Read from POE_SUMMARY_FILE in the read-summary step - Copy content to GITHUB_STEP_SUMMARY for job summary visibility - Maintain backward compatibility by falling back to GITHUB_STEP_SUMMARY This ensures dual visibility: task output appears in both the GitHub job summary and the PR comments. Co-Authored-By: AJ Steers <[email protected]>

github-actions · 2025-10-11T04:17:00Z

Running poe test-with-summary...

Link to job logs.

🟦 Poe command test-with-summary completed successfully.

✳️ Show/Hide Summary Output

Test Results

This is a test of the GITHUB_STEP_SUMMARY feature.

✅ Item 1: Success
✅ Item 2: Success
✅ Item 3: Success

Status: All tests passed! 🎉

github-actions · 2025-10-11T04:17:01Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

action.yml

Co-Authored-By: AJ Steers <[email protected]>

github-actions · 2025-10-11T05:05:16Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

github-actions · 2025-10-11T05:05:20Z

Running poe test-with-summary...

Link to job logs.

🟦 Poe command test-with-summary completed successfully.

✳️ Show/Hide Summary Output

Test Results

This is a test of the GITHUB_STEP_SUMMARY feature.

✅ Item 1: Success
✅ Item 2: Success
✅ Item 3: Success

Status: All tests passed! 🎉

…MMARY_FILE Co-Authored-By: AJ Steers <[email protected]>

github-actions · 2025-10-11T05:10:03Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

github-actions · 2025-10-11T05:10:06Z

Running poe test-with-summary...

Link to job logs.

🟦 Poe command test-with-summary completed successfully.

Copilot

Pull Request Overview

This PR adds support for Poe tasks to publish markdown output that appears in both GitHub Actions job summaries and as expandable sections in PR comments via the GITHUB_STEP_SUMMARY environment variable.

Enhanced action.yml to read and include GITHUB_STEP_SUMMARY content in PR comments as expandable details sections
Added comprehensive documentation and usage examples for the new feature
Implemented CI testing workflow to validate both with-summary and without-summary scenarios

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
action.yml	Added step to read `GITHUB_STEP_SUMMARY` and include content in success/failure PR comments with expandable sections
README.md	Added "Publishing Task Output" documentation with usage examples and implementation guidance
poe_tasks.toml	Added test tasks `test-with-summary` and `test-without-summary` for feature validation
.github/workflows/ci-tests.yml	Added CI workflow to test both summary scenarios automatically on PRs

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

action.yml

github-actions · 2025-10-11T12:57:18Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

github-actions · 2025-10-11T12:57:19Z

Running poe test-with-summary...

Link to job logs.

🟦 Poe command test-with-summary completed successfully.

✳️ Show/Hide Summary Output

action.yml

github-actions · 2025-10-11T12:59:39Z

Running poe test-with-summary...

Link to job logs.

🟦 Poe command test-with-summary completed successfully.

✳️ Show/Hide Summary Output

Test Results

This is a test of the GITHUB_STEP_SUMMARY feature.

✅ Item 1: Success
✅ Item 2: Success
✅ Item 3: Success

Status: All tests passed! 🎉

github-actions · 2025-10-11T12:59:42Z

Running poe test-without-summary...

Link to job logs.

🟦 Poe command test-without-summary completed successfully.

github-actions · 2025-10-11T13:07:45Z

Running poe test-fail-with-summary...

Link to job logs.

❌ Poe command test-fail-with-summary failed. Please inspect the logs.

✴️ Show/Hide Summary Output

Test Results

This is a test of the GITHUB_STEP_SUMMARY feature.

✅ Item 1: Success
✅ Item 2: Success
😵 Item 3: Failure!!

ERROR: Something bad happened ☠️

github-actions · 2025-10-11T13:07:45Z

Running poe test-success-with-summary...

Link to job logs.

🟦 Poe command test-success-with-summary completed successfully.

✳️ Show/Hide Summary Output

Test Results

This is a test of the GITHUB_STEP_SUMMARY feature.

✅ Item 1: Success
✅ Item 2: Success
✅ Item 3: Success

Status: All tests passed! 🎉

github-actions · 2025-10-11T13:07:46Z

Running poe test-success...

Link to job logs.

🟦 Poe command test-success completed successfully.

devin-ai-integration bot assigned aaronsteers Oct 11, 2025

devin-ai-integration bot and others added 2 commits October 11, 2025 01:37

aaronsteers commented Oct 11, 2025

View reviewed changes

action.yml Outdated Show resolved Hide resolved

docs: Add GITHUB_STEP_SUMMARY usage to README

d51646a

Co-Authored-By: AJ Steers <[email protected]>

refactor: Simplify to use GITHUB_STEP_SUMMARY directly, remove POE_SU…

a28a3bc

…MMARY_FILE Co-Authored-By: AJ Steers <[email protected]>

aaronsteers requested a review from Copilot October 11, 2025 12:14

Copilot AI reviewed Oct 11, 2025

View reviewed changes

action.yml Outdated Show resolved Hide resolved

action.yml Outdated Show resolved Hide resolved

combine into one step

76ea61f

aaronsteers commented Oct 11, 2025

View reviewed changes

action.yml Outdated Show resolved Hide resolved

Apply suggestion from @aaronsteers

98d6b85

add test-fail-with-summary

f154f42

aaronsteers merged commit 7e15ba3 into main Oct 11, 2025
3 checks passed

aaronsteers deleted the devin/1760146134-add-step-summary-output branch October 11, 2025 13:10

feat: Add GITHUB_STEP_SUMMARY output to PR comments #17

feat: Add GITHUB_STEP_SUMMARY output to PR comments #17

Conversation

aaronsteers commented Oct 11, 2025 • edited by devin-ai-integration bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

feat: Add task output publishing to PR comments via GITHUB_STEP_SUMMARY

Summary

Review & Testing Checklist for Human

Notes

Uh oh!

devin-ai-integration bot commented Oct 11, 2025

Uh oh!

devin-ai-integration bot commented Oct 11, 2025

🤖 Devin AI Engineer

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

github-actions bot commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aaronsteers commented Oct 11, 2025 •

edited by devin-ai-integration bot

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading

github-actions bot commented Oct 11, 2025 •

edited

Loading