Skip to content

Enhance E2E test to validate failed execution success field#1904

Merged
vcastellm merged 2 commits intofeature/e2e-testingfrom
copilot/sub-pr-1895-again
Jan 29, 2026
Merged

Enhance E2E test to validate failed execution success field#1904
vcastellm merged 2 commits intofeature/e2e-testingfrom
copilot/sub-pr-1895-again

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jan 29, 2026

Proposed changes

The "Failed job execution has success=false" test in e2e/tests/03_job_execution.bats was only checking that executions exist, not validating the actual success field value.

Enhanced test to assert execution failure:

  • Added assertion that verifies success == false in execution response
  • Follows same pattern as successful execution test using jq filtering
  • Now properly validates both execution count and failure status
# Before: Only checked execution exists
count=$(json_array_length "$body")
[ "$count" -ge 1 ]

# After: Also validates success field
failure_count=$(echo "$body" | jq '[.[] | select(.success == false)] | length')
[ "$failure_count" -ge 1 ]

Types of changes

What types of changes does your code introduce?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>
Copilot AI changed the title [WIP] Add comprehensive E2E test suite for Dkron API Enhance E2E test to validate failed execution success field Jan 29, 2026
Copilot AI requested a review from vcastellm January 29, 2026 09:45
@vcastellm vcastellm marked this pull request as ready for review January 29, 2026 20:39
@vcastellm vcastellm merged commit fcc11f0 into feature/e2e-testing Jan 29, 2026
2 of 4 checks passed
@vcastellm vcastellm deleted the copilot/sub-pr-1895-again branch January 29, 2026 20:39
vcastellm added a commit that referenced this pull request Jan 29, 2026
* Add comprehensive E2E test suite for Dkron API

- Create e2e test infrastructure using Bats (Bash Automated Testing System)
- Add Docker Compose setup for isolated e2e testing environment
- Add 71 tests covering:
  - Health, cluster, leader, and debug endpoints (19 tests)
  - Job CRUD operations including timezone, retries, concurrency (24 tests)
  - Job execution and scheduling (17 tests)
  - Pause/unpause functionality (11 tests)
- Add helper functions for API requests, response parsing, and assertions
- Add test runner script with options for filtering, verbose output, and debugging
- Add Makefile targets: e2e, e2e-quick, e2e-debug

Key findings documented in tests:
- Disabled jobs cannot be manually triggered via /run endpoint
- Pause only blocks new job creation, not scheduled execution
- Timezone jobs require 6-field cron expressions (with seconds)

* Add Algolia site verification meta tag

* Apply suggestion from @coderabbitai[bot]

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Add GitHub Actions workflow for E2E tests (#1899)

* Initial plan

* Add GitHub Actions workflow for E2E tests

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Fix workflow: remove invalid tags-ignore and use apt for bats

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Add explicit permissions to e2e workflow

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Merge main branch into E2E test suite PR (#1902)

* Initial plan

* Merge main branch into PR branch - resolve conflicts

Merged latest changes from main branch (commit 28f91f1) into the E2E test PR branch.

Conflict resolution:
- Accepted main branch version for all source files (Go code, UI, configs)
- Preserved E2E test infrastructure (e2e/ directory and all test files)
- Added e2e, e2e-quick, e2e-debug targets back to Makefile
- Kept .github/workflows/e2e.yml workflow file

New features from main now included:
- Execution statistics storage and dashboard graph
- Plugin health monitoring
- Retry logic with exponential backoff for AgentRun RPC
- Various dependency updates

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Update E2E test to use bare endpoint /v1/jobs/:job (#1897)

* Initial plan

* Update first test to use bare endpoint /v1/jobs/:job

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Fix flaky E2E test using deterministic polling instead of fixed sleep (#1898)

* Initial plan

* Replace sleep 3 with wait_for_execution for deterministic test behavior

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Update dkron/api_test.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Fix syntax error in TestHealthEndpoint causing compilation failure (#1906)

* Initial plan

* Fix syntax error in api_test.go - remove duplicate closing brace

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Fix retry loop skipping when AgentRunMaxRetries is negative (#1905)

* Initial plan

* Fix retry loop to ensure at least one attempt when maxRetries is negative

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Enhance E2E test to validate failed execution success field (#1904)

* Initial plan

* Enhance failed job execution test to assert success=false

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Remove beta dependency by downgrading protoreflect to v1.17.0 (#1903)

* Initial plan

* Downgrade protoreflect from v1.18.0 to v1.17.0 to remove beta v2 dependency

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Add comprehensive E2E test suite for Dkron API (#1907)

* Initial plan

* Merge main branch into E2E test suite PR

Merged latest changes from main branch (commit 2b5f80b) into the E2E test PR branch.

Conflict resolution strategy:
- Accepted main branch version for all source files (Go code, UI, configs)
- Preserved E2E test infrastructure (e2e/ directory and all test files)
- Added e2e, e2e-quick, e2e-debug targets back to Makefile
- Kept .github/workflows/e2e.yml workflow file

New features from main now included:
- Redesigned UI with modern professional styling (#1894)
- Execution statistics storage and dashboard graph (#1896)
- Plugin health monitoring in /health endpoint (#1884)
- Retry logic with exponential backoff for AgentRun RPC (#1882)
- Various dependency updates

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

* Fix unresolved merge conflict in ui/src/layout/index.css

Replaced file with clean version from main branch to remove conflict markers.

Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vcastellm <47026+vcastellm@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants