Skip to content

judge: support score-based subtask dependencies#1192

Open
gtn1024 wants to merge 1 commit into
hydro-dev:masterfrom
gtn1024:feat/score-based-subtask-dependencies
Open

judge: support score-based subtask dependencies#1192
gtn1024 wants to merge 1 commit into
hydro-dev:masterfrom
gtn1024:feat/score-based-subtask-dependencies

Conversation

@gtn1024

@gtn1024 gtn1024 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Motivation

The existing if dependency is based on acceptance status. This does not cover partial-scoring checkers, where a subtask can receive a positive final score without being accepted and that score should still be able to unlock a dependent subtask.

Summary

Add score-based subtask dependencies through a new if_score configuration field.

A subtask with if_score is judged only when every referenced subtask has a final score greater than zero.

subtasks:
- id: 1
  # ...

- id: 2
  if_score: [1]
  # ...

Here, subtask 2 is judged when subtask 1 receives a positive score, including partial credit. It is skipped when subtask 1 receives zero points.

Summary by CodeRabbit

  • New Features
    • Added positive-score dependencies for subtasks via if_score, enabling subtasks to run only when selected prerequisite subtasks achieve scores greater than 0.
    • Updated scheduling to respect score dependencies and detect circular dependency configurations.
    • Extended UI and editor/YAML/schema support to configure both dependency types.
  • Documentation
    • Added Simplified Chinese translations for the new positive-score dependency settings.

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Adds the optional if_score subtask configuration across shared types, normalization, schema, YAML formatting, and the problem configuration editor. The judging flow resolves score-dependent subtasks in dependency order, skips subtasks whose prerequisites do not qualify, detects circular dependencies, and aggregates effective scores. The editor supports configuring both regular and positive-score dependencies with localized labels.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding score-based subtask dependency support to judging.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install timed out. The project may have too many dependencies for the sandbox.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gtn1024

gtn1024 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@gtn1024
gtn1024 force-pushed the feat/score-based-subtask-dependencies branch from 5e0b297 to efdc955 Compare July 21, 2026 07:48

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/hydrojudge/src/flow.ts (1)

98-110: 🚀 Performance & Scalability | 🔵 Trivial | ⚖️ Poor tradeoff

Consider Promise-chaining to maximize queue throughput.

The current topological sort schedules subtasks layer-by-layer, awaiting Promise.all(ready.map(...)) before advancing to the next graph depth. If one subtask in a layer takes significantly longer than the others, it will stall execution of the next layer. If the judge is configured with singleTaskParallelism > 1, this will leave processing slots idle.

Consider mapping each subtask to a localized Promise that directly awaits its specific dependencies. This ensures independent subtasks execute as soon as their unique prerequisites are met, maximizing queue throughput. (A quick DFS or the existing while loop can still be run beforehand to detect cycles).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/hydrojudge/src/flow.ts` around lines 98 - 110, Replace the
layer-wide Promise.all scheduling in the subtask execution flow with per-subtask
promises that await only the promises for each subtask’s declared dependencies
before invoking judgeSubtask. Preserve the existing dependency acceptance and
score checks, failure handling, and cycle detection (the current traversal may
remain as a preliminary validation), while allowing independent subtasks to
start as soon as their own prerequisites complete.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/hydrojudge/src/flow.ts`:
- Line 103: Update the if_score dependency evaluation in the scored expression
so non-existent subtask IDs are ignored, matching the accepted check’s behavior.
Guard the infos[id]?.score comparison with the corresponding subtask existence
check, while preserving the requirement that existing dependencies must have a
positive score.

---

Nitpick comments:
In `@packages/hydrojudge/src/flow.ts`:
- Around line 98-110: Replace the layer-wide Promise.all scheduling in the
subtask execution flow with per-subtask promises that await only the promises
for each subtask’s declared dependencies before invoking judgeSubtask. Preserve
the existing dependency acceptance and score checks, failure handling, and cycle
detection (the current traversal may remain as a preliminary validation), while
allowing independent subtasks to start as soon as their own prerequisites
complete.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f68677e1-14a3-4cfb-9eeb-084631acb63d

📥 Commits

Reviewing files that changed from the base of the PR and between 5e0b297 and efdc955.

📒 Files selected for processing (9)
  • packages/common/subtask.ts
  • packages/common/types.ts
  • packages/hydrojudge/src/flow.ts
  • packages/hydrojudge/src/judge/run.ts
  • packages/ui-default/components/monaco/schema/problemconfig.ts
  • packages/ui-default/components/problemconfig/ProblemConfigEditor.tsx
  • packages/ui-default/components/problemconfig/reducer/config.ts
  • packages/ui-default/components/problemconfig/tree/SubtaskSettings.tsx
  • packages/ui-default/locales/zh.yaml
🚧 Files skipped from review as they are similar to previous changes (7)
  • packages/common/types.ts
  • packages/ui-default/components/monaco/schema/problemconfig.ts
  • packages/hydrojudge/src/judge/run.ts
  • packages/ui-default/locales/zh.yaml
  • packages/ui-default/components/problemconfig/ProblemConfigEditor.tsx
  • packages/common/subtask.ts
  • packages/ui-default/components/problemconfig/tree/SubtaskSettings.tsx

const accepted = (subtask.if || []).every((id) => (
!subtasks[id] || (infos[id] && infos[id].status <= STATUS.STATUS_ACCEPTED)
));
const scored = (subtask.if_score || []).every((id) => infos[id]?.score > 0);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Ignore non-existent subtasks in if_score dependencies.

If an if_score dependency references a non-existent subtask ID, infos[id] will be undefined, causing scored to evaluate to false and unintentionally fail the subtask. In contrast, the accepted check (lines 100-101) correctly ignores non-existent subtasks (!subtasks[id] || ...).

To maintain consistent behavior and prevent typographical errors in the problem configuration from failing valid subtasks, if_score should also gracefully ignore invalid subtask IDs.

🐛 Proposed fix
-            const scored = (subtask.if_score || []).every((id) => infos[id]?.score > 0);
+            const scored = (subtask.if_score || []).every((id) => !subtasks[id] || infos[id]?.score > 0);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const scored = (subtask.if_score || []).every((id) => infos[id]?.score > 0);
const scored = (subtask.if_score || []).every((id) => !subtasks[id] || infos[id]?.score > 0);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/hydrojudge/src/flow.ts` at line 103, Update the if_score dependency
evaluation in the scored expression so non-existent subtask IDs are ignored,
matching the accepted check’s behavior. Guard the infos[id]?.score comparison
with the corresponding subtask existence check, while preserving the requirement
that existing dependencies must have a positive score.

@undefined-moe

Copy link
Copy Markdown
Member

Maybe it's better to use an expression parser:

if:
- '$1 > 0'
- '$2 > 10'
- '$3 == 0'

cc @moesoha

@moesoha

moesoha commented Jul 23, 2026

Copy link
Copy Markdown

i think the expression is too complex and is not meaningful.

we already have statuses on subtasks, and the status can be used to make that decision. from my point of view, if_ac: [task id] is enough for such a need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants