Skip to content

Fix GCN validation and test evaluation - #1442

Open
tandede wants to merge 1 commit into
ml-explore:mainfrom
tandede:fix/gcn-evaluation-mode
Open

Fix GCN validation and test evaluation#1442
tandede wants to merge 1 commit into
ml-explore:mainfrom
tandede:fix/gcn-evaluation-mode

Conversation

@tandede

@tandede tandede commented Aug 19, 2026

Copy link
Copy Markdown

Summary

  • evaluate GCN validation and test metrics with the current model parameters in evaluation mode
  • restore the model's prior training state after each evaluation
  • stop returning full training logits once they are no longer reused
  • add a regression test for evaluation mode and state restoration

Problem

The training step returned logits produced before optimizer.update and while Dropout was active. The validation path reused those stale, stochastic logits for early stopping. The test path computed a fresh test_y_hat, but then accidentally calculated both metrics from the old training-loop y_hat instead.

The new evaluate helper temporarily disables training behavior, performs a fresh forward pass with the current parameters, materializes the metrics, and restores the caller's original train/eval state.

Validation

  • python gcn/test.py: 1 test passed
  • python gcn/main.py --epochs 5 --patience 5 --hidden_dim 8 --nb_layers 1 --dropout 0.5: completed Cora training and final evaluation
  • pre-commit run --files gcn/main.py gcn/test.py: Black and isort passed
  • python -m compileall -q gcn: passed

In a seeded five-step reproduction, the old pre-update training-mode output reported test loss/accuracy of 1.955827 / 0.199, while a fresh evaluation-mode pass with the final parameters reported 1.928995 / 0.236; the maximum logit difference was 0.410258.

@tandede
tandede marked this pull request as ready for review August 19, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant