Skip to content

torch/jax/keras: apply dropout in the monotone stack to match the paper #122

Description

@davorrunje

Context

The base-result bake-off (PR #111) and its spec deferred adding dropout inside the monotone stack to match the original paper (Runje & Shankaranarayana, ICML 2023). The published mononet monotone layers do not apply dropout — grep dropout mononet/torch/layers.py is empty — even though the benchmark config carries a dropout hyperparameter (benchmarks/_common/config.py, tuned by the search). The paper's reference applies dropout within the monotone stack (see the legacy reference mononet/legacy/mono_dense_layer.py:395-396).

Goal

Apply dropout within the monotone stack (matching the paper's placement) across all three backends, so the tuned dropout HP actually acts on the monotone layers.

Where

  • mononet/torch/layers.py, mononet/jax/layers.py, mononet/keras/layers.py — the public monotone layers (MonoLinear/MonoDense, MonoResidual).
  • mononet/core/reference.py — NumPy reference (must stay the arithmetic ground truth; dropout is train-time only, so reference/equivalence at eval must be unaffected).
  • Legacy reference for placement: mononet/legacy/mono_dense_layer.py:395-396.

Proposed approach

Add train-time dropout in the monotone stack matching the paper's location. Keep it inactive at eval so the cross-backend equivalence harness (tests/equivalence/) still matches the reference within tolerance.

Acceptance criteria

  • Dropout applied in the monotone stack per the paper, in all three backends.
  • Cross-backend equivalence tests still pass (eval-time behavior unchanged).
  • A unit test asserts dropout is active in train mode and inactive in eval mode.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfollow-upDeferred work captured as a self-contained issue

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions