Skip to content

era5 encoder#577

Draft
Farbum wants to merge 32 commits into
mainfrom
hadriens/era5_encoder
Draft

era5 encoder#577
Farbum wants to merge 32 commits into
mainfrom
hadriens/era5_encoder

Conversation

@Farbum

@Farbum Farbum commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Era5 encoder pretraining strategy with 3 objectives:

  • Objective A: supervised loss on multiple tasks
  • Objective B: reconstruction loss on SWT
  • Objective C: forecast of s2 in latent space

Hadrien Sablon and others added 21 commits June 15, 2026 14:24
learned masked embeddings + STEM tokenizer with more capacity
fdc6edd
minor renaming
7ca7011
Merge remote-tracking branch 'origin/main' into hadriens/era5_encoder
972a422
adding burn risk and lfmc eval tasks
66884b8
adding z-norm for regression tasks
be12366
Merge branch 'hadriens/era5_encoder' of https://github.com/allenai/ol…
29c1b5f 
…moearth_pretrain into hadriens/era5_encoder
fix to allow local beaker job launch
abda5b0
refactoring recon loss WIP + minor update on evals to debug
867af71
cleansed and improved 2-stage reconstruction policies and loss
73a293e
448 input sequence + unit tests + naive masking baseline + minor swt …
963ddc3 
…refactoring
minor cleaning update + unit tests
4e854a7
making naivemaskpolicy a subclass of maskpolicy to accomodate for olm…
716bf6b 
…o constraints
Enable torch.compile and increase microbatch size for faster training…
b673497 
…; update ERA5 layer configs and simplify optimizer state dict
reconstruction clean up + exposing knobs
6ebdd73
fixing regression evals for era5 with width/height shape requirements
1b5e11a
killed unecesssary warnings in logs + fixed burnrik_eval in registry …
6de583a 
…for the training set
Snapshot model YAMLs to save_folder at launch time and make registry …
2ec4b69 
…paths relative so jobs survive branch switches and repo relocation
fixed shape/dim problem for nbac eval once and for all
e187ef2
fixed lint problem on unit tests
f3ad139
minor readability updates + rescaling losses per variable so that swt…
bc415c8 
…_lambda=1 gives identical weight to raw vs swt loss
exposing num_mask knob for all masking strategies + control knob for …
7f0d44f 
…raw reconstruction (lambda_raw)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

size/xl

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Farbum