Draft
Conversation
Replace legacy time-index + month embeddings with a multi-frequency sinusoidal temporal encoding based on fractional year. The encoding uses geometric-spaced frequencies from ~128-year periods to daily resolution, making it slot-position invariant (same calendar date always gets the same encoding regardless of time slot index). - Add TimestampEncodingMode enum and get_static_temporal_encoding() - Thread timestamp_encoding_mode through CompositeEncodings, FlexiVitBase, Encoder, PredictorBase, EncoderConfig, PredictorConfig - Add experiment script (vnext base_band_dropout variant) - Add 8 unit tests covering shape, determinism, forward pass, and legacy compat Made-with: Cursor
- Precompute temporal frequencies as a registered buffer instead of recomputing linspace/exp on every forward pass - Use TimestampEncodingMode enum as single source of truth for config validation instead of duplicated string literals - Remove duplicate self.embedding_size assignment - Fix docstring for get_static_temporal_encoding (clarify approximate DOY) - Use fixed timestamps in test_static_temporal_differs_from_legacy - Add edge case tests: odd dim assertion, invalid mode, freqs buffer shape Made-with: Cursor
Simpler implementation: get_static_temporal_encoding now takes encoding_dim and computes frequencies inline rather than requiring a precomputed buffer. The computation is trivial (linspace on ~64 elements) so caching provides no meaningful benefit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…device bug Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
static_temporalencoding mode that replaces legacy time-index + month embeddings with multi-frequency sinusoidal encoding based on fractional yeartimestamp_encoding_modethroughCompositeEncodings,FlexiVitBase,Encoder,PredictorBase,EncoderConfig,PredictorConfig"legacy"so all existing code/checkpoints are unaffectedTest plan
Made with Cursor