Skip to content

RTide 1.0.0

Latest
Latest

Choose a tag to compare

View all tags
@thomasmonahan thomasmonahan released this 02 Feb 09:37
v1.0.0
31e1f42

RTide 1.0.0 Release Notes

Major Release: RTide 0.1.0 → 1.0.0

RTide 1.0.0 represents a significant evolution of the package with major new features, performance improvements, and architectural enhancements while maintaining full backward compatibility with 0.1.0.

🚀 Major New Features

1. Current Velocity Support (u/v Components)

Previous: RTide only supported scalar sea level elevation
Now: Full support for 2D current velocities

# Current velocity analysis
df = pd.DataFrame({'u': u_data, 'v': v_data}, index=times)
model = RTide(df, lat=42.0, lon=-70.0)
model.Prepare_Inputs()
model.Train()
model.Predict(future_df)

Features:

  • Automatic detection of elevation vs. currents based on column names
  • Multi-output neural network architecture for simultaneous u/v prediction
  • UTide integration for both elevation and currents
  • Visualization tools handle both modes automatically
  • SHAP analysis supports multi-output models

Impact: RTide now handles the full range of tidal applications (elevation and currents)

2. Simultaneous Trend Estimation 🆕

Major Feature: Estimate linear or quadratic trends during training (not post-hoc)

# Linear trend (e.g., sea level rise)
model.Train(trend='linear', standard_epochs=500)

# Quadratic trend (e.g., accelerating sea level rise)
model.Train(trend='quadratic', standard_epochs=500)

# No trend (legacy behavior)
model.Train(trend=None)

How It Works:

  • Dual-input architecture: [forcing_features, normalized_time]
  • Polynomial warm-start initialization using np.polyfit()
  • TrendLayer learns coefficients via backpropagation
  • Automatic time normalization [0, 1] over training period
  • Proper extrapolation beyond training period

Benefits:

  • Simultaneous learning improves trend-tide separation
  • Better generalization than post-hoc detrending
  • Transparent handling in predictions
  • Polynomial warm-start ensures robust convergence

3. SIREN Architecture Support 🆕

New Architecture: Sinusoidal Representation Networks (SIREN)

# SIREN architecture (sine activations)
model.Train(architecture='siren', siren_w0=30.0)

# Response architecture (tanh activations, default)
model.Train(architecture='response')

Features:

  • Sine activation functions: sin(w0 * (Wx + b))
  • SIREN-specific weight initialization
  • Compatible with trend estimation
  • Configurable frequency parameter (w0) -- recommend not changing this.

Benefits:

  • Better representation of nonlinear tides in some cases.
  • Can capture high-frequency tidal components
  • Alternative to standard tanh networks
  • Shown to be useful for fast-moving tidal currents.

4. Fixed Location Mode & Precomputed Inputs ⚡

Major Performance Enhancement: 10-100x speedup for input computation

Fixed Location Mode

model = RTide(df, lat=42.0, lon=-70.0)
model.location_mode = 'fixed'
model.fixed_lat = 45.0
model.fixed_lon = 0.0
model.Prepare_Inputs()

Benefits:

  • Forcing computed at fixed reference location (not station-specific)
  • Enables precomputation and caching
  • No loss in accuracy for most applications
  • Significant speedup for multi-station analysis

Precomputed Input Cache

model = RTide(df, lat=42.0, lon=-70.0)
model.location_mode = 'fixed'
model.use_precomputed_inputs = True
model.Prepare_Inputs()

Features:

  • Gravitational/radiational forcing precomputed on 6-minute grid
  • Compressed storage (float16 in .npz format)
  • Automatic temporal interpolation for any frequency
  • User-configurable cache directory
  • Automatic cache generation on first use

Performance:

Legacy (station mode):  ~30-60 seconds for 1 year hourly
Fixed mode (no cache):  ~10-15 seconds
Fixed mode (cached):    ~1-2 seconds ⚡

Impact: Near-instantaneous input preparation after initial cache generation

5. Enhanced SHAP Analysis

Improvement: SHAP now works seamlessly with all model types

# Works with all configurations
model.Train(trend='linear', architecture='siren')
model.Shap_Analysis(plot=True, output_index=0)

Features:

  • Automatic detection of model type (trend vs. no-trend)
  • Intelligent handling of dual-input models
  • Time held constant at median value for trend models
  • Multi-output support (analyze u or v separately)
  • Clear interpretation of forcing effects

Technical:

  • For trend models: analyzes forcing features at representative time point
  • Prevents issues with SHAP's perturbation sampling
  • Maintains backward compatibility with legacy models

🔧 Architecture & Code Improvements

Model Building Refactor

Centralized Architecture:

# All model building now through factory function
model = models.build_model(
    architecture='siren',
    input_dims=152,
    n_outputs=2,
    hidden_nodes=152,
    depth=3,
    trend='linear',
    trend_init_coeffs={'slope': 0.003, 'intercept': 0.01}
)

Benefits:

  • Single entry point for all architectures
  • Consistent API across model types
  • Easier to add new architectures
  • Better code organization

Custom Keras Layers

New Serializable Layers:

  • SineLayer - SIREN sine activation layer
  • SineActivation - Standalone sine activation
  • TrendLayer - Polynomial trend estimation
  • Proper serialization with @tf.keras.utils.register_keras_serializable

Benefits:

  • Models save/load correctly with custom layers
  • No manual custom_objects needed for standard use
  • Full Keras ecosystem compatibility

Input/Output Schema Inference

Automatic Detection:

# Elevation (single column named 'observations')
df = pd.DataFrame({'observations': data}, index=times)

# Currents (columns 'u' and 'v')
df = pd.DataFrame({'u': u_data, 'v': v_data}, index=times)

# RTide automatically detects and configures
model = RTide(df, lat, lon)  # Auto-detects schema

🐛 Critical Bug Fixes

1. Input Saving in Fixed Location Mode

Bug: Pickle file not saved when using location_mode='fixed'
Impact: Predict() failed with FileNotFoundError
Fix: Inputs now saved for all location modes

2. Trend Initialization

Bug: TrendLayer weights initialized to zeros → never learned
Fix: Polynomial warm-start initialization from np.polyfit()

3. SHAP with Trend Models

Bug: SHAP failed on dual-input trend models
Fix: Intelligent wrapper provides both inputs to model

⚙️ API Enhancements

Prepare_Inputs Configuration

More Flexible:

model.Prepare_Inputs(
    location_mode='fixed',           # NEW: 'station' or 'fixed'
    use_precomputed_inputs=True,     # NEW: Use cached inputs
    precomputed_cache_dir='~/.cache/rtide',  # NEW: Cache location
    fixed_lat=45.0,                  # NEW: Reference latitude
    fixed_lon=0.0,                   # NEW: Reference longitude
    uniform_lags=[1, 12],            # Existing
    symmetrical=True,                # Existing
    radiational=True,                # Existing
)

Train Configuration

Extended Options:

model.Train(
    architecture='siren',            # NEW: 'response' or 'siren'
    siren_w0=30.0,                  # NEW: SIREN frequency
    trend='linear',                  # NEW: 'linear', 'quadratic', or None
    featurewise_scaling=False,       # NEW: Feature-wise X scaling
    standard_epochs=500,             # Existing
    lr=0.0001,                       # Existing
    regularization_strength=0.001,   # Existing
)

📊 Performance Improvements

Input Computation Speed

Configuration Time (1 year hourly) Improvement
0.1.0 (station mode) 30-60s baseline
1.0.0 (station mode) 30-60s same
1.0.0 (fixed, no cache) 10-15s 3-4x faster
1.0.0 (fixed, cached) 1-2s 20-30x faster ⚡

Training Stability

  • Polynomial warm-start → faster convergence for trend models
  • Better weight initialization → fewer epochs needed
  • More robust to hyperparameter choices

📈 Enhanced Visualizations

Multi-Output Support

Visualizations now handle both elevation and currents:

# Automatically adapts to data type
model.Visualize_Predictions(verbose=True)
model.Visualize_Residuals()

For currents:

  • Separate lines for u and v components
  • Magnitude and direction plots
  • Component-wise statistics

Trend Visualization

New output during training (when trend specified):

#### Fitting Initial Linear Trend ####
  Initial linear trend: y = 0.002543 * t + -0.015432
  Estimated trend change over training period: 0.0025 units
  Note: Coefficients will be refined during training via backpropagation

📝 Documentation

New Documentation

  • TREND_ESTIMATION_DOCUMENTATION.md - Complete trend feature guide
  • POLYNOMIAL_WARMSTART_DOCUMENTATION.md - Initialization details
  • SHAP_WITH_TRENDS_DOCUMENTATION.md - SHAP analysis guide
  • QUICK_REFERENCE.md - Quick usage examples
  • Comprehensive inline documentation

Updated Documentation

  • README with new examples
  • API reference updated
  • Installation instructions
  • Troubleshooting guide

🔄 Migration from 0.1.0

Backward Compatibility

✅ 100% backward compatible - All 0.1.0 code runs unchanged on 1.0.0

# 0.1.0 code works identically in 1.0.0
model = RTide(df, lat=42.0, lon=-70.0)
model.Prepare_Inputs()
model.Train()
model.Predict(test_df)

Taking Advantage of New Features

Minimal changes to use new features:

# Add trend estimation
model.Train(trend='linear')

# Use SIREN architecture
model.Train(architecture='siren')

# Enable fast mode
model.location_mode = 'fixed'
model.use_precomputed_inputs = True
model.Prepare_Inputs()

# Handle currents
df = pd.DataFrame({'u': u_data, 'v': v_data}, index=times)
model = RTide(df, lat, lon)  # That's it!

🎯 Use Cases Enabled

1. Current Velocity Prediction

# Full 2D current analysis
df = pd.DataFrame({'u': u_obs, 'v': v_obs}, index=times)
model = RTide(df, lat, lon)
model.Train()

2. Multi-Station Studies

# Process 100s of stations efficiently
model.location_mode = 'fixed'
model.use_precomputed_inputs = True
# Near-instant input preparation for each station

3. High-Frequency Tidal Analysis

# SIREN better captures high-frequency components
model.Train(architecture='siren', siren_w0=30.0)

5. Sea Level Rise Analysis

# Simultaneously estimate tides and long-term trend
model.Train(trend='linear', standard_epochs=500)
# Trend learned during training, not post-hoc

📦 Dependencies

Updated Requirements

tensorflow>=2.0
pandas>=1.0
numpy>=1.18
scikit-learn>=0.22
matplotlib>=3.0
utide>=0.3
shap>=0.40
skyfield>=1.30
joblib>=1.0

Note: No new dependencies added, only version updates

🚧 Known Limitations

1. Trend-Y Scaling Interaction

Trend coefficients initialized from raw data but train in scaled space. Polynomial warm-start handles this well, but users should be aware.

2. Precomputed Cache Size

Cache files can be 10-100 MB depending on time range and resolution. Users can configure cache location and manually delete if needed.

3. SIREN Learning Rate Sensitivity

SIREN may require tuning of learning rate and regularisation for optimal performance on specific datasets.

🔮 Future Directions

Potential enhancements for future releases:

  • Multi-location models
  • GPU acceleration for large datasets
  • Uncertainty quantification
  • Real-time prediction API

📄 License

RTide 1.0.0 uses a business source license but remains freely available for academic and non-commercial use.

🔗 Links

🎉 Summary

RTide 1.0.0 is a major release that:

  • ✅ Adds current velocity support (2D u/v)
  • ✅ Implements simultaneous trend estimation (linear/quadratic)
  • ✅ Provides SIREN architecture option
  • ✅ Achieves 20-30x speedup with precomputed inputs
  • ✅ Enhances SHAP analysis for all model types
  • ✅ Maintains 100% backward compatibility
  • ✅ Fixes critical bugs from 0.1.0

Recommended for all users - users with operational code-bases should upgrade with care.

Installation:

pip install rtide==1.0.0

Upgrade from 0.1.0:

pip install --upgrade rtide
Assets 2
Loading