Skip to content

Generalize Random feature defaults for greater stability#375

Merged
odunbar merged 13 commits intomainfrom
orad/RF-stabilize
Sep 23, 2025
Merged

Generalize Random feature defaults for greater stability#375
odunbar merged 13 commits intomainfrom
orad/RF-stabilize

Conversation

@odunbar
Copy link
Member

@odunbar odunbar commented Aug 18, 2025

Purpose

Closes #368

Content

  • Removed learning of "sigma" which is not stably learnt!
  • Applied some data standardization to ensure reasonable prior ranges, then chose some defaults that stay in this range
  • Helped scale defaults with 1/sqrt(dimension) and 1/rank, for more general performance
  • Allow for users to intentionally override overfit regularization (if over-regularization appears during optimization)
  • Change default covariance regularization from "shrinkage" to "nice" (in ScalarRF)
  • Changed (Corrected?) the "LR" factorization R*D*R' + eps*I to "LR plus D" R*R' + (D+eps)*I to rely less on the definiteness correction nugget eps
  • Updated unit tests for latest EKP.
  • We already can incorporate tikhonov with the prior distribution. therefore I have removed all mentions of other tikhonov regularization. the existing code was untested and not used anywhere.

MISC

  • Added packages into Cloudy_emulate_sample that are missing.

Cloudy results (RF-scalar, RF-nonsep, gp-gpjl)

pairplot_posterior_constr_gp-gpjl pairplot_posterior_constr_rf-nonsep pairplot_posterior_constr_rf-scalar

Regression 2d-2d results (true, RF-scalar, RF-nonsep, gp-skljl)

Plotting only 1st output component
g1_true rf-lr-scalar_no-proc_y1_predictions
rf-nonsep_no-proc_y1_predictions gp-skljl_no-proc_y1_predictions

L63 (RF-diag-scalar, RF-nonsep, gp-gpjl)

CDFs of long time trajectories (over 5 repeats for RF)
rf-diag-scalar_l63_cdfs RF-nonsep_l63_cdfs GP_l63_cdfs


  • I have read and checked the items on the review checklist.

new verbose flags and default data minmax-like data scaling

update regression example

update ishigami example

update g-function example

update Lorenz spatial dep example

missing verbose flag

typo

vectorising scalings

format

Cloudy example add required pkgs

Updated LR factorization to LR + D

reduce RF nonsep

removed sigma learning, adjust prior ranges

tweak examples for new priors

fix plots bug

tweak setups for RF-scalar and nonsep

add edge-case protection for 1D->1D

format
@codecov
Copy link

codecov bot commented Sep 20, 2025

Codecov Report

❌ Patch coverage is 90.32258% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.61%. Comparing base (14e03ab) to head (1e0fca1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/RandomFeature.jl 91.93% 5 Missing ⚠️
src/VectorRandomFeature.jl 86.11% 5 Missing ⚠️
src/ScalarRandomFeature.jl 92.30% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #375      +/-   ##
==========================================
+ Coverage   90.98%   93.61%   +2.62%     
==========================================
  Files          10       10              
  Lines        1609     1628      +19     
==========================================
+ Hits         1464     1524      +60     
+ Misses        145      104      -41     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@odunbar odunbar changed the title [WIP]: Generalize Random feature defaults for greater stability Generalize Random feature defaults for greater stability Sep 22, 2025
@odunbar
Copy link
Member Author

odunbar commented Sep 23, 2025

I am satisfied by the consistent examples/ stability improvements. Merging on successful tests and coverage.

@odunbar odunbar merged commit 101d3f6 into main Sep 23, 2025
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate/improve changes in RF performance under new toolings

1 participant