Transfer of Observations to CES (particular focus on high-dimensional covariances)

[odunbar]

Purpose

• Closes Extend new data processing tools to handle the high-dimensional output covariance structures from EKP #366

Content

For dimension reduction

• Add encoder_kwargs_from(...) utility to easily get the obs_noise_cov,observation, prior_cov etc. for dimension reduction in the correct forms (extensible for the likelihood informed quantities)
• Conversion of all StructureMatrix objects into LinearMap objects.
• Change svd into truncated/iterative svd operations for application to Linear Maps. Implement new total-variance estimation to truncate with retain_var in situations where not all s.v.s are computed
• Ensure that StructureMatrices continue to be LinearMaps under repeated application
• Added unit tests that show framework can be applied to (lower-rank) 10^5 dimensional covariances on observations and many different types of observational covariances.
• Adds TSVD, LowRankApprox, LinearMaps to project svds for larger matrices
• When interfacing with an emulator for regularlization the encoded structure matrices are converted to matrices again
• When decoding, (e.g. in predict(...,transform_to_real=true)) the encoded structure matrices are decoded and provided back as full matrices

For MCMC

• Allow passing of Observations, and ObservationSeries objects to MCMC
• Unit tests

---

• I have read and checked the items on the review checklist.

[codecov]

Codecov Report

❌ Patch coverage is 96.72131% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.24%. Comparing base (779854a) to head (ac0fc62).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/Utilities.jl	95.68%	5 Missing ⚠️
src/Utilities/decorrelator.jl	96.10%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #384      +/-   ##
==========================================
+ Coverage   94.09%   94.24%   +0.15%     
==========================================
  Files          10       10              
  Lines        1625     1807     +182     
==========================================
+ Hits         1529     1703     +174     
- Misses         96      104       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ArneBouillon]

LGTM