Lining up par obs ensembles

[wkitlasten]

Sorry, I am struggling with this again. I have read section 9.2.6 and my tiny brain can't quite grok it.

In this particular case I am passing ies_parameter_ensemble, ies_restart_observation_ensemble, and ies_observation_ensemble from a base run (noptmax=0). Some runs were dropped at the start of the HM run due to bad_phi:

The subset idx:pe does not line up with idx:oe (0:1 vs 1:1) even though realization 0 was not dropped and all ensembles have the same number of realizations (same index 0:nreals).

So, how do I convince myself the realization number are properly aligned during the HM runs?

[jtwhite79]

You could tell ies to save lambda ensembles and then use those to check the phi values ies is reporting for a given par realization?

That 0:1 is telling you that for the 45 subset realizations, the first realization that was selected was the realization named "1". The 1:1 is telling you that the second realization in the obs ensemble is named "1"...I'll leave convincing yourself to you! Haha

[wkitlasten]

I suppose some of my confusion comes from the simulated results ensemble vs obs+noise ensemble, or "restart observation ensemble" vs "observation ensemble" in pestpp parlance (usually differentiated by using oe vs ne in the scripts I have seen?). In the context of the subset reporting, oe is the obs+noise ensemble, right?

A specific example of my confusion (from a single run):
iteration 1: idx:pe = 0:61 and idx:oe = 7:61
iteration 3: idx:pe = 7:61 and idx:oe = 58:61

iteration 1: idx:pe = 4:BASE and idx:oe = 44:BASE
iteration 2: idx:pe = 3:BASE and idx:oe = 44:BASE

The names (61 and BASE) line up, so cool!(?). I guess it doesn't matter that the idx changes between iterations, even though realizations (bad_phi) were only dropped in iteration 1?

iteration 1:
...subset idx:pe real name:  **0:61**, 1:116, 2:290, 3:410, _4:BASE_, 5:14, 6:26, 7:31, 8:35, 9:37, 10:81, 11:86, 12:89, 13:90, 
...subset idx:oe real name:  **7:61**, 16:116, 27:290, 38:410, _44:BASE_, 57:14, 67:26, 71:31, 75:35, 76:37, 107:81, 110:86, 

iteration 2:
...subset idx:pe real name:  0:127, 1:283, 2:344, _3:BASE_, 4:5, 5:12, 6:25, 7:42, 8:89, 9:153, 10:154, 11:163, 12:167, 
...subset idx:oe real name:  17:127, 26:283, 32:344, _44:BASE_, 49:5, 55:12, 66:25, 78:42, 113:89, 161:153, 162:154, 

iteration 3 (always pushing the boundaries!):
...subset idx:pe real name:  0:1, 1:6, 2:18, 3:23, 4:36, 5:38, 6:41, **7:61**, 8:72, 9:73, 10:76, 11:78, 12:80, 13:84, 14:109,
...subset idx:oe real name:  1:1, 6:6, 18:18, 23:23, 35:36, 37:38, 39:41, **58:61**, 67:72, 68:73, 70:76, 72:78, 74:80, 77:84,

I confirmed with my sons that I am counting correctly, although the 4 year old was a little incredulous. I passed the same ensemble (produced from the pe) as ies_restart_observation_ensemble and ies_observation_ensemble to produce phi=0, so I am convinced they are all lining up (even though the indexes change, I'm sure there is some computerizing reason).

Thanks for listening.