Skip to content

Commit 3493fab

Browse files
jeremy-baierHazboun6Joseph SimonJeremy Baierblarsen10
authored
update: chromatic noise models & feat: hypermodel empirical distributions (#242)
* Added sunssb calculatin to theta impact. * added to theta_impact docstring * Added sunssb arg to functions. * Added sunssb arg to solar_wind * Added sunssb arg to create_fourier_designmatrix_sw * Added dm_solar_wind_r_to_p * Added flag to be able to use log10_ne * Added flag to model2a, model3a, modelgen to turn off rn model. * Merged with master 2. * Added option for dropout of intrinsic red noise. * Added dropout params to docstring * More merging * More merging * More merging * No changes. * space deletion * Tweaks to WN model for CHIME ECORR * Adding option to set dt for linear_interp_basis in single_pulsar_model for dm_gp and chrom_gp * Fixed syntax * Update binned sw model. * Update binned sw model 2. * Update binned sw model 3. * Update binned sw model 4. * Update binned sw model 5. * Update binned sw model 6. * Update binned sw model 7. * Update binned sw model 8. * Update binned sw model 9. * Added combine flag to red noise blocks * set zero to 1e-40 * Added model 4a * fixed typo with rn in model general * fixed typo with ss in model general * fix typ in chromatic blk * fix typ in chromatic blk * adding broken powerlaw to model_general * fix delta_common flag * more conflict resolution * fix merge conflict common_red * add gw_crn to sampler flags * fix sampler issue * fixed broken power law * make fbend prior based on Tspan * added general linear interpolant basis * further tweaks to general linear inter * change names in model_general * added white noise dropout models * fixed variables * add cov.npy load in * small tweaks to cusp model * starting to add new emp dist jump prop * playing with new jump proposal * small work on jp * adding pulsar based jump props * adding pulsar based jump props 2 * fix psr emp dist jp * fix psr emp dist jp 2 * adding new chromatic gp JP * revert to standard dropout_rn * up the coverage * adding pulsars with sunssb attr * various solar wind upgrades * linting * tests for new solar wind functions * fix to signal * rm wn_dropout to another branch * linting * matching WN models * multiple dt definitions * rm names from docstring * multiple dt definitions * f bend frequency range * added option for tnequad, default t2equad * move to enterprise-v3.3.0 for T2EQUAD support * added vary flag for DM and chrom models * added vary flag to annual dm sinusoid * fixing gw log uniform draw for models with size!=0 * raise lower prior on periodic log10_p * tweak priors on chrom gp params * tweak priors on chrom gp params 2 * tweak to auto jump proposals dm_gp * adding joe's fix * added checks for fixed point * const dmgp fix for jump proposals additions * fix elif in chromatic block for fixed point * adding Nfreqs options for freq DM/Chrom GP * add varying chromatic index ability * move chromatic modeling out of dm * edit docstring * change docstring * fix typo * add Tspan kwarg to dmgp and chromgp * add ridge prior dmx-like GP to chromatic GP block * added chrom_gp to setup sampler JP auto adds * adding the ridge rename * starting the custom jump proposal for HM empirical distribution * hm emp distr JP * hm emp bug fixes * tryna fix da hypermodel * closing bracket * closing bracket correctly * closing next bracket * another fix for the for loop * another fix for the for loop2 * changed it in lnprior too * tryna rename ridge pt 2 * yonk * yonk * fixing some broken things * automatically save hm weights to chain directory * just a comment * changing jump prop * adding bjorns changes to move the log_weights to the PRIOR instead of the lnlike in order to not bias the PTMCMC * fixing something in the print for hypermodel weights * add: add gp_ecorr kwarg to model single puslar noise * reformat: used black on sampler.py * Revert "reformat: used black on sampler.py" This reverts commit 33d2b26. * reformat: passing flake for sampler.py * reformat: update code to pass flake8 for blocks.py * reformat: update code to pass flake8 for chromatic.py * reformat: update code so that hypermodel.py is passing flake8 * reformat: update code so that models.py is passing flake8 * update tests: updated the tests so that less tests will be breaking * add kwarg: added variable upper prior limit to chrom_gp_idx. default is to 7. * add kwarg: added variable upper prior limit to chrom_gp_idx. default is to 7. * fixed linting * add VEGAS to channelized backends * tests: add and fix tests; mend blocks accordingly * fix: add maximum scipy to avoid deprecation error in tests * update: update ci_test.yml * add Nitu+2024 SW model + test * attempt to fix accidental undoing of changes to gp_kernels and tests * linting, imports * fixed bug in gp_ecorr kwarg * Update setup.py Add the limit on the `scipy` version * changing gp ecorr to be false by default * 500-->50000 * remove upper bound on scipy requirements * fix deprecation warning in pytests * add partial caching of the chromatic fourier designmatrix, tests * attemping to lint w/ copilot * copilot got most of the way there * refactor solar wind noise block * make sure solar_wind is visible when importing chromatic module * error handling for missing swgp basis * allow option to vary swgp or hold its hyperparams fixed * model_utils update: fix underscore in mask_filter * changing solar wind prior * bug fix in the solar wind block. components should be 2 in the powerlaw prior -- not the number of fourier modes * flake * bug fix: fix things which were borked in the tests * address paul baker's comments on the MR * fix prior draws * lint * fix double chromatic noise block in model single pulsar noise * add idx as a constant parameter in chromatic noise block so that it gets logged to the PTA summary * update sw block in model single puslar noise; add better doc strings to sw nosie block * fix tests for solar wind block and model single pulsar noise * fix test to use const.yr --------- Co-authored-by: Hazboun6 <jeffrey.hazboun@gmail.com> Co-authored-by: Joseph Simon <joseph.simon@pcdev3.nemo.uwm.edu> Co-authored-by: Jeremy Baier <baierj@rogue.tail1dab3.ts.net> Co-authored-by: Bjorn Larsen <bjorn.larsen@yale.edu> Co-authored-by: Jeremy Baier <jeremybaier@nanograv.org>
1 parent 4e1ce07 commit 3493fab

11 files changed

Lines changed: 887 additions & 349 deletions

File tree

enterprise_extensions/blocks.py

Lines changed: 178 additions & 60 deletions
Large diffs are not rendered by default.

enterprise_extensions/chromatic/chromatic.py

Lines changed: 159 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
import numpy as np
44
from enterprise import constants as const
5-
from enterprise.signals import deterministic_signals, parameter, signal_base
5+
from enterprise.signals import deterministic_signals, parameter, signal_base, gp_bases
66

77
__all__ = [
88
"chrom_exp_decay",
@@ -17,6 +17,8 @@
1717
"dm_dual_exp_cusp",
1818
"dmx_signal",
1919
"dm_annual_signal",
20+
"construct_chromatic_cached_parts",
21+
"createfourierdesignmatrix_chromatic_with_additional_caching"
2022
]
2123

2224

@@ -237,7 +239,7 @@ def dmx_delay(toas, freqs, dmx_ids, **kwargs):
237239
return wf
238240

239241

240-
def dm_exponential_dip(tmin, tmax, idx=2, sign="negative", name="dmexp"):
242+
def dm_exponential_dip(tmin, tmax, idx=2, sign="negative", name="dmexp", vary=True):
241243
"""
242244
Returns chromatic exponential dip (i.e. TOA advance):
243245
@@ -249,19 +251,30 @@ def dm_exponential_dip(tmin, tmax, idx=2, sign="negative", name="dmexp"):
249251
:param sign:
250252
set sign of dip: 'positive', 'negative', or 'vary'
251253
:param name: Name of signal
254+
:param vary: Whether to vary the parameters or use constant values.
252255
253256
:return dmexp:
254257
chromatic exponential dip waveform.
255258
"""
256-
t0_dmexp = parameter.Uniform(tmin, tmax)
257-
log10_Amp_dmexp = parameter.Uniform(-10, -2)
258-
log10_tau_dmexp = parameter.Uniform(0, 2.5)
259-
if sign == "vary":
259+
if vary:
260+
t0_dmexp = parameter.Uniform(tmin, tmax)
261+
log10_Amp_dmexp = parameter.Uniform(-10, -2)
262+
log10_tau_dmexp = parameter.Uniform(0, 2.5)
263+
else:
264+
t0_dmexp = parameter.Constant()
265+
log10_Amp_dmexp = parameter.Constant()
266+
log10_tau_dmexp = parameter.Constant()
267+
268+
if sign == "vary" and vary:
260269
sign_param = parameter.Uniform(-1.0, 1.0)
270+
elif sign == "vary" and not vary:
271+
sign_param = parameter.Constant()
261272
elif sign == "positive":
262273
sign_param = 1.0
263-
else:
274+
elif sign == "negative":
264275
sign_param = -1.0
276+
else:
277+
raise ValueError("Invalid option specified for chromatic dip sign.")
265278
wf = chrom_exp_decay(
266279
log10_Amp=log10_Amp_dmexp,
267280
t0=t0_dmexp,
@@ -275,7 +288,7 @@ def dm_exponential_dip(tmin, tmax, idx=2, sign="negative", name="dmexp"):
275288

276289

277290
def dm_exponential_cusp(
278-
tmin, tmax, idx=2, sign="negative", symmetric=False, name="dm_cusp"
291+
tmin, tmax, idx=2, sign="negative", symmetric=False, name="dm_cusp", vary=True
279292
):
280293
"""
281294
Returns chromatic exponential cusp (i.e. TOA advance):
@@ -288,25 +301,37 @@ def dm_exponential_cusp(
288301
:param sign:
289302
set sign of dip: 'positive', 'negative', or 'vary'
290303
:param name: Name of signal
304+
:param vary: Whether to vary the parameters or use constant values.
291305
292306
:return dmexp:
293307
chromatic exponential dip waveform.
294308
"""
295-
t0_dm_cusp = parameter.Uniform(tmin, tmax)
296-
log10_Amp_dm_cusp = parameter.Uniform(-10, -2)
297-
log10_tau_dm_cusp_pre = parameter.Uniform(0, 2.5)
309+
if vary:
310+
t0_dm_cusp = parameter.Uniform(tmin, tmax)
311+
log10_Amp_dm_cusp = parameter.Uniform(-10, -2)
312+
log10_tau_dm_cusp_pre = parameter.Uniform(0, 2.5)
313+
else:
314+
t0_dm_cusp = parameter.Constant()
315+
log10_Amp_dm_cusp = parameter.Constant()
316+
log10_tau_dm_cusp_pre = parameter.Constant()
298317

299-
if sign == "vary":
318+
if sign == "vary" and vary:
300319
sign_param = parameter.Uniform(-1.0, 1.0)
320+
elif sign == "vary" and not vary:
321+
sign_param = parameter.Constant()
301322
elif sign == "positive":
302323
sign_param = 1.0
303-
else:
324+
elif sign == "negative":
304325
sign_param = -1.0
326+
else:
327+
raise ValueError("Invalid option specified for dm exponential cusp sign parameter")
305328

306329
if symmetric:
307330
log10_tau_dm_cusp_post = 1
308-
else:
331+
elif vary:
309332
log10_tau_dm_cusp_post = parameter.Uniform(0, 2.5)
333+
else:
334+
log10_tau_dm_cusp_post = parameter.Constant()
310335

311336
wf = chrom_exp_cusp(
312337
log10_Amp=log10_Amp_dm_cusp,
@@ -323,7 +348,7 @@ def dm_exponential_cusp(
323348

324349

325350
def dm_dual_exp_cusp(
326-
tmin, tmax, idx1=2, idx2=4, sign="negative", symmetric=False, name="dual_dm_cusp"
351+
tmin, tmax, idx1=2, idx2=4, sign="negative", symmetric=False, name="dual_dm_cusp", vary=True
327352
):
328353
"""
329354
Returns chromatic exponential cusp (i.e. TOA advance):
@@ -336,29 +361,44 @@ def dm_dual_exp_cusp(
336361
:param sign:
337362
set sign of dip: 'positive', 'negative', or 'vary'
338363
:param name: Name of signal
364+
:param vary: Whether to vary the parameters or use constant values.
339365
340366
:return dmexp:
341367
chromatic exponential dip waveform.
342368
"""
343-
t0_dual_cusp = parameter.Uniform(tmin, tmax)
344-
log10_Amp_dual_cusp_1 = parameter.Uniform(-10, -2)
345-
log10_Amp_dual_cusp_2 = parameter.Uniform(-10, -2)
346-
log10_tau_dual_cusp_pre_1 = parameter.Uniform(0, 2.5)
347-
log10_tau_dual_cusp_pre_2 = parameter.Uniform(0, 2.5)
369+
if vary:
370+
t0_dual_cusp = parameter.Uniform(tmin, tmax)
371+
log10_Amp_dual_cusp_1 = parameter.Uniform(-10, -2)
372+
log10_Amp_dual_cusp_2 = parameter.Uniform(-10, -2)
373+
log10_tau_dual_cusp_pre_1 = parameter.Uniform(0, 2.5)
374+
log10_tau_dual_cusp_pre_2 = parameter.Uniform(0, 2.5)
375+
else:
376+
t0_dual_cusp = parameter.Constant()
377+
log10_Amp_dual_cusp_1 = parameter.Constant()
378+
log10_Amp_dual_cusp_2 = parameter.Constant()
379+
log10_tau_dual_cusp_pre_1 = parameter.Constant()
380+
log10_tau_dual_cusp_pre_2 = parameter.Constant()
348381

349-
if sign == "vary":
382+
if sign == "vary" and vary:
350383
sign_param = parameter.Uniform(-1.0, 1.0)
351-
elif sign == "positive":
384+
elif sign == "vary" and not vary:
385+
sign_param = parameter.Constant()
386+
elif sign == 'positive':
352387
sign_param = 1.0
353-
else:
388+
elif sign == "negative":
354389
sign_param = -1.0
390+
else:
391+
raise ValueError("Invalid option specified for dm exponential cusp sign parameter")
355392

356393
if symmetric:
357394
log10_tau_dual_cusp_post_1 = 1
358395
log10_tau_dual_cusp_post_2 = 1
359-
else:
396+
elif vary:
360397
log10_tau_dual_cusp_post_1 = parameter.Uniform(0, 2.5)
361398
log10_tau_dual_cusp_post_2 = parameter.Uniform(0, 2.5)
399+
else:
400+
log10_tau_dual_cusp_post_1 = parameter.Constant()
401+
log10_tau_dual_cusp_post_2 = parameter.Constant()
362402

363403
wf = chrom_dual_exp_cusp(
364404
t0=t0_dual_cusp,
@@ -378,33 +418,39 @@ def dm_dual_exp_cusp(
378418
return dm_cusp
379419

380420

381-
def dmx_signal(dmx_data, name="dmx_signal"):
421+
def dmx_signal(dmx_data, name="dmx_signal", vary=True):
382422
"""
383423
Returns DMX signal:
384424
385425
:param dmx_data: dictionary of DMX data for each pulsar from parfile.
386426
:param name: Name of signal.
427+
:param vary: Whether to vary the parameters or use constant values.
387428
388429
:return dmx_sig:
389430
dmx signal waveform.
390431
"""
391432
dmx = {}
392-
for dmx_id in sorted(dmx_data):
393-
dmx_data_tmp = dmx_data[dmx_id]
394-
dmx.update(
395-
{
396-
dmx_id: parameter.Normal(
397-
mu=dmx_data_tmp["DMX_VAL"], sigma=dmx_data_tmp["DMX_ERR"]
398-
)
399-
}
400-
)
433+
if vary:
434+
for dmx_id in sorted(dmx_data):
435+
dmx_data_tmp = dmx_data[dmx_id]
436+
dmx.update(
437+
{
438+
dmx_id: parameter.Normal(
439+
mu=dmx_data_tmp["DMX_VAL"], sigma=dmx_data_tmp["DMX_ERR"]
440+
)
441+
}
442+
)
443+
else:
444+
for dmx_id in sorted(dmx_data):
445+
dmx_data_tmp = dmx_data[dmx_id]
446+
dmx.update({dmx_id: parameter.Constant()})
401447
wf = dmx_delay(dmx_ids=dmx_data, **dmx)
402448
dmx_sig = deterministic_signals.Deterministic(wf, name=name)
403449

404450
return dmx_sig
405451

406452

407-
def dm_annual_signal(idx=2, tmin=None, tmax=None, name="dm_s1yr"):
453+
def dm_annual_signal(idx=2, tmin=None, tmax=None, name="dm_s1yr", vary=True):
408454
"""
409455
Returns chromatic annual signal (i.e. TOA advance):
410456
@@ -414,16 +460,92 @@ def dm_annual_signal(idx=2, tmin=None, tmax=None, name="dm_s1yr"):
414460
:param name: Name of signal
415461
:param tmin: earliest TOA for the sinusoid
416462
:param tmax: latest TOA for the sinusoid
463+
:param vary: Whether to vary the parameters or use constant values.
417464
418465
:return dm1yr:
419466
chromatic annual waveform.
420467
"""
421-
log10_Amp_dm1yr = parameter.Uniform(-10, -2)
422-
phase_dm1yr = parameter.Uniform(0, 2 * np.pi)
468+
if vary:
469+
log10_Amp_dm1yr = parameter.Uniform(-10, -2)
470+
phase_dm1yr = parameter.Uniform(0, 2*np.pi)
471+
else:
472+
log10_Amp_dm1yr = parameter.Constant()
473+
phase_dm1yr = parameter.Constant()
423474

424475
wf = chrom_yearly_sinusoid(
425476
log10_Amp=log10_Amp_dm1yr, phase=phase_dm1yr, idx=idx, tmin=tmin, tmax=tmax
426477
)
427478
dm1yr = deterministic_signals.Deterministic(wf, name=name)
428479

429480
return dm1yr
481+
482+
483+
@parameter.function
484+
def construct_chromatic_cached_parts(
485+
toas,
486+
freqs,
487+
nmodes=30,
488+
Tspan=None,
489+
logf=False,
490+
fmin=None,
491+
fmax=None,
492+
modes=None,
493+
fref=1400,
494+
):
495+
"""
496+
Using this function alongside `createfourierdesignmatrix_chromatic_with_additional_caching()`
497+
enables caching of the achromatic portion of the chromatic Fourier designmatrix as well as caching
498+
the division of the reference radio frequency (fref) and observational radio frequency vector (freqs).
499+
The actual caching occurs via the @function decorator on the
500+
`createfourierdesignmatrix_chromatic_with_additional_caching()`, where the decorator is defined in
501+
`enterprise.signals.parameter.function`.
502+
Note that the "achromatic portion of the chromatic Fourier designmatrix" is not related to
503+
the red noise and curn Fourier design matrices.
504+
505+
:param toas: vector of time series in seconds
506+
:param freqs: radio frequencies of observations [MHz], vector N_toa in length.
507+
:param nmodes: number of fourier coefficients to use
508+
:param Tspan: option to some other Tspan
509+
:param logf: use log frequency spacing
510+
:param fmin: lower sampling frequency
511+
:param fmax: upper sampling frequency
512+
:param modes: option to provide explicit list or array of
513+
sampling frequencies
514+
:param idx: Index of chromatic effects
515+
:param fref: reference radio frequency (default 1400 MHz)
516+
:return fmat_red: the achromatic Fmat to build the chromatic Fourier designmatrix from
517+
:return: F: Chromatic-variation fourier design matrix
518+
:return: f: Sampling frequencies
519+
"""
520+
521+
# get base achromatic Fourier design matrix and Fourier frequencies
522+
fmat_red, Ffreqs = gp_bases.createfourierdesignmatrix_red(
523+
toas, nmodes=nmodes, Tspan=Tspan, logf=logf, fmin=fmin, fmax=fmax, modes=modes
524+
)
525+
# compute the reference frequency/toa observational frequency vector
526+
fref_over_radio_freqs = fref / freqs
527+
528+
return fmat_red, Ffreqs, fref_over_radio_freqs
529+
530+
531+
@parameter.function
532+
def createfourierdesignmatrix_chromatic_with_additional_caching(
533+
fmat_red=None, Ffreqs=None, fref_over_radio_freqs=None, idx=4.0
534+
):
535+
"""
536+
Construct Scattering-variation fourier design matrix with a cached achromatic component of the
537+
Fourier design matrix (fmat_red). (Note this is independent of the actual achroamtic basis.)
538+
As a quirk of the @function decorator (which provides the caching) you must pass arguments explicitly
539+
e.g. fmat_red=fmatred, Ffreqs=Ffreqs... etc.
540+
In this construction, the `fmat_red` and `Ffreqs`, and `fref_over_radio_freqs` are not recomputed at every call,
541+
allowing for faster sampling in the chromatic index, alpha.
542+
:param fmat_red: (constant) achromatic Fourier design matrix
543+
:param Ffreqs: Fourier frequencies
544+
:param fref_over_radio_freqs: Reference radio frequency (in MHz) over toa radio frequencies (in MHz)
545+
:param idx: Index of chromatic effects (aka alpha or chi). Pass either a float, Constant, or Parameter
546+
:return: Fmat_chromatic: Chromatic-variation Fourier design matrix
547+
:return: Ffreqs: Fourier modes of the chromatic basis
548+
"""
549+
# give radio frequencies over reference frequency -idx chromatic index
550+
CM = fref_over_radio_freqs**idx
551+
return fmat_red * CM[:, None], Ffreqs

0 commit comments

Comments
 (0)