Response to the Referee #26
Description
Hi all,
Here's my suggested response to the referee. Original comments made by the reviewer are indented, our proposed response in normal text.
Please let me know by the end of next Friday (May 10, 2019) if you have any comments. I am planning to resubmit the paper by end of day on May 10th.
========
Stingray Paper: Response to Referee
We thank the reviewer and the statistics editor for their kind words about the manuscript, and their very helpful suggestions for improvement, which we have taken into account in the new version of the manuscript and present in detail below.
I downloaded the package for both my python 3.0 and python 2.7 environments and tested it on some of my own data. I've also taken a look at the online notebooks available (which are an invaluable resource). I was not able to open the ModelingExamples.ipynb (on any browser) due to errors. I feel, at the time of publication, this resource should be up to date/as complete as possible.
Unfortunately, we have not been able to reproduce the referee’s problem with opening the notebook. Opening the view-only version on GitHub (https://github.com/StingraySoftware/notebooks/blob/master/Modeling/ModelingExamples.ipynb) can sometimes be slow, and sometimes these notebooks don’t render properly on GitHub, which is a known issue. Reloading the page a couple of times allowed us to display the notebook successfully on Safari and Firefox. We also reran the notebook in its entirety using Python 3.6 and the pip-installable version of Stingray, and encountered no errors. If the error persists for the reviewer, may we suggest submitting an issue with some more information about the errors encountered via https://github.com/StingraySoftware/notebooks/issues to help us debug efficiently (perhaps with a dummy account if the reviewer wishes to remain anonymous)?
- I would emphasize in the title and abstract that this package is explicitly designed for X-ray spectral timing, I appreciate this term is used mostly by the X-ray community but it occasionally crops up at other wavelengths. Whilst these tasks are all instrument independent I can't think of an obvious way that this package would transfer to other wavelengths, even if the methods used are the same; the use of event lists and light curves extracted from single stare observations doesn't translate to e.g. ground based optical observing sets. I maybe incorrect on this point though and I recognize that this might change in the future.
While it is true that spectral timing is mostly used in the X-ray community, Fourier analysis has a long history of use in all wavelengths for a wide range of different objects (as the reviewer correctly pointed out in their comment about our introduction). There are natural connections to the AGN community at all wavelengths, and also to stellar variability at optical wavelengths. Fourier-based methods are used quite heavily in stellar structure studies, and we note that both the Kepler and TESS spacecrafts produce data (fairly evenly sampled, long stares at a single field) to what the X-ray community routinely deals with. In fact, in response to the reviewer’s point (3) below, we have included the periodogram of a Kepler light curve of an Active Galactic Nuclei as a proof-of-concept that the methods described and the software itself are applicable to these types of data. We have also included some references in our expanded introduction on research that uses similar timing methods (including power spectra and measurements of the rms-flux relation) using optical data of accreting white dwarf systems. While the package was designed with our own science cases in mind, which certainly skew more towards X-ray astronomy, it wasn’t designed exclusively for X-ray data sets, and recent citations of stingray indicate that researchers using radio and optical observations have begun adopting it for their studies.
- The paper markets the package as a general X-ray spectral timing package, yet the introduction and examples seem a little limited. Moreover, the package has the power to be used to "just" investigate X-ray variability (without the explicit link to energy). In a time where reproducibility and replication are hot topics, open source code such as this is important, beyond the scope of its initial brief, and this should be highlighted to the wider X-ray community, not just those working in spectral timing. The immediate narrowing the introduction to spectral timing (especially after mentioning exoplanents!) and only explicitly discussing QPOs and reverberation mapping doesn't do justice to the breadth of the package. I would like the introduction to be expanded to discuss other exampled of X-ray variability (including X-ray pulsars, which are included in the package, but not eluded to in the introduction). What about X-ray variability in HMXBs? Or X-ray emission from colliding wind systems (e.g. Eta Carina, Gamma Velorum)? Even the X-ray emission from isolated stars. I do not wish to turn the introduction into a review article, but I think the addition of a paragraph to highlight just how much X-ray variability there is and, therefore just how important this package is.
We thank the reviewer for this very important point. In response, we have significantly expanded the introduction to include a wider range of examples where (spectral) timing is successfully being applied, including studies on GRBs, magnetar bursts, and solar flares, as well as asteroseismology, young stellar objects and accretion studies of white dwarf systems.
- Related to point 2 - the two datasets in used to demonstrate the functionality of the package are both LMXBs, which again I feel does the package a disservice, and may discourage a user who works with AGN or HMXBs to use stingray. I appreciate that there are no suitable AGN data to demonstrate reverberation mapping, but an AGN as an example for the power spectrum and including an HMXB to demonstrate the pulsar package would much better showcase the flexibility and power of this package.
We thank the reviewer for making this point! As a group of researchers all working on similar science cases, it is sometimes easy to stay in our own narrow field of study! In order to better illustrate the breadth of the package, we have included an example of a Kepler observation of an AGN in the paper. As an IMXB, Her X-1 was specifically chosen as a pulsar source that might appeal to both researchers working on LMXBs and HMXBs, since it is well-known to both communities. We have also included a reference to recent work that employs Stingray on SMC X-1, which is indeed an HMXB.
Intro: was there a reason you didn't include XMM-Newton in this list? It's omission seems a little glaring given how many X-ray timing studies came out of this telescope?
Indeed, this was an omission, which has been fixed in the new version of the manuscript.
Page 12, col 1: You state that "stingray has functionality to fit these periodograms with a sinc^2 function or alternatively a Gaussian model, and find the mean frequency with high precision." Fitting a Gaussian to a periodogram will give the mean frequency, but I think this statement should be followed with an explicit warning that, as the data points in a periodogram are not independent, fitting should be done with caution and the width on this Gaussian is not the error on this measurement (see Section 7 of VanderPlas 2018)
As suggested, we have added a warning to the section on characterizing pulsar behaviour.
Statistics Editor Report
The methodology presented here is entirely in the frequency domain (spectral analysis) using either classical (e.g. Fourier power spectrum) or specialized (e.g. epoch folding) methods. Time series analyst treating light curves as in Fig 1 could also try time domain, parametric, stochastic, autoregressive models in the ARIMA family. ARIMA is very widely used, described in textbooks on time series analysis (e.g. Box, Jenkins et al. 2015, 5th ed), and has a python implementation.
ARIMA itself treats short-memory linear autocorrelation (AR & MA components) together with a simple detrender (differencing operator, I component). If long-memory autocorrelation is also present, then the ARFIMA extension can be used; it gives a maximum likelihood value for alpha in 1/f^apha red noise jointly with short-memory processes. ARIMA & ARFIMA processes often produce quasi-periodic oscillations without an explicit periodic component (SARIMA models). So this formalism provides an opportunity to evaluate whether alpha is truly nonzero, and a QPO is truly present, that cannot be attributed to stochastic aperiodic autocorrelation. The entire astronomical community, particularly those characterizing QPO accretion systems, would benefit in making such tests.
As such, it would be very valuable to include them in stingray.
We agree with the statistics editor that this would be a very valuable addition. Because this would be a fairly major undertaking, we have opened an Issue on GitHub (StingraySoftware/stingray#389) and included this in our long-term planning for Stingray v2. We have also added a paragraph about time series methods more generally to the “future work” section of the manuscript. As an open-source package, stingray of course welcomes contributions from the community, and the Issue itself has already generated some interest from some of our contributors.