Report on wfm stitching #30
Conversation
description of peak finding
|
|
||
| ## 1. Introduction | ||
|
|
||
| There are currently two different methods that can be employed to perform what is commonly known as 'stitching' of neutron data that was recorded in wave-frame multiplication (WFM) mode at the ESS V20 test beamline. |
There was a problem hiding this comment.
Is it worth briefly saying something about why WFM is important. i.e. Long pulse at ESS gives poor wavelength resolution and this is unsuitable for many technique areas. By generating sub pulses at WFM choppers, each opening acts like a new source pulse with tighter wavelength band selection ... or something similar maybe lifted from the papers below?
There was a problem hiding this comment.
Yes, i guess it would probably be good to have a but more in there. Feel free to write something, if you have already an idea on how to phrase it.
|
|
||
| Because WFM is essentially making 6 new pulses from a single long pulse, the position of the neutron source (or source chopper in the case of V20) is considered to be half-way between the two WFM choppers. In order to conserve the wavelength of the neutrons, the data needs to be converted from 'Arrival time at detector' to real 'Time-of-flight'. | ||
|
|
||
| Each frame needs to be shifted by a different value, as described in [Woracek et al. (2016)](https://doi.org/10.1016/j.nima.2016.09.034). This article also outlines the method used to compute the different frame shifts, and this will thus not be repeated here. We simply list the numbers by which the frames were shifted: |
There was a problem hiding this comment.
The calculation is trivial, but is incorrect in the paper your cite. I think it is worth providing here.
There was a problem hiding this comment.
Ah, I didn't realise that. Do you have the correct calculation written down somewhere or should i extract it from the code?
There was a problem hiding this comment.
What is done here is correct https://git.esss.dk/wedel/wfm_stitching/-/blob/calculate_relative_frame_shifts/wfm_stitching/wfm_processing.py#L24-69 that was just a correction of the formula in robins paper
There was a problem hiding this comment.
So I suggest taking (3) from Robin's paper but correcting. I think it's just the division by 360 that's wrong, should be multiplied.
| - Contact: Neil Vaytet (ESS/DMSC), Owen Arnold (STFC, Tessella) | ||
|
|
||
| ### 2.2 Applying the conversion to time-of-flight | ||
|
|
There was a problem hiding this comment.
I think you either need to say here or specifically somewhere above that this procedure only applies to histogrammed data in which the concept of pulse time has been lost. Only the TOF is shifted. This distinguishes the event mode operation in which the pulse time should also be corrected to preserve the absolute time.
There was a problem hiding this comment.
Actually, maybe instead we should mention that we also need to shift the pulse times in the case of event data. I guess I haven't made it very clear that the stitching we present here would work for both event and histogrammed data, as I'm mostly talking about events.
| Until 2018, WFM frame boundaries had to be given manually as an input to the [ESS WFM processing module](https://git.esss.dk/wedel/wfm_stitching). | ||
| This first method aimed to automatize the frame boundary detection process by using some signal processing techniques (including peak-finding). | ||
|
|
||
| Note that throughout this document, we assume that the long pulse is being split into 6 frames, as this is the number of frames given by the WFM choppers at V20. There is however no reason the number has to be 6, and some ESS instruments are indeed planning to use more than 6 frames (e.g. FREIA has a chopper design for 7 frames). The procedures outlined below do not depend on the numebr of frames, and would remain identical. |
There was a problem hiding this comment.
The V20 WFM choppers have 6 slit openings each.
|
|
||
| The programs to perform the WFM stitching should be distributed as follows: | ||
|
|
||
| 1. A separate package containing the frame boundary determination, which would be distributed on `pip` and `conda`. This should aim to remain Python-only, with no large dependencies such as Mantid. This package would allow a user to decide which method they want to use to obtain the frame boundaries: either using the peaking finding, or the TOF diagram, or just a manual input. This package would also provide a basic (non-optimized) method to perform the TOF conversion, that could be used by e.g. Mantid users who do not wish to use the Scipp package. |
There was a problem hiding this comment.
Are we going to make this independent of the loading. The present WFM package loads the frames using the fits format (we had problems with this as you know). Do we exclude the loading from the WFM?
There was a problem hiding this comment.
I think it should be independent of loading if possible, since as you say there is fits files loading, and also loading with Mantid for nexus files.
I guess in Python it should be easy to make it so that it can accept different kinds of inputs. It could accept a Mantid workspace, or some raw count data. Not sure how complicated/maintanable that would get...
|
|
||
| In principle, stitching is only required to be run once, as very little tweaking of the stitching parameters should be necessary. In addition, some users were carrying out a study that was comparing WFM and single-pulse data using the same sample set-up, and were having to maintain two different (but very similar) reduction scripts, depending on whether they were reducing WFM or single-pulse data. | ||
|
|
||
| We thus propose that WFM stitching could be implemented as a post-processing step in the data acquisition. The raw data could be read in, stitched using the parameters from the beamline (the Nexus geometry is contained inside the data file), and a new post-processed file would then be written to disk (and added to SciCat). All the parameters that were used for the stitching would be included in the file metadata. This would effectively alleviate all the issues listed above. |
There was a problem hiding this comment.
Open question - could we consider event compression? i.e. weighting events within some pulse time tolerance to reduce file size? Or is that out of scope?
In any case I feel it is important to have BOTH the un-corrected file and the WFM corrected file as you suggest.
There was a problem hiding this comment.
I guess if file size does become an issue we could consider event comrpession. Maybe it's worth just mentioning that we have thought about this.
There was a problem hiding this comment.
I think in the first pass we wouldn't do this, but yes do mention
| The users would still have access to the raw un-stitched data, so that they are able to perform the stitching themselves, if they suspect something went wrong during that phase. But for 99% of cases, assuming auto-reduction is operating to satisfaction at an ESS facilty in full production mode, automatic post-processed stitching should work nicely. | ||
|
|
||
| Going one step further, one could even imagine including the event-by-event stitching (although it makes less sense to call it 'stitching' here, 'conversion to time-of-flight' may be more appropriate) as part of the filewriter. It should be possible to write the raw and stitched data files simultaneously, thus cutting out the time needed to re-read the raw data to perform the conversion. | ||
|
|
There was a problem hiding this comment.
WFM correction as part of the live stream is also very important. Using the TOF calculation approach based on beamline geometry, this should be possible. Do we bake that into the decoder?
This document is a detailed summary of the different methods that can be used to stitch WMF data and how we plan to move forward in the future for ESS in terms of implementation and distribution of the software.