Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/jwst/persistence/api_ref.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
===
API
===

Public Step API
===============

.. automodapi:: jwst.persistence.persistence_step
:no-inheritance-diagram:

Complete Developer API
======================

.. automodapi:: jwst.persistence.persistence
:no-inheritance-diagram:
37 changes: 18 additions & 19 deletions docs/jwst/persistence/arguments.rst
Original file line number Diff line number Diff line change
@@ -1,26 +1,25 @@
.. _persistence_step_args:

Step Arguments
==============

The persistence step has three step-specific arguments.

* ``--input_trapsfilled``

``input_trapsfilled`` is the name of the most recent trapsfilled file
for the current detector. If this is not specified, an array of zeros
will be used as an initial value. If this is specified, it will be used
to predict persistence for the input science file.
The step writes an output trapsfilled file, and that could be used
as input to the persistence step for a subsequent exposure.

* ``--flag_pers_cutoff``

If this floating-point value is specified, pixels that receive a
persistence correction greater than or equal to ``flag_pers_cutoff`` DN
(the default is 40) are flagged in the PIXELDQ array of the
output file with the DQ value "PERSISTENCE".
``input_trapsfilled [string]``
Name of the most recent "trapsfilled" file
for the current detector. If this is not specified, an array of zeros
will be used as an initial value. If this is specified, it will be used
to predict persistence for the input science file.
The step writes an output "trapsfilled" file, and that could be used
as input to the persistence step for a subsequent exposure.

* ``--save_persistence``
``flag_pers_cutoff [float]``
If this floating-point value is specified, pixels that receive a
persistence correction greater than or equal to ``flag_pers_cutoff`` DN
(the default is 40) are flagged in the PIXELDQ array of the
output file with the DQ value "PERSISTENCE".

If this boolean parameter is specified and is True (the default is False),
the persistence that was subtracted (group by group, integration by
integration) will be written to an output file with suffix "_output_pers".
``save_persistence [boolean]``
If this parameter is specified and is `True` (the default is `False`),
the persistence that was subtracted (group by group, integration by
integration) will be written to an output file with suffix "_output_pers".
96 changes: 52 additions & 44 deletions docs/jwst/persistence/description.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Description
===========

:Class: `jwst.persistence.PersistenceStep`
:Class: `jwst.persistence.persistence_step.PersistenceStep`
:Alias: persistence

Based on a model, this step computes the number of traps that are
Expand All @@ -15,8 +15,9 @@ correcting the persistence of a subsequent exposure.
There may be an input traps-filled file (defaults to 0), giving the number
of traps that are filled in each pixel. There is one plane of this 3-D image
for each "trap family," sets of traps having similar capture and decay
parameters. The traps-filled file is therefore coupled with the trappars
reference table, which gives parameters family-by-family. There are currently
parameters. The traps-filled file is therefore coupled with the
:ref:`trappars reference table <trappars_reffile>`,
which gives parameters family-by-family. There are currently
three trap families.

If an input traps-filled file was specified, the contents of that file will
Expand All @@ -43,7 +44,7 @@ The number of trap decays in a given time interval is computed as follows:
.. math::
n\_decays = trapsfilled \cdot (1 - exp(-\Delta t / \tau))

where trapsfilled is the number of filled traps, i.e. the value of the
where *trapsfilled* is the number of filled traps, i.e., the value of the
traps-filled image at the
beginning of the time interval, for the current trap family and at the
current pixel; :math:`\Delta t` is the time interval (seconds) over which
Expand All @@ -56,20 +57,20 @@ For each pixel, the persistence in a group is the sum of the trap decays
over all trap families. This persistence is subtracted from the science
data for the current group. Pixels that have large persistence values
subtracted from them are flagged in the DQ array, as information to the
user (see the Step Arguments section).
user (see :ref:`persistence_step_args`).

Trap capture is more involved than is trap decay. The computation of trap
capture is different for an impulse (e.g. a cosmic-ray event) than for a
capture is different for an impulse (e.g., a cosmic-ray event) than for a
ramp, and saturation also affects capture. Computing trap capture needs
an estimate of the ramp slope, and it needs the locations (pixel number and
group number) of cosmic-ray jumps. At the time of writing, the ``persistence``
step is run before the ``jump`` step, so the GROUPDQ array in the input to
group number) of cosmic-ray (CR) jumps. At the time of writing, the ``persistence``
step is run before the :ref:`jump step <jump_step>`, so the GROUPDQ array in the input to
``persistence`` does not contain the information that is required to account
for cosmic-ray events.

Because the ``persistence`` step is run before ``ramp_fit``, the persistence step does
Because the ``persistence`` step is run before :ref:`ramp_fit <ramp_fitting_step>`, the persistence step does
not have the value of the slope, so the step must compute its own estimate
of the slope. The algorithm is as follows. First of all, the slope must be
of the slope. The algorithm is as follows: First of all, the slope must be
computed before the loop over groups in which trap decay is computed and
persistence is corrected, since that correction will in general change the
slope. Within an integration, the difference is taken between groups of the
Expand All @@ -83,39 +84,43 @@ knowing how many of them there are (which we know from the GROUPDQ array).
The average of the remaining differences is the slope. The slope is needed
with two different units. The ``grp_slope`` is the slope in units of DN
(data numbers) per group. The ``slope`` is in units of
(DN / persistence saturation limit) / second, where "persistence saturation
limit" is the (pixel-dependent) value (in DN) from the PERSAT reference file.
``(DN / persistence saturation limit) / second``, where "persistence saturation
limit" is the (pixel-dependent) value (in DN) from the :ref:`persat_reffile`.

The number of traps that capture charge is computed at the end of each
integration. The number of captures is computed in three phases: the
portion of the ramp that is increasing smoothly from group to group;
the saturated portion (if any) of the ramp; the contribution from
cosmic-ray events.
integration. The number of captures is computed in three phases:

1. the portion of the ramp that is increasing smoothly from group to group;
2. the saturated portion (if any) of the ramp;
3. the contribution from cosmic-ray events.

For the smoothly increasing portion of the ramp, the time interval over
which traps capture charge is
nominally :math:`nresets \cdot tframe + ngroups \cdot tgroup`
where nresets is the number of resets at the beginning of the integration,
tframe is the frame time, and tgroup is the group time.
which traps capture charge is nominally:

.. math::
nresets \cdot tframe + ngroups \cdot tgroup

where *nresets* is the number of resets at the beginning of the integration,
*tframe* is the frame time, and *tgroup* is the group time.
However, this time must be reduced by the group time multiplied by the
number of groups for which the data value exceeds the persistence saturation
limit. This reduced value is :math:`Delta t` in the expression below.
limit. This reduced value is :math:`\Delta t` in the expression below.

The number of captures in each pixel during the integration is:

.. math::
trapsfilled = 2 \cdot &(trapdensity \cdot slope^2 \\
&\cdot (\Delta t^2 \cdot (par0 + par2) / 2
+ par0 \cdot (\Delta t \cdot \tau + \tau^2) \\
&\cdot exp(-\Delta t / \tau) - par0 \cdot \tau^2))
&\cdot (\Delta t^2 \cdot (par0 + par2) / 2
+ par0 \cdot (\Delta t \cdot \tau + \tau^2) \\
&\cdot exp(-\Delta t / \tau) - par0 \cdot \tau^2))

where par0 and par2 are the values from columns "capture0" and "capture2"
respectively, from the trappars reference table, and :math:`\tau` is the
where *par0* and *par2* are the values from columns "capture0" and "capture2",
respectively, from the :ref:`trappars_reffile`, and :math:`\tau` is the
reciprocal of the absolute value from column "capture1", for the row
corresponding to the current trap family. trapdensity is the
corresponding to the current trap family. *trapdensity* is the
relative density of traps, normalized to a median of 1. :math:`\Delta t`
is the time interval in seconds over which
the charge capture is to be computed, as described above. slope is the
the charge capture is to be computed, as described above. *slope* is the
ramp slope (computed before the loop over groups), in units of fraction
of the persistence saturation limit per second. This returns the number
of traps that were predicted to be filled during the integration, due to
Expand All @@ -124,12 +129,12 @@ the function that computes the additional traps that were filled due to
the saturated portion of the ramp.

"Saturation" in this context means that the data value in a group exceeds
the persistence saturation limit, i.e. the value in the PERSAT reference
file. filled_during_integration is (initially) the array of the number of
the persistence saturation limit, i.e., the value in the :ref:`persat_reffile`.
*filled_during_integration* is (initially) the array of the number of
pixels that were filled, as returned by the function for the smoothly
increasing portion of the ramp. In the function for computing decays
for the saturated part of the ramp, for pixels that are saturated in the
first group, filled_during_integration
first group, *filled_during_integration*
is set to :math:`trapdensity \cdot par2` (column "capture2"). This accounts
for "instantaneous" traps, ones that fill over a negligible time scale.

Expand All @@ -148,7 +153,7 @@ so the traps that are filled depending on the exponential component is:
.. math::
new\_filled\_traps = empty\_traps \cdot (1 - exp(-sattime / \tau))

where sattime is the duration in seconds over which the pixel was saturated.
where *sattime* is the duration in seconds over which the pixel was saturated.

Therefore, the total number of traps filled during the current integration is:

Expand All @@ -168,7 +173,7 @@ pixels, the amplitude of the jump is computed to be the difference between
the current and previous groups minus grp_slope (the slope in DN per group).
If a jump is negative, it will be set to zero.

If there was a cosmic-ray hit in group number k, then
If there was a cosmic-ray hit in group number *k*, then:

.. math::
\Delta t = (ngroups - k - 0.5) \cdot tgroup
Expand All @@ -185,32 +190,35 @@ and the number of filled traps for the current pixel will be incremented
by that amount.

Input
=====
The input science file is a RampModel.
-----
The input science file is a `~stdatamodels.jwst.datamodels.RampModel`.

A trapsfilled file (TrapsFilledModel) may optionally be passed as input
A "trapsfilled" file (`~stdatamodels.jwst.datamodels.TrapsFilledModel`)
may optionally be passed as input
as well. This normally would be specified unless the previous exposure
with the current detector was taken more than several hours previously,
that is, so long ago that persistence from that exposure could be ignored.
If none is provided, an array filled with 0 will be used as the starting
point for computing new traps-filled information.

Output
======
The output science file is a RampModel, a persistence-corrected copy of
the input data.
------
The output science file is a `~stdatamodels.jwst.datamodels.RampModel`,
a persistence-corrected copy of the input data.

A second output file will be written, with suffix "_trapsfilled". This
is a TrapsFilledModel, the number of filled traps at each pixel at the end
is a `~stdatamodels.jwst.datamodels.TrapsFilledModel`,
the number of filled traps at each pixel at the end
of the exposure. This takes into account the capture of charge by traps
due to the current science exposure, as well as the release of charge
from traps given in the input trapsfilled file, if one was specified. Note
that this file will always be written, even if no input_trapsfilled file
from traps given in the input "trapsfilled" file, if one was specified. Note
that this file will always be written, even if no input "trapsfilled" file
was specified. This file should be passed as input to the next run of the
persistence step for data that used the same detector as the current run.
Pass this file using the input_trapsfilled argument.
Pass this file using the ``input_trapsfilled`` argument.

If the user specifies ``save_persistence=True``, a third output file will
be written, with suffix "_output_pers". This is a RampModel matching the
be written, with suffix "_output_pers".
This is a `~stdatamodels.jwst.datamodels.RampModel` matching the
output science file, but this gives the persistence that was subtracted
from each group in each integration.
3 changes: 1 addition & 2 deletions docs/jwst/persistence/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,4 @@ Persistence Correction
description.rst
arguments.rst
reference_files.rst

.. automodapi:: jwst.persistence
api_ref.rst
9 changes: 5 additions & 4 deletions docs/jwst/persistence/reference_files.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
Reference Files
===============

The ``persistence`` step uses
:ref:`TRAPDENSITY <trapdensity_reffile>`,
:ref:`PERSAT <persat_reffile>`, and
:ref:`TRAPPARS <trappars_reffile>` reference files.
The ``persistence`` step uses the following reference files:

* :ref:`TRAPDENSITY <trapdensity_reffile>`
* :ref:`PERSAT <persat_reffile>`
* :ref:`TRAPPARS <trappars_reffile>`

.. include:: ../references_general/trapdensity_reffile.inc

Expand Down
2 changes: 1 addition & 1 deletion docs/jwst/references_general/persat_reffile.inc
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ PERSAT Reference File
:Data model: `~stdatamodels.jwst.datamodels.PersistenceSatModel`

The PERSAT reference file contains a pixel-by-pixel map of the
persistence saturation (full well) threshold.
persistence saturation (full-well) threshold.

.. include:: ../references_general/persat_selection.inc

Expand Down
9 changes: 4 additions & 5 deletions docs/jwst/references_general/persat_selection.inc
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@ PERSAT is not applicable for instruments not in the table.
All keywords used for file selection are *required*.

========== ======================================
Instrument Keywords
Instrument Keywords
========== ======================================
FGS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRCam INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRISS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
FGS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRCam INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRISS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
========== ======================================

9 changes: 4 additions & 5 deletions docs/jwst/references_general/trapdensity_selection.inc
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@ TRAPDENSITY is not applicable for instruments not in the table.
All keywords used for file selection are *required*.

========== ======================================
Instrument Keywords
Instrument Keywords
========== ======================================
FGS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRCam INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRISS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
FGS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRCam INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRISS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
========== ======================================

9 changes: 4 additions & 5 deletions docs/jwst/references_general/trappars_selection.inc
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@ TRAPPARS is not applicable for instruments not in the table.
All keywords used for file selection are *required*.

========== ======================================
Instrument Keywords
Instrument Keywords
========== ======================================
FGS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRCam INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRISS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
FGS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRCam INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
NIRISS INSTRUME, DETECTOR, DATE-OBS, TIME-OBS
========== ======================================

Loading
Loading