M92 VAC (DR2)

[weaverba137]

Contact Person: Amanda Byström
1 catalog file (fits), 1 README, tutorial in .ipynb + .html form.

Initial Checks:

• Includes README
• Columns in ALLCAPS
• Extension names in ALLCAPS
• Files include units
• VAC publication passed CWR

Initial Notes:

• I suggest renaming this to "m92-survey". Typically, we do not include "vac" in the name, and the word "tertiary" is meaningful only internally to DESI.
• The FITS file has a number of inconsistencies. We definitely want units throughout, but in other cases it's a matter of picking a convention and applying it consistently to all HDUs.
  • Although units are included in some HDUs, some columns in some HDUs that should have units do not.
  • Some HDUs have DATASUM/CHECKSUM header keywords, but not all of them. For the HDUs that do have them, they are invalid.
  • HDU 1 has TCOMMxx header keywords, but the other HDUs do not.
• The notebook should have a top-level title and meaningful subheadings, i.e. the layout should make sense when the outline view is opened.
• The README file only describes one of the 6 HDUs in detail. Nominally the other HDUs are based on https://desi-mws-dr1-datamodel.readthedocs.io/en/latest/rv_output/RVRUN/rvpix.html, but since this is DR2 data, are the HDUs based on the DR2 version of the MWS catalog?

[weaverba137]

PS, since we also prefer version numbers, I suggest:
m92_tertiary_vac.fits --> m92-survey-v1.0.fits.

[abystrom]

Thanks Benjamin, I've fixed these comments now:

• File has been renamed to m92-survey-v1.0.fits
• I had previously copied the unit structure of the DR1 VAC to not confuse users. I've added units now throughout the entire file, so currently the units differ from the DR1 VAC, but I think this is more internally consistent. Please let me know if there are still inconsistencies/you disagree. I wasn't sure how much to add, because adding more units deviates from DR1 more...
• DATASUM and CHECKSUM header keywords have been added.
• The TCOMMxx header keywords have been removed.
• I've rewritten the tutorial headers.
• The HDU columns structures are based on the DR1 RVS VAC now (but the processing is DR2). The only thing that's added that's different from internal DR2 are Gaia telescope columns; so these are not DESI data. I spoke to Sergey, and he currently has not finalised the RVS set of columns to be released with DR2. The HDUs will be the same, but primarily columns in GAIA will change. So we agreed that even if I match the internal DR2 catalogue as it stands now, once release comes, that may change and I'd have to redo it anyway. So to avoid confusion for users, I'm currently using the DR1 structure, but this can be changed easily for the real DR2.

I've updated the directory now. The updated files can now be found at: /global/cfs/cdirs/desi/users/abystrom/m92-survey.

[weaverba137]

Thank you for these changes, things are looking very good so far, but I still have a few more suggestions:

• The notebook doesn't have a title. This could be as simple as a cell containing one (MarkDown) line: # M92 Survey VAC Tutorial. Or that line could be added to the first cell.
• Other headings should be level 2 or lower, depending on the structure of the notebook, e.g. ## Loading the M92 data.
• There are still a number of columns that (it seems apparent to me) should have units, but don't:
  • HDU GAIA
    • *_ERROR: Unless these are inverse variances, the uncertainties should have the same units as the corresponding data column.
    • *_CORR: I think these represent a correlation matrix? Please double-check what units those values should have.
    • There are also a number of columns that look like they should be flux values.
  • HDU SCORES
    • Several columns appear to be flux values.
  • Other HDUs look OK in terms of units.

[weaverba137]

I have created PR desihub/desidatadocs#189 to add the documentation for this VAC. Please check for invitations to join the GitHub groups necessary to view this PR.

[abystrom]

Ok, I've now changed:

• The tutorial headings
• Added GAIA HDU units (using https://irsa.ipac.caltech.edu/data/Gaia/dr3/gaia_dr3_source_colDescriptions.html)
• The SCORES HDU does not have any units according to https://desi-mws-dr1-datamodel.readthedocs.io/en/latest/rv_output/RVRUN/rvpix.html#hdu3 so I've kept it as it is. I don't know if this is a mistake or not. I don't know where to find documentation about where these columns originate from. If tricky, I can just delete this HDU as I've never used it once, and the columns are not relevant for my paper.

I've accepted the desihub invitation as well!

[weaverba137]

The tutorial headings

Looks good, I don't think further changes are needed there.

Added GAIA HDU units

Looks good, I don't think further changes are needed there.

The SCORES HDU does not have any units according to

If it's not used in the notebook nor mentioned (even if unused) in the paper, I would consider deleting that HDU. Otherwise, people will ask why it's there.

[abystrom]

I agree with deleting SCORES (I deleted it from the BHB DR1 VAC for the same reason). I've removed that HDU now, and rewritten the README and tutorial accordingly. It should all be consistent now!

[weaverba137]

Good, thank you. I have copied the files to a staging area, /global/cfs/cdirs/desi/vac/dr2/m92-survey, to demonstrate the final layout.

When your paper is ready, we will also copy that version of the files to a public area specifically for files associated with papers.

At this point, please review desihub/desidatadocs#189.

[abystrom]

Nice, and the rendered documentation version looks great. The staging area looks fine to me.

I've reviewed the datadocs now.

I'll let you know when the paper finished Final Reading, and we get the green light to submit to a journal.

[weaverba137]

@abystrom in keeping with a recent policy change, VACs are now expected to be included in the publication CWR process. If your publication has already been through CWR, please go ahead and check the "VAC publication passed CWR" box above.