KiDZ-”DR1” release

[cheymans]

@kuijken @AngusWright - in the notes from the meeting with Nausicaa you discussed having KiDZ as a separate release? My vote is not to do this (see below), but if we do want to separate them, let me know asap. The prog_ids (#9) go in quite early in the ESO-fication process, and I would need to strip out the KiDZ ids if they are going elsewhere.

Copied from the gdoc:

---

Should KiDZ release be in a separate Phase 3 collection?

• Note (post-meeting): Each Phase 3 collection gets a different DOI: https://archive.eso.org/wdb/wdb/doi/collections/query
• How to encode the provenance of the KiDZ cross-match catalogue: it will be a mix of KiDS and KiDZ coadds. I checked and it is possible to mix both ARCFILE and ORIGFILE PROVi references within the same file.
• Check if there are any proprietary data in the KiDZ fields.

---

Was there a very strong steer from ESO to do this? My vote would be to have the KiDZ tiles as part of the KiDS release.

1. We don't release mosaic catalogues with ESO, so the fact that the KiDZ tiles are outwith the main footprint isn't an issue
2. We don't release shear measurements with ESO, because ESO requires measurements for every detected object. We only release shear measurements for "safe" objects, so no-one makes a mistake measuring cosmic shear with everything KiDS has ever detected. This means the main KiDZ and KiDS per-tile catalogues and images released through ESO will be identical.
3. Our lives will be much easier if there is just one release: one release document, one list, one metadata file, one upload site

I recognise the bonus of separating them is that the cross-match spec-phot catalogue can then be released for KIDZ, providing an excellent test bed for SOM development. But so long as we make a nice webpage for KiDZ, people can download an all-tile combined KIDZ spec-phot catalogue from the KiDS webpage anyway? This would also allow us to better handle access to any proprietary spec-z in the cats.

[cheymans]

I've looked into KiDZ, it will sadly have to be a separate release, as we don't have the THELI stacks for these images, and you need to have an identical data product format for each tile in an ESO release.

Option 1: Worry about KIDZ when we've finished the DR5 release (tempting)
Option 2: Do the two releases together

I think if we go for Option 1, a KIDZ release won't happen through ESO as it's so fiddly. Do we really want to release this extra data through ESO though? My sense is that there is lots of imaging of these spec-z fields already, so the imaging itself isn't that exciting. What people really want is the cross-matched photometry/spec-z catalogue which we can release on the KiDS website.

Thoughts @kuijken @hendrik1008 ?

[hendrik1008]

Option 1 looks like the sensible choice here indeed. A merged catalogue is probably what the vast majority of people care about. We can make images available upon request. This is only interesting for 1-2 people in the world, max.

[kuijken]

Agreed! Let's keep it simple(r)

…

On 29 Jan 2024, at 09:01, hendrik1008 ***@***.***> wrote: Option 1 looks like the sensible choice here indeed. A merged catalogue is probably what the vast majority of people care about. We can make images available upon request. This is only interesting for 1-2 people in the world, max. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[cheymans]

Closing issue:

We won't release KiDSZ through ESO. Checking the DR5 paper we say:

Is think that is OK because it's ambiguous what we mean by release here. We will have to organise the website though, which is where I think most people will land.

[cheymans]

Re-opening issue:

This message from Nausicaa at ESO:

After checking Table 2 in the KiDS DR5, it seems most of the KiDZ data seem to fall under a large programme ID so the data would qualify for delivery to Phase 3 / ESO archive. Magda will be attending the meeting in Leiden on April 9-10 and so she will have the opportunity to discuss the policy aspects further with the KiDS team.

@AngusWright @kuijken @hendrik1008 - see you if can convince Magda that KiDZ shouldn't be part of the release? If we must, however, then please discuss how we should be do it, given the slightly different data structure to the main release.

[cheymans]

@kuijken - could you post an update here on the discussion with Magda. (after the long weekend holiday is complete!)
Do we need to do a KiDZ-release?

[cheymans]

From Nausicaa (e-mail dated 12th June)

I understand Magda could talk in Leiden with some of the team and a consensus was reached to publish everything on the ESO archive. This also follows the recommendations of the PSP following the review of the public surveys program in 2019.

@kuijken - could you update us on what you remember from this discussion? To recap the issue for us is that, as we don't have the "ASSON" THELI stacks for the KiDZ fields, KiDZ would have to be a separate release from DR5. It's not just a case of adding the few extra tiles to the DR5. In addition I'm assuming they would also want the VISTA images included as they are not part of VIKING?