htsget: using htsget server to mask regions from users (i.e clinical usecase(s))

[brainstorm]

Context: within a clinical WGS setting, there is a need to mask out sensitive information from a particular sample and/or just provide the regions that a user has asked for. At present, htsget does not have client (i.e IGV.js/ igv-desktop / tgv) hints that support permission-based data retrieval and appropriate semantic visualization.

1. Return different result codes (or additional fields in the response that provide hints on where the allowed range is?) for "no reads present" vs "forbidden (suppressed)" on a particular range. Rationale: at present, the client cannot discern whether there's no data in that range or just forbidden by the "range permissions" given a priori.
2. Make sure returned blocks from htsget DO NOT contain reads that pertain to those forbidden/suppressed ranges, see Edit boundary blocks on the server side to drop non-requested data umccr/htsget-rs#238 for details.
3. Potentially have two modes of operation depending on privacy concern levels:
   1. Suppressed mode: Implements aforementioned semantic differentiation between result codes.
   2. Strict mode: Always returns errors, not "leaking" whether a region is permitted to be seen or just not present.

/cc @mmalenic @ohofmann @andrewpatto @jrobinso

[brainstorm]

This feature might additionally (require?) implementing BED support for htsget, otherwise those allowed/forbidden regions BED file might face CORS issues since they have to be provided out-of-band currently, see umccr/htsget-igv.js@a524a4c.

[jmarshall]

Discussion from the September meeting:

We talked through the use cases for this and discussed what it might need to look like.

• The spec would need some additional text to explain all this, probably near the paragraph containing “The response may however contain a superset of the desired results […]”. To the effect of: When the client does not have access to particular genomic regions (e.g. because they are clinically sensitive), the response may contain a subset of the requested results, being the results in the requested range outside the forbidden sensitive ranges.
• When this happens, the ticket should (optionally, perhaps) contain additional fields explaining what happened. This would probably take the form of an additional field saying what region(s) the returned data actually span, where that is smaller than the requested region. It would be good to have a concrete proposal of such a field.