[Issue]: Verified implementations of GA4GH products #74
Description
Issue Title
Verified implementations of GA4GH products
Issue Type
Directive Document
Problem Statement
Current state
Currently GA4GH has no way of verifying the level of compatibility or depth of implementation of a GA4GH product. We have a listing of implementations on our central website but what it means to be an implementation is unclear. Finally for those wanting to implement a product we have little to no guidance beyond the documents accompanying our products as they flow through PDAP
Desired state
We seek to have a state where we can define a set of criteria and point to verified implementations of products. In the first instance we want to solve this for open source software (since this is the most tangible) but then expand to other types of products. The same is true for all that we want anyone who creates an implementation of a product to be tagged with this status if it meets the laid out criteria
Impact
One basic impact is a lack of any kind of end to end implementations demonstrating the interoperability of our products. Projects such as FASP and now those in GIF seek to provide some of this. A suite of implementations that are known to be good would allow for these known demonstration flows to be constructed and demonstrate interoperability. If there are issues they would be flagged as part of this process.
Scope Validation
The work affects everyone in GA4GH who produces products. The PDAP asks for two implementations of a product to review. There is no criteria set to ensure they are "good" implementations of a product though. Plus closed-source or closed network implementations can cause issues in the approval process.
We target this work because it will
- Highlight the need for harmonization in our products and what we consider to be "good"
- Recognises those who produce "good" implementations
- Span across multiple work streams who likely already have implementations that can fit this criteria
Proposed Solution(s)
See our in-progress doc and this has been discussed at the July in person TASC meeting
Estimated Effort Level
Low (1-2 months, minimal resources)
Success Criteria
- Approval of base criteria to assess implementation
- Execution of the criteria against at least three example implementations and feedback into the documentation
Post release:
- External projects submit at least three implementations to go through the process
- Creation of at least one compliance suite in response to this
- Engagement with one closed source project to adapt criteria
How will this issue aid GA4GH harmonization?
- How does this aid harmonization of GA4GH products?
- What barriers to organization-wide harmonization does this address?
- Which specific alignment challenges does this solve?
- Does this require cross-work stream development?Please describe the issue here
Additional context
Please provide any additional pieces of information you feel is relevant to this issue
Work Streams Raising This Issue
- Clinical & Phenotypic Data (Clin/Pheno)
- Cloud Work Stream
- Data Security
- Data Use & Researcher IDs (DURI)
- Discovery
- Genomic Knowledge Standards (GKS)
- Large Scale Genomics (LSG)
- Regulatory & Ethics (REWS)
- Data Models & Schemas Committee (DaMaSC)
- Genomic Implementation Forum (GIF)
- Technical Team
- Other (specify below)
Other Groups Raising This Issue
No response
Work Streams That Will Be Impacted
- Clinical & Phenotypic Data (Clin/Pheno)
- Cloud Work Stream
- Data Security
- Data Use & Researcher IDs (DURI)
- Discovery
- Genomic Knowledge Standards (GKS)
- Large Scale Genomics (LSG)
- Regulatory & Ethics (REWS)
- Data Models & Schemas Committee (DaMaSC)
- Genomic Implementation Forum (GIF)
- Technical Team
- Other (specify below)
Other Groups That Will Be Impacted
No response
Key Stakeholders to Consult
Org/communities
- Implementation writers
- Products going through PDAP now
Tech experts
- Tech team members
- Implementation developers
- REWS experts
Decision makers
- PSC
- Exec
- TASC
Products affected
We will target htsget (server implementation), VRS (schema), refget (server implementation) and DRS (server implementation) to go through this process. We also want to engage with REWS to understand if such a criteria actually works for their products too
Additional Context
No response
Priority Level
Medium (should be addressed within 3-6 months)
Additional Tags
- Documentation
- API
- Schema
- Security
- Performance
- Interoperability
- Compliance
- User Experience
- Infrastructure
- Testing