Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Contributing

To add to this repository, please follow these steps:

1. Determine a folder or project with an access requirement need
2. Create an AR with `subjectsDefinedByAnnotations` checked
3. Determine combination of DUO codes and add the template to the template folder here
4. Add to the config.yml with the synapse id of a folder or project linked to the template you just added in step 3.
5. TODO: the code will execute and bind the schema specificed in step 3 to the project specified in step 4.
17 changes: 0 additions & 17 deletions Introduction.md

This file was deleted.

25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1 +1,26 @@
The Data Use Ontology (DUO) provides a helpful framework for gating access to data managed by Sage Bionetworks on the Synapse platform.

[DUO was developed by members of the Global Alliance for Genomic Health (GA4GH)](https://github.com/EBISPOT/DUO/blob/master/README.md): "DUO allows [users] to semantically tag datasets with restriction about their usage, making them discoverable automatically based on the authorization level of users, or intended usage".

At Sage, we extended DUO modifiers for our use cases and incorporated [derived annotations](https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/2597617665/API+Changes+to+support+Extension+of+Data+Access+Management+to+Users+outside+of+Sage+ACT) as a way of scaling governance support on projects by assigning access requirements (ARs)* to entities based on its DUO annotation.

_*ARs are applied in the form of a clickwrap (i.e., the user must agree to terms) and/or a managed access requirement (i.e., the user must provide evidence). Managed ARs may require evidence in the form of **Authentication** (e.g., training certification, profile validation, two-factor authorization) and/or **Authorization** (e.g., intended data use (IDU) statement, data use certificate (DUC), ethics approval letter from an institutional review board (IRB) or independent ethics committee (IEC))._


Learn more about [metadata structures](metadata_structure.md)

## PoC

The `add_duo_schema_bindings.py` is a proof-of-concept for how to add DUO schema bindings to Synapse projects. The script reads a configuration file (`config.yml`) that specifies the Synapse project ID and the DUO schema ID to bind to that project. The script then binds the specified DUO schema to the specified Synapse project or folder.

What's expected is the annotation key "duoCodes" and the codes can be found here: https://github.com/mc2-center/data-models/blob/main/modules/shared/duo.csv.


# Resources
- [EBISPOT DUO](https://github.com/EBISPOT/DUO/blob/master/README.md)
- [Extension of Data Access Management](https://sagebionetworks.jira.com/wiki/spaces/PLFM/pages/2597617665/API+Changes+to+support+Extension+of+Data+Access+Management+to+Users+outside+of+Sage+ACT)


# Publications
- https://doi.org/10.1016/j.xgen.2023.100381
- https://www.semantic-web-journal.net/system/files/swj3583.pdf
44 changes: 44 additions & 0 deletions add_duo_schema_bindings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import asyncio
from synapseclient.models import Project, File
import synapseclient
import json
import os

import yaml


def main():
"""Invoke adding DUO schema bindings"""
syn = synapseclient.Synapse()
# syn.setEndpoints(**synapseclient.client.STAGING_ENDPOINTS)
syn.login()
with open("config.yml", "r") as f:
config = yaml.safe_load(f)

Copy link
Member Author

@thomasyu888 thomasyu888 Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add another script to select the DUO codes, and to auto generate the json schema template + ARs

for configuration in config:
syn.get_available_services()
js = syn.service("json_schema")
my_org = js.JsonSchemaOrganization("governance.schemas")
my_org.create()
with open(configuration['filepath'], 'r') as f:
example_schema = json.load(f)
# Create a new JSON schema version for an existing organization
schema_name = os.path.basename(configuration['filepath']).replace(".json", '').replace("_", '.')
# TODO: need to figure out only pushing new schemas
print(example_schema)
print(schema_name)
new_version1 = my_org.create_json_schema(example_schema, schema_name)
print(new_version1.uri)
synapse_id = configuration['synapse_id']
request_body = {
"entityId": synapse_id,
"schema$id": new_version1.uri,
"enableDerivedAnnotations": True
}
syn.restPUT(
f"/entity/{synapse_id}/schema/binding", body=json.dumps(request_body)
)


if __name__ == "__main__":
main()
6 changes: 6 additions & 0 deletions config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# - filepath: templates/elite_study1.json
# synapse_id: syn59199129
# - filepath: templates/elite_long_life.json
# synapse_id: syn234324324
- filepath: templates/genie.json
synapse_id: syn63886516
File renamed without changes.
58 changes: 58 additions & 0 deletions templates/genie.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
{
"$schema": "http://json-schema.org/draft-07/schema",
"title": "DUO schema",
"$id": "some.project-main-1.3",
"description": "This schema defines how DUO should be used with Some Project.",
"allOf": [
{
"if": {
"properties": {
"duoCodes": {
"type": "array",
"items": {
"type": "string"
},
"contains": {
"const": "IRB"
}
}
}
},
"then": {
"properties": {
"_accessRequirementIds": {
"type": "array",
"contains": {
"const": 9606538
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a fresh set of ARs specific to that project

}
}
}
}
},
{
"if": {
"properties": {
"duoCodes": {
"type": "array",
"items": {
"type": "string"
},
"contains": {
"const": "HMB"
}
}
}
},
"then": {
"properties": {
"_accessRequirementIds": {
"type": "array",
"contains": {
"const": 9606537
}
}
}
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
}
},
,
{
"if": {
"properties": {
"duoCodes": {
"type": "array",
"items": {
"type": "string"
},
"contains": {
"const": "NPU"
}
}
}
},
"then": {
"properties": {
"_accessRequirementIds": {
"type": "array",
"contains": {
"const": sdfsdfs
}
}
}
}
}

]
}