Skip to content

Commit 4a15656

Browse files
beforanbeforan
authored
D20.2 Content drop (#4)
* Overview hopefully complete * analysis * Audience * Deployment and Project Setup * first pass of content complete
1 parent 00ea2d5 commit 4a15656

22 files changed

Lines changed: 545 additions & 27 deletions

File tree

‎app/_meta.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ export default {
44
type: "page",
55
display: "hidden",
66
},
7-
v1: {
8-
title: "Version 1",
7+
"d20.2": {
8+
title: "1st Training Package",
99
type: "page",
1010
},
1111
};

‎app/d20.2/_meta.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
export default {
2+
index: "Overview",
3+
audience: "Audience",
4+
concepts: "Concepts & Standards",
5+
analysis: "Analysis Types for TREs",
6+
"five-safes-tes": "Five Safes TES Tools",
7+
};

‎app/d20.2/analysis/page.mdx‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
import { Cards } from "nextra/components";
2+
import { ChevronRightCircleIcon } from "lucide-react";
3+
4+
# Analysis Types for TREs
5+
6+
There are a number of factors which affect what analysis can be performed firstly within a TRE and secondly when federating that analysis.
7+
8+
The [Federated Research Patterns](https://docs.federated-analytics.ac.uk/federated_research_patterns/patterns) as referred to in [Concepts & Standards](concepts) help with communication of these factors and their impact.
9+
10+
Factors affecting Analytical Patterns include:
11+
12+
- The type of analysis (typically determined by the algorithm(s) in use, not the desired statistics)
13+
- How that analysis will be initiated
14+
- What data movement is required to achieve the analysis
15+
- What data egress would occur
16+
17+
Additonally, all of the above must be considered in the context of the approach to federation of analysis and the capability of the tools implementing that approach.
18+
19+
TREs must find the above factors acceptable within their own governance frameworks and processes in order for analysis to be achievable.
20+
21+
## An interactive illustration
22+
23+
To help TREs understand the impact of their acceptance of these factors on what analysis may be achievable, we have developed the following resource:
24+
25+
<Cards.Card
26+
className="mt-4"
27+
icon={<ChevronRightCircleIcon />}
28+
title="Analysis Types for TREs"
29+
href="https://health-informatics-uon.github.io/analysis-types-for-TREs/"
30+
/>
31+
32+
The site contains a description of [Categories of Analysis methods](https://health-informatics-uon.github.io/analysis-types-for-TREs/Categorisation), and its approach to that categorisation (determined by the factors detailed above).
33+
34+
It also provides an interactive resource that, using the categories, allows the user to [get achievable analyses by their acceptance requirements](https://health-informatics-uon.github.io/analysis-types-for-TREs/analysis-breakdown).

‎app/d20.2/audience/page.mdx‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Training Package Audience
2+
3+
As described in the [overview](/d20.2), this training package has been developed primarily with **TRE Administrators** in mind.
4+
5+
However, there are adjacent audiences for whom some of the referenced material is relevant, discussed below.
6+
7+
Since this resource presents its content through a lens for TRE Administrators, there is also value in clarifying the roles considered to be performed within the label of "TRE Administrator".
8+
9+
## Relevant audiences
10+
11+
- **Federation Administrators**, people responsible for co-ordinating a Federation of TREs
12+
- For the purposes of this training package, this would include managing the configuration of Projects and their TRE and User memberships in a Federation-centralised Submission Service
13+
- This role may be performed by a TRE Administrator, e.g. within the "Front Door" TRE of a Federation, and as such is within the scope of this training package for TRE Administrators.
14+
- **TRE Administrators**, people responsible for managing and operating a TRE.
15+
- The material covers different aspects of TRE Administration that may be handled by different roles:
16+
- **Installers**, responsible for deploying and configuring the technical infrastructure and tools
17+
- **Administrators**, responsible for managing projects within the TRE
18+
- In the context of this training package and Five Safes TES, TRE Administrators will interact with the deployed tooling
19+
- **Egress Officers**, responsible for approving or rejecting project outputs for egress from the TRE.
20+
- **Researchers**, people who want to submit analyses to TREs within a Federation in which they are an approved member of a Project, using this tooling.
21+
- This training package does not focus on this audience, however some Researcher activities are described within the material, to provide TRE Administrators with context and understanding of the process from a Researcher's perspective:
22+
- How to submit and run analyses
23+
- How to retrieve approved outputs.

‎app/d20.2/concepts/page.mdx‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
import { Cards } from "nextra/components";
2+
import {
3+
PocketKnifeIcon,
4+
ChevronRightCircleIcon,
5+
LockIcon,
6+
HeartPulseIcon,
7+
GlobeIcon,
8+
} from "lucide-react";
9+
10+
# Concepts & Standards
11+
12+
There are a number of concepts and standards that TRE Administrators should be familiar with in the context of Federated Workflow Processing.
13+
14+
## Concepts
15+
16+
### Federated Research Patterns
17+
18+
The Federated Research documentation provides an overview of its concepts of "Patterns" and "Weaves":
19+
20+
<Cards.Card
21+
className="mt-2"
22+
icon={<ChevronRightCircleIcon />}
23+
title="Federated Research Patterns"
24+
href="https://docs.federated-analytics.ac.uk/federated_research_patterns/patterns"
25+
/>
26+
27+
<Cards.Card
28+
className="mt-2"
29+
icon={<ChevronRightCircleIcon />}
30+
title="Federated Research Patterns - Weaves"
31+
href="https://docs.federated-analytics.ac.uk/federated_research_patterns/weaves"
32+
/>
33+
34+
> [!NOTE]
35+
>
36+
> This training package focuses on documentation for the deployment and usage of tooling for the Five Safes TES Weave.
37+
38+
### Five Safes TES
39+
40+
## Standards
41+
42+
These standards are used by the Five Safes TES Weave and Tools. Included below is a brief description of what these standards are and why they have been adopted.
43+
44+
### Five Safes
45+
46+
- A set of principles to describe how decisions about granting access to data should be made. Covering Safe People, Data, Project, Outputs, Settings it is a set of interlocking principles
47+
- HDR UK have been a major proponent of the adoption of these principles across the UK health data eco-system. It has become the default mechanism across most domains and is the governance framework in which access decisions are made for HDR UK and DARE UK.
48+
49+
<Cards.Card
50+
className="mt-2"
51+
icon={<LockIcon />}
52+
title="Read more about Five Safes"
53+
href="https://ukdataservice.ac.uk/help/secure-lab/what-is-the-five-safes-framework/"
54+
/>
55+
56+
### Global Alliance for Genomics and Health (GA4GH)
57+
58+
- A joint academic and industry programme
59+
- Health Data Research UK and GA4GH have a strategic partnership
60+
- Pre-competitive work to enable global analysis of genomic data
61+
- Produced international standards
62+
63+
HDR UK and DARE UK have funded programmes to understand if the standards developed in GA4GH can be used in wider data environments, and especially within secure data environments for sensitive data.
64+
65+
The Federated Analytics programme in HDR UK and the TREvolution programme in DARE UK have MVPs to demonstrate the utility of these standards beyond health.
66+
67+
<Cards.Card
68+
className="mt-2"
69+
icon={<GlobeIcon />}
70+
title="Read more about GA4GH"
71+
href="https://www.ga4gh.org"
72+
/>
73+
74+
#### Task Execution Service (TES)
75+
76+
- Provides a standard mechanism for orchestrating complex analyses across different compute environments
77+
78+
<Cards.Card
79+
className="mt-2"
80+
icon={<PocketKnifeIcon />}
81+
title="Read more about TES"
82+
href="https://www.ga4gh.org/product/task-execution-service-tes/"
83+
/>
84+
85+
### Observational Medical Outcomes Partnership (OMOP)
86+
87+
- A standard that describes how health data should be grouped together in a set of common tables (a common data model)
88+
- Created and maintained by Observational Health Data Sciences and Informatics (OHDSI)
89+
- Adopted by NHS England Secure Data Environment programme
90+
- Used as a global standard for real world data studies
91+
92+
HDR UK have supercharged the UK efforts in the adoption of OMOP, by funding the conversion of data to OMOP, helping to establish OHDSI UK, and by the establishment of the real-world evidence framework.
93+
94+
The Federated Analytics programme in HDR UK have open-source tools in production that support the conversion of data to OMOP and the discovery of OMOP datasets globally. These are being developed further in DARE UK to demonstrate how they can be deployed across multiple secure data environments.
95+
96+
<Cards.Card
97+
className="mt-2"
98+
icon={<HeartPulseIcon />}
99+
title="Read more about OMOP"
100+
href="https://www.ohdsi.org/data-standardization/"
101+
/>

‎app/d20.2/five-safes-tes/_meta.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
export default {
2+
index: "Overview",
3+
deployment: "Deployment",
4+
"project-setup": "Project Setup",
5+
"workflow-processing": "Workflow Processing",
6+
};

‎app/d20.2/five-safes-tes/deployment/_meta.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
export default {
2+
index: "Overview",
3+
submission: "Submission",
4+
tre: "TRE",
5+
};

‎app/d20.2/five-safes-tes/deployment/page.mdx‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import { Cards } from "nextra/components";
2+
import { DoorOpenIcon, DoorClosedLockedIcon } from "lucide-react";
3+
4+
# Deploying the Five Safes TES Tools
5+
6+
There are quite a few components that make up the Five Safes TES Tools, but there are some common things to consider when deploying the tools.
7+
8+
Firstly, all the components except the TES Backend are available as OCI Container Images, and the deployment samples use Docker Compose.
9+
10+
The TES Backend is the installer's choice, and is expected to be deployed separately, but network-available to the other Five Safes TES TRE Layer components.
11+
12+
Finally, it is up to the installer how they distribute the installation of the components, though the guidance and samples here refer to separate environments for:
13+
14+
- Submission
15+
- Each TRE's components except TES (Agent, Egress, Storage, Auth, etc.)
16+
- Each TRE's TES Backend
17+
18+
## Requirements
19+
20+
Given the above there are some requirements for deployment, which are covered in each deployment guide.
21+
22+
## Deployment Guides
23+
24+
<Cards>
25+
<Cards.Card
26+
icon={<DoorOpenIcon />}
27+
title="Deploy Submission components"
28+
href="deployment/submission"
29+
/>
30+
<Cards.Card
31+
icon={<DoorClosedLockedIcon />}
32+
title="Deploy TRE components"
33+
href="deployment/tre"
34+
/>
35+
</Cards>

‎app/d20.2/five-safes-tes/deployment/submission/page.mdx‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Deploying Submission Components
2+
3+
The Submission Layer consists of the **Submission API** and **UI** components, along with some supporting components, such as **KeyCloak** and **MinIO**.
4+
5+
It is straight forward to run all the Submission Layer components on a single Virtual Machine via Docker Compose.
6+
7+
> [!TIP]
8+
>
9+
> Some of the components may be fulfilled by existing services, such as an existing KeyCloak or MinIO installation; these can be omitted from the Compose file used for deployment, and the environment settings changed to point at the existing services.
10+
11+
## Requirements
12+
13+
[Requirements for a Submission Layer](https://docs.federated-analytics.ac.uk/submission/requirements) deployment are provided in the Five Safes TES Documentation.
14+
15+
## Deploy
16+
17+
[Deployment](https://docs.federated-analytics.ac.uk/submission/deploy) guides and samples are provided in the Five Safes TES Documentation.

‎app/d20.2/five-safes-tes/deployment/tre/page.mdx‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Deploying TRE Components
2+
3+
Inside the TRE the following components are installed:
4+
5+
- the **TRE Agent API** and **UI**
6+
- the **Egress API** and **UI**
7+
- a **TES Backend**
8+
- some supporting components, such as **KeyCloak** and **MinIO**
9+
10+
It is straight forward to run most of the TRE components on a single Virtual Machine via Docker Compose.
11+
12+
The main exception to this is the TES Backend, which is expected to be deployed separately, but network-available to the other Five Safes TES TRE Layer components.
13+
14+
> [!TIP]
15+
>
16+
> Some of the components may be fulfilled by existing services, such as an existing KeyCloak or MinIO installation; these can be omitted from the Compose file used for deployment, and the environment settings changed to point at the existing services.
17+
18+
## Requirements
19+
20+
[Requirements for the TRE Components](https://docs.federated-analytics.ac.uk/tre_agent/requirements) deployment are provided in the Five Safes TES Documentation.
21+
22+
The Five Safes TES Documentation guides and samples also deploy [Funnel](https://ohsu-comp-bio.github.io/funnel/) as a standalone TES Backend on its own VM of the same specification.
23+
24+
## Deploy
25+
26+
Deployment guides and samples are provided in the Five Safes TES Documentation:
27+
28+
- [TRE Agent, Egress and supporting components](https://docs.federated-analytics.ac.uk/tre_agent/deploy)
29+
- [Funnel](https://docs.federated-analytics.ac.uk/five_safes_tes/TES/install_funnel) as a Five Safes TES Backend
30+
31+
## Connecting components
32+
33+
It is wise at the point of deployment to configure the components within a TRE to connect to each other.
34+
35+
The Five Safes TES Documentation provides guidance on this.
36+
37+
The main connection points are between:
38+
39+
- the **TRE Agent** and **Egress** applications - as described in [this guide](https://docs.federated-analytics.ac.uk/tre_agent/guides/connecttoegress)
40+
- the **TES Backend** and **MinIO** - as described in [this guide](https://docs.federated-analytics.ac.uk/five_safes_tes/TES/install_funnel)

0 commit comments

Comments
 (0)