Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behaviour:
1. Run command '...'
2. with input '....'
4. See error

**Expected behaviour**
A clear and concise description of what you expected to happen.

**Versions (please complete the following information):**
- OS: [e.g. centos07]
- Python: [e.g. 3.6.1]
- Version [e.g. 1.3.4]
- URL [e.g. https://graphkb-api.bcgsc.ca/api]

**Additional context**
Add any other context about the problem here.
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.
2 changes: 1 addition & 1 deletion .github/workflows/compose-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest
name: test docker-compose start
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- run: mkdir -p databases/{postgres,orientdb}/{backup,data}
- run: mkdir keys
- run: docker-compose -f "docker-compose.dev.yml" up -d --build
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/npm-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
name: docker build
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: build the keycloak docker container
run: |
docker build --file demo/Dockerfile.auth --tag bcgsc/pori-auth .
Expand All @@ -21,11 +21,11 @@ jobs:
python-version: [3.8]
steps:
- name: checkout
uses: actions/checkout@v1
uses: actions/checkout@v4
with:
submodules: true
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: install dependencies
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ jobs:
python-version: [3.8]
steps:
- name: checkout
uses: actions/checkout@v1
uses: actions/checkout@v4
with:
submodules: true
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: install dependencies
Expand All @@ -27,7 +27,7 @@ jobs:
- run: bash docs/build_external.sh
- run: mkdocs build
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./dist
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,5 @@ build-docs
docs/_pori*
docs/*/_pori*
docs/*/python_adapter
docs/developer_reference/ipr
docs/developer_reference/graphkb
docs/developer_reference/pori_python
docs/ipr/includes
26 changes: 6 additions & 20 deletions docs/build_external.sh
Original file line number Diff line number Diff line change
@@ -1,31 +1,18 @@
# Pulls the Python adapter documentation from their respective repositories
# and builds them into the main documentation

# clone the IPR python adapter if it does not exist, otherwise update
if ! [ -d docs/_pori_ipr_python ];
then
git clone https://github.com/bcgsc/pori_ipr_python.git docs/_pori_ipr_python
echo "hide: true" > docs/_pori_ipr_python/.pages
else
cd docs/_pori_ipr_python
git checkout master
git pull
cd ../..
fi

# clone the GraphKB Python adapter if it does not exist, otherwise update
if ! [ -d docs/_pori_graphkb_python ];
if ! [ -d docs/_pori_python ];
then
git clone https://github.com/bcgsc/pori_graphkb_python.git docs/_pori_graphkb_python
echo "hide: true" > docs/_pori_graphkb_python/.pages
git clone https://github.com/bcgsc/pori_python.git docs/_pori_python
echo "hide: true" > docs/_pori_python/.pages
else
cd docs/_pori_graphkb_python
git checkout master
cd docs/_pori_python
git checkout main
git pull
cd ../..
fi


# clone the loaders repo if it does not exist, otherwise update
if ! [ -d docs/graphkb/_pori_graphkb_loader ];
then
Expand All @@ -40,8 +27,7 @@ fi

# now build the reference python module API files
markdown_refdocs \
docs/_pori_graphkb_python/graphkb \
docs/_pori_ipr_python/ipr \
docs/_pori_python/pori_python \
-o docs/developer_reference \
--link

Expand Down
8 changes: 4 additions & 4 deletions docs/demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,14 @@ By clicking on a row in the above table you will be brought to the corresponding

## GraphKB

The demo instance of IPR is at [pori-demo.bcgsc.ca/graphkb](https://pori-demo.bcgsc.ca/graphkb). This will bring you to the login page of the GraphKB client application. The `iprdemo` credentials will also work with GraphKB to demonstrate single sign on. However, we also provide an admin demo account for GraphKB so that users can explore the admin-only sections of the application. The credentials for the administrative account are
The demo instance of GraphKB is at [pori-demo.bcgsc.ca/graphkb](https://pori-demo.bcgsc.ca/graphkb). This will bring you to the login page of the GraphKB client application. The `iprdemo` credentials will also work with GraphKB to demonstrate single sign on. However, we also provide an admin demo account for GraphKB so that users can explore the admin-only sections of the application. The credentials for the administrative account are

- username: `graphkb_admin`
- password: `graphkb_admin`

## APIs

If you are a developer looking for a demo of the APIs for GraphKB and IPR their documentation can be found here
If you are a developer looking for a demo of the APIs for GraphKB and IPR, their documentation can be found here

- [IPR OpenAPI Spec](https://pori-demo.bcgsc.ca/ipr-api/api/spec/)
- [GraphKB OpenAPI Spec](https://pori-demo.bcgsc.ca/graphkb-api/api/spec) ([swagger version](https://pori-demo.bcgsc.ca/graphkb-api/api/spec/swagger))
- [IPR Specifications](https://pori-demo.bcgsc.ca/ipr-api/api/spec/)
- [GraphKB Specifications](https://pori-demo.bcgsc.ca/graphkb-api/api/spec)
2 changes: 1 addition & 1 deletion docs/developer_reference/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Every PORI repository has a corresponding github actions plan. These plans autom

### On Release

- Python Adapters are published to [PyPI](https://pypi.org/project/pip/)
- Python Adapters are published to [PyPI](https://pypi.org/)
- Docker containers are published to [DockerHub](https://hub.docker.com/)

## Documentation
Expand Down
73 changes: 1 addition & 72 deletions docs/developer_reference/getting_started.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,3 @@
# Getting Started

For working on most of the PORI-related projects you will need to have a number of the components set up. For example, to work on the GraphKB API you will need both an OrientDB server and a Keycloak server already running.

If your institution regularly works on PORI related projects then we recommend setting up a development instance of the PORI platform which your developers can point their applications to. If you do not have access to something like this, then the easiest way to get the dependencies for whatever part of the PORI platform you are working on up and running is by running the development version of the docker compose configuration found in this repository: [docker-compose.dev.yml](https://github.com/bcgsc/pori/blob/master/docker-compose.dev.yml).

```yaml title="docker-compose.dev.yml"
--8<-- "./docker-compose.dev.yml"
```

The demo uses a default keycloak setup with a realm "PORI" and two clients: "GraphKB" and "IPR".
For convenience there are also a number of default users which all have the default password of "secret".

![default users](../images/pori-keycloak-default-users.png)

## Run docker-compose

First thing you should do is create new/empty directories for the data stored by GraphKB and IPR.

```bash
mkdir -p databases/{postgres,orientdb}/{backup,data}
```

You should also create a new directory for storing the public key from keycloak. This key will be downloaded and stored so that it can be used in checking incoming tokens by the GraphKB and IPR APIs. If this directory already exists you should delete and remake it.

```bash
mkdir keys
```

Now you are ready to start up with the dev compose yml

```bash
docker-compose -f docker-compose.dev.yml up -d
```

It will take a minute or two for all of the servers to start. You can check how they look with docker

```bash
docker ps
```

If any of them show "(health: starting)" then they are not ready yet.

### Viewing Log Files

Sometimes you will need to check the logs from the various servers, this can be done with the docker logs command. First find the container ID (or name) by listing all the running containers with `docker ps` and then run the following

```bash
docker logs <CONTAINER ID>
```

### Loading Data into GraphKB

If you are running the GraphKB loader via its docker container you will need to tell it to use the host network so that it is able to find the GraphKB API.

Here is an example of running the GraphKB Loader on the vocabulary terms using the docker container and the docker-compose setup described above.

First download the vocabulary terms data

```bash
wget https://raw.githubusercontent.com/bcgsc/pori_graphkb_loader/develop/data/vocab.json
```

Then you can load these terms using the ontology file loader

```bash
docker run --net host bcgsc/pori-graphkb-loader:latest \
-u graphkb_importer \
-p secret \
-g http://localhost:8888/api \
file \
ontology \
vocab.json
```
Please refers to the [Install page](../install.md) for instructions on how to use Docker and Docker Compose to set up the PORI platform for development.
4 changes: 3 additions & 1 deletion docs/developer_reference/index.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# About the Developer Reference

This section contains auto-generated documentation from package implementations and docstrings generated via the python package [markdown_refdocs](https://github.com/creisle/markdown_refdocs).
The **Pori python** sub-section (see side menu) contains auto-generated documentation from the PORI python package implementation and docstrings. It has been generated by [markdown refdocs](https://github.com/creisle/markdown_refdocs).

We recommand reading the [Contributing Guidelines](contributing.md) and [Getting Started](getting_started.md) sections first.
5 changes: 2 additions & 3 deletions docs/graphkb/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ GraphKB is a graph-based implementation of a cancer knowledge base. An overview

## Features


### Dual Aggregate and Standalone KB

GraphKB functions both as an aggregate knowledge base as well as a standalone application. Data can be imported from many external sources (see the [data loading page](./loading_data.md)), but it can also be input directly. This also allows users to make manual updates and changes to imported content which is helpful when there may be errors or missing data that needs to be addressed in a time-sensitive manner.
Expand Down Expand Up @@ -67,6 +68,4 @@ modules to simplify this process. See the [data loading page](./loading_data.md)

## Using the Python Adapter

The python adapter to GraphKB is provided for users who would like to incorporate an instance of
GraphKB into their own scripts and pipelines. Additionally it is used by the IPR python adapter to
connect to GraphKB.
The PORI Python Adapter is provided for users who would like to incorporate an instance of GraphKB (and/or IPR) into their own scripts and pipelines.
20 changes: 15 additions & 5 deletions docs/graphkb/loading_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Most popular resources which have pre-built loaders provided for GraphKB are lis

This is an external knowledge base which can be imported as statements into GraphKB.

- ![logo](https://clinicaltrials.gov/ct2/html/images/ct.gov-nlm-nih-logo.png)
- ![logo](https://clinicaltrials.gov/assets/images/ctg-header-logo.svg)

### ClinicalTrials.gov

Expand All @@ -60,7 +60,7 @@ Most popular resources which have pre-built loaders provided for GraphKB are lis

Catalogue of Somatic Mutations in Cancer. Loaders are written for importing both the resistance mutations as well as recurrent fusions information.

- ![cosmic logo]()
- ![dgidb logo]()

### DGIdb

Expand Down Expand Up @@ -168,7 +168,7 @@ Most popular resources which have pre-built loaders provided for GraphKB are lis

A collection of putative alteration/action relationships identified in clinical, preclinical, and inferential studies.

- ![logo](https://ncit.nci.nih.gov/ncitbrowser/images/thesaurus_browser_logo.jpg)
- ![logo](https://evsexplore.semantics.cancer.gov/evsexplore/assets/images/NCItTile.png)

### NCIt

Expand All @@ -190,13 +190,23 @@ Most popular resources which have pre-built loaders provided for GraphKB are lis
As this is not an open data resource, using this loader will require licensing specific to your
user/instance. This is an external knowledge base which can be imported as statements into GraphKB.

- ![logo](https://oncotree.mskcc.org/assets/favicon-CLRaiHsg.svg)

### OncoTree

[https://oncotree.mskcc.org](https://oncotree.mskcc.org/)

[Attribution](https://oncotree.mskcc.org/about.html#useful-links)

Classify cancers by both histological and molecular traits.

- ![logo](http://uberon.github.io/images/u-logo.jpg)

### Uberon

[https://uberon.github.io](https://uberon.github.io/)
[https://obophenotype.github.io/uberon/](https://obophenotype.github.io/uberon/)

[CC BY 3.0](https://github.com/obophenotype/uberon/issues/1139)
[CC BY 3.0](https://github.com/obophenotype/uberon/?tab=License-1-ov-file#readme)

The uberon ontology contains anatomical entity definitions.

Expand Down
3 changes: 1 addition & 2 deletions docs/graphkb/matching/gene.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Gene Example

The [`get_equivalent_features()`](./../reference/graphkb/match/#get_equivalent_features) method is
used to find genes *equivalent* to the input/target feature.
The [`get_equivalent_features()`](../../developer_reference/pori_python/graphkb/match.md#get\_equivalent\_features) method is used to find genes *equivalent* to the input/target feature.

```python
from graphkb.match import get_equivalent_features
Expand Down
2 changes: 1 addition & 1 deletion docs/graphkb/matching/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Ontology Algorithm

The matching algorithm implemented by this adapter heavily uses the Graph Structure of GraphKB to resolve aliases, generalisms, etc. The default behaviour of this algorithm is described below and shown in the related examples. This is primarily accomplished via the `similarTo` query type provided by the GraphKB API. This algorithm can be viewed interactively in GraphKB via the matching section of the about pages
The matching algorithm implemented by this adapter heavily uses the Graph Structure of GraphKB to resolve aliases, generalisms, etc. The default behaviour of this algorithm is described below and shown in the related examples. This is primarily accomplished via the `similarTo` query type provided by the GraphKB API. This algorithm can be viewed interactively in GraphKB via the matching section of the about page.

![matching gif](../images/graphkb_client.matching_demo.gif)

Expand Down
23 changes: 23 additions & 0 deletions docs/graphkb/matching/subgraphs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Subgraphs

Introduced with the GraphKB API [v3.16.0](https://github.com/bcgsc/pori_graphkb_api/releases/tag/v3.16.0) release, the new `/subgraphs` API route traverses an ontology graph and returns a subgraph of all traversed nodes and edges (see the [specifications](https://graphkb-api.bcgsc.ca/api/spec#section/Subgraphs) for usage examples).

It aims to overcome two limitations of the fixed subqueries, such as the `similarTo` query previously described in the [Ontology Algorithm](../) and [Gene Example](./gene.md) sections:

1. #### A graph traversal that is *potentially incomplete*
After resolving aliases for the second and final time, any remaining inheritance-like edges are left untraversed. This is particularly a problem when, for a given ontology, many sources and versions are referencing each other (e.g. [OncoTree](../loading_data.md#oncotree), [NCIt](../loading_data.md#ncit) and [Disease Ontology](../loading_data.md#disease-ontology) for Diseases).

2. #### Results that are *challenging to interpret*
Since only vertices are returned, one can only wonder which edges were traversed during graph traversal.

### Thorough graph traversal

By leveraging OrientDB's graph traversal engine, the subgraphs route ensures that all relevant vertices are reached (subject to the depth limit, which defaults to 100 but can be overridden).

### Results returned as a subgraph

Both vertices and edges are returned, alongside an adjacency list. Since the returned subgraph can be disconnected (this can ossure if the base records of the traversal were disconnected in the first place), a list of the different components is also returned.

### Virtualization

A simplified 'virtual' version of the traversed graph can also be returned, where each group of similar nodes (nodes linked together by similarity edges) is collapsed into a single virtual node, and hierarchical edges are collapsed into virtual edges and redirected from and to the corresponding virtual nodes. This functionality, when coupled with a graph visualization tool, greatly simplifies graph traversal interpretation.
Loading
Loading