External platforms

VIP executions take files as inputs and outputs, and VIP downloads and uploads from/to different data storage. The default storage used in VIP is a distributed one accessible from the VIP portal through the "File Transfer" tool. This is transparent for the user, and this corresponds to this simple data flow :

1. user uploads input data on VIP's distributed storage
2. user executes an application that uses these inputs to produce outputs on VIP's distributed storage
3. user downloads the results from VIP's distributed storage

However, in the cases where the inputs is already present in an external storage platform, it is more efficient to skip the first step and to make the execution use directly the data from the external storage. In a similar manner, the last step is also skipped by uploading the results directly on the external platform.

Currently, 2 external platforms are supported on VIP: Girder and Shanoir. Though not exactly an external platform, it is also possible to use some basic file management protocol to manage data directly from/to a local storage directly accessible from the execution platform.

Girder

VIP Configuration

To use an existing girder platform as an external platform, a VIP admin must declare it in VIP (in database). This requires:

• a unique prefix (like awesommeGirder)
• the girder platform URL

Note that a VIP plugin for Girder exists. In this case, the Girder platform must also be declared in the VIP CORS configuration.

User configuration

An user willing to use a girder platform configured in VIP must provide a Girder API key to VIP. The key must be created in its Girder account page, and provided to VIP through a REST API.

If the VIP plugin exists in Girder, the plugin can do it automatically in a specific VIP tab in the Girder account page.

Usage

If well configured, an user can use girder file as inputs and girder folders as inputs through the VIP REST API using this format:

"fileInput" : "awesommeGirder:xxxxxxxxx"

where xxxxxxxx is the girder id (sha1) of the desired file or folder.

Shanoir

VIP Configuration

To use an existing girder platform as an external platform, a VIP admin must declare it in VIP (in database). This requires:

• a unique prefix (like awesommeShanoir)
• some Shanoir URLs (download and upload)
• the associated Keycloak URL.

In most cases, the executions will be launched from a Shanoir portal, which requires:

• that the Shanoir URL must be declared in the VIP's CORS configuration
• that the associated keycloak must be declared in VIP configuration as an authentication method.

Also:

• a special "service" account must exist in keycloak to allow Shanoir to request VIP in its behalf and get the status of the running executions. This account must have the "SERVICE" role and a VIP account with the same email must exist in VIP.

User configuration

No specific configuration is necessary for an user to use a configured Shanoir instance. It just needs to have a VIP account with the same email address as in Shanoir.

Usage

If well configured, an user can use shanoir ressources as inputs and through the VIP REST API using this format:

"fileInput": "awesommeShanoir:filename.txt?resourceId=zzzzzz&format=dcm&converterId=4&clientId=shanoir-ng-front&&token=xxxx&refreshToken=yyyyy"

The format for output folder is simpler.

"resultsLocation": "awesommeShanoir:shanoirUploadId?&type=File&clientId=shanoir-ng-front&&token=xxxx&refreshToken=yyyyy"

File transfers

In some cases, if:

• the VIP execution resource is known in advance
• there is a storage directly accessible from it

Then inputs and outputs can be specified with this format:

"fileInput" : "file:/path/to/afile.txt"

"resultsLocation": "file:/path/to/a/folder"

---

Connecting a Docker Container to Shanoir

1. Create the vip-oidc.json File in /vip/.vip

1. Log in as the vip user:

   sudo su - vip

2. Create the file /vip/.vip/vip-oidc.json

3. Fill it using the following template:

   {
     "servers": [
       {
         "url": "https://prod.fr"
       },
       {
         "url": "https://qualif.fr"
       }
     ]
   }

   Replace the URLs with the instance you wish to connect to.

2. Configure Keycloak

1. Open the configuration file:

   vi /vip/.vip/vip.conf

2. Scroll to the bottom of the file and locate the variable keycloak.active

3. Change its value from False to True

3. Insert Instance Information into the Database

1. Connect to the VIP database:

   ./mariadb/bin/mariadb vip

2. Replace the variables in the VALUES section below and run this SQL query to insert the Shanoir instance configuration:

   INSERT INTO VIPExternalPlatforms
       (identifier, type, description, url, upload_url, keycloak_client_id, refresh_token_url)
       VALUES
       ('identifier', 'type', 'description',
        'url', 'upload_url', 'keycloak_client_id',
        'refresh_token_url')
       ON DUPLICATE KEY UPDATE
       type = VALUES(type),
       description = VALUES(description),
       url = VALUES(url),
       upload_url = VALUES(upload_url),
       keycloak_client_id = VALUES(keycloak_client_id),
       refresh_token_url = VALUES(refresh_token_url);

For example :

Field	Value
identifier	qualif
type	SHANOIR
description	Shanoir qualification
url	https://qualif.fr
upload_url	https://upload-qualif.fr
keycloak_client_id	uploader
refresh_token_url	https://qualif.fr/token

4. Create an Account on VIP

• Create your account on VIP using the same email address as the one used in Keycloak/Shanoir.