Years ago we wrote a bash script that spins up Dataverse in EC2. Here's the PR where added it as well as the docs:
It works great for the following use cases:
- CI, called by Jenkins
- one-off spin ups of an env for a demo (JSF only)
These days we have a new React UI at https://github.com/IQSS/dataverse-frontend that requires Keycloak, which means we can't use our bash script, which knows nothing about Keycloak.
In the "dev-env" directory of the frontend repo, we have a way to spin up the backend, frontend, Keycloak (and other necessary services) all configured and allowing login. The idea is to get something similar working in the cloud, like we have now using the bash script above.
I'm not picky about the technology we use, as long as it's open source. Originally, I figured we'd just add a "container mode" to the bash script with -c or whatever, as I've described here:
However, @srmanda-cs and @poikilotherm seem to favor modern tools, as discussed at https://dataverse.zulipchat.com/#narrow/channel/375812-containers/topic/Docker.20mode.20when.20spinning.20up.20EC2.20instances/near/591807528
Whatever tools we use, part of the definition of done is to have sufficient documentation (as above) that clearly steps through how to both get set up with tooling and do the spin up.
We can certainly create sub-issues for smaller chunks of this work. I'll make some suggestions below.
Phase 1: backend
Get just the backend (which still includes JSF, currenly) spun up. I know I mentioned the "dev-env" directory of the frontend repo above, but we could even start with the compose.yml file at https://guides.dataverse.org/en/6.10.1/container/running/demo.html#quickstart
We're planning to try OpenTofu ( https://opentofu.org ) first. Let's put the OpenTofu files in /docker/opentofu.
Phase 2: HTTPS
The spun-up environment should have a valid HTTPS cert.
Related:
Phase 3: frontend (no auth), arbitrary branches for both backend and frontend
Add the frontend but don't let people log into it yet.
Let's say we want to spin up this branch from the frontend:
It's already merged but let's say it requires this branch from the backend:
Phase 3: frontend (with auth)
Add Keycloak and let the frontend log in with users that are preconfigured in Keycloak. For example, see below from https://github.com/IQSS/dataverse-frontend/blob/v.0.3.0/dev-env/keycloak/test-realm.json#L439
"users": [
{
"id": "e5531496-cfb8-498c-a902-50c98d649e79",
"createdTimestamp": 1684755721064,
"username": "curator",
"enabled": true,
"totp": false,
"emailVerified": true,
"firstName": "Dataverse",
"lastName": "Curator",
"email": "dataverse-curator@mailinator.com",
"credentials": [
{
"id": "664546b4-b936-45cf-a4cf-5e98b743fc7f",
"type": "password",
"userLabel": "My password",
"createdDate": 1684755740776,
"secretData": "{\"value\":\"AvVqybCNtCBVAdLEeJKresy9tc3c4BBUQvu5uHVQw4IjVagN6FpKGlDEKOrxhzdSM8skEvthOEqJkloPo1w+NQ==\",\"salt\":\"2em2DDRRlNEYsNR3xDqehw==\",\"additionalParameters\":{}}",
"credentialData": "{\"hashIterations\":27500,\"algorithm\":\"pbkdf2-sha256\",\"additionalParameters\":{}}"
}
],
"disableableCredentialTypes": [],
"requiredActions": [],
"realmRoles": ["default-roles-test"],
"notBefore": 0,
"groups": ["/curators"]
}
],
Future plans/ideas
Years ago we wrote a bash script that spins up Dataverse in EC2. Here's the PR where added it as well as the docs:
It works great for the following use cases:
These days we have a new React UI at https://github.com/IQSS/dataverse-frontend that requires Keycloak, which means we can't use our bash script, which knows nothing about Keycloak.
In the "dev-env" directory of the frontend repo, we have a way to spin up the backend, frontend, Keycloak (and other necessary services) all configured and allowing login. The idea is to get something similar working in the cloud, like we have now using the bash script above.
I'm not picky about the technology we use, as long as it's open source. Originally, I figured we'd just add a "container mode" to the bash script with
-cor whatever, as I've described here:However, @srmanda-cs and @poikilotherm seem to favor modern tools, as discussed at https://dataverse.zulipchat.com/#narrow/channel/375812-containers/topic/Docker.20mode.20when.20spinning.20up.20EC2.20instances/near/591807528
Whatever tools we use, part of the definition of done is to have sufficient documentation (as above) that clearly steps through how to both get set up with tooling and do the spin up.
We can certainly create sub-issues for smaller chunks of this work. I'll make some suggestions below.
Phase 1: backend
Get just the backend (which still includes JSF, currenly) spun up. I know I mentioned the "dev-env" directory of the frontend repo above, but we could even start with the compose.yml file at https://guides.dataverse.org/en/6.10.1/container/running/demo.html#quickstart
We're planning to try OpenTofu ( https://opentofu.org ) first. Let's put the OpenTofu files in /docker/opentofu.
Phase 2: HTTPS
The spun-up environment should have a valid HTTPS cert.
Related:
Phase 3: frontend (no auth), arbitrary branches for both backend and frontend
Add the frontend but don't let people log into it yet.
Let's say we want to spin up this branch from the frontend:
It's already merged but let's say it requires this branch from the backend:
Phase 3: frontend (with auth)
Add Keycloak and let the frontend log in with users that are preconfigured in Keycloak. For example, see below from https://github.com/IQSS/dataverse-frontend/blob/v.0.3.0/dev-env/keycloak/test-realm.json#L439
Future plans/ideas