Merge pull request #726 from GenSpectrum/main

Update prod from main

Author: chaoran-chen

.github/pull_request_template.md

@@ -16,3 +16,7 @@ When applicable, add a screenshot showing the main effect this PR has, even if "
 e.g. changed layout, changed button text, different logging in backend.
 This helps others quickly grasping what you did even if they are not familiar with the code base.
 -->
+
+## PR Checklist
+- [ ] All necessary documentation has been adapted.
+- [ ] The implemented feature is covered by an appropriate test.

CONTRIBUTING.md

@@ -34,7 +34,7 @@ Every view has a section that lets the user set filters on the sequence data tha
 - selecting a region (sequences from Germany, the US or Asia),
 - considering only sequences with a certain mutation.
 
-Every view also has a section that shows a number of diagrams that visualize the data that match the filters that the user set.
+Every view also has a section that shows diagrams that visualize the data that match the filters that the user set.
 
 The currently set filters are considered the "state" of the view.
 
@@ -59,7 +59,7 @@ Think: `prevalence = count(variant in Germany) / count(all sequences in Germany)
 
 ### Basic guidelines
 
-We make use of some major libraries: Astro, React, Daisyui, and Tailwindcss.
+We make use of some major libraries: Astro, React, DaisyUI, and Tailwindcss.
 
 - Render pages using Astro when possible
     - Astro render HTML on the server, which is faster than rendering on the client.

README.md

@@ -19,6 +19,10 @@ Currently, we support the following organisms:
 - West Nile Virus
 - RSV-A
 - RSV-B
+- Mpox
+- Ebola Sudan
+- Ebola Zaire
+- Crimean-Congo Hemorrhagic Fever (CCHF)
 
 The dashboards also allow users to create subscriptions to receive notifications
 when there is new data above a configurable threshold for certain variant of an organism.

docs/arc42/01-introduction.md

@@ -0,0 +1,19 @@
+# Introduction and Goals
+
+> Files in this folder document the architecture of the project using the [arc42](https://arc42.org/) template.
+
+The GenSpectrum dashboards visualize pathogen genomic data obtained from [LAPIS](https://github.com/GenSpectrum/LAPIS) instances.
+
+Important requirements are:
+
+* **interactive**: 
+  Users can set filters on the data (such as restricting the data to a certain date range, region, or mutation).
+  This allows users to explore the data in a flexible way.
+* **realtime**:
+  The visualizations update in realtime when the user changes the filters.
+  This allows users to be ahead of time since they don't depend on us providing them with precomputed data.
+  
+One important use case is that users should be able to search for new emerging variants of pathogens.
+Interactivity and realtime updates are crucial for this use case,
+since users can browse the data.
+

docs/arc42/02-architecture-constraints.md

@@ -0,0 +1,13 @@
+# Architecture Constraints
+
+* The dashboards are an app that run in a standard web browser.
+  We host an instance that is accessible via the internet.
+* The genomic data is obtained from LAPIS instances.
+* We want to use the [Dashboard Components](https://github.com/GenSpectrum/dashboard-components/tree/main/components)
+  as much as possible.
+
+Non-constraints:
+
+* In contrast to other GenSpectrum applications, the dashboards do not need to be reusable.
+  It is not intended to be hosted by anyone else (although of course it is possible to do so).
+  It is not intended to be configurable by the user.

docs/arc42/03-context.md

@@ -0,0 +1,12 @@
+# System Context
+
+![context](../plantuml/03-context.svg)
+
+* Users access the dashboards website via HTTP.
+* The backend is not publicly accessible since it does not have authorization implemented.
+  All access should be proxied through the website.
+* To log in, we use a GitHub OAuth app.
+  Users can log in with their GitHub account to access additional account related features.
+  * Since the subscription feature is currently on hold and not fully implemented yet,
+    the backend is not used and the login functionality is disabled.
+  * No login is required to access the visualization pages.

docs/arc42/04-solution-strategy.md

@@ -0,0 +1,20 @@
+# Solution Strategy
+
+We decided to use Astro and React for the frontend.
+Astro comes with a server application that is deployed to our servers.
+Thus, we don't need a separate infrastructure that serves files for the frontend,
+and we can use it as a proxy for the backend.
+Where more interactivity is needed, we use React components within the static Astro pages.
+
+For data visualization, we use the [GenSpectrum Dashboards Components](https://github.com/GenSpectrum/dashboard-components).
+Every organism that the dashboards support has several pages that are composed of dashboard components
+to support a certain view on the data.
+A filter section lets the user specify which data should be considered.
+The filter is persisted in the URL as query parameters so that users can share the current view with others or bookmark it.
+
+The backend is a Spring Boot application that provides a REST API. Data is stored in a PostgreSQL database.
+The backend was developed to support the subscriptions features of the dashboards.
+The subscriptions are stored in the database and can be managed via the backend API.
+They are linked to the GitHub account of the user. 
+At the time of implementation, setting up a GitHub app was the easiest way to authenticate users.
+

docs/arc42/06-runtime-view.md

@@ -0,0 +1,10 @@
+# Runtime View
+
+This depicts an example use case of a user using the GenSpectrum dashboards to explore pathogen genomic data:
+
+![runtime-view](../plantuml/06-runtime-view-filter-sequences.svg)
+
+This diagram shows the login process that is required before adding a subscription:
+
+![add-subscription](../plantuml/06-runtime-view-add-subscription.svg)
+

docs/arc42/07-deployment-view.md

@@ -0,0 +1,10 @@
+# Deployment View
+
+The dashboards are deployed on the staging and production servers of GenSpectrum.
+Every commit builds a Docker image of the backend and the website.
+The Docker images are then deployed via Docker compose.
+
+The staging server has the `latest` Docker tag deployed which is built on the `main` branch,
+while the production server has the `prod` tag deployed which is built on the `prod` branch.
+Deployment happens automatically via [Watchtower](https://github.com/containrrr/watchtower). 
+It pulls and starts new images once they are available.

docs/arc42/09-architecture-decisions.md

@@ -0,0 +1,31 @@
+# Architecture Decisions
+
+## Login Solution
+
+### Context
+
+For the subscriptions feature, we need a way to authenticate users.
+Every user should be able to manage their subscriptions and see their data.
+We also need a way to notify users in case the subscription triggers (via e-mail or Slack hook).
+
+We considered several options for authentication:
+
+* Using Keycloak
+* Using GitHub OAuth
+* Using a managed service
+
+### Decision
+
+We decided to use a GitHub OAuth app for authentication.
+It was an easy and straightforward way to authenticate users.
+We don't need to host or maintain an authentication service ourselves.
+
+We also decided to use [auth-astro](https://github.com/nowaythatworked/auth-astro)
+to handle the authentication flow in the Astro application.
+It supports GitHub OAuth out of the box.
+
+We refrained from using Keycloak since setting it up is more effort.
+We would have needed to host it on one of our servers and configure it properly.
+Although is it relatively easy to get a Keycloak Docker image running,
+it is still difficult to get the configuration right.
+None of the team members was an expert in Keycloak and misconfigurations can lead to security issues.