Merge pull request #86 from ScilifelabDataCentre/PATs-for-cli-users

SQ-889: Personal access tokens for programmatic access

Author: RMCrean

docs/development/personal_access_tokens.md

@@ -0,0 +1,37 @@
+# Personal Access Tokens (PATs)
+
+Personal Access Tokens (PATs) provide an alternative authentication mechanism to password and JSON web tokens (JWTs) for users who need to make use of DivBase programmatically. For example, from HPC job scripts or pipelines.
+
+A user never logs in with a PAT, they instead create a PAT via the frontend and pass the PAT in each API call. PATs can be configured to expire or not expire. On every API call, a db lookup is made in the `personal_access_token` table to verify the token and check its permissions - and get the corresponding user. This means that PATs can be revoked immediately by deleting/soft-deleting them from the db.
+
+## PATs vs JWTs
+
+JWTs: When an interactive user logs in via the CLI or the frontend, the server issues a short-lived **JWT access token** and a longer-lived **JWT refresh token**. The CLI auto-refreshes the access token when needed using the refresh token.
+
+PATs have no refresh cycle and no concept of login. A PAT is much closer to an API key. A user creates a PAT via the frontend, copies the plaintext token to their device (probably making it an environment variable) and each API call to DivBase will use the PAT as a `Bearer` token in the `Authorization` header. The `get_current_user` in `deps.py` is responsible for handling the two diff auth mechanisms in the API (JWTs or PATs). All PATs are prefixed with `divbase_pat_` followed by a long random string. The token is only shown to the user at creation time and a hash of it is stored in the db.
+
+## Permissions model
+
+Permissions are stored in the `permissions` column of the `personal_access_token` table as a **JSONB**, validated against the `PATPermissions` Pydantic model (defined in `divbase-lib`):
+
+```python
+class PATPermissions(BaseModel):
+    all_projects: bool = False
+    projects: dict[str, str] = Field(default_factory=dict)
+    task_history: bool = False
+```
+
+| Scope | Meaning |
+|---|---|
+| `all_projects` | Access all projects the user is a member of, at their membership role |
+| `projects` | Access specific projects only; each project mapped to a max role (`read`, `edit`, `manage`) |
+| `task_history` | Access the task history endpoints which are not project specific |
+
+Note that almost all divbase-api routes called by the CLI tool are project scoped. So they rely on the `get_project_member` dependency. The only exceptions to that are:
+
+1. `divbase-cli auth whoami`
+2. `divbase-cli task-history user` and `divbase-cli task-history id [job_id]`
+
+For whoami, decision made that all PATs can call this endpoint so can just rely on `get_current_user` dependency which checks if the token is valid but does not check any scopes/permissions. For the task history endpoints, we have a `require_task_history_scope` dependency that checks if the token has the `task_history` scope is set to True. Note that task history project is dependent on the `projects` scope like all other endpoints.
+
+The effective project role for a scoped PAT is `min(pat_role, user_membership_role)` — the PAT can never escalate beyond the user's actual membership.

docs/user-guides/account-management.md

@@ -1,5 +1,52 @@
-# Account management
+# Account Management
 
-Simple guide on how to use the website and how to request a project etc...
+## Getting a project
 
-TODO
+!!! Warning
+    DivBase is under active development and not in production yet. We are shortly planning to create a number of projects for some pilot users. If you would like to be involved in this pilot testing phase, please let us know!
+
+Projects in DivBase are created by the DivBase team. To request a new project, you need to contact us. Please include a very short description of what the project would be for and who you plan to give access too. Please email us with your university affiliated email address. please make sure you have already registered for a DivBase account and verified your email before contacting us about creating a project.
+
+Once your project is created, you will be added as a member with the **Manage** role, which allows you to add and remove other members.
+
+## Adding members to a project
+
+Project members can be added by anyone with the **Manage** role on that project.
+
+1. Navigate to your project page via **Projects** in the navigation bar.
+2. Open the project you want to manage.
+3. In the **Members** section, click **Add Member**.
+4. Enter the email address of the person you want to add and select their role (`read`, `edit`, or `manage`).
+5. Click **Add Member** to confirm.
+
+!!! note "The user must already have a DivBase account"
+    The person must have registered and verified their email before they can be added. If they haven't signed up yet, ask them to do so first.
+
+You can also change a member's role or remove them from the project from the same Members section.
+
+## Resetting your password
+
+If you've forgotten your password, click **Forgot your password?** on the login page. Enter your email address and you'll receive a reset link. The link expires after a short period — if it has expired you can simply request a new one.
+
+You can also change your password at any time from your **Profile** page once logged in.
+
+## Personal Access Tokens
+
+Personal Access Tokens (PATs) let you authenticate with DivBase programmatically without having to store your password on the device. This could be useful in pipelines, or HPC jobs that need to interact with DivBase. You can manage your personal access tokens from the **Profile** → **Personal Access Tokens** page.
+
+!!! Info
+    To learn more about how to use a PAT in scripts/pipelines and/or HPC jobs, see [Using DivBase Programmatically](./using-divbase-programmatically.md#use-personal-access-tokens-to-authenticate-programmatically).
+
+To create a PAT:
+
+1. Log in to DivBase and navigate to your **Profile** page. From there you'll see a section to manage your Personal Access Tokens.
+2. When adding a new token, you need to give it a name, an optional description and an expiry date.
+3. You can also configure the scope of the PAT (What permissions the token has):
+    - **All projects** — the token can access all projects you're a member of, with same role as you have.
+    - **Specific projects** — restrict the token to selected projects, each with a maximum role.
+    - **Task history** — Allow you to use the token to view your entire task history, not filtered by project.
+4. Click **Create**. Copy the token immediately — it is shown **only once** and cannot be recovered afterwards.
+
+You can have up to **5 active tokens** at a time. To revoke a token, click **Revoke** next to it on the Personal Access Tokens page.
+
+For how to use a PAT in scripts and HPC jobs, see [Using DivBase Programmatically](./using-divbase-programmatically.md#use-personal-access-tokens-to-authenticate-programmatically).

docs/user-guides/using-divbase-programmatically.md

@@ -6,6 +6,50 @@ If you have any tips or suggestions to add to this page or any desired features,
 
 TODO: E.G. how to wait for a query to be complete, and download the query results programmatically.
 
+## Use Personal Access Tokens to Authenticate programmatically
+
+For scripts, pipelines, and HPC jobs the recommended approach is to use a **Personal Access Token (PAT)**. A PAT is a static bearer token that you create once via the website and pass to `divbase-cli` via an environment variable. When using a PAT, there is no login step and no password storage required.
+
+See [Account Management — Personal Access Tokens](./account-management.md#personal-access-tokens) for how to create/and remove PATs.
+
+Once you have a token, set the `DIVBASE_API_PAT` environment variable to it. `divbase-cli` will automatically use it in every request.
+
+!!! question "What if I have both an active login session and a Personal Access Token set?"
+    `divbase-cli` prioritises an active login session over a PAT. If you have both, the CLI will use the active session and ignore the PAT. To use the PAT, you would need to run `divbase-cli auth logout` first.
+
+```bash
+export DIVBASE_API_PAT="divbase_pat_your_token_here"
+divbase-cli files ls
+```
+
+When `DIVBASE_API_PAT` is set, `divbase-cli` does not need you to be logged in.
+
+### Example: Slurm job script
+
+The cleanest way to use a PAT in a SLURM job is to store the token in a restricted file and load it at job start:
+
+```bash
+echo "divbase_pat_your_token_here" > ~/.divbase_pat
+chmod 600 ~/.divbase_pat  # only readable/writeable by the owner
+```
+
+Then in your SLURM script:
+
+```bash
+#!/bin/bash
+#SBATCH --job-name=my_divbase_job
+#SBATCH --time=24:00:00
+# ....
+
+export DIVBASE_API_PAT=$(cat ~/.divbase_pat)
+
+# Download the files you need
+divbase-cli files download my_data.vcf.gz
+```
+
+!!! tip "Scope your token to what the job needs and when you need it for"
+    When creating the PAT, restrict it to the specific project(s) you need it for. Consider also setting an appropriate expiry date for the token. You can always revoke the token immediately if needed from the divbase website.
+
 ## Parse divbase-cli files ls/info output programmatically
 
 1. You can make the output of the `divbase-cli files info` and `divbase-cli files ls` commands in TSV format for easier parsing. Use the `--tsv` flag:
@@ -34,30 +78,3 @@ TODO: E.G. how to wait for a query to be complete, and download the query result
         ```bash
         divbase-cli files stream my_file.vcf.gz | bcftools view -h -
         ```
-
-## Login programmatically
-
-If you want to use DivBase in a script/job that may take **longer than 1 week** to complete from the time of submission, you can login to DivBase programmatically in the script itself. For security reasons (not having to store your password) it is preferable to instead log in interactively before submitting the script, which will keep you logged in for 1 week.
-
-To login programmatically, use the `--password-stdin` (or `-p`) flag to provide your password via standard input (STDIN) when logging into DivBase with the `divbase-cli auth login` command. Using STDIN prevents the password from ending up in the shell's history, or log-files.
-
-If you use a password manager that allows you to output your password to standard output, you can pipe the password directly into the `divbase-cli auth login` command. For example, if you use `pass` as your password manager, you could do:
-
-```bash
-pass show my_divbase_password | divbase-cli auth login EMAIL_ADDRESS --password-stdin
-```
-
-An alternative (but less secure) way to do this is to store your password in a plain text file (make sure to set the appropriate permissions on the file to help make it more secure) and then read the password from the file and pipe it into the `divbase-cli auth login` command. For example:
-
-```bash
-chmod 600 ~/my_password.txt  # only readable/writeable by the owner
-cat ~/my_password.txt | divbase-cli auth login EMAIL_ADDRESS --password-stdin
-```
-
-Another less secure alternative is to use environment variables to store your password and then echo the password from the environment variable and pipe it into the `divbase-cli auth login` command. For example:
-
-```bash
-echo $DIVBASE_PASSWORD | divbase-cli auth login EMAIL_ADDRESS --password-stdin
-```
-
-Where `DIVBASE_PASSWORD` is an environment variable that contains your password.

mkdocs.yml

@@ -104,6 +104,7 @@ nav:
     - Deployment: development/deployment.md
     - Documentation: development/documentation.md
     - Sending Emails: development/sending-emails.md
+    - Personal Access Tokens: development/personal_access_tokens.md
     - S3 Transfers: development/s3_transfers.md
     - Testing: development/testing.md
     - Monitoring and observability: development/monitoring_and_observability.md

packages/divbase-api/src/divbase_api/admin_panel.py

@@ -38,9 +38,10 @@
 from divbase_api.deps import _authenticate_frontend_user_from_tokens
 from divbase_api.frontend_routes.auth import get_login, post_logout
 from divbase_api.models.announcements import AnnouncementDB, AnnouncementLevel, AnnouncementTarget
+from divbase_api.models.personal_access_tokens import PersonalAccessTokenDB
 from divbase_api.models.project_versions import ProjectVersionDB
 from divbase_api.models.projects import ProjectDB, ProjectMembershipDB, ProjectRoles
-from divbase_api.models.queue_status import QueueStatus
+from divbase_api.models.queue_status import QueueStatusDB
 from divbase_api.models.revoked_tokens import RevokedTokenDB, TokenRevokeReason
 from divbase_api.models.task_history import CeleryTaskMeta, TaskHistoryDB, TaskStartedAtDB
 from divbase_api.models.users import UserDB
@@ -631,7 +632,7 @@ class AnnouncementView(ModelView):
 
 class QueueStatusView(ModelView):
     """
-    Custom admin panel View for the QueueStatus model.
+    Custom admin panel View for the QueueStatusDB model.
 
     This is a singleton table (only 1 row allowed).
     Deletion and creation are disabled - only editing the existing row is allowed.
@@ -724,6 +725,76 @@ async def validate(self, request: Request, data: dict[str, Any]) -> None:
     # as does not work well when modifying the timestamp in edit view, easier to just display as UTC
 
 
+class PersonalAccessTokenView(ModelView):
+    """
+    Custom admin panel View for the PersonalAccessTokenDB model.
+    As with passwords, the hashed_token is never displayed in the admin panel on purpose.
+    """
+
+    page_size_options = PAGINATION_DEFAULTS
+    fields = [
+        IntegerField("id", label="ID", disabled=True),
+        StringField("name", label="Name", required=True),
+        TextAreaField("description", required=False, label="Description"),
+        JSONField(
+            "permissions",
+            required=False,
+            label="Permissions",
+            help_text="Permissions associated with the PAT.",
+            disabled=True,
+        ),
+        DateTimeField(
+            "expires_at",
+            help_text="Timestamp when the PAT expires. Value can be left empty for no expiration. Value determined by system, cannot be edited.",
+            required=False,
+            disabled=True,
+        ),
+        DateTimeField(
+            "last_used_at",
+            help_text="Timestamp when the PAT was last used. Value empty if not yet used. Value determined by system, cannot be edited.",
+            required=False,
+            disabled=True,
+        ),
+        BooleanField("is_deleted", required=True, label="Is Deleted", help_text="Mark the PAT as deleted or not."),
+        DateTimeField(
+            "date_deleted",
+            help_text="Timestamp when the PAT was soft deleted (else None). Value determined by system, cannot be edited.",
+            disabled=True,
+        ),
+        IntegerField("user_id", label="User ID", required=False),
+        HasOne("user", identity="user", label="User"),
+        DateTimeField("created_at", label="Created At", disabled=True),
+        DateTimeField("updated_at", label="Updated At", disabled=True),
+    ]
+
+    exclude_fields_from_list = ["hashed_token", "permissions", "user_id"]
+    exclude_fields_from_edit = ["hashed_token", "id", "created_at", "updated_at", "user_id", "user", "permissions"]
+    exclude_fields_from_detail = ["hashed_token", "user_id"]
+
+    def can_create(self, request: Request) -> bool:
+        """Disable manual creation of PATs."""
+        return False
+
+    async def edit(self, request: Request, pk: Any, data: dict) -> Any:
+        """
+        Override the default edit method to ensure that the `date_deleted` field is updated
+        when/if a users soft deletion status is changed.
+        """
+        if "is_deleted" in data:
+            if data["is_deleted"]:
+                data["date_deleted"] = datetime.now(tz=timezone.utc)
+            else:
+                data["date_deleted"] = None
+
+        return await super().edit(request=request, pk=pk, data=data)
+
+    async def serialize_field_value(self, value: Any, field: Any, action: RequestAction, request: Request) -> Any:
+        formatted = _format_cet_datetime(value, field, ["created_at", "updated_at", "expires_at", "last_used_at"])
+        if formatted is not None:
+            return formatted
+        return await super().serialize_field_value(value, field, action, request)
+
+
 class DivBaseAuthProvider(AuthProvider):
     """
     This class enables starlette-admin to make use of DivBase's pre-existing auth system.
@@ -807,5 +878,12 @@ def register_admin_panel(app: FastAPI, engine: AsyncEngine) -> None:
     admin.add_view(
         AnnouncementView(AnnouncementDB, icon="fas fa-bullhorn", label="Announcements", identity="announcement")
     )
-    admin.add_view(QueueStatusView(QueueStatus, icon="fas fa-power-off", label="Queue Status", identity="queue-status"))
+    admin.add_view(
+        QueueStatusView(QueueStatusDB, icon="fas fa-power-off", label="Queue Status", identity="queue-status")
+    )
+    admin.add_view(
+        PersonalAccessTokenView(
+            PersonalAccessTokenDB, icon="fas fa-key", label="Personal Access Tokens", identity="personal-access-token"
+        )
+    )
     admin.mount_to(app)