diff --git a/frontend/pages/docs/administration/federation/peer-integration.mdx b/frontend/pages/docs/administration/federation/peer-integration.mdx new file mode 100644 index 0000000000..bdb6803aa5 --- /dev/null +++ b/frontend/pages/docs/administration/federation/peer-integration.mdx @@ -0,0 +1,70 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Federation and Peer Integration + +> **Common questions this page answers:** +> +> - How do I connect Bailo to other instances? +> - What is federation in Bailo? +> - What federation states are available? + +Federation connects your Bailo instance to other Bailo instances or external model repositories, allowing users to +discover and mirror models from external sources. + +## Federation states + +Your Bailo instance can operate in one of three federation states, controlled by the administrator: + +- **Disabled** - Federation is turned off. No external sources are visible +- **Read Only** - Your instance can browse and mirror models from connected peers, but does not share its own models +- **Enabled** - Full federation: your instance both reads from and shares models with connected peers + +The federation state is configured by your administrator in the application configuration. + +## Peer types + +Bailo supports connecting to different types of external model sources. + +- **Bailo** - Another Bailo instance +- **HuggingFace Hub** - The HuggingFace model repository + +Additional sources can be set up by your Bailo administrator. + +## Configuring peers + +Peers are configured in the application's backend configuration. Each peer requires: + +- **Base URL**: The URL of the external Bailo instance or repository +- **Label**: A display name for the peer (shown in the Marketplace filter) +- **Kind**: The type of peer (Bailo or HuggingFace Hub) +- **State**: Whether federation with this peer is disabled, read-only, or enabled + +Peer configuration is done through the app configuration file - see +[App Configuration](/docs/administration/getting-started/app-configuration). + +## How federation appears to users + +Users see external models alongside local models in the Marketplace, with filtering by external source. + +When federation is enabled and peers are configured: + +- Users see a **filter by external sources** option in the Marketplace +- Models from external sources appear alongside local models +- Each external model shows its source +- External models can be clicked to view the source + +## Checking peer status + +Check the reachability of connected peers through the API. + +You can check the status of connected peers [via the API](/api/docs//#/system). + +Each peer has a reachability status indicating whether Bailo can communicate with it. + +## Related pages + +- [App Configuration](/docs/administration/getting-started/app-configuration) - Where peers are configured +- [Creating a Mirrored Model](/docs/users/model-mirroring/creating-a-mirrored-model) - Mirroring external models +- [Browsing the Marketplace](/docs/users/using-a-model/browsing-the-marketplace) - Filtering by external sources + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/getting-started/app-configuration.mdx b/frontend/pages/docs/administration/getting-started/app-configuration.mdx index 9dcefce47f..6af5eadc95 100644 --- a/frontend/pages/docs/administration/getting-started/app-configuration.mdx +++ b/frontend/pages/docs/administration/getting-started/app-configuration.mdx @@ -1,12 +1,23 @@ import DocsWrapper from 'src/docs/DocsWrapper' +import Alert from '@mui/material/Alert' # App Configuration -## Configuration Files +> **Common questions this page answers:** +> +> - How do I configure Bailo? +> - Where are the configuration files? +> - How does configuration inheritance work? -> ⚠️ Note: If you use Helm this information does not apply to you. We use Helm to automatically configure the -> application for you. Within helm you can manually alter this configuration in -> `infrastructure/helm/bailo/templates/bailo/bailo.configmap.yaml` +## Configuration files + + +If you use Helm this information does not apply to you. + +We use Helm to automatically configure the application for you. Within helm you can manually alter this configuration in +`infrastructure/helm/bailo/templates/bailo/bailo.configmap.yaml` + + Bailo uses [`node-config`](https://github.com/node-config/node-config#readme) for application configuration. `node-config` organizes hierarchical configurations, allowing the user to set a series of overrides over a base diff --git a/frontend/pages/docs/administration/getting-started/deployment-architecture.mdx b/frontend/pages/docs/administration/getting-started/deployment-architecture.mdx new file mode 100644 index 0000000000..7ea1118e6a --- /dev/null +++ b/frontend/pages/docs/administration/getting-started/deployment-architecture.mdx @@ -0,0 +1,133 @@ +import DocsWrapper from 'src/docs/DocsWrapper' +import Alert from '@mui/material/Alert' + +# Deployment Architecture + +> **Common questions this page answers:** +> +> - What services make up Bailo? +> - How does Bailo's architecture work? +> - What are the deployment options? + +Bailo is composed of several services that work together. Understanding this architecture helps when configuring, +deploying, and troubleshooting a Bailo instance. + +## Core services + +Bailo requires six core services to operate. + +- **Frontend** (Next.js / Node.js) - Serves the web application UI +- **Backend** (Node.js) - API (Application Programming Interface) server handling all business logic and data access +- **NGINX** - Reverse proxy routing requests to frontend, backend, and registry +- **MongoDB** - Stores all metadata: models, releases, schemas, reviews, users, tokens +- **Object Storage** (MinIO, S3-compatible) - Stores model files and registry blobs. S3 stands for Simple Storage + Service. +- **Container Registry** (Docker Registry) - Hosts container (Docker) images pushed to models + +## Optional services + +These services add security scanning, email notifications, and webhook testing capabilities. + +- **ArtefactScan** (FastAPI / Python) - Security scanning microservice wrapping ModelScan and Trivy +- **ClamD** (ClamAV) - Antivirus daemon used to scan uploaded files for malware +- **Mail Service** (SMTP-compatible) - Sends email notifications for reviews and access requests. SMTP stands for Simple + Mail Transfer Protocol. +- **Webhook** (Webhook Tester) - Development tool for receiving and inspecting webhook requests + +## How requests flow + +All requests enter through the NGINX reverse proxy and are routed to the appropriate service. + +1. Users access Bailo through the **NGINX** reverse proxy (default port 8080) +2. NGINX routes requests: + - UI page requests go to the **Frontend** + - API requests (`/api/...`) go to the **Backend** + - Container registry requests (`/v2/...`) go to the **Container Registry** +3. The **Backend** reads and writes metadata to **MongoDB** +4. File uploads and downloads go through the **Backend** to **Object Storage** (MinIO) +5. Container image push/pull operations go through the **Container Registry**, which also uses **Object Storage** +6. When files or images are uploaded, the **Backend** triggers scans via the **ClamD** and **ArtefactScan** service (if + configured) + +## Deployment options + +Bailo can be deployed using Docker Compose for development or Helm for production Kubernetes environments. + +### Docker compose (Development & Small Deployments) + +The repository includes a `compose.yaml` that runs the core services locally. This is the simplest way to get started. +There is also ``compose.prod.yaml` to run all services locally. This is also used in the repository's GitHub workflows +for testing. + +### Helm (Production & Kubernetes) + +For production deployments, Bailo provides Helm charts for deploying to Kubernetes. See: + +- [Helm Basic Usage](/docs/administration/helm/basic-usage) +- [Helm Configuration](/docs/administration/helm/configuration) +- [Isolated Environments](/docs/administration/helm/isolated-environments) + +### Standalone dockerfile + + + `Dockerfile.standalone` is intended for local testing and secondary-instance scenarios. It should **not** be used as + the recommended production deployment model. Use Helm for production Kubernetes deployments. + + +`Dockerfile.standalone` builds a single container that runs a self-contained Bailo instance. It packages the frontend, +backend, NGINX, MongoDB, MinIO, the container registry, and Mailcrab into one image using `supervisord`. + +This is not the normal development path. For day-to-day local development, use `compose.yaml`, because it runs each +service separately and supports rebuilding, inspecting, and restarting individual services more easily. + +Use the standalone Dockerfile when you need an additional Bailo instance alongside your main Docker Compose instance, +for example: + +- Testing cross-instance functionality, such as searching for models in another Bailo instance +- Running a second local instance without starting another full Compose project +- Creating a compact demo or smoke-test instance +- Checking behaviour where one Bailo deployment needs to communicate with another + +To use it, first start your main Bailo instance with Docker Compose: + +```bash +docker compose build --parallel +docker compose up -d +``` + +The main instance is then available at `http://localhost:8080`. + +Build and run the standalone image from the repository root: + +```bash +docker build . -t second-bailo -f Dockerfile.standalone +docker run --rm -p 4318:8080 --network dev_internal --name second-bailo second-bailo:latest +``` + +The standalone instance is then available at `http://localhost:4318`. + +If your Compose instance uses a different `INSTANCE_NAME`, replace `dev_internal` with the matching internal network +name, for example `_internal`. + +## Data storage + +Bailo stores structured metadata in MongoDB and binary artefacts in S3-compatible object storage: + +- **MongoDB** holds all structured metadata (models, schemas, reviews, tokens, etc.) +- **Object Storage (MinIO/S3)** holds binary artefacts (model files, container image layers) + +Both need to be backed up to prevent data loss. + +## Configuration + +Bailo uses [node-config](https://github.com/node-config/node-config) for backend configuration. Configuration files can +be overridden via environment variables or Helm values. See +[App Configuration](/docs/administration/getting-started/app-configuration) for details. + +## Related pages + +- [App Configuration](/docs/administration/getting-started/app-configuration) - Configuring the backend +- [Helm Basic Usage](/docs/administration/helm/basic-usage) - Deploying with Helm +- [Artefact Scanners](/docs/administration/microservices/artefact-scanners) - Configuring security scanning + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/helm/basic-usage.mdx b/frontend/pages/docs/administration/helm/basic-usage.mdx index ada52f9d50..574b63111c 100644 --- a/frontend/pages/docs/administration/helm/basic-usage.mdx +++ b/frontend/pages/docs/administration/helm/basic-usage.mdx @@ -8,6 +8,12 @@ same base templates for deploying Bailo, configuring overrides to values as need For more details, see the [Helm project page](https://helm.sh/). +> **Common questions this page answers:** +> +> - How do I deploy Bailo with Helm? +> - What are the requirements for a Helm deployment? +> - How do I upgrade or remove a Helm deployment? + ## Requirements - [HELM CLI](https://github.com/helm/helm/releases) @@ -19,6 +25,8 @@ If you are deploying to OpenShift, you will also need OC: ## Deployment +Deploy Bailo to Kubernetes or OpenShift using standard Helm commands. + ### Setup All commands assume they are run in the `infrastructure/helm/bailo` directory. You should have already authenticated diff --git a/frontend/pages/docs/administration/helm/configuration.mdx b/frontend/pages/docs/administration/helm/configuration.mdx index 5055126998..bdc24b7349 100644 --- a/frontend/pages/docs/administration/helm/configuration.mdx +++ b/frontend/pages/docs/administration/helm/configuration.mdx @@ -2,6 +2,12 @@ import DocsWrapper from 'src/docs/DocsWrapper' # Helm Configuration +> **Common questions this page answers:** +> +> - How do I configure Helm values for Bailo? +> - How do I override the default Helm configuration? +> - What minimum configuration is required? + The default Helm configuration is stored within `values.yaml`. You should not alter this configuration directly, instead overriding individual values via the command line `--set` flag, or by providing a custom `--values` file. diff --git a/frontend/pages/docs/administration/helm/isolated-environments.mdx b/frontend/pages/docs/administration/helm/isolated-environments.mdx index e0cc517e97..4665ec5e18 100644 --- a/frontend/pages/docs/administration/helm/isolated-environments.mdx +++ b/frontend/pages/docs/administration/helm/isolated-environments.mdx @@ -5,23 +5,29 @@ import DocsWrapper from 'src/docs/DocsWrapper' Bailo is built to be deployed on isolated environments, where access to the internet is strictly regulated. To install Bailo on a segregated environment you will need the following dependencies. -## Helm Charts +> **Common questions this page answers:** +> +> - How do I deploy Bailo in an air-gapped environment? +> - What dependencies does Bailo need for offline deployment? +> - How do I configure the Trivy database in an isolated environment? -We rely on the following [Bitnami charts](https://charts.bitnami.com/): +## Helm charts + +Bailo's Helm chart depends on [Bitnami charts](https://charts.bitnami.com/): - MinIO https://charts.bitnami.com/bitnami - MongoDB https://charts.bitnami.com/bitnami -## Docker Images +## Docker images -We rely on the following images: +These container images must be available in your connected registry: -- mongodb/mongodb-community-server:8.2.2-ubi9 -- bitnami/minio:2025.7.23 -- marlonb/mailcrab:v1.6.2 -- nginxinc/nginx-unprivileged:1.28.0-alpine3.21-slim -- registry:3.0.0 -- node:24.11.1-alpine +- mongodb/mongodb-community-server:8.x-ubi9 +- bitnamilegacy/minio:2025.x +- marlonb/mailcrab:v1.x +- nginxinc/nginx-unprivileged:1.x-alpine3.x-slim +- registry:3.x +- node:24.x-alpine These are regularly updated. To retrieve the latest versions used, check `./compose.yaml` and `./backend/Dockerfile`. These versions can be configured using the `tag` attribute for each image in `values.yaml` to override the defaults. In @@ -29,17 +35,19 @@ general, keeping to the same major version will maintain compatibility with Bail ## NPM -We rely on many NPM packages. The full list is available in `package-lock.json`, `frontend/package-lock.json` and -`backend/package-lock.json`. Most are standard packages with the exception of: +Bailo relies on many NPM (Node Package Manager) packages. The full list is available in `package-lock.json`, +`frontend/package-lock.json` and `backend/package-lock.json`. Most are standard packages with the exception of: -- `sharp`, which is an optimised image transformer and requires some compilation tools. -- `cypress`, which is a user interface testing tool and requires a Chromium download. +- `sharp`, which is an optimised image transformer and requires some compilation tools +- `cypress`, which is a user interface testing tool and requires a Chromium download `sharp` includes instructions on installing / building without internet [here](https://sharp.pixelplumbing.com/install). Instructions on installing `cypress` without internet are available [here](https://docs.cypress.io/guides/references/advanced-installation). -## Trivy Database +## Trivy database + +In isolated environments, Trivy may need custom configuration to access its vulnerability database. To give greater visibility of Bailo's registry and it's containing images, some environments may use Trivy to scan for threats, packages and more. @@ -53,7 +61,8 @@ There are a number of environment variables that can be configured to customise - `TRIVY_DB_HOSTNAME=ghcr.io` - Host to retrieve the database from - `TRIVY_DB_IMAGE=ghcr.io/aquasecurity/trivy-db:2` - Image to retrieve the database from -- `TRIVY_DB_TLS_VERIFY=True` - Enable or disable TLS verification or use a path to point to a custom certificate +- `TRIVY_DB_TLS_VERIFY=True` - Enable or disable TLS (Transport Layer Security) verification or use a path to point to a + custom certificate - `TRIVY_DB_INSECURE=False` - Enable if database host is not using TLS or has a custom certificate implementation - `TRIVY_DB_USERNAME` - Registry login username diff --git a/frontend/pages/docs/administration/microservices/artefact-scanners.mdx b/frontend/pages/docs/administration/microservices/artefact-scanners.mdx index 0df72fa790..34ddd1b7b6 100644 --- a/frontend/pages/docs/administration/microservices/artefact-scanners.mdx +++ b/frontend/pages/docs/administration/microservices/artefact-scanners.mdx @@ -2,6 +2,12 @@ import DocsWrapper from 'src/docs/DocsWrapper' # Artefact Scanners +> **Common questions this page answers:** +> +> - What scanners does Bailo use? +> - How do I configure artefact scanning? +> - What is the ArtefactScan service? + Within Bailo, it is possible to optionally deploy artefact scanners which are designed to help manage any potential risk of artefacts being uploaded with potentially malicious contents. @@ -11,12 +17,12 @@ The scanners currently supported by Bailo are: - [ArtefactScan](https://github.com/gchq/Bailo/tree/main/lib/artefactscan_api) - Bailo scanning service that orchestrates multiple scanners - [ModelScan](https://github.com/protectai/modelscan) - malicious or unsafe content detection in model artefacts - - [Trivy](https://github.com/aquasecurity/trivy) - vulnerability scanning and SBOM generation for container image - layers + - [Trivy](https://github.com/aquasecurity/trivy) - vulnerability scanning and SBOM (Software Bill of Materials) + generation for container image layers ## ClamAV -ClamAV is used for traditional malware detection on uploaded files. +ClamAV provides traditional antivirus malware detection for uploaded files. - Uses the official [ClamAV Docker image](https://hub.docker.com/r/clamav/clamav) - Integrated via a @@ -62,16 +68,18 @@ Trivy is used to analyse container image layers for known vulnerabilities and to - Scans uploaded image layer tarballs (overlay filesystems) - Uses a locally cached Trivy vulnerability database -> Trivy is an open‑source vulnerability scanner from Aqua Security for containers and other artefacts. +> Trivy is an open-source vulnerability scanner from Aqua Security for containers and other artefacts. + +## Minimal configuration -## Minimal Configuration +Configure scanner settings in the backend application configuration. These settings will apply to all scanners. -| Name | Description | Value | -| ------------------------------------------------- | ----------------------------------------------------- | ------ | -| `connectors.artefactScanners.kinds` | List of enabled artefact scanner kinds | `[]` | -| `connectors.artefactScanners.retryDelayInMinutes` | Minutes between repeated scans of the same artefact | `60` | -| `connectors.artefactScanners.maxInitRetries` | Number of startup connection attempts before failing | `5` | -| `connectors.artefactScanners.initRetryDelay` | Delay between successive startup pings (milliseconds) | `5000` | +- **connectors.artefactScanners.kinds** - List of enabled artefact scanner kinds. Default: `[]` +- **connectors.artefactScanners.retryDelayInMinutes** - Minutes between repeated scans of the same artefact. Default: + `60`. +- **connectors.artefactScanners.maxInitRetries** - Number of startup connection attempts before failing. Default: `5` +- **connectors.artefactScanners.initRetryDelay** - Delay between successive startup pings (milliseconds). Default: + `5000`. Enabling or disabling specific scanners is handled via connector configuration and deployed microservices. diff --git a/frontend/pages/docs/administration/migrations/bailo-0.4.mdx b/frontend/pages/docs/administration/migrations/bailo-0.4.mdx index b46bfa4bf0..6cc6ec5561 100644 --- a/frontend/pages/docs/administration/migrations/bailo-0.4.mdx +++ b/frontend/pages/docs/administration/migrations/bailo-0.4.mdx @@ -1,6 +1,17 @@ import DocsWrapper from 'src/docs/DocsWrapper' +import Alert from '@mui/material/Alert' -## Bailo Migration to v0.4 +# Bailo Migration to v0.4 + + + Bailo v0.4 is no longer supported. This page may include legacy and out-of-date information. + + +> **Common questions this page answers:** +> +> - What changed in Bailo v0.4? +> - How do I migrate to Bailo v0.4? +> - What are the configuration changes in v0.4? Whilst Bailo v0.4 does not introduce any database or major code changes, it does feature a reorganisation of the project. Instead of the existing format where we had a single 'app', we now split it into two images: @@ -8,15 +19,17 @@ project. Instead of the existing format where we had a single 'app', we now spli - frontend - backend -The frontend section handles rendering the UI. When run in production, it is entirely statically built, simply sending -raw HTML files to the user. The backend handles all API requests and provides the interactivity of the service. +The frontend section handles rendering the UI (User Interface). When run in production, it is entirely statically built, +simply sending raw HTML files to the user. The backend handles all API requests and provides the interactivity of the +service. The code originally in `server` is now stored in the `backend` folder. ### Turbo -To handle multiple repositories, we now use the `turborepo` build system. It provides us with high performance and -reproducible builds, only compiling what has changed since the last compilation. The monorepo is comprised of 'apps': +Bailo v0.4 uses `turborepo` to manage the monorepo structure with separate frontend and backend apps. It provides us +with high performance and reproducible builds, only compiling what has changed since the last compilation. The monorepo +is comprised of 'apps': - frontend - backend @@ -27,8 +40,8 @@ And libraries: - lib/p-mongo-queue - lib/node -These are each individual npm packages, with their own `package.json` files, scripts and more. A script can be run in -individual repositories by using `--workspace`, or in all by omitting it: +These are each individual NPM (Node Package Manager) packages, with their own `package.json` files, scripts and more. A +script can be run in individual repositories by using `--workspace`, or in all by omitting it: ```bash # From the root directory @@ -71,9 +84,9 @@ remain compatible with both old and new packages. #### What is ESM? -ESM, or ECMEAScript Modules, is the latest standard for packaging / modularising JS code, as opposed to CJS, or Common -JavaScript. ESM has a number of benefits that include nicer import/export syntax, better code reusability, and also the -ability to import modules asynchronously. +ESM, or ECMAScript Modules, is the latest standard for packaging / modularising JS code, as opposed to CJS (CommonJS), +or Common JavaScript. ESM has a number of benefits that include nicer import/export syntax, better code reusability, and +also the ability to import modules asynchronously. For more information about ESM, we recommend visiting: @@ -82,11 +95,13 @@ For more information about ESM, we recommend visiting: ### Helm +{/* Helm charts were updated to support the new two-container architecture. */} + To support running two containers, minor changes have been made to our `helm` charts. These changes add a new 'frontend' container, which requires no environment variables / setup. All traffic should route to the frontend container EXCEPT: -- `/api`, which should be sent to the backend container. -- `/v2`, which should be sent to the Docker registry. +- `/api`, which should be sent to the backend container +- `/v2`, which should be sent to the Docker registry The backend environment variables / mount locations remain unchanged. The frontend serves static files, so requires minimal setup. @@ -96,23 +111,25 @@ methods. ### Config +Configuration options were reorganised and standardised in v0.4. + We've taken the time to reduce the number of duplicate configuration options we have, and try to standardise the existing values. If you are using a service such as helm, you may not need to make any changes. The configuration is now also labelled to help advise on what each setting changes. Some major changes: -- `minio` connection options are now stored under `minio.connection` and passed directly to the `minio` library. +- `minio` connection options are now stored under `minio.connection` and passed directly to the `minio` library - `minio.uploadBucket` is now called `minio.buckets.uploads` - `minio.registryBucket` is now called `minio.buckets.registry` - `minio.createBuckets` is now called `minio.automaticallyCreateBuckets` -- `smtp` connection options are now stored under `smtp.connection` and passed directly to the `node-mailer` library. +- `smtp` connection options are now stored under `smtp.connection` and passed directly to the `node-mailer` library - `s2i.builderImage` has been moved to `ui.seldonVersions` and is now an array of names and images. These are to allow users to select the correct Seldon version to build from - `openshift.appPublicRoute` is removed in favour of `app.protocol`, `app.host` and `app.port` -- `openshift` is now under `build.openshift`. +- `openshift` is now under `build.openshift` - `uiConfig` is now `ui` - `listen` has been split into `app.port` for the frontend and `api.port` for the backend diff --git a/frontend/pages/docs/administration/migrations/bailo-2.0.mdx b/frontend/pages/docs/administration/migrations/bailo-2.0.mdx index 63d7c27eac..df3c34deeb 100644 --- a/frontend/pages/docs/administration/migrations/bailo-2.0.mdx +++ b/frontend/pages/docs/administration/migrations/bailo-2.0.mdx @@ -1,23 +1,32 @@ import DocsWrapper from 'src/docs/DocsWrapper' -## Bailo Migration to v2.0 +# Bailo Migration to v2.0 + +> **Common questions this page answers:** +> +> - What changed in Bailo v2.0? +> - How do I migrate from Bailo v1 to v2? +> - Can I run v1 and v2 simultaneously? Bailo v2 is a complete rewrite of all aspects of the model management platform in order to meet modern demands. To try to make the migration as easy as possible, Bailo can simultaneously run V1 and V2, allowing you to schedule the migration over time. +Bailo v2 is a complete rewrite that maintains the same architecture, allowing v1 and v2 to run simultaneously for +gradual migration. + No architecture has significantly changed. We still require two application containers (frontend and backend), a registry container and an nginx routing container. We also continue to store all data in: -- MongoDB for all metadata storage. -- An S3 compatible platform for all binary storage. +- MongoDB for all metadata storage +- An S3 (Simple Storage Service) compatible platform for all binary storage Whilst storage methods remain the same, all tables and keys are unique between V1 and V2 which allows for simultaneously running multiple versions: - All MongoDB collections are now prefixed with `v2_` -- The registry now uses `model_id` as a namespace, instead of either `internal` or `deployment_id`. -- Models are now kept within the `beta` s3 path. +- The registry now uses `model_id` as a namespace, instead of either `internal` or `deployment_id` +- Models are now kept within the `beta` s3 path Bailo provides the `migrateV2` script, which can be run in order to convert existing models from the older format to the new system. Users who do not use the default schemas will have to implement their own conversion functions. See the diff --git a/frontend/pages/docs/administration/migrations/scripts.mdx b/frontend/pages/docs/administration/migrations/scripts.mdx index d0dd969abb..5e67b84ce4 100644 --- a/frontend/pages/docs/administration/migrations/scripts.mdx +++ b/frontend/pages/docs/administration/migrations/scripts.mdx @@ -1,6 +1,12 @@ import DocsWrapper from 'src/docs/DocsWrapper' -## Bailo DataBase Migration Scripts +# Database Migration Scripts + +> **Common questions this page answers:** +> +> - How do Bailo migration scripts work? +> - How do I write a new migration script? +> - What are common pitfalls when writing migrations? As Bailo develops, the requirements for storing data also change. Sometimes this will simply be adding a new property or Document to MongoDB, and other times it will require migrating a collection of properties to a new object and @@ -47,7 +53,9 @@ data. ### Common gotchas -- Using `{lean: true}` in a `find` query returns POJOs (not Mongoose Documents) as per +These are common pitfalls to avoid when writing or updating migration scripts. + +- Using `{lean: true}` in a `find` query returns POJOs (Plain Old JavaScript Objects) (not Mongoose Documents) as per [the docs](). The POJO does not have some commonly needed such as `save`, not getters/setters which are commonly used in migration scripts when working with properties that have been removed from a Document. diff --git a/frontend/pages/docs/administration/review-roles/assigning-roles-to-schemas.mdx b/frontend/pages/docs/administration/review-roles/assigning-roles-to-schemas.mdx new file mode 100644 index 0000000000..3e5ca40cb8 --- /dev/null +++ b/frontend/pages/docs/administration/review-roles/assigning-roles-to-schemas.mdx @@ -0,0 +1,88 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Assigning Review Roles to Schemas + +> **Common questions this page answers:** +> +> - How do I attach review roles to a schema? +> - What happens when I change role assignments on a schema? +> - How do review roles propagate from schemas to models? + +Review roles and schemas work together to define your governance process. When you attach review roles to a schema, +every model that uses that schema will require those roles to approve releases and access requests. + +## How the connection works + +Review roles are attached to schemas, and models inherit role requirements when they select a schema. + +``` +Review Role ──attached to──> Schema ──selected by──> Model +``` + +1. An administrator creates review roles (e.g. "Model Technical Reviewer (MTR)", "Model Senior Responsible Officer + (MSRO)") +2. Those roles are attached to a schema when the schema is created or edited +3. When a model owner selects that schema, the model inherits the review role requirements +4. The model owner then assigns specific people to each review role on their model + +## Attaching roles during schema creation + +Select review roles during schema creation from the Default Review Roles section: + +1. Navigate to **Schemas** from the navigation sidebar +2. Click **Upload a new schema** +3. Fill in the schema details (ID, name, description, type, uploaded JSON schema file) +4. In the **Default Review Roles** section, select which roles this schema requires +5. Upload the schema + +## Attaching roles to an existing schema + +Modify review role requirements on existing schemas from the schema's Actions menu: + +1. Navigate to **Schemas** from the navigation sidebar +2. Select the schema you want to edit +3. In the **Actions** menu, click **Update review roles** +4. Make your selection +5. **Save** your changes + +## What happens when you change role assignments + +Changes to role assignments on a schema affect both existing and new models using that schema: + +- **Existing models** using that schema are affected - they will need the updated set of reviewers +- **New models** created with the schema will inherit the current role requirements +- Model owners may need to assign people to any newly added roles + +## Best practices + +Plan role assignments before creating schemas and use consistent roles across related schemas. + +- **Plan role assignments before creating schemas** - It's easier to set up roles correctly from the start than to + change them later +- **Use consistent roles across related schemas** - If you have multiple model card schemas, using the same review roles + makes the governance process predictable +- **Document role expectations** - Use the review role description field to clearly explain what each reviewer should + check + +## Example configuration + +A typical governance setup: + +1. Create two review roles: + - **Model Technical Reviewer (MTR)** - reviews technical quality + - **Model Senior Responsible Officer (MSRO)** - reviews governance and compliance +2. Create a model card schema and attach both MTR and MSRO +3. Create an access request schema and attach MSRO only + +This means: + +- Every release requires both technical and governance approval +- Access requests only need governance approval + +## Related pages + +- [Managing Review Roles](/docs/administration/review-roles/managing-review-roles) - Creating and editing roles +- [Understanding Schemas](/docs/administration/schemas/understanding-schemas) - How schemas work +- [Uploading a Schema](/docs/administration/schemas/upload-a-schema) - Uploading a schema to Bailo + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/review-roles/managing-review-roles.mdx b/frontend/pages/docs/administration/review-roles/managing-review-roles.mdx new file mode 100644 index 0000000000..52e7eb2134 --- /dev/null +++ b/frontend/pages/docs/administration/review-roles/managing-review-roles.mdx @@ -0,0 +1,122 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Managing Review Roles + +> **Common questions this page answers:** +> +> - How do I create a review role in Bailo? +> - What is a system role on a review role? +> - How do I set up Model Technical Reviewer (MTR) and Model Senior Responsible Officer (MSRO)? + +Review roles define who must approve releases and access requests for models in your organisation. As an administrator, +you can create, edit, and delete review roles to match your governance requirements. + +Only administrators will have the following options available to them. + +## Viewing existing roles + +View all configured review roles from the Review Roles navigation item. + +To see all review roles configured in your Bailo instance, select **Review Roles** from the navigation sidebar. + +This page lists all review roles with their name, short name, description, and associated system role. + +## Creating a review role + +To create a new review role: + +1. Navigate to **Review Roles** from the navigation sidebar +2. Click **Create new review role** +3. Populate the following fields: + - **Name** - The display name for the role. Example: "Model Technical Reviewer". + - **Short Name** - A brief identifier used internally. Example: "mtr". + - **Description** - What this role is responsible for reviewing. Example: "Reviews the technical quality and fitness + for purpose of models". + - **System Role** - The minimum system role a user must have to be assigned this review role. Options: Owner, + Contributor, Consumer, or none. Example: "Contributor". + - **Default collaborators** - Users or groups automatically assigned to this role when new models are created. + Example: "team-leads". +4. Click **Submit** + +### Choosing a system role + +The **System Role** field links a review role to an system role. This means: + +- If set to **Contributor**, only users who are at least contributors on a model can be assigned this review role +- If set to **Owner**, only owners can be assigned +- If left empty, any user can be assigned regardless of their system role + +### Default collaborators + +Default collaborators are pre-populated when a model owner sets up review roles on their model. This is useful for roles +that should always be assigned to the same team or person across all models. + +## Editing a review role + +To edit an existing role: + +1. Navigate to **Review Roles** +2. Click on the role you want to edit +3. Click **Edit Role** +4. Modify the fields as needed +5. Click **Submit** to save your changes + +Changes to review roles apply to all schemas that use the role and all models that use those schemas. + +## Deleting a review role + +Deleting a review role removes it from all schemas that reference it: + +1. Navigate to **Review Roles** +2. Click on the role you want to delete +3. Click **Delete role** +4. Click **Confirm** in the confirmation popup + +> **Warning**: Deleting a review role removes it from all schemas that reference it. Models using those schemas will no +> longer require approval from that role. Make sure this is intended before deleting. + +## Connecting roles to schemas + +After creating review roles, attach them to schemas so that models using those schemas require the roles: + +1. Navigate to **Schemas** from the navigation sidebar +2. Create or edit a schema +3. In the **Review Roles** section, select which roles should be required for this schema +4. Save the schema + +Every model that uses this schema will then require all attached review roles to approve releases before they are +finalised. + +## Common configurations + +These examples show typical governance setups using review roles. + +### Two-role governance (MTR + MSRO) + +The most common setup uses two review roles: + +- **Model Technical Reviewer (MTR)**: Reviews technical quality. System role: Contributor +- **Model Senior Responsible Officer (MSRO)**: Reviews governance and compliance. System role: Owner + +Both roles are attached to the model card schema. A release is only approved when both the MTR and MSRO approve it. + +### Single-role governance + +For simpler workflows, a single review role may be sufficient: + +- **Model Reviewer**: Reviews all aspects. System role: Contributor + +### Fixed reviewer + +For strict compliance requirements: + +- Create a role with **Default Entities** set to your compliance team +- This ensures the same team always reviews every model + +## Related pages + +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - How the review process works +- [Roles & Permissions](/docs/reference/roles-and-permissions) - System roles vs review roles +- [Understanding Schemas](/docs/administration/schemas/understanding-schemas) - How schemas and review roles connect + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/schemas/create-a-schema.mdx b/frontend/pages/docs/administration/schemas/create-a-schema.mdx index 930aa9eee4..6b70baaa15 100644 --- a/frontend/pages/docs/administration/schemas/create-a-schema.mdx +++ b/frontend/pages/docs/administration/schemas/create-a-schema.mdx @@ -1,22 +1,101 @@ import DocsWrapper from 'src/docs/DocsWrapper' -# Creating and Uploading schemas +# Creating a Schema -Bailo makes use of the [RJSF](https://react-jsonschema-form.readthedocs.io/en/latest/) (react-jsonschema-form) library -in order to dynamically create its upload & access request forms. These forms are constructed using existing widgets, or -custom widgets that have been created specifically for Bailo. +Schemas define the structure of model card and access request forms in Bailo. They are built using +[JSON Schema](https://json-schema.org/) and rendered using +[RJSF (React JSON Schema Form)](https://react-jsonschema-form.readthedocs.io/en/latest/). -## Example / minimal schemas +If you're new to schemas, read [Understanding Schemas](/docs/administration/schemas/understanding-schemas) first for a +non-technical overview. -Bailo includes both an example schema for models and for access requests. You can find them in -`backend/src/scripts/example_schemas`. It is important to note that these schemas contain all of mandatory fields that -the application needs in order to run. These fields are hard-coded in various places in the source code. +> **Common questions this page answers:** +> +> - How do I create a schema in Bailo? +> - What question types are available in schemas? +> - How do I add conditional fields to a schema? +> - How do I test a schema? -## Create a schema +## Before you start -### Simple questions +Bailo includes example schemas for both model cards and access requests in `backend/src/scripts/example_schemas`. It is +important to note that these schemas contain all of mandatory fields that the application needs in order to run. These +fields are hard-coded in various places in the source code. Use them as a starting point for your own schemas. -A typical object inside the schema will look like this: +> **Important**: Some fields are required by the application and are referenced in the source code. Always start from an +> example schema rather than building from scratch. + +## Schema structure + +A schema is a JSON object with a `properties` field containing pages, sections, and questions. + +### Pages + +The top-level `properties` define the pages (tabs) of the form: + +```json +{ + "properties": { + "firstPage": { + "type": "object", + "title": "Page 1", + "properties": { + ... + } + }, + "secondPage": { + "type": "object", + "title": "Page 2", + "properties": { + ... + } + } + } +} +``` + +### Sections + +Within each page, you can create sub-sections using nested objects: + +```json +{ + "properties": { + "firstPage": { + "type": "object", + "title": "Page 1", + "properties": { + "sectionOne": { + "type": "object", + "title": "Section 1", + "properties": { + "questionOne": { + "type": "number", + "title": "Enter a number" + } + } + }, + "sectionTwo": { + "type": "object", + "title": "Section 2", + "properties": { + "questionTwo": { + "type": "string", + "title": "Enter a string" + } + } + } + } + } + } +} +``` + +## Question types + +Schemas support several question types including text, dates, booleans, and dropdowns. + +### Simple text questions ```json "firstName": { @@ -27,14 +106,14 @@ A typical object inside the schema will look like this: } ``` -The type property can be altered depending on what value you want the question to return. The title property will be -displayed as the question itself, and the description will appear under the question to include any additional -information that might help the user. Please note that if you do not include a title property, the library will use the -name of the object (in this case firstName) instead. +- **type** - the data type (`string`, `number`, `boolean`, `integer`) +- **title** - the question text displayed to the user +- **description** - help text shown below the question +- **default** - a pre-filled value (optional) -### Dates +If you omit the `title` property, the library uses the property name (e.g. `firstName`). -Dates can be added like this: +### Dates ```json "date": { @@ -44,69 +123,29 @@ Dates can be added like this: } ``` -### Sections - -When using Bailo you will see that the upload form is made up of a number of pages. Each page is a separate object -inside the first "properties" property of the schema. For example: +### Boolean questions ```json -"properties": { - "firstPage": { - "type": "object", - "title": "Page 1", - "properties": { - ... - } - }, - "secondPage": { - "type": "object", - "title": "Page 2", - "properties": { - ... - } - } +"isPublic": { + "type": "boolean", + "title": "Is this model publicly available?" } ``` -Creating sub-sections works identically to how we defined our pages above. If we take the first page of our example -above, and then amend it so it looks like this: +### Dropdown selections ```json -"properties": { - "firstPage": { - "type": "object", - "title": "Page 1", - "properties": { - "sectionOne": { - "type": "object", - "title": "Section 1" - "properties": { - "questionOne": { - "type": "number", - "title": "Enter a number", - } - } - }, - "sectionTwo": { - "type": "object", - "title": "Section 2" - "properties": { - "questionTwo": { - "type": "string", - "title": "Enter a string", - } - } - } - }, - ... - } +"category": { + "type": "string", + "title": "Model category", + "enum": ["Classification", "Regression", "Generation", "Other"] } ``` -#### Hiding sections from the model card +## Hiding sections from the model card -Sections will display on the model card page by default, but in the case whereby a section needs to be hidden from the -model card, then add the following property: +By default, all sections are displayed on the model card page. To hide a section from the model card view while still +collecting the data in the form: ```json "myPage": { @@ -115,17 +154,16 @@ model card, then add the following property: "properties": { ... }, - "displayModelCard": false, + "displayModelCard": false } ``` -Note: This will only hide it from the model page, the data will still persist in the version metadata. +> **Note**: This only hides the section from the model card display. The data still persists in the version metadata. -#### Required fields for model states +## Required fields for model states -Bailo supports conditional required fields based on the state that has been set on a model. This can be used by adding -the property `requiredByModelStates` to the schema and specifying the list of model states that require that question. -For example: +Bailo supports conditional required fields based on the state set on a model. Add the `requiredByModelStates` property +to a question and specify which model states require it: ```json "modelSummary": { @@ -138,18 +176,16 @@ For example: } ``` -When a schema is retrieved by a user they can specify a model state as well as the schema ID and the backend will -automatically set the necessary -[required properties](https://json-schema.org/understanding-json-schema/reference/object#required). +When a user retrieves a schema, they can supply a model state alongside the schema ID. The backend sets the appropriate +[required properties](https://json-schema.org/understanding-json-schema/reference/object#required) for that state. -### Dependencies +## Dependencies (Conditional Questions) -A dependency is useful when the answer to one question might warrant some additional information; for example, a boolean -question might not need a detailed response if the answer is 'no', but if the user answer is 'yes' you might want to ask -them for some additional details. +Dependencies allow you to show or hide questions based on the answer to another question. For example, a boolean +question might not need a detailed response if the answer is 'no', but if the user answers 'yes' you might want to ask +for additional details. -To avoid validation errors, Bailo schema dependencies need to be handled slightly differently to the ones provided in -the RJSF documentation. Here is an example of how we handle dependencies for Bailo schemas: +To avoid validation errors, Bailo schemas handle dependencies differently from the standard RJSF documentation: ```json "myPage": { @@ -158,11 +194,11 @@ the RJSF documentation. Here is an example of how we handle dependencies for Bai "properties": { "questionOne": { "type": "boolean", - "title": "True or false?", + "title": "True or false?" }, "questionTwo": { "title": "More information please!", - "type": "string", + "type": "string" } }, "dependencies": { @@ -201,21 +237,88 @@ the RJSF documentation. Here is an example of how we handle dependencies for Bai } ``` -Inside `dependencies` we have defined a case of only having one of two different situations. Firstly, if `questionOne` -is set to `false`, then `questionTwo` will be set to read only, whereas if we set `questionOne` to `true` then -`questionTwo` will not be set as read only. This method might seem slightly more convoluted than the documentation -online, but it allows for answers to be changed without making the schema invalid. The current design of RJSF's -dependencies works as so: - -- User selects `true` for `questionOne` -- User enters data into `questionTwo` -- User changes their mind and selects `false` for `questionOne` -- User data for `questionTwo` remains in the form data, despite not being visible on the form itself - -This behaviour is so that any user data is retained in case the user wishes to select `true` for `questionOne` again. -From a user perspective this nice as it means there is less chance of data being lost when changing answers, but it does -have a knock-on affect whereby in certain situations it will make the schema fail validation. Our approach is set the -additional questions to read only when they are not in use. The downside to this approach is that we potentially include -unwanted additional data, but this can be removed by the user if this happens. +### Why this pattern? + +The standard RJSF dependency approach can cause validation errors when users change their answers. The issue is: + +1. User selects `true` for `questionOne` +2. User enters data into `questionTwo` +3. User changes their mind and selects `false` for `questionOne` +4. User data for `questionTwo` remains in the form data, despite not being visible + +Our approach sets additional questions to **read-only** instead of hiding them entirely. This preserves user data if +they change their mind, while avoiding schema validation failures. The trade-off is that some unused data may persist, +but it can be cleared by the user if needed. + +## Testing your schema + +Always test a schema by uploading it, creating a test model, and working through every form page. + +Before deploying a schema to production: + +1. Create the schema as a JSON file following the structure above +2. Upload it using the [Upload a Schema](/docs/administration/schemas/upload-a-schema) page +3. Create a test model using the new schema +4. Work through every page of the form to verify: + - All questions appear as expected + - Required fields are properly validated + - Dependencies show and hide correctly + - Section titles and descriptions are clear +5. Check the model card view to confirm the right sections are displayed + +## Complete example + +A minimal but complete schema with two pages, required fields, and basic structure: + +```json +{ + "type": "object", + "properties": { + "overview": { + "type": "object", + "title": "Overview", + "properties": { + "modelName": { + "type": "string", + "title": "Model name" + }, + "description": { + "type": "string", + "title": "Model description" + }, + "tags": { + "type": "array", + "title": "Tags", + "items": { + "type": "string" + } + } + }, + "required": ["modelName", "description"] + }, + "details": { + "type": "object", + "title": "Technical Details", + "properties": { + "framework": { + "type": "string", + "title": "ML Framework", + "enum": ["PyTorch", "TensorFlow", "scikit-learn", "Other"] + }, + "trainingData": { + "type": "string", + "title": "Training data description" + } + } + } + } +} +``` + +## Related pages + +- [Understanding Schemas](/docs/administration/schemas/understanding-schemas) - Non-technical overview +- [Uploading a Schema](/docs/administration/schemas/upload-a-schema) - How to upload your schema to Bailo +- [Schema Migrations](/docs/administration/schemas/schema-migrations) - Migrating data between schema versions export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/schemas/schema-migrations.mdx b/frontend/pages/docs/administration/schemas/schema-migrations.mdx new file mode 100644 index 0000000000..14a4ba5f87 --- /dev/null +++ b/frontend/pages/docs/administration/schemas/schema-migrations.mdx @@ -0,0 +1,111 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Schema Migrations + +As your organisation's requirements evolve, you may need to update your schemas. Schema migrations allow you to +transform existing model card data from one schema version to another, without requiring model owners to manually +re-enter their information. + +Only administrators will have the following options available to them. + +> **Common questions this page answers:** +> +> - How do I migrate models to a new schema version? +> - What is a schema migration plan? +> - What is the difference between draft and final migrations? + +## When to use schema migrations + +Use migrations when you've updated a schema and need to transform existing model card data to match. + +Common scenarios: + +- You've updated a schema to add new required fields +- You're reorganising the section structure of a schema +- You're retiring an old schema and moving models to a newer version + +## How migrations work + +A migration plan maps each question from a source schema to either a new location in the target schema or marks it for +deletion. For each question in the source, you specify one of two actions: + +- **Move** - The answer is copied from the source location to a specified location in the target schema +- **Delete** - The answer is discarded and not carried over to the target schema + +The migration plan preserves all existing data that you choose to move, while allowing you to restructure how it is +organised in the new schema. + +## Comparing schemas + +Compare two schemas side by side to identify differences before planning a migration. + +1. Navigate to **Schemas** from the navigation sidebar +2. Select **Compare** +3. Fill in the comparison schemas: + - **Source Schema**: The schema you are migrating from + - **Target Schema**: The schema you are migrating to + +## Creating a migration plan + +Create a migration plan from the Schemas navigation item by mapping source questions to target locations. + +1. Navigate to **Schemas** from the navigation sidebar +2. Select **Migrations** +3. Click **New schema migration plan** +4. Fill in the migration schemas: + - **Source Schema**: The schema you are migrating from + - **Target Schema**: The schema you are migrating to +5. Click **Begin migration** to proceed to the migration details screen. +6. Fill in the migration details: + - **Migration name**: A descriptive name for the migration (e.g. "v1 to v2 model card migration") + - **Migration description**: What this migration does and why +7. With the migration details saved, you can now map each question. For each question in the source schema, specify: + - **Move**: Map it to a path in the target schema + - **Delete**: Mark it for removal +8. Once all questions have been mapped, verify the intended actions under **View Actions**. +9. Save as **Draft** (for testing) or **Final** (ready to use) + +### Draft vs final + +- **Draft** migrations can be edited and tested but cannot be applied to models +- **Final** migrations are locked and can be applied to models + +It is recommended to save as a draft first, verify the mappings are correct, and then mark as final when ready. + +## Applying a migration + +Once finalised, a migration plan can be applied to individual models that use the source schema: + +- Models with the source schema will display "There is a migration available for this model." with a **Migrate** button +- Model owners may then **Select a Migration Plan** +- This transforms the model's existing model card data according to the migration plan and switches the model to the + target schema + +> **Note**: Applying a migration creates a new model card version. The previous version is preserved in the model's +> history, so you can always see what the data looked like before the migration. + +## Best practices + +Test migrations as drafts, migrate incrementally, and communicate changes to model owners. + +- **Test with a draft first**: Create the migration as a draft and review the mappings carefully before finalising +- **Migrate incrementally**: If you're making large changes, consider breaking them into smaller migration steps +- **Communicate with model owners**: Let them know a migration is happening and that they may need to fill in any new + fields that don't have mappings from the old schema +- **Check after migration**: Verify that the migrated data appears correctly in the model card form + +## Viewing existing migrations + +View all migration plans and their status from the Schemas navigation item: + +1. Navigate to **Schemas** from the navigation sidebar +2. Select **Migrations** +3. The list shows all migrations with their name, source schema, target schema, and draft/final status + +## Related pages + +- [Understanding Schemas](/docs/administration/schemas/understanding-schemas) - What schemas are and how they work +- [Creating a Schema](/docs/administration/schemas/create-a-schema) - How to author a new schema +- [Uploading a Schema](/docs/administration/schemas/upload-a-schema) - How to upload a schema to Bailo + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/schemas/understanding-schemas.mdx b/frontend/pages/docs/administration/schemas/understanding-schemas.mdx new file mode 100644 index 0000000000..cc62b8a1bd --- /dev/null +++ b/frontend/pages/docs/administration/schemas/understanding-schemas.mdx @@ -0,0 +1,109 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Understanding Schemas + +> **Common questions this page answers:** +> +> - What is a schema in Bailo? +> - What types of schemas exist? +> - Why do schemas matter? +> - What is RJSF? + +Schemas are the foundation of Bailo's governance model. They define the questions and structure of model cards and +access request forms, ensuring that every model in your organisation is documented consistently. + +This page explains what schemas are and how they work. For step-by-step instructions on creating a schema, see +[Creating a Schema](/docs/administration/schemas/create-a-schema). + +## What is a schema? + +A schema is a template that defines the questions, structure, and validation rules for model card and access request +forms: + +- What **questions** appear when someone fills in a model card or access request form +- How those questions are **organised** into sections and tabs +- Which questions are **required** and which are optional +- What **format** answers should take (free text, dates, dropdowns, etc.) + +When a user creates a model or submits an access request, they select a schema. That schema then generates the form they +fill in. + +## Schema types + +Bailo supports three types of schema, each serving a different purpose: + +- **Model Card** - Defines the structure of model documentation: intended use, training details, performance metrics, + limitations, etc. +- **Data Card** - Defines the fields for dataset information: size, format, data source, known limitations or biases, + etc. +- **Access Request** - Defines the form users fill in when requesting access to a model: who is requesting, why, what + permissions they need, etc. + +## How schemas relate to other concepts + +Schemas connect to models, review roles, and forms in a structured relationship. + +``` +Schema ──defines──> Model Card Form +Schema ──defines──> Data Card Form +Schema ──defines──> Access Request Form +Schema ──attached to──> Review Roles (which roles must approve) +Model ──selects──> Schema (at setup time) +Data Card ──selects──> Schema (at setup time) +``` + +When an administrator attaches review roles to a schema, every model that uses that schema will require those roles to +be assigned and to approve releases and access requests. + +## Active vs inactive schemas + +Schemas can be toggled between active and inactive states without breaking existing models: + +- **Active** schemas appear in the selection list when users create new models or access requests +- **Inactive** schemas are hidden from the selection list but continue to work for models that already use them + +This allows you to retire old schemas without breaking existing models. + +## Why schemas matter + +Without schemas, every model would be documented differently, making it difficult to: + +- Compare models against each other +- Ensure governance requirements are met +- Automate compliance checks +- Maintain consistent quality across your model inventory + +Schemas give administrators control over what information is captured, while giving model owners a clear and structured +form to fill in. + +## Schema versioning and migrations + +Schema migrations allow model card data to be transformed when schemas are updated. + +As your organisation's requirements evolve, you may need to update schemas. Rather than forcing model owners to re-enter +all their information, Bailo supports **Schema Migrations** - plans that transform model card data from one schema +version to another. + +See [Schema Migrations](/docs/administration/schemas/schema-migrations) for details. + +## Technical details + +Schemas are built using JSON Schema and rendered using RJSF (React JSON Schema Form). This means schemas support: + +- Simple text fields, numbers, and dates +- Dropdown selections and radio buttons +- Conditional fields that appear based on other answers +- Multi-section forms with tab navigation +- Required field validation + +For technical instructions on writing schema JSON, see +[Creating a Schema](/docs/administration/schemas/create-a-schema). + +## Related pages + +- [Creating a Schema](/docs/administration/schemas/create-a-schema) - Step-by-step schema authoring guide +- [Uploading a Schema](/docs/administration/schemas/upload-a-schema) - How to upload a schema to Bailo +- [Schema Migrations](/docs/administration/schemas/schema-migrations) - Migrating data between schema versions +- [Managing Review Roles](/docs/administration/review-roles/managing-review-roles) - Creating roles to attach to schemas + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/administration/schemas/upload-a-schema.mdx b/frontend/pages/docs/administration/schemas/upload-a-schema.mdx index 3fd35973d0..3ac01a44f5 100644 --- a/frontend/pages/docs/administration/schemas/upload-a-schema.mdx +++ b/frontend/pages/docs/administration/schemas/upload-a-schema.mdx @@ -4,31 +4,39 @@ import Box from '@mui/material/Box' import bailoResetApprovals from '../../../../public/docs/bailo_upload_new_schema.png' -## Upload a Schema +# Upload a Schema + +> **Common questions this page answers:** +> +> - How do I upload a schema to Bailo? +> - What fields are required when uploading a schema? +> - How do I deactivate a schema? + +Upload new schemas from the schema management page. You must hold the admin role to manage schemas. By default Bailo comes with a default schema for both a model card and an access request. These are added on startup in `backend/src/services/v2/schema.ts`. These contain a small number of questions, and are not expected to be used on a production system. -To add a new schema, visit [/schemas/list](/schemas/list) on a running Bailo instance. To manage schemas you will need -to hold the 'admin' role on the Bailo instance. Click the 'Upload a new Schema' button in the top left. +1. To add a new schema, visit [/schemas/list](/schemas/list) on a running Bailo instance. +2. Click the **Upload a new Schema** button in the top left. -This will bring up a form to create a new schema. A description of each field is below: +The screenshot above shows the Schemas page on the Model tab, with a populated schema called Minimal Schema v10 +displaying the Actions button to one side. An **Upload a new Schema** is prominently displayed at the top of the page. -| Field | Description | -| ----------- | -------------------------------------------------------------------------------------------------- | -| Id | A globally unique schema identifier, should include the schema version. | -| Name | A user facing name for the schema, should include the schema version. | -| Description | This is displayed to the user to help them decide which schema to pick. This can include markdown. | -| Schema Type | Either 'Model' or 'Access Request'. | -| Schema | Select a 'JSON Schema' that includes the questions you require. | +3. Populate the form to create a new schema with fields: + - **Id** - A globally unique schema identifier, should include the schema version. + - **Name** - A user-facing name for the schema, should include the schema version. + - **Description** - Displayed to the user to help them decide which schema to pick. Can include markdown. + - **Schema Type** - Either 'Model' or 'Access Request'. + - **Schema** - Select a JSON Schema that includes the questions you require. -From the schema list page you can also convert a schema to inactive by pressing the 'make inactive' button. If a schema -is deleted, any models associated with it will be deleted. Converting a schema to 'inactive' reduces it's priority -within the UI but does not break any existing models uploaded with it. +From the schema list page you can also convert a schema to inactive by pressing the **make inactive** button in the +**Actions** menu. If a schema is deleted, any models associated with it will be deleted. Converting a schema to +'inactive' reduces it's priority within the UI but does not break any existing models uploaded with it. export default ({ children }) => {children} diff --git a/frontend/pages/docs/directory.tsx b/frontend/pages/docs/directory.tsx index 215453eb2b..a28135c026 100644 --- a/frontend/pages/docs/directory.tsx +++ b/frontend/pages/docs/directory.tsx @@ -11,64 +11,98 @@ export interface DirectoryEntry { title: string slug: string header?: boolean + level?: number } export const flatDirectory: Array = [ // Main Docs { title: 'Overview', slug: '', header: true }, + // Getting Started + { title: 'Getting Started', slug: 'getting-started', header: true }, + { title: 'Quick Start Guide', slug: 'getting-started/quick-start' }, + { title: 'Core Concepts', slug: 'getting-started/core-concepts' }, + // Users { title: 'Users', slug: 'users', header: true }, - { title: 'Managing Models and Releases', slug: 'users/managing-models-and-releases', header: true }, - { title: 'Uploading Artifacts', slug: 'users/managing-models-and-releases/upload-to-bailo', header: true }, - { title: 'Creating a Model', slug: 'users/managing-models-and-releases/upload-to-bailo/creating-a-model' }, - { title: 'Completing the Model', slug: 'users/managing-models-and-releases/upload-to-bailo/completing-the-model' }, - { title: 'Creating a Release', slug: 'users/managing-models-and-releases/upload-to-bailo/create-a-release' }, - { title: 'Uploading Files', slug: 'users/managing-models-and-releases/upload-to-bailo/files' }, - { title: 'Uploading Images', slug: 'users/managing-models-and-releases/upload-to-bailo/images' }, + { title: 'Models', slug: 'users/models', header: true }, + { title: 'Creating a Model', slug: 'users/models/creating-a-model' }, + { title: 'Model Card', slug: 'users/models/model-card' }, + { title: 'Creating a Release', slug: 'users/models/creating-a-release' }, + { title: 'Uploading Files', slug: 'users/models/uploading-files' }, + { title: 'Uploading Images', slug: 'users/models/uploading-images' }, + { title: 'Model Templating', slug: 'users/models/model-templating' }, + + { title: 'Data Cards', slug: 'users/data-cards', header: true }, + { title: 'Creating a Data Card', slug: 'users/data-cards/creating-a-data-card' }, + { title: 'Managing Data Cards', slug: 'users/data-cards/managing-data-cards' }, { title: 'Using a Model', slug: 'users/using-a-model', header: true }, + { title: 'Browsing the Marketplace', slug: 'users/using-a-model/browsing-the-marketplace' }, { title: 'Requesting Access', slug: 'users/using-a-model/requesting-access' }, - { title: 'Personal Access Tokens', slug: 'users/using-a-model/personal-access-tokens' }, { title: 'Using a Pushed Docker Image', slug: 'users/using-a-model/using-a-pushed-docker-image' }, - { title: 'Downloading files', slug: 'users/using-a-model/downloading-files' }, + { title: 'Downloading Files', slug: 'users/using-a-model/downloading-files' }, - { title: 'Using scanners', slug: 'users/using-scanners', header: true }, - { title: 'File scanning', slug: 'users/using-scanners/file-scanning' }, - { title: 'Image scanning', slug: 'users/using-scanners/image-scanning' }, + { title: 'Reviews', slug: 'users/reviews', header: true }, + { title: 'Understanding Reviews', slug: 'users/reviews/understanding-reviews' }, + { title: 'Reviewing', slug: 'users/reviews/reviewing', header: true }, + { title: 'Reviewing a Release', slug: 'users/reviews/reviewing/releases' }, + { title: 'Reviewing an Access Request', slug: 'users/reviews/reviewing/access' }, + { title: 'Reviewing a Model Card Lifecycle', slug: 'users/reviews/reviewing/model-lifecycle' }, + { title: 'Review Outcomes', slug: 'users/reviews/review-outcomes' }, - { title: 'Deletion', slug: 'users/deletion', header: true }, - { title: 'Deleting a File', slug: 'users/deletion/file-deletion' }, - { title: 'Deleting a Model', slug: 'users/deletion/model-deletion' }, - { title: 'Soft Deletion', slug: 'users/deletion/soft-deletion' }, + { title: 'Security Scanning', slug: 'users/using-scanners', header: true }, + { title: 'File Scanning', slug: 'users/using-scanners/file-scanning' }, + { title: 'Image Scanning', slug: 'users/using-scanners/image-scanning' }, + + { title: 'Inferencing', slug: 'users/inferencing', header: true }, + { title: 'Creating an Inference Service', slug: 'users/inferencing/creating-an-inference-service' }, + { title: 'Managing Inference Services', slug: 'users/inferencing/managing-inference-services' }, { title: 'Model Mirroring', slug: 'users/model-mirroring', header: true }, { title: 'Creating a Mirrored Model', slug: 'users/model-mirroring/creating-a-mirrored-model' }, { title: 'Editing a Mirrored Model Card', slug: 'users/model-mirroring/editing-a-mirrored-model-card' }, - { title: 'Reviews', slug: 'users/reviews', header: true }, - { title: 'Reviewing Releases and Access Requests', slug: 'users/reviews/reviewing', header: true }, - { title: 'Reviewing a Release', slug: 'users/reviews/reviewing/releases' }, - { title: 'Reviewing an Access Request', slug: 'users/reviews/reviewing/access' }, - { title: 'Reviewed Releases and Access Requests', slug: 'users/reviews/reviewed', header: true }, - { title: 'Reviewing model card lifecycle', slug: 'users/reviews/model-lifecycle' }, - { title: 'Releases', slug: 'users/reviews/reviewed/releases' }, - { title: 'Access Requests', slug: 'users/reviews/reviewed/access-request' }, + { title: 'Untrusted Models', slug: 'users/untrusted-models', header: true }, + { title: 'Untrusted Models', slug: 'users/untrusted-models/untrusted-models' }, - { title: 'Programmatically using Bailo', slug: 'users/programmatically-using-bailo', header: true }, + { title: 'Deletion', slug: 'users/deletion', header: true }, + { title: 'Deleting a File', slug: 'users/deletion/file-deletion' }, + { title: 'Deleting a Model', slug: 'users/deletion/model-deletion' }, + { title: 'Soft Deletion', slug: 'users/deletion/soft-deletion' }, + + { title: 'Programmatic Access', slug: 'users/programmatically-using-bailo', header: true }, { title: 'Authentication', slug: 'users/programmatically-using-bailo/authentication' }, - { title: 'Open API', slug: 'users/programmatically-using-bailo/open-api' }, - { title: 'Webhooks', slug: 'users/programmatically-using-bailo/webhooks' }, + { title: 'Personal Access Tokens', slug: 'users/programmatically-using-bailo/personal-access-tokens' }, { title: 'Python Client', slug: 'users/programmatically-using-bailo/python-client' }, + { title: 'OpenAPI Reference', slug: 'users/programmatically-using-bailo/open-api' }, + { title: 'Webhooks', slug: 'users/programmatically-using-bailo/webhooks' }, // Administration { title: 'Administration', slug: 'administration', header: true }, /// Getting Started { title: 'Getting Started', slug: 'administration/getting-started', header: true }, + { title: 'Deployment Architecture', slug: 'administration/getting-started/deployment-architecture' }, { title: 'App Configuration', slug: 'administration/getting-started/app-configuration' }, + /// Schema Management + { title: 'Schemas', slug: 'administration/schemas', header: true }, + { title: 'Understanding Schemas', slug: 'administration/schemas/understanding-schemas' }, + { title: 'Create a Schema', slug: 'administration/schemas/create-a-schema' }, + { title: 'Upload a Schema', slug: 'administration/schemas/upload-a-schema' }, + { title: 'Schema Migrations', slug: 'administration/schemas/schema-migrations' }, + + /// Review Roles + { title: 'Review Roles', slug: 'administration/review-roles', header: true }, + { title: 'Managing Review Roles', slug: 'administration/review-roles/managing-review-roles' }, + { title: 'Assigning Roles to Schemas', slug: 'administration/review-roles/assigning-roles-to-schemas' }, + + /// Federation + { title: 'Federation', slug: 'administration/federation', header: true }, + { title: 'Peer Integration', slug: 'administration/federation/peer-integration' }, + /// Microservices { title: 'Microservices', slug: 'administration/microservices', header: true }, { title: 'Artefact Scanners', slug: 'administration/microservices/artefact-scanners' }, @@ -79,22 +113,24 @@ export const flatDirectory: Array = [ { title: 'Configuration', slug: 'administration/helm/configuration' }, { title: 'Isolated Environments', slug: 'administration/helm/isolated-environments' }, - /// Schema Management - { title: 'Schema', slug: 'administration/schemas', header: true }, - { title: 'Create a Schema', slug: 'administration/schemas/create-a-schema' }, - { title: 'Upload a Schema', slug: 'administration/schemas/upload-a-schema' }, - /// Migrations { title: 'Migrations', slug: 'administration/migrations', header: true }, { title: 'Bailo v0.4', slug: 'administration/migrations/bailo-0.4' }, { title: 'Bailo v2.0', slug: 'administration/migrations/bailo-2.0' }, { title: 'DataBase Scripts', slug: 'administration/migrations/scripts' }, + + // Reference + { title: 'Reference', slug: 'reference', header: true }, + { title: 'Glossary', slug: 'reference/glossary' }, + { title: 'Roles & Permissions', slug: 'reference/roles-and-permissions' }, + { title: 'Troubleshooting & FAQ', slug: 'reference/troubleshooting' }, ] export interface DirectoryTree { slug: string title: string header?: boolean + level: number children?: DirectoryTree[] } @@ -103,6 +139,7 @@ function slugsToTree(paths: Array) { slug: '', title: 'Root', header: true, + level: 0, children: [], } @@ -113,6 +150,7 @@ function slugsToTree(paths: Array) { let currentId = '' for (const part of parts) { + const level = parts.indexOf(part) + 1 const isLastPart = part === parts[parts.length - 1] const isFirstPart = part === parts[0] @@ -129,6 +167,7 @@ function slugsToTree(paths: Array) { slug: currentId, title: path.title, ...(path.header ? { header: true } : {}), + level, ...(isLastPart ? {} : { children: [] }), }) diff --git a/frontend/pages/docs/getting-started/core-concepts.mdx b/frontend/pages/docs/getting-started/core-concepts.mdx new file mode 100644 index 0000000000..38b609b14a --- /dev/null +++ b/frontend/pages/docs/getting-started/core-concepts.mdx @@ -0,0 +1,148 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Core Concepts + +This page explains the key building blocks of Bailo and how they relate to each other. Understanding these concepts will +help you navigate the platform and make the most of its features. + +> **Common questions this page answers:** +> +> - What is a model card? +> - What is the difference between a release and a model? +> - What roles exist in Bailo? +> - How does the review process work? + +## Entry types + +Bailo organises content into four entry types, each serving a different purpose: + +- **Model** - A machine learning model with full lifecycle management - model card, releases, access control, and + reviews +- **Data Card** - Documentation for a dataset, tracking its provenance, storage, and relationship to models +- **Mirrored Model** - A read-only copy of a model from another Bailo instance or external source +- **Untrusted Model** - A model flagged for additional scrutiny, with restricted capabilities + +Some Entry types must be enabled by your Bailo administrator before they can be used. + +## The lifecycle of a model + +A model follows a structured lifecycle from creation through review to consumer access: + +``` +Create Model → Select Schema → Fill In Model Card → Add Collaborators + → Upload Files / Images → Create Release → Review & Approval + → Consumers Request Access → Download / Pull +``` + +### 1. Model + +A **Model** is the top-level container. It has a name, description, visibility (public or private), and a list of +collaborators. Creating a model is the first step. + +### 2. Schema + +A **Schema** defines the structure of a model card - what questions must be answered about a model. Schemas are created +by administrators and selected when setting up a model. There are two kinds: + +- **Model Card schemas** define the metadata form for models +- **Access Request schemas** define the form users fill in when requesting access + +### 3. Model Card + +The **Model Card** is the structured documentation attached to a model. It captures information like intended use, +training methodology, known limitations, and performance metrics. The questions come from the selected schema. + +Model cards are versioned - each edit creates a new version, and you can view the history to see how documentation has +evolved. + +### 4. Release + +A **Release** is a versioned snapshot of a model at a point in time. Releases use +[semantic versioning](https://semver.org/) (e.g. `1.0.0`, `1.1.0`, `2.0.0`) and include: + +- A reference to a specific model card version +- Attached files (model weights, configuration files, etc.) +- Attached container images (e.g. Docker images) +- Release notes + +Creating a release triggers the review process. + +### 5. Review + +When a release or access request is created, it enters a **Review** process. Reviewers assigned to the model evaluate +the submission and either approve it or request changes. Reviews are driven by **Review Roles** - named roles like +"Model Technical Reviewer" or "Model Senior Responsible Officer" that an administrator configures. + +A release is only approved when all required review roles have given their approval. + +### 6. Access Request + +To download files or pull container images from a model, users must create an **Access Request**. This submission goes +through its own review process. Once approved, the user gains the permissions specified in the request. + +Some models may have **ungoverned access** enabled, which allows any user to download files without going through the +access request process. + +## Roles and permissions + +Bailo uses two complementary role systems to control access and governance. + +### System Roles + +These are assigned per-model and control what actions a user can take: + +- **Owner** - Full control: edit model, manage collaborators, create releases, delete the model +- **Contributor** - Edit the model card, upload files and images, create releases +- **Consumer** - Download files and pull images (requires an approved access request, unless ungoverned access is on) + +### Review roles + +These are created by administrators and attached to schemas. When a model uses a schema with review roles, those roles +must be assigned to specific people who will review releases and access requests. Common examples include: + +- **Model Technical Reviewer (MTR)** - reviews the technical quality of the model +- **Model Senior Responsible Officer (MSRO)** - reviews governance and compliance aspects + +For a full reference, see [Roles & Permissions](/docs/reference/roles-and-permissions). + +### Entry roles + +The collective term for all system roles and review roles. + +## Visibility + +Model visibility controls who can discover and browse a model: + +- **Public** models appear in the Marketplace and can be found by any user +- **Private** models are only visible to users who have been explicitly added as collaborators + +Even for public models, downloading artefacts typically requires an approved access request. + +## Security scanning + +Depending on configuration, Bailo automatically scans uploaded files and container images for security issues: + +- **Files** are scanned by ClamAV (malware detection) and ModelScan (serialisation attack detection) +- **Container images** are scanned by Trivy (vulnerability detection with CVE reporting) + +Scan results are displayed alongside each file and image, with severity ratings. + +Your Bailo administrator may change what scanners are enabled and used. + +## Federation + +Federation allows multiple Bailo instances to share models across organisational boundaries. + +Bailo instances can be connected through **Federation**, allowing models from one instance to appear in another. This is +useful for organisations with multiple deployments or for integrating with external model repositories like HuggingFace +Hub. + +Federated models appear as **Mirrored Models** - read-only entries that stay in sync with their source. + +## What's Next? + +- [Quick Start Guide](/docs/getting-started/quick-start) - Walk through creating your first model +- [Glossary](/docs/reference/glossary) - Definitions of all Bailo-specific terms +- [Creating a Model](/docs/users/models/creating-a-model) - Detailed model creation guide + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/getting-started/quick-start.mdx b/frontend/pages/docs/getting-started/quick-start.mdx new file mode 100644 index 0000000000..8168fb199e --- /dev/null +++ b/frontend/pages/docs/getting-started/quick-start.mdx @@ -0,0 +1,138 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Quick Start Guide + +This guide walks you through the essential steps of using Bailo, from browsing models to creating your own. It should +take around 15 minutes to complete. + +> **Common questions this page answers:** +> +> - How do I create my first model in Bailo? +> - How do I get started with Bailo? +> - What is the basic workflow for publishing a model? + +Before you begin, make sure you have access to your organisation's Bailo instance and can log in. + +## Step 1: Browse the Marketplace + +The Marketplace is Bailo's home page where you can search and filter all public models and data cards. + +When you first log in, you'll see the **Marketplace**. This is where all public models and data cards are listed. You +can: + +- **Search** for models by name using the search bar at the top +- **Filter** using common properties, organisations, popular tags etc. shown below the search bar +- **Switch tabs** between Models and Data Cards + +If you're looking for a specific model, try the search bar first. If you're exploring what's available, browse by tags +or scroll through the list. + +## Step 2: Create a Model + +Register a new model entry in Bailo by providing its name, description, and visibility settings: + +1. Click the **+** (Create) button in the top navigation bar or in the Marketplace +2. Select **Model** as the entry type +3. Next, provide the basic information about your model by filling in the required fields: + - **Model Name**: A clear, descriptive name for your model + - **Description**: What the model does and its intended use + - **Access control**: Choose **Public** (visible to all users) or **Private** (visible only to collaborators) +4. Once you have filled in these details:, click **Create Model** + +Your model is now created and visible in the Marketplace (if public). + +## Step 3: Select a Schema and Fill In the Model Card + +After creating your model, you need to select a **schema**. A schema defines the questions you'll answer about your +model - things like intended use, training details, known limitations, and performance metrics. + +1. You'll be prompted to select a schema from the available options +2. Once selected, click **Edit model card** then a form appears with sections to fill in +3. Work through each section, providing as much detail as you can +4. Click **Save** + +The model card is the core documentation for your model. It helps reviewers and consumers understand what the model does +and how it should be used. + +## Step 4: Add Collaborators and Reviewers + +Control who can contribute to your model and who is responsible for reviewing releases: + +1. Navigate to the **Settings** tab on your model page +2. Under **Access**, add users or groups as collaborators +3. Assign the appropriate level of access to each collaborator via roles: + - **Owner**: Full control over the model + - **Contributor**: Can edit the model card and upload files + - **Consumer**: Can download files and images (after access approval) +4. Assign **Review Roles** (e.g. Model Technical Reviewer, Model Senior Responsible Officer) to the appropriate people +5. Click **Save** + +## Step 5: Upload Model Files + +Upload the actual model artefacts such as weights, configuration files, and training data so they can be attached to a +release. + +You can upload model artefacts at any time: + +1. Go to the **File management** tab +2. Click **Add new files** and select a file from your computer +3. Optionally add tags to help organise your files +4. Uploaded files can be attached to releases + +If enabled, files are automatically scanned for security issues after upload. + +## Step 6: Upload Container Images + +If your model includes a containerised application (e.g. a Docker image for inference), you can push it to the Bailo +registry so it can be included in releases: + +1. Go to the **Registry** tab on your model page +2. Click **Push image** +3. If you do not already have a user authentication token, click **Manage user tokens** and create one +4. Authenticate your local Docker client using: + - Use your access key as the username + - Use your user token as the password +5. Tag your local image using the command shown in the dialog: + - Replace `` with your local image name and tag + - Replace `` with the image name you want to use in Bailo + - Replace `` with the image version tag +6. Push the tagged image to the registry using the command shown in the dialog +7. Uploaded container images can be attached to releases + +If enabled, images are automatically scanned for vulnerabilities after upload. + +## Step 7: Create a Release + +A release bundles a specific version of your model card together with its uploaded files and images, and triggers the +review workflow: + +1. Go to the **Releases** tab on your model page +2. Click **Draft new release** +3. Describe this version of the model: + - **Version**: Use semantic versioning (e.g. `1.0.0`) + - **Release notes**: Describe what's included in this release +4. Include the artefacts you uploaded earlier e.g. any files and/or container images you've uploaded +5. Click **Create Release** + +Creating a release triggers the review process. Your assigned reviewers will be notified. + +## Step 8: Review and Approval + +After creating a release, it enters the review process: + +1. Assigned reviewers receive notifications +2. Reviewers examine the model card and release artefacts +3. Each reviewer either **approves** or **requests changes** +4. Once all required reviewers approve, the release is marked as approved + +You can check the status of your reviews from the **Review** page (accessible from the navigation sidebar). + +## What's Next? + +- [Core Concepts](/docs/getting-started/core-concepts) - Understand the data model and key terminology +- [Requesting Access](/docs/users/using-a-model/requesting-access) - Learn how to request access to someone else's model +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Automate your workflow with the Python + client +- [Roles & Permissions](/docs/reference/roles-and-permissions) - Understand what each role can do + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/index.mdx b/frontend/pages/docs/index.mdx index 54249e62bf..1f676a30d3 100644 --- a/frontend/pages/docs/index.mdx +++ b/frontend/pages/docs/index.mdx @@ -2,18 +2,49 @@ import DocsWrapper from 'src/docs/DocsWrapper' # What is Bailo? +> **Common questions this page answers:** +> +> - What does Bailo do and why would I use it? +> - Where should I start as a new user, model consumer, or administrator? +> - How is the documentation organised? + Bailo provides a **consistent**, **managed** platform for the machine learning lifecycle, enabling models to be deployed in a **standardised** and **well orchestrated** way. This improves the use of ML models in operational contexts, and does so in a **well controlled** and **low-risk** manner. The service aims to: -- Providing a centralised repository for machine learning models +- Provide a centralised repository for machine learning models - Encourage discovery and reuse of existing models - Standardise model packaging and deployment - Enforce consistent compliance and review processes - Support monitoring and governance of operational models +## Where to start + +Pick the entry point that best matches your role or goal. + +- **New to Bailo**: [Quick Start Guide](/docs/getting-started/quick-start) - Create your first model in ~15 minutes +- **Looking to understand how Bailo works**: [Core Concepts](/docs/getting-started/core-concepts) - The data model, + roles, and workflows +- **A model consumer**: [Requesting Access](/docs/users/using-a-model/requesting-access) - How to get access to a model +- **Using the API or Python client**: [Programmatic Access](/docs/users/programmatically-using-bailo/authentication) - + Authentication and client libraries +- **An administrator**: [App Configuration](/docs/administration/getting-started/app-configuration) - Setting up and + configuring Bailo + +## Documentation sections + +The documentation is split into four sections covering onboarding, day-to-day usage, platform administration, and +reference material. + +- **Getting Started** - Quick start guide and core concepts for new users +- **Users** - Guides for creating models, managing releases, requesting access, reviews, and more +- **Administration** - Deployment, configuration, schema management, and review role setup +- **Reference** - Glossary, roles and permissions, and troubleshooting + +## Contributing + All of this documentation is available on [our GitHub repository](https://github.com/gchq/Bailo/tree/main/frontend/pages/docs). Contributions and corrections are welcome. diff --git a/frontend/pages/docs/reference/glossary.mdx b/frontend/pages/docs/reference/glossary.mdx new file mode 100644 index 0000000000..24c527cbba --- /dev/null +++ b/frontend/pages/docs/reference/glossary.mdx @@ -0,0 +1,121 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Glossary + +This page defines key terms used throughout Bailo's documentation and interface. Terms are listed alphabetically. + +> **Common questions this page answers:** +> +> - What does MTR mean in Bailo? +> - What is a model card? +> - What is an access request? +> - What is the difference between an Owner, Contributor, and Consumer? +> - What is RJSF? + +--- + +**Access Request** - A formal request submitted by a user to gain permission to download files or pull container images +from a model. Access requests go through a review process and must be approved before access is granted. + +**Artefact** - A general term for any file or container image associated with a model, including model weights, +configuration files, training data, and container images. + +**ClamAV** - An open-source antivirus engine used by Bailo to scan uploaded files for malware. + +**Collaborator** - A user or group that has been granted a role (Owner, Contributor, or Consumer) on a specific model. + +**Consumer** - A system role that allows a user to download files and pull container images from a model, provided they +have an approved access request or the model has ungoverned access enabled. + +**Container image** (e.g. a Docker image) is a standalone, executable artefact used to create a container. This +container image contains all the libraries, dependencies, and files that the container needs to run. + +**Contributor** - A system role that allows a user to edit the model card, upload files and images, and create releases +for a model. + +**Data Card** - An entry type for documenting datasets. Data cards capture information about data provenance, storage, +and relationships to models. + +**Entry** - The top-level entity in Bailo. Models, Data Cards, Mirrored Models, and Untrusted Models are all types of +entry. + +**Entry Role** - The collective term for all system roles and review roles. + +**Federation** - A feature that connects multiple Bailo instances (or external sources like HuggingFace Hub), allowing +models from one to be browsed and mirrored in another. + +**Inference Service** - A running deployment of a model's container image within Bailo, providing a live endpoint for +making predictions without downloading the model. + +**Marketplace** - The main browsing page in Bailo where users discover public models and data cards through search, tag +filtering, and category browsing. + +**Mirrored Model** - A read-only copy of a model from another Bailo instance or external source. Mirrored models are +kept in sync with their source and cannot be directly edited. + +**Model** - The primary entry type in Bailo. A model represents a machine learning model with its documentation (model +card), versioned releases, files, container images, and access controls. + +**Model Card** - The structured documentation attached to a model. Its content is defined by the selected schema and +typically includes intended use, training details, performance metrics, and known limitations. Model cards are +versioned. + +**Model Senior Responsible Officer (MSRO)** - A commonly used review role representing the person responsible for +governance and compliance decisions about a model. This is a configurable review role, not a fixed system role. + +**Model Technical Reviewer (MTR)** - A commonly used review role representing the person responsible for reviewing the +technical quality of a model. This is a configurable review role, not a fixed system role. + +**ModelScan** - A security scanning tool used by Bailo to detect serialisation attacks in uploaded model files (e.g. +malicious pickle files). + +**Owner** - A system role that grants full control over a model: editing, managing collaborators, creating releases, and +deleting the model. + +**Peer** - Another Bailo instance or external model repository connected through federation. + +**Personal Access Token (PAT)** - An API key pair (access key + secret key) that allows programmatic access to Bailo's +API. Tokens can be scoped to specific models and actions. + +**Release** - A versioned snapshot of a model, using semantic versioning (e.g. `1.0.0`). Releases bundle together a +model card version, uploaded files, container images, and release notes. Creating a release triggers the review process. + +**Review** - The evaluation process that releases and access requests go through before approval. Reviews are conducted +by users assigned to review roles on the model. + +**Review Role** - An administrator-defined role (e.g. Model Technical Reviewer, Model Senior Responsible Officer) that +is attached to schemas and assigned to specific people on each model. Review roles determine who must approve releases +and access requests. + +**RJSF (React JSON Schema Form)** - The library Bailo uses to render dynamic forms from JSON Schema definitions. This is +the underlying technology behind schemas and model cards. + +**Schema** - A JSON Schema definition that determines the structure and questions of a model card or access request +form. Schemas are created by administrators and selected when setting up a model. + +**Schema Migration** - A plan for transforming model card data from one schema version to another, allowing schemas to +evolve without losing existing data. + +**Semantic Versioning (Semver)** - A versioning convention used for releases: `MAJOR.MINOR.PATCH` (e.g. `1.2.3`). Major +versions indicate breaking changes, minor versions add functionality, and patch versions fix issues. + +**Soft Deletion** - Bailo's deletion model where deleted items are marked as removed but not immediately destroyed. +Administrators can recover soft-deleted items in exceptional circumstances. + +**System Role** - A system role (e.g. Owner, Contributor, Consumer) that is built into Bailo and determines what actions +a user or group can perform on a model, such as editing metadata, uploading files, creating releases, or downloading +artefacts. + +**Trivy** - An open-source vulnerability scanner used by Bailo to scan container images for known security +vulnerabilities (CVEs). + +**Ungoverned Access** - A model setting that allows any user to download files and pull images without submitting an +access request. + +**Untrusted Model** - An entry type for models that require additional scrutiny. Untrusted models may have restricted +capabilities compared to standard models. This feature must be enabled by an administrator. + +**Webhook** - An HTTP callback that Bailo sends to an external URL when certain events occur (e.g. a release is created, +an access request is submitted). Used for integrating Bailo with external systems. + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/reference/roles-and-permissions.mdx b/frontend/pages/docs/reference/roles-and-permissions.mdx new file mode 100644 index 0000000000..2e33261622 --- /dev/null +++ b/frontend/pages/docs/reference/roles-and-permissions.mdx @@ -0,0 +1,152 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Roles and Permissions + +Bailo uses two complementary role systems: **System Roles** that control what actions a user can take on a model, and +**Review Roles** that determine who must approve releases and access requests. + +> **Common questions this page answers:** +> +> - What can an Owner do in Bailo? +> - What is the difference between system roles and review roles? +> - Who can delete a model? +> - Who can upload files? +> - What is a Model Technical Reviewer (MTR)? + +## System roles + +Every collaborator on a model is assigned one of three system roles that govern day-to-day interactions. + +### Owner + +The **Owner** has full control over a model. There must be at least one owner on every model. + +Owners can: + +- Edit the model name, description, and visibility +- Edit the model card +- Manage collaborators (add, remove, change roles) +- Upload and delete files +- Push and delete container images (e.g. Docker images) +- Create, edit, and delete releases +- Create and manage access requests +- Delete the model +- Configure model settings (e.g. ungoverned access, template eligibility) + +### Contributor + +A **Contributor** can work on the model's content but cannot manage access or delete the model. + +Contributors can: + +- Edit the model card +- Upload and delete files +- Push container images +- Create releases + +Contributors cannot: + +- Change model settings or visibility +- Manage collaborators +- Delete the model + +### Consumer + +A **Consumer** can access the model's artefacts for use, provided they have an approved access request (or the model has +ungoverned access enabled). + +Consumers can: + +- View the model and its model card +- Download files (with approved access request) +- Pull container images (with approved access request) +- Create access requests + +Consumers cannot: + +- Edit the model card +- Upload files or images +- Create releases + +## Permission summary + +This reference lists every action in Bailo and which roles can perform it. + +- **View public model** - Owner: Yes, Contributor: Yes, Consumer: Yes, Public (no role): Yes +- **View private model** - Owner: Yes, Contributor: Yes, Consumer: Yes, Public (no role): No +- **Edit model card** - Owner: Yes, Contributor: Yes, Consumer: No, Public (no role): No +- **Edit model settings** - Owner: Yes, Contributor: No, Consumer: No, Public (no role): No +- **Manage collaborators** - Owner: Yes, Contributor: No, Consumer: No, Public (no role): No +- **Upload files** - Owner: Yes, Contributor: Yes, Consumer: No, Public (no role): No +- **Delete files** - Owner: Yes, Contributor: Yes, Consumer: No, Public (no role): No +- **Push container images** - Owner: Yes, Contributor: Yes, Consumer: No, Public (no role): No +- **Create releases** - Owner: Yes, Contributor: Yes, Consumer: No, Public (no role): No +- **Delete releases** - Owner: Yes, Contributor: No, Consumer: No, Public (no role): No +- **Download files** - Owner: Yes, Contributor: Yes, Consumer: With access request, Public (no role): With ungoverned + access +- **Pull container images** - Owner: Yes, Contributor: Yes, Consumer: With access request, Public (no role): With + ungoverned access +- **Create access requests** - Owner: Yes, Contributor: Yes, Consumer: Yes, Public (no role): No +- **Delete model** - Owner: Yes, Contributor: No, Consumer: No, Public (no role): No + +## Review roles + +Review roles are created by administrators and attached to schemas. Unlike system roles, review roles are not fixed - +your organisation can define whatever roles make sense for your governance process. + +### How review roles work + +1. An administrator creates review roles (e.g. "Model Technical Reviewer", "Model Senior Responsible Officer") +2. The administrator attaches these roles to a schema +3. When a model uses that schema, the model owner assigns specific people to each review role +4. When a release or access request is created, the assigned reviewers are notified +5. Each review role must approve before the submission is considered fully approved + +### Common review roles + +While organisations can create any review roles they need, two are commonly used: + +- **Model Technical Reviewer (MTR)**: Reviews the technical quality of the model - architecture, training process, + evaluation metrics, and fitness for purpose +- **Model Senior Responsible Officer (MSRO)**: Reviews governance and compliance aspects - data handling, ethical + considerations, risk assessment, and organisational policies + +### Review role properties + +Each review role has: + +- **Name** - Display name (e.g. "Model Technical Reviewer") +- **Short Name** - Abbreviated identifier (e.g. "mtr") +- **Description** - Explanation of the role's purpose +- **System Role** - The minimum system role needed (owner, contributor, consumer, or none) +- **Default Entities** - Users or groups automatically assigned to this role on new models + +### Who can review what? + +- **Release reviews** can be performed by any assigned review role +- **Access request reviews** are typically restricted to specific review roles (commonly MSRO) + +The exact review requirements depend on how your administrator has configured the schemas and review roles. + +## Entry roles + +The collective term for all system roles and review roles. + +## Assigning roles + +All roles on a model are managed from the model's Settings tab: + +1. Open the model page +2. Go to the **Settings** tab +3. Under **Access**, search for users or groups +4. Select the appropriate role(s) +5. **Save** changes + +## Related pages + +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - How the review process works +- [Managing Review Roles](/docs/administration/review-roles/managing-review-roles) - Creating and configuring review + roles (administrators) +- [Glossary](/docs/reference/glossary) - Definitions of all Bailo terms + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/reference/troubleshooting.mdx b/frontend/pages/docs/reference/troubleshooting.mdx new file mode 100644 index 0000000000..05fd7ee71e --- /dev/null +++ b/frontend/pages/docs/reference/troubleshooting.mdx @@ -0,0 +1,149 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Troubleshooting and FAQ + +> **Common questions this page answers:** +> +> - Why can't I see a model in the Marketplace? +> - Why is my release stuck in review? +> - Why can't I push or pull a container image? +> - How do I fix token authentication errors? + +Common issues and answers for Bailo users and administrators. + +## Models and the marketplace + +Common issues with finding, editing, and deleting models. + +### I can't see a model in the Marketplace + +- **Private models** are only visible to collaborators. Ask the model owner to add you, or check whether the model is + set to public. +- **Filters may be active.** Clear any search terms, tag filters, or organisation filters +- If the model was recently created, try refreshing the page + +### I can't edit a model + +- Only **Owners** and **Contributors** can edit model cards. Check your role on the model's Settings tab +- If you need access, ask the model owner to add you as a collaborator + +### I can't delete a model + +- Only the model **Owner** can delete a model +- Deletion is permanent for the model and all its associated releases, files, and access requests + +## Releases + +Common issues with creating releases and the review process. + +### My release is stuck in review + +- Check that **all required review roles** have been assigned to specific people in the model's Settings tab +- If reviewers haven't been assigned, no one will be notified of the review +- Contact the assigned reviewers directly if the review has been pending for a long time + +### I can't create a release + +- Only **Owners** and **Contributors** can create releases +- Make sure you have filled in the required fields: version (semver format), and release notes + +## Access requests + +Common issues with access request approval and file downloads. + +### My access request hasn't been approved + +- Access requests are reviewed by the Model Senior Responsible Officer (or equivalent review role). Check who is + assigned to that role in the model's Settings tab. +- Contact the reviewer directly if the request has been pending + +### I can't download files even though I have access + +- Your access request must be **approved**, not just submitted +- Check the status of your access request on the model page +- If the model has **ungoverned access** enabled, you should be able to download without an access request + +## Container (Docker) Images + +Common issues with pushing and pulling container images. + +### I can't push a container image + +- Make sure you're using the correct registry address (check with your administrator) +- Verify your personal access token has the `image:push` permission +- Ensure your token is scoped to the correct model (or set to "All models") +- Check that you're logged into the registry with `docker login` + +### I can't pull a container image + +- You need an **approved access request** for the model (unless ungoverned access is enabled) +- Verify your personal access token has the `image:read` permission +- Check the registry address is correct + +## Personal Access tokens + +Common issues with creating and using Personal Access Tokens (PATs). + +### How do i create a token? + +Navigate to **User (top right) > Settings > Authentication** and click **Add token**. See +[Personal Access Tokens](/docs/users/programmatically-using-bailo/personal-access-tokens) for details. + +### I lost my secret key + +Secret keys are only shown once when the token is created. If you've lost it, delete the old token and create a new one. + +### My token isn't working + +- Check that the token has the correct **permissions** for the action you're trying to perform +- Check that the token is **scoped to the correct model** (or set to "All models") +- Verify you're using the correct access key and secret key pair + +## Schemas and forms + +Common issues with schema validation and form fields. + +### Schema validation errors when filling in a model card + +- Required fields are marked - make sure all required fields are filled in +- Check that dates are in the correct format +- If a field appears read-only, it may be controlled by a dependency on another field's value + +### I can't see a schema when creating a model + +- Only **active** schemas appear in the selection list. An administrator may have deactivated the schema +- Contact your administrator if you need a specific schema + +## Security scanning + +Common issues with file and container image scan results. + +### A file scan shows issues + +- **ClamAV** detects malware. If a file is flagged, do not distribute it until the issue is investigated +- **ModelScan** detects serialisation attacks (e.g. in pickle files). Review the flagged file for unsafe deserialisation + patterns. +- You can **rescan** a file after a cooldown period if you believe the result is a false positive + +### A container image scan shows vulnerabilities + +- **Trivy** reports known vulnerabilities (CVEs) with severity ratings +- Review the vulnerability details to determine impact +- Update base images or dependencies to address critical vulnerabilities +- You can rescan an image after updating it + +## Administration + +Common administration questions. + +### How do i contact an administrator? + +Check the **Help** page in Bailo for support links and contact information specific to your deployment. + +## Related pages + +- [Quick Start Guide](/docs/getting-started/quick-start) - Getting started with Bailo +- [Roles & Permissions](/docs/reference/roles-and-permissions) - Understanding what each role allows +- [Glossary](/docs/reference/glossary) - Definitions of Bailo terms + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/style-guide.mdx b/frontend/pages/docs/style-guide.mdx new file mode 100644 index 0000000000..29f5827793 --- /dev/null +++ b/frontend/pages/docs/style-guide.mdx @@ -0,0 +1,274 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Documentation Style Guide + +This guide defines the writing, formatting, and structural conventions for all documentation under +`frontend/pages/docs`. Follow these guidelines when writing new pages or editing existing ones to ensure consistency +across the documentation. + +--- + +## File format + +All documentation pages are written in MDX (`.mdx`) and must: + +- Import and use `DocsWrapper` as the default export wrapper +- End with the default export statement: + + ```mdx + export default ({ children }) => {children} + ``` + +- Use the `next/legacy/image` `Image` component for all images (never standard Markdown image syntax for in-page + screenshots) +- Import MUI components (e.g. `Box`) for layout wrapping where needed + +--- + +## Page structure + +Every page must follow this order: + +1. **H1 title** (`#`) - one per page, at the top +2. **Common questions callout** - immediately below the title +3. **Body content** - sections using H2 (`##`) and H3 (`###`) headings +4. **Related or next steps section** - at the bottom, before the export + +### H1 title + +- Title Case +- Concise and descriptive +- One per page only + +```mdx +# Creating a Model +``` + +### Common questions callout + +Every page must include a blockquote immediately after the H1 that lists two to five questions the page answers. + +Use bold for the label: + +> Common questions this page answers: +> +> - What does this feature do? +> - How do I configure it? +> - Who can use it? + +--- + +## Headings + +- Use `##` (H2) for major sections +- Use `###` (H3) for subsections within a major section +- Do not skip heading levels +- Sentence case for H2 and below (Title Case for H1 only) +- Do not use H4 or deeper - restructure the content instead + +```mdx +## Security scanning + +### ClamAV +``` + +## Writing style + +- **Audience**: assume a technical but non-expert reader unless the page is explicitly under `administration/` +- **Voice**: active voice, second person ("you", "your") +- **Tone**: professional and direct; avoid filler phrases and unnecessary pleasantries +- **UK English**: use UK spelling throughout (e.g. artefact, organise, colour) +- **Sentence length**: prefer short, clear sentences; break long sentences into lists where appropriate +- **Tense**: present tense throughout + +--- + +## Lists + +Use unordered lists (`-`) for: + +- Features, options, or capabilities with no inherent order +- Role or permission enumerations +- Sets of related items + +Use ordered lists (`1.`) for: + +- Step-by-step instructions where sequence matters + +Nested lists are permitted up to one level of nesting. Avoid deeper nesting. + +Format list items consistently within the same list - either all fragments or all full sentences, not mixed. + +**Bold** the lead term for definition-style lists, followed by a dash and a description: + +```mdx +- **Owner** - Full control: edit model, manage collaborators, create releases, delete the model +- **Contributor** - Edit the model card, upload files and images, create releases +``` + +Avoid ending list items with punctuation (e.g. no full stops at the end of sentences). + +--- + +## Code and technical terms + +- Use backticks for inline code, filenames, configuration keys, version strings, CLI commands, and technical identifiers +- Use fenced code blocks for multi-line code or configuration, with a language identifier +- Use semantic versioning examples consistently: `1.0.0`, `1.1.0`, `2.0.0` + + ```mdx + - **connectors.artefactScanners.kinds** - List of enabled artefact scanner kinds. Default: `[]` + ``` + +````mdx +``` +Create Model → Select Schema → Fill In Model Card → Add Collaborators +``` +```` + +--- + +## Emphasis + +- **Bold** (`**text**`): UI element names, key terms on first use, lead terms in definition lists +- _Italic_: use sparingly, only for genuine emphasis or titles of external works +- Do not use bold for general emphasis or decoration + +--- + +## Links + +- Use descriptive link text that reflects the destination page title +- Do not use "click here" or bare URLs as link text +- Internal links use root-relative paths starting with `/docs/` +- External links use full URLs + +```mdx +[Quick Start Guide](/docs/getting-started/quick-start) [ClamAV](https://www.clamav.net/) +``` + +--- + +## Images + +Wrap all images in a `Box` component to control width, centred using `margin: 'auto'`: + +```mdx + + descriptive alt text + +``` + +- Import images at the top of the file using static imports +- Always provide descriptive `alt` text in lowercase (not title case) +- Follow every image with a plain-text caption sentence on the line immediately below the closing `` tag, + describing what the screenshot shows + + ```mdx + The screenshot above shows the file actions menu with a Delete File option. + ``` + +## Callouts and notes + +Use blockquotes for callouts. Reserve them for: + +- The common questions block (required, at top of every page) +- Quoted external definitions or descriptions of third-party tools + +Do not use blockquotes for general warnings or tips - integrate these into the body text instead. + +--- + +## Section endings + +End pages with one of the following sections (or multiple if appropriate), using an H2 heading: + +### "What's Next?" (conceptual/overview pages) + +Use for pages that introduce concepts and naturally lead the reader forward: + +```mdx +## What's Next? + +- [Quick Start Guide](/docs/getting-started/quick-start) - Walk through creating your first model +- [Glossary](/docs/reference/glossary) - Definitions of all Bailo-specific terms +``` + +### "Next steps" (task/how-to pages) + +Use for pages that describe completing a task, showing what to do after: + +```mdx +## Next steps + +After creating a model and selecting a schema: + +1. [Fill in the model card](/docs/users/models/model-card) - document your model +2. [Upload files](/docs/users/models/uploading-files) - add model artefacts +3. [Create a release](/docs/users/models/creating-a-release) - version your model for distribution +``` + +### "Related pages" + +Optionally included on any page to surface related documentation: + +```mdx +## Related pages + +- [Model Card](/docs/users/models/model-card) - Filling in the model card and configuring access +- [Core Concepts](/docs/getting-started/core-concepts) - Understanding the Bailo data model +``` + +--- + +## Directory conventions + +| Path | Content type | +| ------------------ | -------------------------------------------------- | +| `getting-started/` | Onboarding guides and conceptual overviews | +| `users/` | Task-based guides for day-to-day platform use | +| `administration/` | Deployment, configuration, and platform management | +| `reference/` | Glossary, roles and permissions, troubleshooting | + +Page filenames use kebab-case and match the H1 title of the page as closely as possible. + +--- + +## AWS RAG writing best practices + +We also follow the +[AWS documentation best practices for RAG applications](https://docs.aws.amazon.com/prescriptive-guidance/latest/writing-best-practices-rag/best-practices.html). +The key practices are summarised below: + +- **Use headings and subheadings consistently** - Clear, hierarchical headings help RAG models understand document + structure and extract relevant information accurately +- **Keep numbered lists sequential** - Do not skip numbers in ordered lists; gaps cause confusion for both human readers + and LLMs processing the content +- **Add transitions between list items** - Where steps or items are related, use bridging phrases (e.g. "Once you have + completed the above...") to connect ideas and preserve logical flow +- **Avoid tables** - Replace tabular content with multi-level bullet lists or flat-level syntax (items arranged at the + same hierarchical level, without nesting); this format is more reliably processed by LLMs +- **Describe graphical content in text** - Summarise or describe the content of images and diagrams in accompanying + text; this preserves meaning for RAG models that may not process images, and avoids unnecessary token consumption +- **Add session starters for common queries** - Where a section answers a predictable question, open with a short + sentence that frames the content (e.g. "If you are looking to configure X, follow the steps below"); this improves + semantic matching during retrieval +- **Summarise each section** - Include a brief summary immediately after each heading; this reinforces key points, + improves embedding-space similarity search, and increases the likelihood of accurate retrieval +- **Keep content focused and unambiguous** - Each page or section should cover one topic clearly; RAG models generate + responses from retrieved excerpts, so avoiding ambiguity leads to more accurate and useful outputs +- **Define abbreviations and domain-specific terms** - LLMs do not have inherent knowledge of internal terminology; + define acronyms and Bailo-specific language on first use to reduce the risk of misinterpretation or hallucination +- **Break large documents into smaller, self-contained pages** - Avoid pages that span multiple unrelated subtopics; + smaller, clearly titled documents improve indexing, tagging, and retrieval precision + +--- + +## Contributing + +All documentation source files are in the +[Bailo GitHub repository](https://github.com/gchq/Bailo/tree/main/frontend/pages/docs). + +Contributions and corrections are welcome via pull request. + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/data-cards/creating-a-data-card.mdx b/frontend/pages/docs/users/data-cards/creating-a-data-card.mdx new file mode 100644 index 0000000000..5b8643ed44 --- /dev/null +++ b/frontend/pages/docs/users/data-cards/creating-a-data-card.mdx @@ -0,0 +1,89 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Creating a Data Card + +A **Data Card** documents a dataset used in machine learning - its provenance, storage location, contents, and +relationship to models. Data cards help your organisation track what data is being used, where it comes from, and how it +should be handled. + +> **Common questions this page answers:** +> +> - How do I create a data card in Bailo? +> - When should I use a data card instead of a model? +> - How do I document a dataset? + +## When to use a data card + +Create a data card when you want to: + +- Document a training or evaluation dataset +- Track data provenance and lineage +- Record data storage locations and access requirements +- Link datasets to the models that use them +- Meet compliance or governance requirements around data handling + +## Creating a data card + +Create a data card from the navigation bar by selecting the Data Card entry type: + +1. Click the **+ Create** button in the navigation bar +2. Select **Data Card** as the entry type +3. Provide the basic information about your dataset through the required fields: + - **Data Card Name**: A clear, descriptive name for the dataset + - **Description**: What the dataset contains and its intended use + - **Access control**: Choose **Public** (visible to all users) or **Private** (visible only to collaborators) +4. Once you have filled in these details, click **Create data card** + +## Selecting a schema + +After creating the data card, you'll be prompted to select a **schema**. The schema defines what information you need to +provide about your dataset. Your administrator will have set up schemas appropriate for your organisation. + +Common fields in a data card schema might include: + +- Data source and collection methodology +- Size and format +- Storage location +- Sensitivity classification +- Retention period +- Known limitations or biases + +## Filling in the data card + +Work through the form sections to document your dataset, then save to create a versioned record. + +Once you've selected a schema, the data card form appears. Click **Edit data card** then work through each section, +providing as much detail as you can. The form is divided into tabs based on the schema structure. + +Click **Save** once you are happy with the form. This creates a new data card version which is automatically shown to +anyone viewing the data card. + +## Using the Python client + +You can also create and manage data cards programmatically using the Bailo Python client: + +```python +from bailo import Client, Datacard + +client = Client("https://your-bailo-instance.com") + +# Create a new data card +data_card = Datacard.create( + client=client, + name="Training Dataset v2", + description="Curated training data for image classification", +) + +# Update the data card metadata +data_card.update_data_card(metadata={"overview": {"tags": ["image", "classification"]}}) +``` + +A full example is available in the [Data Cards demo notebook](/docs/python/notebooks/datacards_demo/index.html). + +## Related pages + +- [Managing Data Cards](/docs/users/data-cards/managing-data-cards) - Editing, versioning, and linking data cards +- [Core Concepts](/docs/getting-started/core-concepts) - How data cards fit into the Bailo data model +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Programmatic access to Bailo + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/data-cards/managing-data-cards.mdx b/frontend/pages/docs/users/data-cards/managing-data-cards.mdx new file mode 100644 index 0000000000..bb5d744fc3 --- /dev/null +++ b/frontend/pages/docs/users/data-cards/managing-data-cards.mdx @@ -0,0 +1,84 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Managing Data Cards + +Once a data card has been created, you can edit its content, view its version history, manage collaborators, and link it +to models. + +> **Common questions this page answers:** +> +> - How do I edit a data card? +> - How do I add collaborators to a data card? +> - Can I change the schema on a data card? +> - How do I link a data card to a model? + +## Editing a data card + +Edit a data card by navigating to it and using the Edit data card button: + +1. Navigate to the data card page +2. The **Overview** tab shows the current content +3. Click **Edit data card** and edit any section of the data card form +4. Click **Save** to create a new version + +## Viewing version history + +Every edit creates a new version, and you can view the full history to track changes over time: + +1. Open the data card page +2. Navigate to the version history +3. Select a previous version to view what the data card looked like at that point in time + +This is useful for tracking how documentation has evolved and understanding what changed between versions. + +## Managing collaborators + +Add users or groups as collaborators with specific roles to control access to the data card: + +1. Navigate to the **Settings** tab on the data card page +2. Under **Access**, search for users or groups to add as collaborators +3. Assign an appropriate role: + - **Owner**: Full control, including editing and deleting the data card + - **Contributor**: Can edit the data card content + - **Consumer**: Can view the data card + +## Changing the schema + +Schema changes are managed through administrator-created migration plans rather than direct switching. + +It is not possible to directly switch the schema used by a data card, however it is possible for a newer schema to be +published by the Bailo administrators (for example, if your organisation has updated its data governance requirements). +See [Schema Migrations](/docs/administration/schemas/schema-migrations) for further details. Note that changing schemas +may require you to re-enter some information if the new schema has different fields. + +## Linking data cards to models + +Data cards and models can reference each other to maintain traceability between datasets and the models trained on them. +When filling in a model card, look for fields that allow you to reference related data cards. + +## Using the Python client + +Use the Bailo Python client to retrieve and update data cards programmatically: + +```python +from bailo import Client, Datacard + +client = Client("https://your-bailo-instance.com") + +# Get an existing data card +data_card = Datacard.from_id(client=client, data_card_id="my-data-card-id") + +# Update metadata +data_card.update_data_card(metadata={"overview": {"tags": ["updated"]}}) + +# Get the current data card content +card = data_card.data_card +``` + +## Related pages + +- [Creating a Data Card](/docs/users/data-cards/creating-a-data-card) - How to create a new data card +- [Roles & Permissions](/docs/reference/roles-and-permissions) - What each role allows +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Full programmatic access guide + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/deletion/file-deletion.mdx b/frontend/pages/docs/users/deletion/file-deletion.mdx index 669d406305..7972dec29b 100644 --- a/frontend/pages/docs/users/deletion/file-deletion.mdx +++ b/frontend/pages/docs/users/deletion/file-deletion.mdx @@ -7,6 +7,12 @@ import bailoDeleteFileConfirm from 'public/docs/bailo_delete_file_confirm.png' # Deleting a file +> **Common questions this page answers:** +> +> - How do I delete a file from a model? +> - What happens when I delete a file? +> - Can I recover a deleted file? + Files uploaded to a model can be deleted by authorised users. Deleting a file removes it from use across the model, including any associated Releases. @@ -26,6 +32,8 @@ Releases remain, but will no longer reference the deleted file. ## How to delete a file +Delete a file from the File Management tab using the file actions menu: + 1. Open the model 2. Navigate to the **Files** tab 3. Locate the file you want to delete @@ -36,9 +44,11 @@ Releases remain, but will no longer reference the deleted file. file actions menu with delete option +The screenshot above shows the file actions menu with a Delete File option. + ## Confirming deletion -A confirmation dialog is shown before deletion: +A confirmation dialog lists affected releases before the file is permanently removed: - If the file is used by releases, those releases are listed - You must explicitly confirm to proceed @@ -47,6 +57,8 @@ A confirmation dialog is shown before deletion: delete file confirmation dialog showing affected releases +The screenshot above shows the delete confirmation dialog listing releases that reference the file. + If an error occurs (for example, insufficient permissions), an error message will be displayed and the file will not be deleted. @@ -58,10 +70,10 @@ deleted. ## About recovery -File deletion is intended to be **final**. +File deletion is intended to be **final**, with only rare administrator recovery possible. -In rare and exceptional circumstances, platform administrators may be able to restore a deleted file. This is **not a -standard service** and users should only request recovery if absolutely required. +In exceptional circumstances, platform administrators may be able to restore a deleted file. This is **not a standard +service** and users should only request recovery if absolutely required. Before deleting a file, ensure it is no longer required by any releases or users. diff --git a/frontend/pages/docs/users/deletion/model-deletion.mdx b/frontend/pages/docs/users/deletion/model-deletion.mdx index ac8170bf69..a20aa81834 100644 --- a/frontend/pages/docs/users/deletion/model-deletion.mdx +++ b/frontend/pages/docs/users/deletion/model-deletion.mdx @@ -7,6 +7,12 @@ import bailoDeleteConfirm from 'public/docs/bailo_delete_model_confirm.png' # Deleting a model +> **Common questions this page answers:** +> +> - How do I delete a model in Bailo? +> - What happens when I delete a model? +> - Can I recover a deleted model? + Models can be deleted from Bailo by users with the appropriate permissions. Deleting a model removes it from use and visibility across the platform. @@ -40,6 +46,8 @@ From a user perspective, the model is gone and cannot be used. ## How to delete a model +Delete a model from the Settings tab in the Deletion section: + 1. Open the model 2. Navigate to **Settings** 3. Scroll to the **Deletion** section @@ -49,9 +57,11 @@ From a user perspective, the model is gone and cannot be used. model settings deletion section +The screenshot above shows the model Settings page with the Deletion section and a Delete Model button. + ## Confirming deletion -To prevent accidental deletion, you must confirm the action: +To prevent accidental deletion, you must type the exact model name to confirm deletion: 1. A confirmation dialog will appear 2. Type the **exact model name** into the confirmation field @@ -61,6 +71,8 @@ To prevent accidental deletion, you must confirm the action: delete model confirmation dialog +The screenshot above shows the delete confirmation dialog requiring the exact model name to be typed. + If the name does not match exactly, the delete button remains disabled. ## After deletion @@ -71,10 +83,10 @@ If the name does not match exactly, the delete button remains disabled. ## About recovery -Model deletion is designed to be **final**. +Model deletion is designed to be **final**, with only rare administrator recovery possible. -In rare and exceptional circumstances, platform administrators may be able to restore a deleted model. This is **not a -standard service** and users should only request recovery if absolutely required. +In exceptional circumstances, platform administrators may be able to restore a deleted model. This is **not a standard +service** and users should only request recovery if absolutely required. Before deleting a model, ensure it is no longer needed. diff --git a/frontend/pages/docs/users/deletion/soft-deletion.mdx b/frontend/pages/docs/users/deletion/soft-deletion.mdx index 69877716b0..7413499e73 100644 --- a/frontend/pages/docs/users/deletion/soft-deletion.mdx +++ b/frontend/pages/docs/users/deletion/soft-deletion.mdx @@ -7,13 +7,19 @@ import bailoDeleteConfirm from 'public/docs/bailo_delete_model_confirm.png' # Soft deletion +> **Common questions this page answers:** +> +> - What is soft deletion in Bailo? +> - Can I recover a deleted model or file? +> - Is deletion permanent? + Soft deletion is how Bailo safely removes models and files without immediately destroying them behind the scenes. -From a user point of view, a soft‑deleted item should be considered **deleted**. +From a user point of view, a soft-deleted item should be considered **deleted**. ## What you will see -When something is soft deleted: +From a user's perspective, soft-deleted items are completely removed from the interface: - It disappears from the Marketplace and search - It can no longer be accessed, downloaded, or used @@ -33,6 +39,8 @@ This approach protects the platform, not individual workflows. ## About recovery +Recovery is not a standard feature and should not be relied upon. + Soft deletion **does not mean undo**. While platform administrators may technically be able to restore a deleted item, this is: @@ -45,7 +53,7 @@ Users should **not rely on recovery** and should avoid requesting restoration af ## What you should do before deleting -Before deleting anything, check: +Check that the item is no longer needed before proceeding with deletion: - Whether it is still used by any releases - Whether other users depend on it @@ -55,6 +63,6 @@ If in doubt, consider keeping the item or creating a new version instead. ## Key takeaway -If you delete something in Bailo, assume it is **gone for good**. +Treat deleted items as **permanently gone**. export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/inferencing/creating-an-inference-service.mdx b/frontend/pages/docs/users/inferencing/creating-an-inference-service.mdx new file mode 100644 index 0000000000..ee03521a8e --- /dev/null +++ b/frontend/pages/docs/users/inferencing/creating-an-inference-service.mdx @@ -0,0 +1,78 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Creating an Inference Service + +An inference service deploys a model's container (Docker) image as a live endpoint, allowing you to make predictions +without downloading the model locally. This feature must be enabled by your administrator. + +> **Common questions this page answers:** +> +> - How do I deploy a model as an inference service? +> - What processor types are available for inference? +> - How are requests to an inference service authenticated? + +## Prerequisites + +Before creating an inference service, you need: + +- A container (Docker) image pushed to the model's registry (see + [Uploading Images](/docs/users/models/uploading-images)) +- Appropriate permissions on the model (Owner or Contributor) + +## Creating a service + +Create an inference service from the Inferencing tab on your model page. + +1. Open your model page +2. Navigate to the **Inferencing** tab +3. Click **Create a new Inferencing Service** +4. Populate the following required fields: + - **Description**: What this inference service does. Required + - **Image**: Select from the model's available container images. Required + - **Port**: The port your model exposes for inference requests (1-65535). Required + - **Processor Type**: Choose CPU or an available GPU type. Required + - **Memory**: Memory allocation in GB (1-8 GB, shown for CPU only). Required for CPU +5. Once all fields are filled in, click **Create Inferencing Service** + +### Choosing a processor type + +- **CPU**: Available on all deployments. You'll need to specify memory allocation (1-8 GB) +- **GPU**: Available GPU types depend on your deployment's configuration. GPU options are configured by your + administrator + +### Choosing a port + +The port should match the port your model's container exposes for serving predictions. Common ports include: + +- `8080` or `8000` for HTTP-based inference servers +- `5000` for Flask-based applications + +Check your model's container configuration if you're unsure which port to use. + +## After creation + +Once created, the service is provisioned and accessible via an endpoint on the Inferencing tab. You can: + +- View the service status on the model's **Inferencing** tab +- Access the service endpoint to make prediction requests +- Use your existing Bailo access to authenticate to the service + +## Authenticating requests + +Authentication is handled automatically by the platform. Inference services sit behind an authentication proxy in the +service mesh. When a request is made to an inference service, the proxy checks with Bailo to confirm that the user has +the required access for the relevant model scope, similar to how container registry authentication works. + +You do not normally need to create or provide a Personal Access Token (PAT) for inference service requests. Access is +tied to the permissions you already have in Bailo for the model. If you do not have the required model access, the +request is denied. + +CORS is enabled on inference services so that Bailo can embed an externally hosted service interface in an `iframe`. + +## Related pages + +- [Managing Inference Services](/docs/users/inferencing/managing-inference-services) - Updating and deleting services +- [Uploading Images](/docs/users/models/uploading-images) - How to push container images +- [Requesting Access](/docs/users/using-a-model/requesting-access) - How to request access to a model + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/inferencing/managing-inference-services.mdx b/frontend/pages/docs/users/inferencing/managing-inference-services.mdx new file mode 100644 index 0000000000..6d780a24e2 --- /dev/null +++ b/frontend/pages/docs/users/inferencing/managing-inference-services.mdx @@ -0,0 +1,57 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Managing Inference Services + +Once an inference service has been created, you can view its details, update its configuration, or delete it. + +> **Common questions this page answers:** +> +> - How do I view my inference services? +> - How do I update an inference service? +> - Can I change the container image on an inference service? + +## Viewing inference services + +View all active inference services for a model from the Inferencing tab: + +1. Open the model page +2. Navigate to the **Inferencing** tab - this is where all services are managed +3. All active inference services are listed with their image, description, and status + +## Updating a service + +Update an inference service's configuration from the service's settings panel: + +1. Navigate to the **Inferencing** tab on the model page +2. Select the service you want to update - this opens the service detail view +3. Click **View Settings** to open the configuration panel +4. Modify the fields as needed: + - **Description** + - **Port** + - **Processor Type** + - **Memory** (CPU only) +5. Save your changes + +> **Note**: You cannot change the container image of an existing inference service. To use a different image, delete the +> current service and create a new one. + +## Deleting a service + +Deleting an inference service can only be done via the API: + +``` +DELETE /api/v2/model/{modelId}/inference/{image}/{tag} +``` + +Deleting a service stops it and removes the deployment. This action cannot be undone. + +## Programmatic access + +Inference services can also be managed [via the API](/api/docs//#/inference). + +## Related pages + +- [Creating an Inference Service](/docs/users/inferencing/creating-an-inference-service) - Setting up a new service +- [OpenAPI Reference](/docs/users/programmatically-using-bailo/open-api) - Full API documentation + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/completing-the-model.mdx b/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/completing-the-model.mdx deleted file mode 100644 index 0a370c283f..0000000000 --- a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/completing-the-model.mdx +++ /dev/null @@ -1,22 +0,0 @@ -import Image from 'next/legacy/image' -import Box from '@mui/material/Box' -import bailoModelAccess from 'public/docs/bailo_model_access.png' -import DocsWrapper from 'src/docs/DocsWrapper' - -# Completing the Model - -Once your model has been uploaded and documented, it must be configured for review. - -1. Open the model and navigate to the **Settings** tab. -2. Select **Model access** from the side menu. -3. Use the **Add a user or group to the Model access list** dropdown to add any additional users and/or groups to the - model access. -4. Use the **Select roles** dropdown to choose the roles responsible for reviewing and approving the model. - - - selecting user model roles - - -The selected roles will be notified when the model or its releases require review. - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/create-a-release.mdx b/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/create-a-release.mdx deleted file mode 100644 index 0bb817d226..0000000000 --- a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/create-a-release.mdx +++ /dev/null @@ -1,24 +0,0 @@ -import Image from 'next/legacy/image' -import Box from '@mui/material/Box' -import bailoNewRelease from 'public/docs/bailo_new_release.png' -import bailoNewReleaseForm from 'public/docs/bailo_new_release_form.png' -import DocsWrapper from 'src/docs/DocsWrapper' - -# Creating a release - -Once you have uploaded a model, go to the **_Releases_** tab, and click on **_Draft new Release_**. - - - empty model releases tab with new release button - - -This will bring up a small form that will require you to choose a semver, add release notes and gives you the option of -uploading files and images. - - - draft new release form - - -Once the form is filled out, click on **_Create Release_** to submit all details. - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/creating-a-model.mdx b/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/creating-a-model.mdx deleted file mode 100644 index 3c01fcd561..0000000000 --- a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/creating-a-model.mdx +++ /dev/null @@ -1,63 +0,0 @@ -import Image from 'next/legacy/image' -import Box from '@mui/material/Box' -import bailoMarketIcon from 'public/docs/bailo_market_icon.png' -import DocsWrapper from 'src/docs/DocsWrapper' - -# Bailo Marketplace - -Models added to Bailo are displayed in the **Bailo Marketplace**, where they can be discovered and reused by other -users. - - - view of the marketplace - - -Models marked as **Private** are excluded from the Marketplace and are only visible to users who have been explicitly -granted access. - -## Finding models - -Models can be discovered by: - -- Using the search bar -- Filtering by tags - -## Adding a model to the Marketplace - -You can add a new model using either: - -- The **Create** button on the Marketplace page, or -- The **Create** button in the application header - -Creating a model requires completing three fields: - -- **Model name** -- **Model description** -- **Visibility** (Public or Private) - -Public models are visible to all users. Private models are only visible to users with the appropriate permissions, -configured in the model settings. - -Once these fields are completed, the model is created and added to Bailo. - -## Adding Model Card detail - -After creating the model, you must select a **schema**. The selected schema defines the structure of the model card. - -The model card captures important information such as: - -- Intended use -- Training and evaluation details -- Known limitations or biases - -Model card content can be updated over time as more information becomes available. - -## Changing the Model Name or Description - -To update the model name or description: - -1. Open the model -2. Navigate to **Settings** -3. Edit the **Name** and/or **Description** fields - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/images.mdx b/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/images.mdx deleted file mode 100644 index 58957a263d..0000000000 --- a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/images.mdx +++ /dev/null @@ -1,38 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' -import Box from '@mui/material/Box' -import Image from 'next/legacy/image' - -import bailoPushImage from 'public/docs/bailo_push_image_to_docker_container.png' -import bailoImageRelease from 'public/docs/bailo_add_image_to_release.png' - -# Uploading docker images - -Bailo uses a Docker registry to store container images associated with models. Unlike files, Docker images are -**model‑specific**: you push an image to a model first, and then explicitly attach it to one or more releases. - -## Pushing an image to a model - -To push a Docker image to Bailo, navigate to your model and open the **Registry** tab. This tab provides step‑by‑step -instructions tailored to your model and registry. - - - instructions on how to push image to model - - -## Adding an image to a release - -Once the image has been successfully pushed to Bailo’s registry, it becomes available to attach to a release. - -Edit an existing release or create a new one, then select the image from the available list. - - - release being edited with new image attached - - -## What is a Docker container? - -In the context of Bailo, a Docker container is an isolated, self‑contained application environment used to run a model. -Packaging models as containers allows them to be started quickly, run consistently across environments, and used for -inference without additional setup. - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/model-mirroring/creating-a-mirrored-model.mdx b/frontend/pages/docs/users/model-mirroring/creating-a-mirrored-model.mdx index b7f039541a..3574c60ab2 100644 --- a/frontend/pages/docs/users/model-mirroring/creating-a-mirrored-model.mdx +++ b/frontend/pages/docs/users/model-mirroring/creating-a-mirrored-model.mdx @@ -9,10 +9,16 @@ import tokens from 'public/docs/bailo_tokens.png' # Creating a mirrored model +> **Common questions this page answers:** +> +> - How do I create a mirrored model in Bailo? +> - What is model mirroring? +> - How do I synchronise a mirrored model with its source? + ## What is model mirroring? -Model mirroring allows you to create a read-only copy of an existing model that is synchronised from an external source. -The source model may exist: +Model mirroring allows you to create a read-only copy of an existing model that is manually synchronised from an +external source. The source model may exist: - In another Bailo instance owned by the same organisation, or - In an external system that supports Bailo model exports @@ -21,22 +27,26 @@ The mirrored model uses the source model as the single source of truth. Core mod cannot be edited locally. This ensures consistency while still allowing controlled reuse of model metadata. You may optionally add additional local context to a mirrored model without modifying the source model. -## How to create a Mirrored Model +## How to create a mirrored model -If model mirroring is enabled for your instance, you will be able to create a mirrored model on the model creation page: +Create a mirrored model from the model creation page if mirroring is enabled for your Bailo instance: - access request schema selection + model creation page with mirrored model option +The screenshot above shows the model creation page with the Mirrored Model option available. + 1. Select **Create mirrored model** 2. Provide the basic model details as prompted 3. Enter the **Source Model ID** of the model you want to mirror - access request schema selection + mirrored model creation form with source model ID field +The screenshot above shows the mirrored model creation form with fields for basic details and the Source Model ID. + The Source Model ID must exactly match the ID of the source model. This is used to establish the synchronisation link. ## Exporting the source model @@ -49,9 +59,11 @@ source model: 3. Export the model - access request schema selection + source model export settings +The screenshot above shows the source model's export settings used to synchronise the mirrored copy. + Once exported, the mirrored model will be populated with the source model's data and kept in sync on subsequent exports. export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/model-mirroring/editing-a-mirrored-model-card.mdx b/frontend/pages/docs/users/model-mirroring/editing-a-mirrored-model-card.mdx index fcb2543636..23fe8e5d3c 100644 --- a/frontend/pages/docs/users/model-mirroring/editing-a-mirrored-model-card.mdx +++ b/frontend/pages/docs/users/model-mirroring/editing-a-mirrored-model-card.mdx @@ -9,6 +9,12 @@ import tokens from 'public/docs/bailo_tokens.png' # Editing a mirrored model card +> **Common questions this page answers:** +> +> - Can I edit a mirrored model card? +> - How do I add local information to a mirrored model? +> - Does editing a mirrored model overwrite the source data? + Mirrored models are read-only with respect to information originating from the source model. This ensures the source model remains the authoritative version. @@ -16,18 +22,24 @@ You cannot directly edit source content in a mirrored model, however, you can ad provide context that is specific to your environment or use case. - access request schema selection + mirrored model card with read-only source content +The screenshot above shows a mirrored model card with read-only source content and an Add additional information button. + ## Adding additional information +Add local context to a mirrored model without modifying the source data, using the Add additional information button. + Instead of an **Edit** button, mirrored models display an **Add additional information** button. Selecting this option allows you to add local responses alongside the source content: - access request schema selection + mirrored model additional information form +The screenshot above shows the form for adding local information alongside the source model's content. + Key points: - Additional information does not overwrite source data @@ -39,13 +51,14 @@ Key points: ## How additional information is displayed -Once saved, your additional information is shown beneath the source answer. This makes it clear which content originates -from the source model and which content is locally added. +Local additions appear beneath the source content, making the origin of each piece of information clear. - access request schema selection + mirrored model card showing source and local content +The screenshot above shows the model card with source content and locally added information displayed separately. + Additional information can be updated at any time and will persist across future synchronisations when the source model is re-exported. diff --git a/frontend/pages/docs/users/models/creating-a-model.mdx b/frontend/pages/docs/users/models/creating-a-model.mdx new file mode 100644 index 0000000000..d94492b703 --- /dev/null +++ b/frontend/pages/docs/users/models/creating-a-model.mdx @@ -0,0 +1,88 @@ +import Image from 'next/legacy/image' +import Box from '@mui/material/Box' +import bailoMarketIcon from 'public/docs/bailo_market_icon.png' +import DocsWrapper from 'src/docs/DocsWrapper' + +# Creating a Model + +This guide walks you through creating a new model in Bailo, from the initial setup through to selecting a schema and +filling in the model card. + +> **Common questions this page answers:** +> +> - How do I create a new model in Bailo? +> - What entry types are available? +> - How do I choose between public and private visibility? + +## Choosing an entry type + +Bailo supports four entry types. Select the one that matches your use case. + +Click the **+ Create** button in the navigation bar. You'll see the available entry types: + +- **Model** - A standard machine learning model with full lifecycle management +- **Mirrored Model** - A read-only copy of a model from an external source (see + [Model Mirroring](/docs/users/model-mirroring/creating-a-mirrored-model)) +- **Untrusted Model** - A model requiring additional scrutiny (if enabled - see + [Untrusted Models](/docs/users/untrusted-models)) +- **Data Card** - Documentation for a dataset (see [Creating a Data Card](/docs/users/data-cards/creating-a-data-card)) + +Select **Model** to create a standard model. + +## Basic details + +Every model requires a name, description, and visibility setting: + +- **Name** - a clear, descriptive name for your model +- **Description** - what the model does and its intended use +- **Visibility** - choose **Public** (visible in the Marketplace) or **Private** (visible only to collaborators) + +Public models are visible to all users. Private models are only visible to users with the appropriate permissions, +configured in the model settings. + + + view of the marketplace + + +The screenshot above shows the marketplace with the model creation button. + +## Selecting a schema + +The schema determines the structure of your model card - the questions you will answer about your model. + +After creating the model, you must select a **schema**. You have two options: + +- **Create from schema** - select a schema from the available list and start with a blank form +- **Create from template** - copy the model card structure and content from an existing model (see + [Model Templating](/docs/users/models/model-templating)) + +The model card captures important information such as: + +- Intended use +- Training and evaluation details +- Performance metrics +- Known limitations or biases + +## Next steps + +After creating a model and selecting a schema: + +1. [Fill in the model card](/docs/users/models/model-card) - document your model +2. [Upload files](/docs/users/models/uploading-files) - add model artefacts +3. [Create a release](/docs/users/models/creating-a-release) - version your model for distribution + +## Changing the model name or description + +To update the model name or description after creation: + +1. Open the model +2. Navigate to **Settings** +3. Edit the **Name** and/or **Description** fields + +## Related pages + +- [Model Card](/docs/users/models/model-card) - Filling in the model card and configuring access +- [Core Concepts](/docs/getting-started/core-concepts) - Understanding the Bailo data model +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Creating models programmatically + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/models/creating-a-release.mdx b/frontend/pages/docs/users/models/creating-a-release.mdx new file mode 100644 index 0000000000..eab6297b5d --- /dev/null +++ b/frontend/pages/docs/users/models/creating-a-release.mdx @@ -0,0 +1,110 @@ +import Image from 'next/legacy/image' +import Box from '@mui/material/Box' +import bailoNewRelease from 'public/docs/bailo_new_release.png' +import bailoNewReleaseForm from 'public/docs/bailo_new_release_form.png' +import DocsWrapper from 'src/docs/DocsWrapper' + +# Creating a Release + +A release is a versioned snapshot of your model at a point in time. Releases bundle together files, container images, +and a reference to a specific model card version. Creating a release triggers the review process. + +> **Common questions this page answers:** +> +> - How do I create a release in Bailo? +> - What is semantic versioning? +> - What is a minor release? +> - What happens after I create a release? + +## What is a release? + +Releases use [semantic versioning](https://semver.org/) (e.g. `1.0.0`, `1.1.0`, `2.0.0`) to track changes to a model +over time: + +- **Major** version (first number): significant changes or breaking changes +- **Minor** version (second number): new features or improvements +- **Patch** version (third number): bug fixes or small updates + +## Creating a release + +You create a release from the Releases tab on your model page by filling in a version number, release notes, and +attaching files or images: + +1. Open your model page and go to the **Releases** tab +2. Click **Draft new Release** + + + empty model releases tab with new release button + + +The screenshot above shows the Releases tab with no releases yet and a 'Draft new Release' button. + +3. The release form opens with the following fields to complete: + - **Semver** - the version number for this release (e.g. `1.0.0`) + - **Release notes** - describe what is included in this release + - **Files** - attach uploaded files to this release + - **Images** - attach container (Docker) images to this release + + + draft new release form + + +The screenshot above shows the release creation form with fields for version number, release notes, file attachments, +and container image attachments. + +4. Once you have filled in the release details and attached any artefacts, click **Create Release** to submit + +## Draft vs published releases + +Releases are initially created as **drafts**. Draft releases can be edited before they are finalised. Once all required +reviews are approved, the release is considered published. + +## Minor releases + +You can mark a release as a **minor release** to bypass the review workflow. This is intended for small, low-risk +updates that do not materially change the behaviour or purpose of the model. + +Typical examples include: + +- Correcting documentation or release notes +- Updating non-functional files (e.g. README updates) +- Minor packaging or metadata adjustments +- Replacing files with equivalent versions that do not change model behaviour + +Key points: + +- **No review is created** when a release is marked as minor +- The release becomes available immediately after creation +- The **minor flag cannot be changed** after the release is created +- Minor releases should **not** be used for model updates, behavioural changes, or anything requiring governance review + +If a change could affect model performance, outputs, safety, or intended use, you should create a **standard release** +so that it goes through the full review process. + +## What happens after creation + +Creating a release triggers the review process, notifying all assigned reviewers: + +1. All review roles assigned to the model are notified +2. Each reviewer evaluates the release +3. Once all required roles approve, the release is finalised +4. If any reviewer requests changes, you'll be notified to update the release + +## Editing a release + +You can update a release's notes, files, and images after creation: + +1. Navigate to the **Releases** tab +2. Select the release you want to edit +3. Click **Edit release** +4. Update the release notes, files, or images +5. **Save** your changes + +## Related pages + +- [Uploading Files](/docs/users/models/uploading-files) - Adding files to your model +- [Uploading Images](/docs/users/models/uploading-images) - Pushing container images +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - The review process +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Creating releases programmatically + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/models/model-card.mdx b/frontend/pages/docs/users/models/model-card.mdx new file mode 100644 index 0000000000..6c8d6cdbba --- /dev/null +++ b/frontend/pages/docs/users/models/model-card.mdx @@ -0,0 +1,85 @@ +import Image from 'next/legacy/image' +import Box from '@mui/material/Box' +import bailoModelAccess from 'public/docs/bailo_model_access.png' +import DocsWrapper from 'src/docs/DocsWrapper' + +# Filling In the Model Card + +The model card is the structured documentation that describes your model. It captures information like intended use, +training methodology, performance metrics, and known limitations. The questions you see come from the schema you +selected when creating the model. + +> **Common questions this page answers:** +> +> - How do I fill in a model card? +> - How do I add collaborators to a model? +> - How do I assign review roles? +> - What is the model card version history? + +## What is a model card? + +A model card serves as the authoritative documentation for a machine learning model. It helps: + +- **Reviewers** evaluate whether the model is fit for purpose +- **Consumers** understand what the model does and its limitations +- **Auditors** verify compliance with organisational policies + +## Filling in the form + +The model card form is divided into sections displayed as tabs, where you answer questions defined by the selected +schema. + +- **Required fields** are marked and must be completed +- Your progress is saved only when you click **Save** +- You can return and edit the model card at any time + +Each edit creates a **new version** of the model card. You can view previous versions in the model's version history. + +## Configuring model access + +After documenting your model, configure who can access the model and who will review releases and access requests. + +1. Open the model and navigate to the **Settings** tab +2. Select **Access** from the side menu +3. Use the **Add a user or group to the Model access list** dropdown to add collaborators +4. Use the **Select roles** dropdown to choose the roles responsible for reviewing and approving the model +5. **Save** your changes + + + selecting user model roles + + +The screenshot above shows the model Settings tab with the Access section, where users and groups can be added as +collaborators and assigned system roles and review roles. + +The selected roles will be notified when the model or its releases require review. + +### System roles + +When adding collaborators, assign one of: + +- **Owner** - Full control over the model +- **Contributor** - Edit model card, upload files, create releases +- **Consumer** - Download files and pull images (with approved access request) + +### Review roles + +Assign specific people to each review role required by the schema (e.g. Model Technical Reviewer, Model Senior +Responsible Officer). These people will review releases and access requests. + +## Version history + +Every edit to the model card creates a new version that is preserved in the model's history: + +1. Open the model page +2. Navigate to the version history +3. Select a previous version to see the model card at that point in time + +## Related pages + +- [Creating a Model](/docs/users/models/creating-a-model) - Setting up a new model +- [Creating a Release](/docs/users/models/creating-a-release) - Versioning your model +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - The review process +- [Roles & Permissions](/docs/reference/roles-and-permissions) - Detailed role reference + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/models/model-templating.mdx b/frontend/pages/docs/users/models/model-templating.mdx new file mode 100644 index 0000000000..6fedea39a3 --- /dev/null +++ b/frontend/pages/docs/users/models/model-templating.mdx @@ -0,0 +1,75 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Creating a Model from a Template + +Model templating allows you to create a new model based on an existing model's structure and content. This is useful +when you want to create models that share a similar model card layout or when your organisation has a standard model +card format. + +> **Common questions this page answers:** +> +> - How do I create a model from a template? +> - How do I enable templating on my model? +> - Can I copy a model card from an existing model? + +## How templating works + +Templating copies the model card structure and content from an existing model to a new model: + +1. A new model is created with your specified name and description +2. The template model's model card structure and content are copied to the new model +3. You can then edit the copied content to reflect your specific model's details + +The template model is not affected - it remains unchanged. + +## Prerequisites + +Both the template model and the destination model must meet certain requirements: + +- The template model must have **Allow templating** enabled in its settings +- You must have already created the new model (templating is applied to an existing model) + +## Using a template + +Apply a template when setting up the model card for a newly created model: + +1. Create a new model (see [Creating a Model](/docs/users/models/creating-a-model)) +2. When prompted to set up the model card, choose **Create from template** +3. Select a template model from the dropdown (only models with templating enabled are shown) +4. Click **Create from template** +5. The model card content from the template is copied to your model +6. Edit the content to reflect your specific model's details + +## Enabling templating on a model + +Model owners can allow their model to be used as a template by other users: + +1. Open your model page +2. Navigate to the **Settings** tab +3. Enable **Allow templating** +4. Save your changes + +Once enabled, your model will appear in the template selection dropdown for other users. + +## Programmatic access + +You can also create a model from a template using the API or the Python client: + +**API:** + +``` +POST /api/v2/model/{modelId}/setup/from-template +``` + +**Python client:** + +```python +model.card_from_template(template_id="source-model-id") +``` + +## Related pages + +- [Creating a Model](/docs/users/models/creating-a-model) - Standard model creation +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Programmatic workflows + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/files.mdx b/frontend/pages/docs/users/models/uploading-files.mdx similarity index 64% rename from frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/files.mdx rename to frontend/pages/docs/users/models/uploading-files.mdx index c4621a0696..8d48016c85 100644 --- a/frontend/pages/docs/users/managing-models-and-releases/upload-to-bailo/files.mdx +++ b/frontend/pages/docs/users/models/uploading-files.mdx @@ -6,16 +6,22 @@ import bailoUploadFileRelease from 'public/docs/bailo_release_file_upload.png' import bailoUploadedFileRelease from 'public/docs/bailo_created_release_upload_file.png' import bailoUploadFile from 'public/docs/bailo_file_upload.png' -# Uploading files +# Uploading Files Files represent any artefacts related to a model, such as binaries, configuration files, or supporting documentation. These files allow others to build, evaluate, and use your model. +> **Common questions this page answers:** +> +> - How do I upload files to a model in Bailo? +> - How do I attach files to a release? +> - How do I upload files programmatically? + ## Adding files to a model -Files are uploaded from the **File Management** tab of a model. +Upload files from the **File Management** tab on your model page: -1. Click **Add New Files** +1. Click **Add new files** 2. Select files to upload 3. Optionally assign tags to each file @@ -23,27 +29,33 @@ Files are uploaded from the **File Management** tab of a model. view of files being uploaded +The screenshot above shows the file upload interface with file selection, tag assignment, and an Upload files button. + +4. Click **Upload files** + ## Adding files to a new release When creating a new release, you can attach both newly uploaded files and files that already exist on the model. -
- new release form with file attached +The screenshot above shows the new release form with a file already attached. + ## Adding files to an existing release -Existing releases can be edited to add or remove files. +You can edit existing releases to add or remove file attachments. view of existing release with edit release button +The screenshot above shows an existing release with an Edit release button for adding or removing files. + ## Uploading files programmatically -To upload or download files programmatically, use the Bailo Python Client. Examples are available +Use the Bailo Python Client to upload or download files via code. Examples are available [here](/docs/python/notebooks/models_and_releases_demo_pytorch/index.html). export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/models/uploading-images.mdx b/frontend/pages/docs/users/models/uploading-images.mdx new file mode 100644 index 0000000000..3b65c5016e --- /dev/null +++ b/frontend/pages/docs/users/models/uploading-images.mdx @@ -0,0 +1,51 @@ +import DocsWrapper from 'src/docs/DocsWrapper' +import Box from '@mui/material/Box' +import Image from 'next/legacy/image' + +import bailoPushImage from 'public/docs/bailo_push_image_to_docker_container.png' +import bailoImageRelease from 'public/docs/bailo_add_image_to_release.png' + +# Uploading Container Images + +Bailo uses a container (Docker) registry to store container images associated with models. Unlike files, container +images are **model-specific**: you push an image to a model first, and then explicitly attach it to one or more +releases. + +> **Common questions this page answers:** +> +> - How do I push a container (Docker) image to Bailo? +> - How do I add a container image to a release? +> - What is a container image? + +## Pushing an image to a model + +Push container images to the Bailo registry from the **Registry** tab on your model page. + +This tab provides step-by-step instructions tailored to your model and registry. + + + instructions on how to push image to model + + +The screenshot above shows the Registry tab with step-by-step instructions for logging in, tagging, and pushing a +container image. + +## Adding an image to a release + +Once pushed, container images can be attached to new or existing releases. + +Edit an existing release or create a new one, then select the image from the available list. + + + release being edited with new image attached + + +The screenshot above shows a release being edited with a container image attached from the available list. + +## What is a container? + +In the context of Bailo, a container is an isolated, self-contained application environment used to run a model. +Packaging models as containers allows them to be started quickly, run consistently across environments, and used for +inference without additional setup. + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/programmatically-using-bailo/authentication.mdx b/frontend/pages/docs/users/programmatically-using-bailo/authentication.mdx index c3c2b3400f..24a127b214 100644 --- a/frontend/pages/docs/users/programmatically-using-bailo/authentication.mdx +++ b/frontend/pages/docs/users/programmatically-using-bailo/authentication.mdx @@ -7,58 +7,144 @@ import tokens from 'public/docs/bailo_tokens.png' # Programmatic Authentication with Bailo +Bailo supports two authentication methods for programmatic access: PKI (Public Key Infrastructure) certificates and +personal access tokens. Choose the method that suits your environment. + +> **Common questions this page answers:** +> +> - How do I authenticate with the Bailo API? +> - Should I use PKI certificates or personal access tokens? +> - How do I use the Python client with authentication? +> - How do I use curl to call the Bailo API? + +## Which method should I use? + +Choose between PKI certificate authentication and token-based authentication depending on your environment. + +- **PKI (Public Key Infrastructure) certificates** - Best for organisations with PKI infrastructure and automated + pipelines. Requires a client certificate, private key, and CA (Certificate Authority) certificate. +- **Token-based (Personal Access Tokens)** - Best for individual users, quick setup, and scoped access. Requires a Bailo + account to generate tokens. + ## Public Key Infrastructure (PKI) -Bailo supports PKI‑based authentication. The Python client can sign requests using certificates. +PKI-based authentication uses mutual TLS (Transport Layer Security) with client certificates to sign requests. This is +the recommended method for environments where mutual TLS is standard. + +### Setup + +You will need: + +- A **client certificate** (`.pem` file) +- A **private key** (`.pem` file) +- A **certificate authority (CA) file** (`.pem` file) + +### Using PKI with the Python client ```python from bailo import PkiAgent, Client -pki_agent = PkiAgent() - -Client( - "http://127.0.0.1:8080", - cert="path/to/cert.pem", - key="path/to/key.pem", - auth="path/to/certificate/authority/file.pem" +agent = PkiAgent( + cert="path/to/client-cert.pem", + key="path/to/client-key.pem", + auth="path/to/ca-cert.pem", ) + +client = Client("https://your-bailo-instance.com", agent=agent) +``` + +### Using PKI with curl + +```bash +curl --cert path/to/client-cert.pem \ + --key path/to/client-key.pem \ + --cacert path/to/ca-cert.pem \ + https://your-bailo-instance.com/api/v2/models/search ``` -## Token‑based authentication +## Token-based Authentication + +Token-based authentication uses Personal Access Tokens (PATs) for fine-grained, scoped API access. + +If you are not using PKI, Bailo allows the use of PATs for fine-grained access. Tokens grant access to specific models +and specific actions. -If you are not using PKI then Bailo allows the usage of access tokens to allow for fine-grained access. Tokens are used -to grant access to models within Bailo to specific models and specific actions. These can be found in -`Settings > Authentication`. +### Creating a token + +1. Navigate to **User (top right) > Settings > Authentication** +2. Click **Add token** Authentication tab listing personal access tokens and add token button -To create a new access token, press `Add token` and select the options from below. - -- `image:read` pull Docker images from Bailo -- `file:read` download files from Bailo +The screenshot above shows the Authentication tab in Bailo's user settings, listing existing personal access tokens with +an 'Add token' button. -Copy and paste the `Access Key` and `Secret Key` and store these safely. +3. Configure the token: + - **Description** - a name to help you identify this token + - **Scope** - choose "All models" or select specific models + - **Permissions** - select the actions this token allows +4. Click **Generate Token** +5. **Copy both the Access Key and Secret Key immediately** - the secret key is only shown once popup confirming token creation +The screenshot above shows the token creation confirmation dialog displaying the access key and secret key. + ### Using tokens with the Python client ```python from bailo import TokenAgent, Client -token_agent = TokenAgent(ACCESS_KEY, SECRET_KEY) +agent = TokenAgent( + access_key="your-access-key", + secret_key="your-secret-key", +) + +client = Client("https://your-bailo-instance.com", agent=agent) +``` + +### Using tokens with curl + +Tokens use HTTP Basic Authentication with the access key as the username and secret key as the password: -Client("http://127.0.0.1:8080", agent=token_agent) +```bash +curl -u "your-access-key:your-secret-key" \ + https://your-bailo-instance.com/api/v2/models/search ``` -## Base Agent +## Security best practices + +Follow these practices to keep your API credentials secure. + +- **Never commit tokens to source control** - use environment variables or secret managers +- **Use the minimum required permissions** - only grant the actions your workflow needs +- **Scope tokens to specific models** when possible, rather than "All models" +- **Rotate tokens regularly** - delete old tokens and create new ones periodically +- **Store secret keys securely** - they cannot be retrieved after creation + +## Base agent The full Python docs for the `Agent` class and subsequent child-classes is available [here](../../python/bailo.core.html#bailo.core.agent.Agent). It is possible to pass `kwargs` to the base `Agent` class such as `verify`. +## Troubleshooting + +Common authentication errors and their solutions. + +- **401 Unauthorized** - Check your credentials are correct and the token hasn't been deleted +- **403 Forbidden** - Your token may not have the right permissions or model scope +- **Certificate errors** - Verify your certificate paths and that the CA certificate matches your Bailo instance +- **Connection refused** - Check the Bailo URL is correct and the instance is running + +## Related pages + +- [Personal Access Tokens](/docs/users/programmatically-using-bailo/personal-access-tokens) - Detailed token management +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Using the Python client library +- [Swagger API](/api/docs/) - All available API endpoints + export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/programmatically-using-bailo/open-api.mdx b/frontend/pages/docs/users/programmatically-using-bailo/open-api.mdx index bfaf9b58fb..d6bc7acf03 100644 --- a/frontend/pages/docs/users/programmatically-using-bailo/open-api.mdx +++ b/frontend/pages/docs/users/programmatically-using-bailo/open-api.mdx @@ -12,6 +12,61 @@ Bailo provides an OpenAPI specification for all Bailo endpoints viewable [here]( open api documentation page with example request parameters -For guidance on using Swagger UI, see the [official documentation](https://swagger.io/tools/swagger-ui/). +The screenshot above shows the Swagger UI interface with grouped API endpoints, request parameters, and response +schemas. + +## Using Swagger UI + +Swagger UI lets you browse, understand, and test API (Application Programming Interface) endpoints directly in your +browser. + +### Browsing endpoints + +Endpoints are grouped by category (Models, Releases, Files, Schemas, etc.). Click on a group to expand it and see all +available endpoints. Each endpoint shows: + +- The HTTP method and path +- A description of what it does +- Required and optional parameters +- Request body schema (for POST/PUT/PATCH) +- Response schema and status codes + +### Making test requests + +You can test endpoints directly from the Swagger UI: + +1. Click on an endpoint to expand it +2. Click **Try it out** +3. Fill in the required parameters and request body +4. Click **Execute** +5. View the response below + +> **Note**: You must be authenticated to make requests. If you're logged into Bailo in the same browser, your session +> credentials will be used automatically. + +## Downloading the OpenAPI specification + +Download the raw OpenAPI specification in JSON format for use with code generation tools or API clients. + +- Navigate to `/api/v2/specification` on your Bailo instance +- The response is the full OpenAPI JSON specification + +You can use this specification with tools like: + +- [OpenAPI Generator](https://openapi-generator.tech/) to generate client libraries +- [Postman](https://www.postman.com/) to import all endpoints as a collection +- [Insomnia](https://insomnia.rest/) for API testing + +## Endpoint summary + +For a complete list of all API endpoints, see the interactive [Swagger API](/api/docs/). + +For guidance on Swagger UI itself, see the [official Swagger documentation](https://swagger.io/tools/swagger-ui/). + +## Related pages + +- [Swagger API](/api/docs/) - All available API endpoints +- [Authentication](/docs/users/programmatically-using-bailo/authentication) - Setting up API authentication +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Python wrapper for the API export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/programmatically-using-bailo/personal-access-tokens.mdx b/frontend/pages/docs/users/programmatically-using-bailo/personal-access-tokens.mdx new file mode 100644 index 0000000000..b87c0399ef --- /dev/null +++ b/frontend/pages/docs/users/programmatically-using-bailo/personal-access-tokens.mdx @@ -0,0 +1,136 @@ +import DocsWrapper from 'src/docs/DocsWrapper' +import Image from 'next/legacy/image' +import Box from '@mui/material/Box' + +import personalAccessTokens from 'public/docs/bailo_personal_access_tokens.png' +import createPersonalAccessToken from 'public/docs/bailo_create_personal_access_token.png' + +# Personal Access Tokens + +Personal Access Tokens (PATs) allow you to authenticate with Bailo's API and container registry programmatically. Each +token can be scoped to specific models and actions, giving you fine-grained control over what the token can do. + +> **Common questions this page answers:** +> +> - How do I create a Personal Access Token (PAT)? +> - What permissions can I assign to a token? +> - How do I use a token with curl or the Python client? +> - How do I revoke a token? + +## Creating a token + +Create tokens from the Authentication tab in your user settings. + +1. Navigate to **User (top right) > Settings > Authentication** +2. Click **Add Token** + + + Authentication tab listing personal access tokens and add token button + + +The screenshot above shows the Authentication tab listing existing personal access tokens with their descriptions, +access keys, and a delete button for each. + +3. Fill in the form: + - **Description** - a name to help you identify this token (e.g. "CI/CD pipeline", "Local development") + - **Models** - choose **All models** or select specific models + - **Actions** - select the permitted actions this token can perform + + + generate personal token form + + +The screenshot above shows the token creation form with fields for description, model scope, and action permissions. + +4. Click **Generate Token** +5. **Copy both the Access Key and Secret Key immediately** - the secret key is only shown once and cannot be retrieved + later + +## Available permissions + +Each token can be scoped to specific actions using these permission types. + +- **model:read** - View model and data card settings +- **model:write** - Create and update model and data card settings +- **model:export** - Export model and data card settings +- **release:read** - View release information +- **release:write** - Create and update releases +- **release:export** - Export release information +- **access_request:read** - View and list access requests +- **access_request:write** - Create, approve, and comment on access requests +- **file:read** - Download and view files +- **file:write** - Upload and delete files +- **image:read** - List and pull container (Docker) images +- **image:write** - Push and delete container images +- **schema:write** - Upload and modify schemas (administrators only) +- **reviewRole:read** - View review roles +- **reviewRole:write** - Create and update review roles + +> **Note**: Selecting a write permission automatically includes the corresponding read permission. + +## Token scoping + +Tokens can be scoped to all models or restricted to specific models. + +- **All models** - the token works with every model in Bailo +- **Specific models** - the token only works with the models you select + +Use specific model scoping when possible to follow the principle of least privilege. + +## Managing tokens + +View and delete existing tokens from the Authentication tab: + +1. Navigate to **User > Settings > Authentication** +2. Your existing tokens are listed with their description and access key +3. Click the delete button next to a token to revoke it + +> **Important**: Deleting a token is immediate and permanent. Any scripts or applications using that token will stop +> working. + +## Using tokens + +Tokens use HTTP Basic Authentication with the access key as the username and the secret key as the password. + +- **Username**: the access key +- **Password**: the secret key + +### With the Python client + +```python +from bailo import TokenAgent, Client + +agent = TokenAgent(access_key="your-access-key", secret_key="your-secret-key") +client = Client("https://your-bailo-instance.com", agent=agent) +``` + +### With curl + +```bash +curl -u "your-access-key:your-secret-key" \ + https://your-bailo-instance.com/api/v2/models/search +``` + +### With the container registry + +```bash +docker login your-registry-host -u "your-access-key" -p "your-secret-key" +``` + +## Security best practices + +Follow these practices to keep your tokens secure. + +- **Store secret keys securely** - use environment variables or a secrets manager, not source code +- **Use minimum permissions** - only grant the actions your workflow needs +- **Scope to specific models** when possible +- **Rotate tokens regularly** - delete old tokens and create new ones +- **Use separate tokens** for different purposes (CI/CD, local dev, etc.) + +## Related pages + +- [Authentication](/docs/users/programmatically-using-bailo/authentication) - Overview of authentication methods +- [Python Client](/docs/users/programmatically-using-bailo/python-client) - Using tokens with the Python client +- [Swagger API](/api/docs/) - All available API endpoints + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/programmatically-using-bailo/python-client.mdx b/frontend/pages/docs/users/programmatically-using-bailo/python-client.mdx index fce6c1267f..bea8e4aac1 100644 --- a/frontend/pages/docs/users/programmatically-using-bailo/python-client.mdx +++ b/frontend/pages/docs/users/programmatically-using-bailo/python-client.mdx @@ -2,35 +2,253 @@ import DocsWrapper from 'src/docs/DocsWrapper' # Python Client -Bailo provides a Python client that wraps key API endpoints. The package is published on -[PyPI](https://pypi.org/project/bailo/). +Bailo provides a Python client that wraps the API (Application Programming Interface) into high-level helper classes. +The package is published on [PyPI (Python Package Index)](https://pypi.org/project/bailo/). -**You can find the full Python documentation [here](../../python/index.html).** +> **Common questions this page answers:** +> +> - How do I use the Bailo Python client? +> - How do I create a model programmatically? +> - How do I upload files and create releases with Python? +> - How do I authenticate the Python client? + +**Full API documentation is available [here](../../python/index.html).** ## Installation +Install the Bailo Python client from PyPI using pip. + ```bash pip install bailo ``` -## Why use the Python Client? +For MLflow integration: + +```bash +pip install bailo[mlflow] +``` + +## Authentication + +Before using the client, configure authentication using either PKI (Public Key Infrastructure) certificates or Personal +Access Tokens. + +### PKI certificate authentication (Recommended) + +Use a client certificate for mutual TLS: + +```python +from bailo import Client, PkiAgent + +agent = PkiAgent( + cert="path/to/client-cert.pem", + key="path/to/client-key.pem", + auth="path/to/ca-cert.pem", +) +client = Client( + url="https://your-bailo-instance.com", + agent=agent, +) +``` + +### Token-based Authentication + +Use a Personal Access Token created in Bailo's Settings page: + +```python +from bailo import Client, TokenAgent + +agent = TokenAgent( + access_key="your-access-key", + secret_key="your-secret-key", +) +client = Client( + url="https://your-bailo-instance.com", + agent=agent, +) +``` + +## Helper classes + +The Python client provides high-level helper classes that simplify common workflows. + +- **Model** - Create, retrieve, and manage models +- **Datacard** - Create and manage data cards +- **Release** - Create versioned releases with files and images +- **AccessRequest** - Submit and manage access requests +- **Schema** - Create and retrieve schemas +- **Experiment** - Track experiments and metrics + +## Common workflows + +These examples show the most common tasks you can perform with the Python client. + +### Creating a model + +```python +from bailo import Client, Model, TokenAgent + +agent = TokenAgent(access_key="...", secret_key="...") +client = Client(url="https://your-bailo-instance.com", agent=agent) + +# Create a new model +model = Model.create( + client=client, + name="My Image Classifier", + description="A CNN for image classification", +) + +print(f"Model created with ID: {model.model_id}") + +# Set up the model card from a schema +model.card_from_schema(schema_id="your-schema-id") + +# Update the model card metadata +model.update_model_card(metadata={ + "overview": { + "tags": ["image-classification", "cnn"], + } +}) +``` + +### Creating a release and uploading files + +```python +from bailo import Release + +# Upload a file to the model +with open("model_weights.pt", "rb") as f: + model_file_id = client.simple_upload( + model_id=model.model_id, + name="model_weights.pt", + buffer=f, + ) + +# Create a release +release = Release.create( + client=client, + model_id=model.model_id, + version="1.0.0", + notes="Initial release with trained weights", + model_card_version=model.card_version, + files=[model_file_id], + draft=True, +) + +print(f"Release {release.version} created") +``` + +### Downloading files + +```python +# Download a specific file +release.download("model_weights.pt", output_dir="./downloads") + +# Download all files from a release +release.download_all(output_dir="./downloads") +``` + +### Managing access requests + +```python +from bailo import AccessRequest + +# Create an access request +access_request = AccessRequest.create( + client=client, + model_id="target-model-id", + schema_id="access-request-schema-id", + metadata={ + "overview": { + "name": "Research access", + "entities": ["user:your-username"], + } + }, +) +``` + +### Working with data cards -The Python client integrates cleanly with Python tooling and is ideal for: +```python +from bailo import Datacard -- Notebooks and experimentation -- Automation and scripting -- Integrating Bailo into ML workflows +# Create a data card +data_card = Datacard.create( + client=client, + name="Training Dataset v2", + description="Curated training data for image classification", +) -### Scripting and Automation +# Set up data card from a schema +data_card.card_from_schema(schema_id="data-card-schema-id") +``` -This Python Client provides a better way to add additional Bailo functionality to projects. +### Working with schemas -### Using Bailo with MLflow +```python +from bailo import Schema -Optional extras are provided for integration with [MLflow](https://mlflow.org/docs/latest/ml/). +# Get all available model schemas +schemas = Schema.get_all(client=client, kind="model") + +for schema in schemas: + print(f"{schema.name} (ID: {schema.schema_id})") + +# Create a new schema +schema = Schema.create( + client=client, + schema_id="my-new-schema", + name="My Model Card Schema", + description="Custom schema for our team's models", + kind="model", + json_schema={"type": "object", "properties": {...}}, +) +``` + +## Jupyter notebook examples + +Detailed, runnable examples are available as Jupyter notebooks in the Bailo repository. + +- [Models and Releases (PyTorch)](/docs/python/notebooks/models_and_releases_demo_pytorch/index.html) - End-to-end model + creation, file upload, and release management +- [Access Requests](/docs/python/notebooks/access_requests_demo/index.html) - Submitting and managing access requests +- [Data Cards](/docs/python/notebooks/datacards_demo/index.html) - Creating and updating data cards +- [Schemas](/docs/python/notebooks/schemas_demo/index.html) - Schema creation and management +- [Experiment Tracking](/docs/python/notebooks/experiment_tracking_demo/index.html) - Tracking experiments and metrics + +## MLflow integration + +The optional MLflow integration allows you to log models from MLflow directly to Bailo. ```bash pip install bailo[mlflow] ``` +This enables workflows where you train a model using MLflow's tracking capabilities and then publish the resulting +artefacts to Bailo for governance and distribution. + +## Low-Level Client Methods + +For advanced use cases, the `Client` class exposes lower-level methods that map directly to API endpoints. + +- **Models** - `post_model`, `get_model`, `get_models`, `patch_model`, `delete_model` +- **Model Cards** - `get_model_card`, `put_model_card`, `model_card_from_schema`, `model_card_from_template` +- **Releases** - `post_release`, `get_release`, `get_all_releases`, `put_release`, `delete_release` +- **Files** - `simple_upload`, `get_files`, `get_download_file`, `get_download_by_filename`, `delete_file` +- **Access Requests** - `post_access_request`, `get_access_request`, `get_access_requests`, `patch_access_request`, + `delete_access_request` +- **Schemas** - `get_all_schemas`, `get_schema`, `post_schema` +- **Reviews** - `get_reviews`, `post_release_review`, `post_access_request_review` +- **Scanning** - `put_file_scan`, `put_image_scan` +- **Roles** - `get_model_roles` + +Refer to the [full API documentation](../../python/index.html) for detailed method signatures and parameters. + +## Related pages + +- [Authentication](/docs/users/programmatically-using-bailo/authentication) - Detailed authentication setup +- [Personal Access Tokens](/docs/users/programmatically-using-bailo/personal-access-tokens) - Creating tokens for API + access +- [OpenAPI Reference](/docs/users/programmatically-using-bailo/open-api) - REST API documentation + export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/programmatically-using-bailo/webhooks.mdx b/frontend/pages/docs/users/programmatically-using-bailo/webhooks.mdx index 8766a4962b..e0229bd684 100644 --- a/frontend/pages/docs/users/programmatically-using-bailo/webhooks.mdx +++ b/frontend/pages/docs/users/programmatically-using-bailo/webhooks.mdx @@ -2,27 +2,60 @@ import DocsWrapper from 'src/docs/DocsWrapper' # Webhooks -Bailo provides webhooks for programmatic, event-driven interactions with individual models for interfacing with external -applications. +> **Common questions this page answers:** +> +> - How do I set up a webhook in Bailo? +> - What events can trigger webhooks? +> - How do I update or delete a webhook? +> - How do I secure webhook requests? + +Bailo provides webhooks for programmatic, event-driven interactions with individual models. When certain events occur on +a model, Bailo sends an HTTP POST request to a URL you specify, allowing you to integrate with external applications and +workflows. + +## Setting up a webhook + +Create webhooks through the API by sending a POST request with the webhook configuration: + +```bash +curl -X POST https://your-bailo-instance.com/api/v2/model/{modelId}/webhooks \ + -H "Content-Type: application/json" \ + -u "your-access-key:your-secret-key" \ + -d '{ + "name": "My CI/CD webhook", + "uri": "https://your-service.com/webhook", + "token": "your-bearer-token", + "events": ["createRelease", "updateRelease"], + "active": true, + "insecureSSL": false + }' +``` -## Events +### Webhook configuration options -A Bailo model's webhook events are: +- **name** - A descriptive name for the webhook. Required +- **uri** - The URI (Uniform Resource Identifier) that will receive the POST requests. Required +- **token** - A bearer token included in the `Authorization` header of webhook requests. Optional +- **events** - Which events trigger this webhook (see Events below). Optional, defaults to all events +- **active** - Whether the webhook is enabled. Optional, defaults to true +- **insecureSSL** - Skip SSL (Secure Sockets Layer) verification for the target URL. Optional, defaults to false -- `createRelease` -- `updateRelease` -- `createReviewResponse` -- `createAccessRequest` +## Events -To view and configure webhooks for a given model, refer to the Webhook section of our [api docs](./open-api). +A Bailo model's webhook can be triggered by the following events: -## Request Format +- **createRelease** - Fires when a new release is created on the model +- **updateRelease** - Fires when an existing release is updated +- **createReviewResponse** - Fires when a reviewer submits a response (approval or change request) +- **createAccessRequest** - Fires when a user submits an access request for the model -When a webhook is triggered, it will send a `POST` request to the webhook's URI. +## Request format -The webhook will include `"Authorization": "Bearer "` in the request `headers` if the webhook has a token. +When triggered, Bailo sends a POST request to the webhook's URI (Uniform Resource Identifier) with event-specific data. -The body of the webhook's request will vary depending on the type of hook. +The webhook includes `"Authorization": "Bearer "` in the request `headers` if the webhook has a token configured. + +The body of the webhook's request varies depending on the type of event. ### createRelease / updateRelease @@ -60,7 +93,7 @@ The body of the webhook's request will vary depending on the type of hook. "semver": "0.0.1", "accessRequestId": "123456789abcdef012345678", "modelId": "abc123", - "kind": "{release|access}" + "kind": "{release|access}", "role": "mtr", "createdAt": "2025-01-21T12:00:00.000Z", "updatedAt": "2025-01-21T12:00:00.000Z" @@ -90,4 +123,50 @@ The body of the webhook's request will vary depending on the type of hook. } ``` +## Managing webhooks + +List, update, and delete webhooks using the API. + +### Listing webhooks + +```bash +curl https://your-bailo-instance.com/api/v2/model/{modelId}/webhooks \ + -u "your-access-key:your-secret-key" +``` + +### Updating a webhook + +```bash +curl -X PUT https://your-bailo-instance.com/api/v2/model/{modelId}/webhook/{webhookId} \ + -H "Content-Type: application/json" \ + -u "your-access-key:your-secret-key" \ + -d '{ + "name": "Updated webhook name", + "uri": "https://your-service.com/webhook", + "events": ["createRelease"], + "active": true + }' +``` + +### Deleting a webhook + +```bash +curl -X DELETE https://your-bailo-instance.com/api/v2/model/{modelId}/webhook/{webhookId} \ + -u "your-access-key:your-secret-key" +``` + +## Security considerations + +Follow these practices to secure your webhook endpoints. + +- **Use the token field** to include a bearer token in webhook requests, so your receiving service can verify the + request came from Bailo +- **Use HTTPS** for your webhook URI to prevent eavesdropping on the request payloads +- **Avoid `insecureSSL: true`** in production - only use it for testing with self-signed certificates + +## Related pages + +- [Authentication](/docs/users/programmatically-using-bailo/authentication) - Setting up API authentication +- [Swagger API](/api/docs/) - All available API endpoints + export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/model-lifecycle.mdx b/frontend/pages/docs/users/reviews/model-lifecycle.mdx deleted file mode 100644 index 48f682373b..0000000000 --- a/frontend/pages/docs/users/reviews/model-lifecycle.mdx +++ /dev/null @@ -1,30 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Model Card Lifecycle Review - -## What is a model card lifecycle review? - -Model cards should be periodically reviewed by the model owners so that they can make sure that the information -contained within the form is still accurate and up to date. - -## How are model cards reviewed? - -Model cards are reviewed in a similar manner to releases and access requests, the main difference being that is the -model owner who conducts the review. Once a model owner is happy that a model card is sufficiently filled out, they can -set a review due date within the **details** section to the left of the model card inside the **Overview** tab of their -model. - -If a date is set, a review is automatically created, and can be responded to whenever the model developer feels is -appropriate. - -## Approval process - -The model owner(s) will get notifications nearer the time of the review date, but the review itself can be conducted -sooner if it is deemed appropriate. Once the person conducting the review is happy that the model card is up to date, -they can either click the **review** button on the model card **details** panel, or they can go to the **reviews** page -where this is new section titled **Lifecycle**. - -The review page itself is similar to that of releases and access requests, with the addition of a date selector field -which is mandatory for lifecycle reviews. - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/review-outcomes.mdx b/frontend/pages/docs/users/reviews/review-outcomes.mdx new file mode 100644 index 0000000000..88824073d8 --- /dev/null +++ b/frontend/pages/docs/users/reviews/review-outcomes.mdx @@ -0,0 +1,111 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Review Outcomes + +This page explains what happens after a release, access request, or model card lifecycle review has been reviewed, +including the different outcomes and what to do when changes are requested. + +> **Common questions this page answers:** +> +> - What happens after a release is reviewed? +> - What do I do when changes are requested? +> - How do I view review history? +> - What happens after a model card lifecycle review is completed? + +## Release review outcomes + +A release review can result in approval or a request for changes. + +### Approved + +When all required review roles have approved a release, it is marked as approved and moves to the **Your archived +reviews** tab on the Reviews page. Approved releases are considered final and ready for use. + +Both the **Model Technical Reviewer (MTR)** and the **Model Senior Responsible Officer (MSRO)** (or equivalent review +roles configured for the schema) must approve before a release is fully approved. + +### Changes requested + +If any reviewer requests changes, the model owner is notified. The model owner should: + +1. Review the feedback provided in the reviewer's comments +2. Make the necessary changes to the model card, files, or release notes +3. The reviewer can then re-evaluate and approve + +### Who can create releases? + +Only the **Owner** and **Contributors** of a model can create releases. See +[Roles & Permissions](/docs/reference/roles-and-permissions) for details. + +## Access request review outcomes + +Access request reviews follow a similar approval or change-request flow. + +### Approved + +When the required reviewer approves an access request, the requesting user gains permission to download files and pull +container images from the model. Approved access requests move to the **Your archived reviews** tab. + +### Changes requested + +If the reviewer requests changes, the requesting user is notified. They should review the feedback and update their +access request accordingly. + +### Who reviews access requests? + +Access requests are typically reviewed by the **Model Senior Responsible Officer** (or equivalent). The specific review +role depends on your schema configuration. + +## Model card lifecycle review outcomes + +Model card lifecycle reviews help confirm that a model card remains accurate and up to date. + +### Approved + +When a lifecycle review is completed, the current review is closed and the model card is treated as reviewed. The +reviewer must set a new review due date, which determines when the model card should next be reviewed. + +Approved lifecycle reviews move to the **Your archived reviews** tab. + +### Changes required before completion + +If the model card is not accurate or complete, the model owner should update the model card before completing the +lifecycle review. + +The reviewer should check that any outdated or missing information has been addressed, then submit the lifecycle review +with a new review due date. + +### Who reviews model cards? + +Model card lifecycle reviews are normally completed by **model owners** or another user responsible for maintaining the +model card. + +## Viewing review history + +To see the history of reviews for a model: + +- **Releases**: Open the model page and navigate to the **Releases** tab. Each release shows its review status +- **Access requests**: Open the model page and navigate to the **Access Requests** tab +- **Model card lifecycle reviews**: Open the model page and navigate to the **Overview** tab. The model card details + show lifecycle review information where configured. + +To see all your completed reviews: + +1. Click **Reviews** in the navigation bar +2. Select the **Your archived reviews** tab + +## Python client examples + +Detailed Python examples for working with releases and access requests are available: + +- [Models and Releases demo](/docs/python/notebooks/models_and_releases_demo_pytorch/index.html) +- [Access Requests demo](/docs/python/notebooks/access_requests_demo/index.html) + +## Related pages + +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - How the review process works +- [Reviewing a Release](/docs/users/reviews/reviewing/releases) - Guide for reviewers +- [Reviewing an Access Request](/docs/users/reviews/reviewing/access) - Reviewing access requests +- [Reviewing a Model Card Lifecycle](/docs/users/reviews/reviewing/model-lifecycle) - Reviewing model cards + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/reviewed/access-request.mdx b/frontend/pages/docs/users/reviews/reviewed/access-request.mdx deleted file mode 100644 index f26106df43..0000000000 --- a/frontend/pages/docs/users/reviews/reviewed/access-request.mdx +++ /dev/null @@ -1,20 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Reviewed Access Requests - -## What is an access request? - -An access request grants permission to use a model hosted in Bailo. - -## Who reviews access requests? - -This can only be reviewed and approved by the **Model Senior Responsible officer**. - -## Approval process - -When an access request is created, the Model Senior Responsible Officer can either request changes or approve. Once -approved, the access request will be moved to **Archived**. - -An example Python workflow is available [here](/docs/python/notebooks/access_requests_demo/index.html). - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/reviewed/releases.mdx b/frontend/pages/docs/users/reviews/reviewed/releases.mdx deleted file mode 100644 index a537dcb52f..0000000000 --- a/frontend/pages/docs/users/reviews/reviewed/releases.mdx +++ /dev/null @@ -1,30 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Reviewed Releases - -## What is a release? - -A release represents a specific version of a model. Multiple releases can exist for the same model, allowing changes -without affecting earlier versions. - -## How are releases reviewed? - -Release reviewers are assigned by the model owner, typically including: - -- Model Technical Reviewer -- Model Senior Responsible Officer - -See [Completing The Model](../../managing-models-and-releases/upload-to-bailo/completing-the-model) for details. - -## Who can create a release? - -Only the owner of the model can create releases. - -## Approval process - -When the Model Technical Reviewer and Model Senior Responsible Officer review a release, one or both will leave a -request for changes and/or approve. When approved by both roles, the release will be moved to **Archived**. - -An example Python workflow is available [here](/docs/python/notebooks/models_and_releases_demo_pytorch/index.html). - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/reviewing/access.mdx b/frontend/pages/docs/users/reviews/reviewing/access.mdx index f55ea37005..a7b259275a 100644 --- a/frontend/pages/docs/users/reviews/reviewing/access.mdx +++ b/frontend/pages/docs/users/reviews/reviewing/access.mdx @@ -5,11 +5,60 @@ import DocsWrapper from 'src/docs/DocsWrapper' # Reviewing an Access Request -Access requests appear in the **Access Request Reviews** tab on the **Review** page. The review process mirrors release -reviews. Only the **Model Senior Responsible Officer** can approve access requests. +When a user submits an access request for a model, it enters the review process. Access request reviews are typically +handled by the **Model Senior Responsible Officer (MSRO)** (or equivalent review role configured for the schema). + +> **Common questions this page answers:** +> +> - How do I review an access request? +> - What happens after I approve an access request? +> - Who reviews access requests? + +## Finding access requests to review + +Navigate to the **Reviews** page from the navigation bar. Access requests awaiting your review appear in the **Your open +reviews** tab. open access requests reviews page with a unreviewed request +The screenshot above shows the Reviews page with an open access request awaiting review. + +## Reviewing an access request + +Open the access request, examine the requester's details and justification, and submit your decision. + +1. Select the access request from your review queue +2. Click **View access request** and examine the request details: + - **Who** is requesting access (the requesting user or group) + - **Why** they need access (the justification provided in the form) + - **What** permissions they are requesting + - **When** the access should expire (if an end date is specified) +3. Add a **comment** explaining your decision +4. Choose your decision: + - **Approve** - grant the requested access + - **Request changes** - ask the requester to provide more information or modify their request + +## After approval + +Once an access request is approved: + +- The requesting user gains permission to **download files** and **pull container images** from the model +- The access request moves to the **Your archived reviews** tab on the Reviews page +- The requester is notified of the approval + +## After requesting changes + +If you request changes: + +- The requester is notified and can update their access request +- Once updated, you can review the request again + +## Related pages + +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - How the review process works end-to-end +- [Review Outcomes](/docs/users/reviews/review-outcomes) - What happens after approval or change requests +- [Requesting Access](/docs/users/using-a-model/requesting-access) - How users create access requests + export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/reviewing/model-lifecycle.mdx b/frontend/pages/docs/users/reviews/reviewing/model-lifecycle.mdx new file mode 100644 index 0000000000..3c4b6709f5 --- /dev/null +++ b/frontend/pages/docs/users/reviews/reviewing/model-lifecycle.mdx @@ -0,0 +1,101 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Reviewing a Model Card Lifecycle + +Model card lifecycle reviews help ensure that the information recorded on a model remains accurate, complete, and up to +date. They are typically handled by the **model owner** or another user responsible for maintaining the model card. + +> **Common questions this page answers:** +> +> - What is a model card lifecycle review? +> - Who reviews model cards? +> - How do I complete a lifecycle review? +> - What happens after I submit a lifecycle review? + +## What is a model card lifecycle review? + +A model card lifecycle review is a periodic check of a model's documentation. It gives model owners a structured way to +confirm that the model card still reflects the current state of the model. + +Lifecycle reviews are useful for checking that: + +- The model description is still accurate +- Ownership and contact details are correct +- Intended use, limitations, and ethical considerations are up to date +- Data, training, evaluation, and deployment information still reflects the current model +- Any organisational or governance requirements are still being met + +## Who reviews model cards? + +Model card lifecycle reviews are normally completed by **model owners**. Unlike release reviews and access request +reviews, lifecycle reviews are not usually assigned to a separate reviewer role. + +The model owner is responsible for checking the model card and confirming that it remains accurate. + +## Setting a lifecycle review date + +A lifecycle review is created by setting a review due date on the model. + +To set a lifecycle review date: + +1. Navigate to the model +2. Open the **Overview** tab +3. Find the **Details** section beside the model card +4. Set a **review due date** + +Once a due date is set, Bailo automatically creates a lifecycle review. The review can be completed when the model owner +is ready, including before the due date if appropriate. + +## Finding lifecycle reviews to complete + +Navigate to the **Reviews** page from the navigation bar. Lifecycle reviews awaiting your action appear in the **Your +open reviews** tab, under the **Model card** heading. + +You may also be able to start the review from the model card itself by using the **Review** button in the model card +**Details** panel. + +## Completing a lifecycle review + +Open the lifecycle review, check the model card, and submit your response. + +1. Either: + - Select the lifecycle review from the **Reviews** page + - Navigate to the model's **Overview** tab and click **Review** in the model card **Details** panel +2. Read through the model card and confirm whether the information is still accurate and complete +3. Update the model card if anything is missing, outdated, or incorrect +4. Choose a new **review due date** +5. Add a **comment** explaining the outcome of the review +6. **Approve** the lifecycle review + +The new review due date is mandatory. It determines when the model card should next be reviewed. + +## After submitting your review + +After submitting the lifecycle review: + +- The current lifecycle review is closed +- The model card is treated as reviewed +- A new review due date is recorded +- A future lifecycle review will be created based on the new due date +- The review moves to the **Your archived reviews** tab on the Reviews page + +## What to look for + +When completing a lifecycle review, consider checking: + +- **Model purpose** - Is the description of the model and its intended use still correct? +- **Ownership** - Are the owner, organisation, and contact details up to date? +- **Usage guidance** - Are users given accurate information about how the model should and should not be used? +- **Limitations** - Are known limitations, risks, and assumptions still documented? +- **Evaluation** - Are performance metrics, testing information, and validation results still relevant? +- **Data** - Are training, evaluation, or reference datasets still accurately described? +- **Governance** - Does the model card still meet any required organisational or policy expectations? + +## Related pages + +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - How the review process works end-to-end +- [Reviewing a Release](/docs/users/reviews/reviewing/releases) - How to review model releases +- [Reviewing an Access Request](/docs/users/reviews/reviewing/access) - How to review access requests +- [Review Outcomes](/docs/users/reviews/review-outcomes) - What happens after approval or change requests + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/reviewing/releases.mdx b/frontend/pages/docs/users/reviews/reviewing/releases.mdx index 1fa345bd37..41e4bee720 100644 --- a/frontend/pages/docs/users/reviews/reviewing/releases.mdx +++ b/frontend/pages/docs/users/reviews/reviewing/releases.mdx @@ -7,25 +7,85 @@ import DocsWrapper from 'src/docs/DocsWrapper' # Reviewing a Release -When a release requires review, it appears to the relevant users & groups in the **Release Reviews** tab on the -**Review** page. +When a release is created on a model, it enters the review process. If you have been assigned a review role on that +model, you will be notified and the release will appear in your review queue. + +> **Common questions this page answers:** +> +> - How do I review a release in Bailo? +> - What should I look for when reviewing a release? +> - Where do I find releases to review? + +## Who sees release reviews? + +Release reviews are visible to users who have been assigned a **review role** on the model (e.g. Model Technical +Reviewer, Model Senior Responsible Officer). The specific roles depend on the schema attached to the model. + +## Finding releases to review + +Navigate to the **Reviews** page from the navigation bar. Releases awaiting your review appear in the **Your open +reviews** tab, under the **Releases** heading. open release reviews page with a list of unreviewed releases -Select the relevant release to open its details. A **Review** button appears in the release banner. +The screenshot above shows the Reviews page with a list of open release reviews waiting for action. + +## Reviewing a release + +Open a release from the Reviews page or the model's Releases tab, examine the submission, and submit your decision: + +1. Either: + - Select a review from the **Reviews** page + - Navigate to the **Releases** page for the model. A **Review** button appears in the release banner releases page showing release with review banner visually attached -You have the option of approving or requesting changes. +The screenshot above shows a release on the model's Releases page with a review banner attached. + +2. Click **View full release** to examine the release: + - Review the **model card** for completeness and accuracy + - Check the **release notes** describe what is included + - Verify that appropriate **files** and **container images** are attached +3. After examining the release details, add a **comment** explaining your decision (especially important if requesting + changes) +4. Choose your decision: + - **Approve** - you are satisfied with the release + - **Request changes** - something needs to be addressed release review approval popup -When you have made your decision, an email will be sent to the model owner to notify them of your response. +The screenshot above shows the release review form with options to approve or request changes, and a comment field. + +## After submitting your review + +After submitting, the model owner is notified and other reviewers can see your response. + +- An **email notification** is sent to the model owner with your decision +- If you approved, the release moves closer to full approval (all required roles must approve) +- If you requested changes, the model owner will update the release and you can review again +- Your response is visible to other reviewers and the model owner + +## What to look for + +Different reviewer roles focus on different aspects of the release. + +Depending on your review role, consider checking: + +- **Technical reviewers**: Is the model architecture appropriate? Are evaluation metrics provided? Are known limitations + documented? +- **Governance reviewers**: Are data handling practices documented? Are ethical considerations addressed? Does the model + meet organisational policies? + +## Related pages + +- [Understanding Reviews](/docs/users/reviews/understanding-reviews) - How the review process works end-to-end +- [Review Outcomes](/docs/users/reviews/review-outcomes) - What happens after approval or change requests +- [Roles & Permissions](/docs/reference/roles-and-permissions) - Understanding review roles export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/reviews/understanding-reviews.mdx b/frontend/pages/docs/users/reviews/understanding-reviews.mdx new file mode 100644 index 0000000000..7bd141fbca --- /dev/null +++ b/frontend/pages/docs/users/reviews/understanding-reviews.mdx @@ -0,0 +1,133 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Understanding the Review Process + +Reviews are Bailo's governance mechanism. They ensure that models, access requests, and model card lifecycle checks are +examined by the right people before they are approved or completed. This page explains how the review system works +end-to-end. + +> **Common questions this page answers:** +> +> - How do reviews work in Bailo? +> - Who reviews my release? +> - Who reviews my model card lifecycle review? +> - What triggers a review? +> - What happens when a reviewer requests changes? + +## What triggers a review? + +Reviews are created automatically when: + +- A **release** is created for a model - the release enters review and must be approved before it is considered final +- An **access request** is submitted - the request must be approved before the user gains download or pull permissions +- A **model card lifecycle review date** is set - the model owner is prompted to confirm that the model card remains + accurate and up to date + +You do not need to manually create release or access request reviews. The system handles these based on the review roles +configured on the model. Lifecycle reviews are created from the review due date set on the model card. + +## Who reviews what? + +Review assignments depend on the schema's review roles, the model owner's reviewer assignments, and the type of +submission: + +1. **The schema** attached to the model defines which review roles are required (e.g. "Model Technical Reviewer", "Model + Senior Responsible Officer") +2. **The model owner** assigns specific people or groups to each review role in the model's Settings tab +3. **The type of submission** determines which roles participate: + - **Release reviews**: All assigned review roles participate + - **Access request reviews**: Typically only specific roles (commonly the Model Senior Responsible Officer, or MSRO) + participate + - **Model card lifecycle reviews**: Model owners, or another user responsible for maintaining the model card, + complete the review + +## The review lifecycle + +The review lifecycle follows a structured flow from submission through reviewer evaluation to approval or change +requests. + +### For releases + +1. A model owner or contributor creates a new release +2. Bailo creates review tasks for each required review role +3. Assigned reviewers are notified +4. Each reviewer examines the release and the model card, then either: + - **Approves** - confirms the release meets their requirements + - **Requests changes** - explains what needs to be fixed +5. If changes are requested, the model owner updates the release and the reviewer re-evaluates +6. Once **all** required review roles have approved, the release is fully approved + +### For access requests + +1. A user submits an access request for a model +2. Bailo creates a review task for the appropriate review role(s) +3. The reviewer examines the request details and either approves or requests changes +4. Once approved, the requesting user gains the permissions to download files and pull images + +### For model card lifecycle reviews + +1. A model owner sets a review due date in the model card **Details** section on the model's **Overview** tab +2. Bailo creates a lifecycle review for the model card +3. The model owner is notified nearer the review date +4. The model owner checks that the model card is still accurate, complete, and up to date +5. If information is missing or outdated, the model owner updates the model card +6. The model owner approves the lifecycle review and sets a new review due date + +## Checking your reviews + +You can see all reviews assigned to you from the Reviews page in the navigation bar: + +1. Click **Review** in the navigation bar +2. The **Your open reviews** tab shows reviews waiting for your action, with a count badge +3. The **Your archived reviews** tab shows completed reviews + +Lifecycle reviews can also be opened from the model's **Overview** tab by using the **Review** button in the model card +**Details** panel. + +## Responding to a review + +Reviewers examine the submission, add comments, and choose to approve or request changes: + +1. Open the review from the Reviews page or from the model's release, access request, or model card page +2. Examine the submission - for releases, review the model card and attached artefacts; for lifecycle reviews, check + that the model card remains accurate and up to date +3. Add a **comment** explaining your assessment +4. Choose your decision: + - **Approve** - you are satisfied with the submission + - **Request changes** - something needs to be addressed before you can approve +5. Submit your response + +You can also react to other reviewers' comments to indicate agreement or add further context. + +## Review outcomes + +A review can result in one of three outcomes. + +- **Approved** - All required review roles have approved. The release access request, or model card lifecycle review is + finalised. +- **Changes requested** - One or more reviewers need something addressed. The submitter should update and resubmit +- **Pending** - The review is still in progress. Not all required roles have responded yet + +## Tips for model owners + +These tips help model owners prepare for smoother reviews. + +- **Assign reviewers early** - set up review role assignments in your model's Settings tab before creating your first + release +- **Provide thorough model cards** - the more detail you provide, the faster reviews will go +- **Monitor your reviews** - check the Reviews page regularly or watch for email notifications +- **Set lifecycle review dates** - use model card lifecycle reviews to periodically confirm that model documentation + remains accurate + +## Related pages + +- [Reviewing a Release](/docs/users/reviews/reviewing/releases) - Step-by-step guide for reviewers +- [Reviewing an Access Request](/docs/users/reviews/reviewing/access) - How to review access requests +- [Reviewing a Model Card Lifecycle Review](/docs/users/reviews/reviewing/model-lifecycle) - How to complete lifecycle + reviews +- [Review Outcomes](/docs/users/reviews/review-outcomes) - What happens after reviews are approved or changed +- [Roles & Permissions](/docs/reference/roles-and-permissions) - Understanding review roles and system roles +- [Managing Review Roles](/docs/administration/review-roles/managing-review-roles) - Creating and configuring review + roles (administrators) + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/untrusted-models/untrusted-models.mdx b/frontend/pages/docs/users/untrusted-models/untrusted-models.mdx new file mode 100644 index 0000000000..c18f078d79 --- /dev/null +++ b/frontend/pages/docs/users/untrusted-models/untrusted-models.mdx @@ -0,0 +1,60 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Untrusted Models + +Untrusted models are a special entry type for models that require additional scrutiny before being fully trusted within +your organisation. This feature is optional and must be enabled by your administrator. + +> **Common questions this page answers:** +> +> - What is an untrusted model in Bailo? +> - How do untrusted models differ from regular models? +> - How do I create an untrusted model? + +## What are untrusted models? + +An untrusted model is a model that has not yet been verified or validated to the same standard as a regular model. Your +organisation may use this classification for: + +- Models from external or unknown sources +- Models that have not yet completed a full review cycle + +The specific meaning of "untrusted" depends on your organisation's policies. Your administrator configures a description +that explains what untrusted models mean in your context. + +## Creating an untrusted model + +Create an untrusted model from the model creation page if the feature is enabled by your administrator: + +1. Click the **+ Create** button in the navigation bar +2. Under the **Models** section, select **Untrusted Model** +3. Fill in the required fields: + - **Name** + - **Description** +4. Click **Save** + +The creation process is the same as for a regular model, but the entry is classified as untrusted. + +## How untrusted models differ + +Untrusted models appear in the Marketplace alongside regular models but are clearly marked as untrusted. Key differences +include: + +- Untrusted Models are always Private +- Access requests are not available for untrusted models +- The container (Docker) registry is disabled for untrusted models +- Users should exercise additional caution when using artefacts from untrusted models +- Your organisation may have additional policies governing the use of untrusted models + +## Availability + +This feature is controlled by your administrator. If you don't see the **Untrusted Model** option when creating a new +entry, it means the feature is not enabled in your Bailo instance. + +## Related pages + +- [Creating a Model](/docs/users/models/creating-a-model) - Standard model creation +- [Core Concepts](/docs/getting-started/core-concepts) - Overview of all entry types +- [Glossary](/docs/reference/glossary) - Definition of untrusted models + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/using-a-model/browsing-the-marketplace.mdx b/frontend/pages/docs/users/using-a-model/browsing-the-marketplace.mdx new file mode 100644 index 0000000000..f7049f55d4 --- /dev/null +++ b/frontend/pages/docs/users/using-a-model/browsing-the-marketplace.mdx @@ -0,0 +1,85 @@ +import DocsWrapper from 'src/docs/DocsWrapper' + +# Browsing the Marketplace + +The Marketplace is the main page you see when you log into Bailo. It lists all public models and data cards, providing +search and filtering tools to help you find what you need. + +> **Common questions this page answers:** +> +> - How do I find a model in Bailo? +> - How do I filter models in the Marketplace? +> - Why can't I see a model? +> - Can I share a filtered view of the Marketplace? + +## Searching + +Use the search bar at the top left of the Marketplace to find entries by name or description. You need to enter at least +3 characters for search results to appear. + +By default, the search scans the entire document, including the model card data, to find exact matches to your query. To +search only by title, toggle from **Full Text** to **Name** on the right side of the search bar. This performs a partial +match search on titles only. + +Search results are limited to the currently selected tab (**Models** or **Data Cards**). + +## Filtering + +The Marketplace provides several filter categories to narrow down results. + +### By entry type + +Use the tabs to switch between: + +- **Models** - Machine learning models +- **Data Cards** - Dataset documentation + +### By organisation + +Organisations are displayed below the search bar. Filter models by the organisation they belong to. + +### By state + +Filter models by their development state (e.g. in development, ready for deployment). + +### By external sources (Federation) + +If your Bailo instance is connected to other Bailo instances or external model repositories through federation, you can +filter to see models from specific external sources. External sources only appear if federation is enabled by your +administrator. + +### By tags + +Click a tag to filter the list to models that have been assigned that tag. You can select multiple tags to narrow your +results further. + +### By mirrored models + +Filter the list to show only mirrored models or data cards. Mirrored entries are replicas from external sources and are +only available if mirroring has been enabled by your administrator. + +### By roles + +Filter models based on your assigned role, showing only those where you hold a specific access role. + +## Visibility + +Model visibility determines whether a model appears in the Marketplace. + +- **Public** models are visible to all users in the Marketplace +- **Private** models do not appear in the Marketplace and are only visible to users who have been added as collaborators + +Even for public models, you typically need an approved access request to download files or pull container images. + +## Sharing filtered views + +Filter selections are stored in the page URL as query parameters. You can copy and share the URL to give someone the +same filtered view of the Marketplace. **Role-based filters are not included.** + +## Related pages + +- [Requesting Access](/docs/users/using-a-model/requesting-access) - How to request access to a model +- [Creating a Model](/docs/users/models/creating-a-model) - Adding your own model +- [Core Concepts](/docs/getting-started/core-concepts) - Understanding entry types and visibility + +export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/using-a-model/downloading-files.mdx b/frontend/pages/docs/users/using-a-model/downloading-files.mdx index 544ae9a8ae..1277c4d579 100644 --- a/frontend/pages/docs/users/using-a-model/downloading-files.mdx +++ b/frontend/pages/docs/users/using-a-model/downloading-files.mdx @@ -8,30 +8,43 @@ import downloadfile3 from 'public/docs/bailo_file_download_release_page.png' # Downloading Files -Files can be downloaded in three ways. The final two are exclusively for files with associated releases. +> **Common questions this page answers:** +> +> - How do I download files from a model in Bailo? +> - Where can I find files to download? +> - What are the different ways to download files? -## From the File Management tab +Files can be downloaded from three locations in Bailo. The File Management tab provides access to all files, while the +Releases tab and individual release pages show only files attached to releases. -Navigate to the **File Management** tab and click the file name to download it. +## From the file management tab + +Download files directly from the **File Management** tab by clicking the file name. clickable file in file management tab -## From the Releases tab +The screenshot above shows the File Management tab with clickable file names for direct download. + +## From the releases tab -Open the **Releases** tab and click the download link next to the relevant file. +Download release-associated files from the **Releases** tab using the download link next to each file. view of releases with the clickable file download link +The screenshot above shows the Releases tab with download links next to each file in each release. + ## From a specific release page -Open an individual release and use the file download link provided. +Open an individual release page to access file download links for that release. view of specific release with the clickable file download link +The screenshot above shows a specific release page with a file download link. + export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/using-a-model/personal-access-tokens.mdx b/frontend/pages/docs/users/using-a-model/personal-access-tokens.mdx deleted file mode 100644 index 34108d2b88..0000000000 --- a/frontend/pages/docs/users/using-a-model/personal-access-tokens.mdx +++ /dev/null @@ -1,29 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' -import Image from 'next/legacy/image' -import Box from '@mui/material/Box' - -import personalAccessTokens from 'public/docs/bailo_personal_access_tokens.png' -import createPersonalAccessToken from 'public/docs/bailo_create_personal_access_token.png' - -# Personal Access Tokens - -## What are Personal Access Tokens? - -Personal Access Tokens are used to authenticate users when interacting with the registry. When creating a token you will -be asked to specify which models the token can be used with and which actions the token is able to authenticate. - -## Creating a Personal Access Token - -Open the **Personal Access Tokens** page and click **Add Token**. - - - Authentication tab listing personal access tokens and add token button - - -Complete the form and click **Generate Token**. - - - generate personal token form - - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/users/using-a-model/requesting-access.mdx b/frontend/pages/docs/users/using-a-model/requesting-access.mdx index aaa891c67e..0fa7e1f5b4 100644 --- a/frontend/pages/docs/users/using-a-model/requesting-access.mdx +++ b/frontend/pages/docs/users/using-a-model/requesting-access.mdx @@ -8,31 +8,45 @@ import createAccessRequest from 'public/docs/bailo_create_access_request.png' # Requesting Access to a Model -## What is an Access Request? +> **Common questions this page answers:** +> +> - How do I request access to a model? +> - What is an access request? +> - What happens after I submit an access request? + +## What is an access request? An access request is required before you can use a model. This allows model owners to authorise usage and maintain an audit trail of who has access and for what purpose. ## Creating an access request -Navigate to the model overview page and click **Request Access**. +Submit an access request from the model overview page by selecting a schema and filling in the required form. + +1. Navigate to the model overview page and click **Request Access**. model view with request access button -Access requests require a schema, similar to model creation. Select the appropriate schema when prompted. +The screenshot above shows a model overview page with a 'Request Access' button. + +2. Access requests require a schema, similar to model creation. Select the appropriate schema when prompted. access request schema selection -Complete the form and submit the request. +The screenshot above shows the access request schema selection dialog. + +3. Complete the form and submit the request. access request form +The screenshot above shows the access request form with fields to fill in. + ## Review process Once this request has been approved by the necessary role(s), you will be able to download Release artefacts and pull diff --git a/frontend/pages/docs/users/using-a-model/using-a-pushed-docker-image.mdx b/frontend/pages/docs/users/using-a-model/using-a-pushed-docker-image.mdx index 176698c6e3..459d03ea26 100644 --- a/frontend/pages/docs/users/using-a-model/using-a-pushed-docker-image.mdx +++ b/frontend/pages/docs/users/using-a-model/using-a-pushed-docker-image.mdx @@ -1,5 +1,6 @@ import DocsWrapper from 'src/docs/DocsWrapper' import Image from 'next/legacy/image' +import Box from '@mui/material/Box' import pushingAnImage from '../../../../public/docs/Pushing-an-image.png' import registryTab from '../../../../public/docs/Registry-tab.png' @@ -8,12 +9,22 @@ import ModelReleaseImageList from '../../../../public/docs/Model-release-images. # Model Images +> **Common questions this page answers:** +> +> - How do I push a container (Docker) image to Bailo? +> - How do I pull a container (Docker) image from Bailo? +> - How do I add a container image to a release? + ## Pushing a Docker image -Docker images can be pushed to the Bailo registry and associated with a specific model. These images can later be used -in model releases. +Container (Docker) images can be pushed to the Bailo registry and associated with a specific model. These images can +later be used in model releases. + + + model registry tab with push image button circled + -model registry tab with push image button circled +The screenshot above shows the model Registry tab with a push image button. The process is: @@ -28,13 +39,21 @@ The process is: After pushing an image, it becomes selectable when drafting a release. Choose the image from the dropdown in the release creation form. -image dropdown displaying model-assigned images for use in a release + + image dropdown displaying model-assigned images for use in a release + + +The screenshot above shows the image selection dropdown in the release creation form. ## Accessing an image Images attached to a release are listed in the **Releases** tab. -release in releases tab with attached docker images + + release in releases tab with attached docker images + + +The screenshot above shows a release in the Releases tab with attached container images listed. You can pull an image using: diff --git a/frontend/pages/docs/users/using-scanners/file-scanning.mdx b/frontend/pages/docs/users/using-scanners/file-scanning.mdx index 99e7a5269e..c055ecd46f 100644 --- a/frontend/pages/docs/users/using-scanners/file-scanning.mdx +++ b/frontend/pages/docs/users/using-scanners/file-scanning.mdx @@ -7,7 +7,13 @@ import rescanView from 'public/docs/bailo_file_rescan.png' # Scanning Files -## Using Scanners +> **Common questions this page answers:** +> +> - How are files scanned in Bailo? +> - What scanners are used for file scanning? +> - How do I rescan a file? + +## Using scanners When uploading files to Bailo, your files will be automatically scanned (assuming scanners are configured and initialised). @@ -18,19 +24,22 @@ By default, Bailo is configured to use the following scanners: [here](https://www.clamav.net/). - ModelScan is a tool used to prevent model serialisation attacks, across a variety of formats. You can read more about it [here](https://github.com/protectai/modelscan/tree/main). ModelScan is packaged in - [Bailo Artefact Scan](/docs/administration/microservices/artefact-scanners) + [Bailo ArtefactScan](/docs/administration/microservices/artefact-scanners) For scan results: - Go to your model and click on File Management Tab - Click the Scan chip, towards the right of your file -- View scan results. Each scanning tool will have its results listed separately. +- View scan results. Each scanning tool will have its results listed separately view file scanning chip -## Rescan File +The screenshot above shows a file in the File Management tab with a scan status chip that can be clicked to view scan +results. + +## Rescan file To rescan a file after it has been uploaded, click the 3 dots to the right of the scan results chip and select the Rescan option. @@ -39,7 +48,9 @@ Rescan option. popup confirming file scan results - +The screenshot above shows the file actions menu with a Rescan option for re-running security scans. + + There is a cool-down period for rerunning a scan that is configured by your administrator diff --git a/frontend/pages/docs/users/using-scanners/image-scanning.mdx b/frontend/pages/docs/users/using-scanners/image-scanning.mdx index 2de3a0e047..ed25a0dbb7 100644 --- a/frontend/pages/docs/users/using-scanners/image-scanning.mdx +++ b/frontend/pages/docs/users/using-scanners/image-scanning.mdx @@ -10,13 +10,21 @@ import multiPlatView from 'public/docs/bailo_image_multiplat_view.png' # Scanning Images -## Using Scanner +> **Common questions this page answers:** +> +> - How are container images scanned in Bailo? +> - How do I view vulnerability details for a container image? +> - What is a CVE? +> - How do I rescan an image? -When pushing images up to the Bailo container registry, your images are automatically scanned. By default, Bailo is -configured to use Trivy. Trivy is packaged in -[Bailo Artefact Scan](/docs/administration/microservices/artefact-scanners) Trivy is a static analysis tool focused on -the security of containers, and informing users about potential issues. You can read more about the principles and scope -of Trivy [here](https://trivy.dev/docs/latest/community/principles/). +## Using scanner + +Container images pushed to the Bailo registry are automatically scanned using Trivy, a vulnerability scanner that +detects known security issues called CVEs (Common Vulnerabilities and Exposures). + +Trivy is a static analysis tool focused on the security of containers, and informing users about potential issues, +packaged in [Bailo ArtefactScan](/docs/administration/microservices/artefact-scanners). You can read more about the +principles and scope of Trivy [here](https://trivy.dev/docs/latest/community/principles/). For scan results: @@ -28,7 +36,10 @@ For scan results: view image scanning chip -## Detailed View +The screenshot above shows the scan status chip on a container image in the Registry tab, with vulnerability counts +grouped by severity. + +## Detailed view To view more detail, click the 'See detailed results' button. This will take you to the vulnerability report page, which includes the following information: @@ -36,7 +47,7 @@ includes the following information: - Compressed image size - Vulnerabilities - Scanner used -- Detailed vulnerability report, containing CVE information. +- Detailed vulnerability report, containing CVE information You can filter these results by Severity, using the tags at the top of the page. @@ -44,24 +55,36 @@ You can filter these results by Severity, using the tags at the top of the page. view image scan results -## Rescan Image +The screenshot above shows the detailed vulnerability report page with CVE information and severity filters. + +## Rescan image -If you want to rescan, click the 3 dots to the right of the scan results chip and select the Rescan option. +Trigger a rescan of a container image from the scan results menu. + +1. Click the 3 dots to the right of the scan results chip +2. Click **Rerun image scan** view rescan button -## Multiplatform Images +The screenshot above shows the image actions menu with a Rescan option. + +## Multiplatform images -A multiplatform image will appear with a purple tag next to the name +Bailo supports scanning multiplatform images, which target different architectures. + + + A multiplatform image will appear with a purple tag next to the name + Bailo also allows you to scan multiplatform images, which are images that point to a different set of layers based on -your host architecture. You can also click the 'See detailed results' button, and view the platform-specific, detailed -results page. +your host architecture. Click the **See detailed results** button to view the platform-specific, detailed results page. view multiplatform image summary +The screenshot above shows a multiplatform image summary with platform-specific vulnerability breakdowns. + export default ({ children }) => {children} diff --git a/frontend/pages/docs/v1/administration/getting-started/authentication.mdx b/frontend/pages/docs/v1/administration/getting-started/authentication.mdx deleted file mode 100644 index b7f161dc9b..0000000000 --- a/frontend/pages/docs/v1/administration/getting-started/authentication.mdx +++ /dev/null @@ -1,100 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Authentication - -Bailo is intended to be run in multiple different environments, each of which will have their own standard -authentication and authorisation methods. To enable this, Bailo exposes a class of methods that can be overridden to -integrate with various services. - -By default, we expect the Bailo instance to be run behind a reverse proxy, which sets the following headers to identify -users and their roles: - -- `x-userid` is a unique identifier for a user. -- `x-email` is an optional email address (to send notifications). -- `x-user` is a JSON encoded object containing any other user data. - -Within Nginx, these can be set via the `proxy_set_header` declaration: - -```bash -proxy_set_header X-UserId "user"; -proxy_set_header X-Email "user@example.com"; -proxy_set_header X-User '{"some":"data"}'; -``` - -To support other authentication mechanisms, an administrator can override the `getUserFromReq` method to get this -information from any other method. To do this edit `backend/src/connectors/Authorisation.ts` to override the required -methods from `backend/src/utils/AuthorisationBase.ts`. E.g. - -```javascript -// backend/src/connectors/Authorisation.ts -import AuthorisationBase from '../utils/AuthorisationBase.js' -import { Request } from 'express' - -class Authorisation extends AuthorisationBase { - async getUserFromReq(_req: Request) { - const username = req.get('x-username') - - return { - userId: username, - email: await getUserEmail(username), - data: await getUserData(username), - } - } -} - -export default Authorisation -``` - -In the above example, instead of getting this data from headers passed into the application, we now get the username -from headers and get the users email and data from another method. - -This is called on each request and not cached, so you may want to implement this yourself if you are using expensive -cryptographic functions or making external network requests. - -# Authorisation - -By default Bailo protects models from deployment by requiring an approval by the model owner. Also, only the model owner -is capable of editting a model that has been uploaded. All users can view all models uploaded to the system. - -This is unlikely to be suitable for large organisations, who may wish to integrate their own internal authorisation -system into Bailo. An administrator can integrate with their internal authorisation system by overriding one of the -`canUserSeeX` functions, visible in `backend/src/utils/AuthorisationBase.ts`. Each functino takes in a user object, as -well as the object to view. The function is then expected to return either `true` or `false` depending on if the user -can view that object. - -This authorisation affects both users directly viewing a model, as well as the models visible in the marketplace. - -An example authorisation implemention: - -```javascript -// backend/src/connectors/Authorisation.ts -import AuthorisationBase from '../utils/AuthorisationBase.js' -import { UserInterface } from '../models/User.js' -import { VersionDoc } from '../models/Version.js' - -class Authorisation extends AuthorisationBase { - async getUserFromReq(_req: Request) { - // we now include roles in the user object - return { - userId: 'user', - data: { roles: ['alpha', 'beta', 'gamma'] }, - } - } - - async canUserSeeVersion(user: UserInterface, model: VersionDoc) { - // we can access the roles from the user document - const userRoles = user.data.roles - // our schema has a 'role' field we need users to have to see it - const versionRole = version.metadata.authorisation.role - - // check if the user has the required role - const canAccess = userRoles.includes(versionRole)) - - return canAccess - } -} - -export default Authorisation -``` - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/v1/administration/getting-started/building-the-bailo-image.mdx b/frontend/pages/docs/v1/administration/getting-started/building-the-bailo-image.mdx deleted file mode 100644 index 45cb0f1660..0000000000 --- a/frontend/pages/docs/v1/administration/getting-started/building-the-bailo-image.mdx +++ /dev/null @@ -1,21 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Building the Bailo Image - -Bailo does not provide pre-built images. Building one yourself is easy! Two images should be produced, one for the -frontend and one for the backend. - -```bash -# Bailo uses a monorepo, so images should be built from the root directory -$ docker build --build-context python=./lib/python -t "bailo-backend" -f ./backend/Dockerfile . -$ docker build -t "bailo-frontend" -f ./frontend/Dockerfile . -$ docker build -t "bailo-artefactscan" -f ./lib/artefactscan_api/Dockerfile . -``` - -Ensure you do not use the development stages in Dockerfiles (used by building with `--target dev`) as they expect the -application to be mounted into them at a later point in time (to enable hot code reloading). - -`docker` can be swapped out for any other OCI compliant container build platform. We routinely test with both `podman` -and `OpenShift Builder`. This image should be pushed to a registry that is accessible from your deployment target. - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/v1/administration/getting-started/configuration/app-configuration.mdx b/frontend/pages/docs/v1/administration/getting-started/configuration/app-configuration.mdx deleted file mode 100644 index 4cc1b7398d..0000000000 --- a/frontend/pages/docs/v1/administration/getting-started/configuration/app-configuration.mdx +++ /dev/null @@ -1,48 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# App Configuration - -## Configuration Files - -Bailo uses [`node-config`](https://github.com/node-config/node-config#readme) for application configuration. -`node-config` organizes hierarchical configurations, allowing the user to set a series of overides over a base -configuration. - -The default set of configuration can be found in `config/default.js`, with overrides for other environments included in -the same folder. For production environments, configuration should be placed in either `production.cjs` or `local.js` to -override the default configuration. - -The full order of configuration inheritance can be found -[on the node-config wiki](https://github.com/node-config/node-config/wiki/Configuration-Files). - -Given the following set of files: - -```javascript -// default.js -{ a: 5, b: "ten", c: { d: 20, e: 25 } } - -// production.cjs -{ b: 10, c: { d: "d" } } - -// local.js -{ c: { d: 999 } } -``` - -The result is: - -```javascript -{ - a: 5, // no overrides - b: 10, // overridden in production.cjs - c: { - d: 999, // overriden in production.cjs, then again in local.js - e: 25 // not overwritten, objects are merged together. - } -} -``` - -When deploying using `helm`, configuration is primarily handled through `values.yaml` controlling -`helm/bailo/templates/bailo/bailo.configmap.yaml`. Configuration is loaded when the application starts. The application -must be restarted when configuration changes. - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/v1/administration/getting-started/configuration/making-a-schema.mdx b/frontend/pages/docs/v1/administration/getting-started/configuration/making-a-schema.mdx deleted file mode 100644 index 4fdd708d31..0000000000 --- a/frontend/pages/docs/v1/administration/getting-started/configuration/making-a-schema.mdx +++ /dev/null @@ -1,222 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Creating and Uploading schemas - -Bailo makes use of the [RJSF](https://react-jsonschema-form.readthedocs.io/en/latest/) (react-jsonschema-form) library -in order to dynamically create its upload & deployment forms. These forms can be constructed of various existing -widgets, or even use custom widgets defined in Bailo. - -## Example / minimal schemas - -Bailo includes both an example schema for uploads and for deployments. You can find them in -`backend/src/scripts/example_schemas`. It is important to note that these schemas contain all of mandatory fields that -the application needs in order to run. These fields are hard-coded in various places in the source code. - -## Create a schema - -### Simple questions - -A typical object inside the schema will look like this: - -```json -"firstName": { - "type": "string", - "title": "First name", - "default": "Chuck", - "description": "This is for your first name" -} -``` - -The type property can be altered depending on what value you want the question to return. The title property will be -displayed as the question itself, and the description will appear under the question to include any additional -information that might help the user. Please note that if you do not include a title property, the library will use the -name of the object (in this case firstName) instead. - -### Dates - -Dates can be added like this: - -```json -"date": { - "type": "string", - "title": "Date of birth", - "format": "date" -} -``` - -### Sections - -When using Bailo you will see that the upload form is made up of a number of pages. Each page is a separate object -inside the first "properties" property of the schema. For example: - -```json -"properties": { - "firstPage": { - "type": "object", - "title": "Page 1", - "properties": { - ... - } - }, - "secondPage": { - "type": "object", - "title": "Page 2", - "properties": { - ... - } - } -} -``` - -Creating sub-sections works identically to how we defined our pages above. If we take the first page of our example -above, and then amend it so it looks like this: - -```json -"properties": { - "firstPage": { - "type": "object", - "title": "Page 1", - "properties": { - "sectionOne": { - "type": "object", - "title": "Section 1" - "properties": { - "questionOne": { - "type": "number", - "title": "Enter a number", - } - } - }, - "sectionTwo": { - "type": "object", - "title": "Section 2" - "properties": { - "questionTwo": { - "type": "string", - "title": "Enter a string", - } - } - } - }, - ... -} -``` - -#### Hiding sections from the model card - -Sections will display on the model card page by default, but in the case whereby a section needs to be hidden from the -model card, then add the following property: - -```json -"myPage": { - "title": "Page 1", - "type": "object", - "properties": { - ... - }, - "displayModelCard": false, -} -``` - -Note: This will only hide it from the model page, the data will still persist in the version metadata. - -### Dependencies - -A dependency is useful when the answer to one question might warrant some additional information; for example, a boolean -question might not need a detailed response if the answer is 'no', but if the user answer is 'yes' you might want to ask -them for some additional details. - -To avoid validation errors, Bailo schema dependencies need to be handled slightly differently to the ones provided in -the RJSF documentation. Here is an example of how we handle dependencies for Bailo schemas: - -```json -"myPage": { - "title": "Page 1", - "type": "object", - "properties": { - "questionOne": { - "type": "boolean", - "title": "True or false?", - }, - "questionTwo": { - "title": "More information please!", - "type": "string", - } - }, - "dependencies": { - "questionOne": { - "oneOf": [ - { - "properties": { - "questionOne": { - "enum": [false], - "readOnly": false - }, - "questionTwo": { - "readOnly": true - } - }, - "required": ["questionOne"], - "additionalProperties": false - }, - { - "properties": { - "questionOne": { - "enum": [true], - "readOnly": false - }, - "questionTwo": { - "readOnly": false - } - }, - "required": ["questionOne", "questionTwo"], - "additionalProperties": false - } - ] - } - }, - "required": ["questionOne"] -} -``` - -Inside `dependencies` we have defined a case of only having one of two different situations. Firstly, if `questionOne` -is set to `false`, then `questionTwo` will be set to read only, whereas if we set `questionOne` to `true` then -`questionTwo` will not be set as read only. This method might seem slightly more convoluted than the documentation -online, but it allows for answers to be changed without making the schema invalid. The current design of RJSF's -dependencies works as so: - -- User selects `true` for `questionOne` -- User enters data into `questionTwo` -- User changes their mind and selects `false` for `questionOne` -- User data for `questionTwo` remains in the form data, despite not being visible on the form itself - -This behaviour is so that any user data is retained in case the user wishes to select `true` for `questionOne` again. -From a user perspective this nice as it means there is less chance of data being lost when changing answers, but it does -have a knock-on affect whereby in certain situations it will make the schema fail validation. Our approach is set the -additional questions to read only when they are not in use. The downside to this approach is that we potentially include -unwanted additional data, but this can be removed by the user if this happens. - -## Uploading a schema - -There are two ways of uploading a schema to Bailo. The first is by manually adding it to the `schemas` collection in -Mongo directly, or, by using the addUploadSchemas.ts / addDeploymentScript.ts scripts found within -`backend/src/scripts`. This process is not currently very graceful and requires some minor editing of the scripts -themselves. - -First, you will need to change the import at the top of the file so that it points to your new schema, and not the -minimal schema. Next, where we call `createSchema()` you will need to amend the supplied parameters of `name` and -`reference` as Mongo will not accept multiple schemas with the same name or reference. - -Once this script has been changed, run: - -```bash -npm run script -- addUploadSchema -``` - -Or for deployments: - -```bash -npm run script -- addDeploymentSchema -``` - -export default ({ children }) => {children} diff --git a/frontend/pages/docs/v1/administration/getting-started/helm-deployments.mdx b/frontend/pages/docs/v1/administration/getting-started/helm-deployments.mdx deleted file mode 100644 index 3e5a991a3c..0000000000 --- a/frontend/pages/docs/v1/administration/getting-started/helm-deployments.mdx +++ /dev/null @@ -1,63 +0,0 @@ -import DocsWrapper from 'src/docs/DocsWrapper' - -# Helm Deployments - -> At the moment `helm/bailo` is configured for OpenShift deployment, our default deployment target. Support for -> Kubernetes is on the way. We have heard that external organisations have had success already by disabling 'routes' and -> setting 'ingresses' instead. - -## Requirements - -- [HELM CLI](https://github.com/helm/helm/releases) -- [Kubectl](https://kubernetes.io/docs/tasks/tools/) - -### Optional - -OC is required for deployments to OpenShift - -- [OpenShift CLI](https://mirror.openshift.com/pub/openshift-v4/clients/oc/latest/linux/oc.tar.gz) - -## Deployment - -### Setup - -All commands assume they are run in the `helm/bailo` directory. The user should have already authenticated `kubectl` to -their cluster and changed to the correct context: - -1. `kubectl config set-context --current --namespace=bailo` - -To deploy to `OpenShift`, `oc` must also be configured correctly. - -### Configuration - -Deployment options can be overridden by including a `--values ` to a Helm command, or by -using `--set