Skip to content

Files

Latest commit

 

History

History

compliance-monitor

README.md

SCS compliance monitor

A service with a REST-like interface, backed by Postgresql, to manage data about compliance of subjects with SCS certificate requirements.

Setup

The first two sections have been adapted from https://github.com/SovereignCloudStack/standards/blob/main/Tests/README.md.

Python dependencies

This program uses the Python library psycopg2. As a prerequisite for its build and installation, the following command is needed (on distributions based on Debian):

sudo apt install build-essential libpq-dev python3-dev

Please ensure you have installed the dependencies from requirements.txt. We recommended using Python >= 3.10 and to install the requirements into a virtualenv as follows:

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

Updating Python dependencies

We manage Python dependencies in two files: requirements.in and requirements.txt. The former one is meant to be edited by humans, whereas the latter one is generated by pip-compile and contains an exact listing of all dependencies, including transitive ones.

pip-compile can be installed via pip install pip-tools. It needs to be run in two cases:

  1. You modified requirements.in (e.g., added a new dependency): run pip-compile requirements.in.
  2. You want to bump the pinned dependencies: run pip-compile --upgrade requirements.in.

Note: The Python version used for running pip-compile should be consistent. The currently used version is documented in the header of the requirements.txt.

Postgresql

You need running Postgresql. For instance, run it in a container like so:

docker run --network=host --rm -v $(pwd)/data:/var/lib/postgresql/data -it --name postgres -e POSTGRES_PASSWORD=mysecretpassword postgres

Monitor service

Run the service as follows:

SCM_DB_PASSWORD=mysecretpassword ./monitor.py --port 8080 --bootstrap bootstrap.yaml

(Alternatively, you may set SCM_DB_PASSWORD_FILE to point to a file containing the password.)

The service will automatically create or update the database schema as needed, and it will load any records from the given bootstrap file into the database; this file should at least contain credentials for one user, because otherwise you won't be able to post new data. See the dedicated section for details.

To use the service in production, it is strongly recommended to set up a reverse proxy with SSL.

Bootstrap file

This file will be read and the database updated accordingly when the service is started, as well as upon the corresponding signal.

accounts:
  - subject: admin  # usually the subject under test, but this one is a special account
    api_keys:  # needed for basic auth, this must be the hash as returned by `crypt.crypt`
      - "$2b$12$02j2DtlOXdT/MTVmWG60Yu.MNIJOGxRGKvE3DdA5DylCaHXxAY1Om"
    keys:  # needed to check signature of posted material
      - public_key: "..."
        public_key_type: "ssh-ed25519"
        public_key_name: "primary"
    roles:
      # anyone (even non-authenticated) can read public details for any subject
      # any account can read the non-public details of compliance results of their subject
      # any account can append results for their own subject
      # - append_any  # can append results for any subject
      - read_any  # can read non-public details of compliance results for any subject
      - admin  # can cause reload of the bootstrap file, among other things
      - approve  # can approve non-pass results
  - subject: scs2
    api_keys: []
    keys: []

Endpoints

POST /reports

Needs to be authenticated (via basic auth).

Needs to specify Content-Type, either application/x-signed-json or application/x-signed-yaml.

The actual report must be prefixed by an SSH signature, plus an ampersand character. The full body can be created and sent as follows:

ssh-keygen \
  -Y sign -f ~/.ssh/id_ed25519 -n report myreport.yaml
curl \
  --data-binary @myreport.yaml.sig --data-binary @myreport.yaml \
  -H "Content-Type: application/x-signed-yaml" -H "Authorization: Basic $BASICAUTH" \
  http://127.0.0.1:8080/reports

The tool curl will concatenate the contents of the two files with an ampersand in between.

GET /reports

Returns the most recent reports, by default restricted to the authenticated subject and limited to 10 items.

Needs to be authenticated (via basic auth).

Supports query parameters:

  • subject=SUBJECT: by default, results are restricted to the subject of the authenticated account; if the account has the role read_any, any subject may be specified, or it may be left blank to remove the restriction;
  • limit=N: return at most N items (default: 10);
  • skip=N: skip N items (useful for pagination).

GET /results

Returns the most recent results that are not expired or have been expired for at most 7 days.

Needs to be authenticated (via basic auth).

The return value is a list of objects like the following:

    {
        "reportuuid": "def374a9-56a9-492c-b113-330d491c58c7",
        "subject": "gxscs",
        "checked_at": "2024-03-16T14:13:53.857422",
        "scopeuuid": "50393e6f-2ae1-4c5c-a62c-3b75f2abef3f",
        "version": "v3",
        "check": "image-metadata-check",
        "result": 1,
        "approval": false
    }

Supports query parameters:

  • approved=APPROVED: return only results with approval status APPROVED (either 0 or 1); default: no such restriction is applied;
  • limit=N: return at most N items (default: 10);
  • skip=N: skip N items (useful for pagination).

POST /results

Sets approval state of given results.

Needs to be authenticated (via basic auth).

Needs to specify Content-Type as application/json.

The request body is a list of objects like the following:

    {
        "reportuuid": "def374a9-56a9-492c-b113-330d491c58c7",
        "scopeuuid": "50393e6f-2ae1-4c5c-a62c-3b75f2abef3f",
        "version": "v3",
        "check": "image-metadata-check",
        "approval": true
    }

The final field is the desired state; the other fields are used to determine the result in question (within one report, version and check uniquely determine a result; the scope is given here as well in case reports at some point contain multiple scopes).

GET /status/{subject}

Returns the current status of the subject. Use the Accept header to select desired content type:

  • text/html (default): a snippet of HTML suitable for the end user;
  • image/png: a PNG image of a badge;
  • application/json: a short summary in JSON format.

Query parameters:

  • scopeuuid (optional): restrict scope
  • version (optional): restrict version
  • privileged_view (optional 0 or 1, default 0): request privileged view (see below)

If the privileged view is requested, then this request needs to be authenticated (via basic auth), either for the same subject or for some account with role read_any. This view will immediately show any non-pass result, whereas otherwise, such a result needs to be verified manually.

GET /metrics/{subject}

A Prometheus exporter for the status of the subject.

Needs to be authenticated (via basic auth).

Supports content type text/plain; version=0.0.4; charset=utf-8 only.

GET /{view_type}/table[_full]

Returns the compliance table for all active subjects, where view_type can be one of the following:

  • markdown: return Markdown fragment (mimetype text/markdown)
  • fragment: return HTML fragment (mimetype text/html)
  • page: return full HTML page (mimetype text/html)

If table_full is used, then HTTP basic auth must be performed, and the table will show the privileged view (i.e., any FAIL will be reported regardless of manual approval).

GET /{view_type}/details[_full]/{subject}/{scopeuuid}

Returns compliance details for given subject and scope.

GET /{view_type}/scope/{scopeuuid}

Returns spec overview for the given scope.