IF architecture

[jmcook1186]

@kaito-kameda @YaSuenag

[YaSuenag]

CC: @nttDamien

[YaSuenag]

Summary

I'd like to discuss about implementing Web API for if-run.

Purpose

We want to use IF on Kubernetes, and to leverage it as a part of monitoring system.

IF is useful tool to calculate environmental values like energy consumption or carbon emission. We can use it for both realtime and historical data. It will make monitoring stack more powerful, and we can evaluate them easily with GSF standards like SCI.

Currently IF provides CLI tools only, so it is difficult to integrate to online system like a micro service architecture. We need to create "invoker" of IF CLI tools to handle stdin/stdout/stderr as a parent process.

Proposal

1. Implement Web API for if-run

Implement if-run as a web application.

if-run is a core component of IF. We can use it to calculate manifest, So the Web API endpoint accepts manifest YAML, and then returns the output to the client.

We can implement API endpoint with Express, and it proxies if-run module. It should be much better than spawning process due to performance reason.

Our API proposal is in below. You can see it on Swagger Editor if you copy and paste.

OpenAPI document

openapi: 3.0.3
info:
  title: Impact Framework Web API
  description: |-
    This API provides key feature of Impact Framework (IF) as a Web API.
  license:
    name: MIT license
    url: https://github.com/Green-Software-Foundation/if/blob/main/LICENSE
  version: 1.0.0
externalDocs:
  description: Find out more about Impact Framework
  url: https://if.greensoftware.foundation/
paths:
  /v1/run:
    post:
      summary: Execute `if-run` with given manifest.
      operationId: ifRun
      parameters:
        - in: query
          name: outputonly
          schema:
            type: boolean
          required: false
          description: "`true` if only `.tree.*.outputs` is needed."
      requestBody:
        description: IF Manifests
        content:
          application/vnd.if-manifest+yaml:
            example: |
              name: demo
              initialize:
                plugins:
                  double-a-value:
                    path: 'builtin'
                    method: Coefficient
                    config:
                      input-parameter: "cpu/utilization"
                      coefficient: 2
                      output-parameter: "cpu-utilization-doubled"
              tree:
                children:
                  child-0:
                    pipeline:
                      compute:
                        - double-a-value
                    inputs:
                      - timestamp: 2023-07-06T00:00
                        duration: 1
                        cpu/utilization: 20
                      - timestamp: 2023-07-06T00:01
                        duration: 1
                        cpu/utilization: 80
        required: true
      responses:
        '200':
          description: Successful operation.
          content:
            application/vnd.if-manifest+yaml:
              example: |
                name: demo
                initialize:
                  plugins:
                    double-a-value:
                      path: builtin
                      method: Coefficient
                      config:
                        input-parameter: cpu/utilization
                        coefficient: 2
                        output-parameter: cpu-utilization-doubled
                execution:
                  command: >-
                    /usr/bin/node-22 /home/yasuenag/work/if/node_modules/.bin/if-run --manifest
                    demo.yaml
                  environment:
                    if-version: 1.0.1
                    os: linux
                    os-version: 6.12.11-200.fc41.x86_64
                    node-version: 22.11.0
                    date-time: 2025-03-19T04:14:17.563Z (UTC)
                    dependencies:
                      - '@grnsft/[email protected]'
                  status: success
                tree:
                  children:
                    child-0:
                      pipeline:
                        compute:
                          - double-a-value
                      inputs:
                        - timestamp: 2023-07-06T00:00
                          duration: 1
                          cpu/utilization: 20
                        - timestamp: 2023-07-06T00:01
                          duration: 1
                          cpu/utilization: 80
                      outputs:
                        - timestamp: 2023-07-06T00:00
                          duration: 1
                          cpu/utilization: 20
                          cpu-utilization-doubled: 40
                        - timestamp: 2023-07-06T00:01
                          duration: 1
                          cpu/utilization: 80
                          cpu-utilization-doubled: 160
            application/vnd.if-manifest+json:
              example: |
                {
                  "name": "demo",
                  "initialize": {
                    "plugins": {
                      "double-a-value": {
                        "path": "builtin",
                        "method": "Coefficient",
                        "config": {
                          "input-parameter": "cpu/utilization",
                          "coefficient": 2,
                          "output-parameter": "cpu-utilization-doubled"
                        }
                      }
                    }
                  },
                  "execution": {
                    "command": "/usr/bin/node-22 /home/yasuenag/work/if/node_modules/.bin/if-run --manifest demo.yaml",
                    "environment": {
                      "if-version": "1.0.1",
                      "os": "linux",
                      "os-version": "6.12.11-200.fc41.x86_64",
                      "node-version": "22.11.0",
                      "date-time": "2025-03-19T04:14:17.563Z (UTC)",
                      "dependencies": [
                        "@grnsft/[email protected]"
                      ]
                    },
                    "status": "success"
                  },
                  "tree": {
                    "children": {
                      "child-0": {
                        "pipeline": {
                          "compute": [
                            "double-a-value"
                          ]
                        },
                        "inputs": [
                          {
                            "timestamp": "2023-07-06T00:00",
                            "duration": 1,
                            "cpu/utilization": 20
                          },
                          {
                            "timestamp": "2023-07-06T00:01",
                            "duration": 1,
                            "cpu/utilization": 80
                          }
                        ],
                        "outputs": [
                          {
                            "timestamp": "2023-07-06T00:00",
                            "duration": 1,
                            "cpu/utilization": 20,
                            "cpu-utilization-doubled": 40
                          },
                          {
                            "timestamp": "2023-07-06T00:01",
                            "duration": 1,
                            "cpu/utilization": 80,
                            "cpu-utilization-doubled": 160
                          }
                        ]
                      }
                    }
                  }
                }

2. Containerize

Containerization gives benefits that the user can use IF Web API with single command like docker run, and it would be portable. So we propose to create Dockerfile for Web API.

Kubernetes has been leveraged in modern system. We can delegate container lifecycle to Kubernetes if we deploy the app on it. Thus we would also like to propose to create Helm chart for Web API container.

3. Publish official images

It is better to publish "official" container image and Helm chart from GSF.

In Carbon Aware SDK, official both Web API container image and Helm chart has been provided from GitHub Container Registry ( ghcr.io ). We can use that registry for IF.

Both container image and Helm chart for Carbon Aware SDK would be generated / published by GHA workflow. IF already has GHA workflow for new release. We can integrate new workflow for container image / Helm chart to it.