Create Plugin Installer Container for Headlamp

[knrt10]

Background

Headlamp currently supports plugin installation through the headlamp-plugin CLI tool, which can install plugins to a specified directory. However, for in-cluster mode deployments, we need a more Kubernetes-native way to manage plugins that can:

1. Install plugins during container initialization
2. Support configuration management
3. Handle plugin updates and removals
4. Work seamlessly with Kubernetes deployments

Objective

Create a containerized version of the headlamp-plugin CLI tool that can be used as an init container in Kubernetes deployments. This container will:

• Accept a configuration file (via ConfigMap/Secret/Path) specifying which plugins to install
• Install the specified plugins into a shared volume before Headlamp starts
• Be automatically built and published as part of Headlamp's release process
• Provide a seamless way to manage plugins in Kubernetes environments

The container will be used in Headlamp's Helm chart as an init container, allowing operators to declaratively manage their plugins through Kubernetes-native configuration. This eliminates the need for manual plugin installation and ensures consistent plugin deployment across all Headlamp instances.

Implementation Details

1. Plugin Installer Container

1.1 Configuration Format

Create a YAML-based plugin configuration format:

plugins:
- name: my-plugin
source: https://example.com/plugin.tar.gz
version: 1.0.0
config:
  key1: value1
  key2: value2
- name: another-plugin
source: https://github.com/org/plugin/releases/download/v1.0.0/plugin.tar.gz
version: latest

2. CLI Extensions

2.1 New CLI Arguments

Extend headlamp-plugin to support:

headlamp-plugin install --config /path/to/config.yaml
headlamp-plugin install --source url --config-json '{"key": "value"}'

2.2 Configuration Management

• Add configuration parsing functionality
• Support multiple plugin installations
• Enable plugin-specific configuration
• Add validation for configurations

3. Kubernetes Integration

3.1 Init Container Setup

Update Headlamp's Helm chart to support plugin installation:

initContainers:
- name: plugin-installer
image: headlamp/plugin-installer:latest
volumeMounts:
  - name: plugins-dir
   mountPath: /plugins
  - name: plugin-config
   mountPath: /config
env:
  - name: PLUGIN_CONFIG_PATH
   value: /config/plugins.yaml
volumes:
- name: plugins-dir
emptyDir: {}
- name: plugin-config
configMap:
  name: headlamp-plugin-config

3.2 ConfigMap for Plugin Configuration

apiVersion: v1
kind: ConfigMap
metadata:
name: headlamp-plugin-config
data:
plugins.yaml: |
plugins:
  - name: my-plugin
   source: https://example.com/plugin.tar.gz
   version: 1.0.0

4. Implementation Steps

4.1 Extend Plugin Manager

• Add configuration file support to PluginManager class
• Implement batch installation functionality
• Add configuration validation
• Support plugin-specific settings

4.2 Create Docker Image

• Set up Docker build process
• Create installation scripts
• Add configuration parsing
• Include necessary dependencies

4.3 Update Helm Chart

• Add init container configuration
• Create ConfigMap templates
• Add volume management
• Update documentation

4.4 Add CI/CD Support

• Automate Docker image builds
• Add image versioning
• Set up container scanning
• Configure release automation

5. Plugin Installation Process

5.1 Container Startup

#!/bin/sh
# Read config file
CONFIG_FILE=${PLUGIN_CONFIG_PATH:-/config/plugins.yaml}

# Parse and install plugins
headlamp-plugin install --config $CONFIG_FILE

5.2 Volume Management

• Use emptyDir for single-node setups
• Support PersistentVolume for production
• Handle plugin updates
• Manage plugin configurations

Success Criteria

• Plugins can be installed via configuration files
• Init container successfully installs plugins during pod startup
• Plugin configurations are properly managed
• Updates and removals work as expected
• Process is documented and maintainable

Technical Requirements

• Kubernetes 1.16+
• Helm 3+
• Node.js 18+
• Access to plugin repositories
• Sufficient storage for plugins

Notes

• Consider security implications of plugin downloads
• Plan for plugin version compatibility
• Document upgrade procedures
• Consider backup and restore procedures
• Add monitoring for plugin installation status

cc @headlamp-k8s/maintainers

[adwait-godbole]

Hi @knrt10. Thanks for bringing up this project in LFX. Are there any pre-tests for the selection criteria of mentees?

[vickybesra]

Dear @knrt10 , I hope you’re doing well. I’m writing to express my keen interest in the containerized headlamp-plugin CLI project for LFX Term 1, as I’m excited by the challenge of creating a Kubernetes-native solution that automates plugin installation during container initialization while managing configurations and updates seamlessly through Helm and ConfigMaps; my background in Kubernetes, container orchestration, and CI/CD has prepared me to tackle this project, and I’m eager to contribute my skills in developing, testing, and documenting a robust solution that enhances in-cluster plugin management for Headlamp—please let me know if I may take on this opportunity under your guidance.

[knrt10]

Hello folks, happy to see you interested in the issue. Please feel free to apply on the LFX website. Thanks

[illume]

@adwait-godbole wrote:

Thanks for bringing up this project in LFX. Are there any pre-tests for the selection criteria of mentees?

Good question. The application is a cover letter and a resume. Applications for this project through https://mentorship.lfx.linuxfoundation.org/project/992fc67e-ff9e-41fd-8062-28ec8733903f

There's no pre-tests.

There's these two other Headlamp projects that are part of the mentorship program:

• Headlamp plugin for KEDA #2791
• RFC: Telemetry in Headlamp with OpenTelemetry #2799

[mgalesloot]

An alternative approach. Usable today already. Works great with GitOps and Renovate.

Note the ultimate solution should be based on OCI Artifacts as K8s Volumes. This is since v1.31 in alpha, so we have to wait for GA a bit. See https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-image-volume-source/.

• Container Image
  Plugins build a container image in their CICD. The container image will have the main.js and package.json and a minimal OS. Version the container image properly.
• InitContainer
  In the Headlamp installation, we can now make an initContainer for each plugin:

- command:
     - /bin/sh
     - -c
     - mkdir -p /build/plugins && cp -r /plugins/* /build/plugins/
  image: ghcr.io/kubebeam/kubescape-headlamp-plugin:latest ## you can specify a version such as 0.5.0-beta
  name: kubescape-plugin
  volumeMounts:
   - mountPath: /build/plugins
     name: headlamp-plugins

• emptyDir
  Put the plugin files on an emptydir. There is no point in making a PV for a few .js files. Also, a rolling update of the Headlamp pod would not like it when this PVC is ReadWriteOnce.

volumes:
  - name: headlamp-plugins
    emptyDir: {}

• Helm Chart
  In the headlamp helm chart we can add a list of plugins and let the helm template generate the initContainers.
  e.g.

plugins: 
  - name: kubescape-plugin 
     image: ghcr.io/kubebeam/kubescape-headlamp-plugin:latest
  - name: flux
     image: ghcr.io/headlamp/plugins/flux:v1.0.0

[Faakhir30]

An alternative approach. Usable today already. Works great with GitOps and Renovate.

Note the ultimate solution should be based on OCI Artifacts as K8s Volumes. This is since v1.31 in alpha, so we have to wait for GA a bit. See https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-image-volume-source/.

Indeed, this approach simplifies some things while following the OCI objects/artifacts format.

• Ig, we don't have to do plugin-cli containerization with this alternate approach.
• That would also make work of 4.4 Add CI/CD Support section of initial issue

However, here are some concerns:

• What headlamp plugins' primary distribution channel is used mostly? Isn't it the tarballs installations with artifactshub. docs?
  • So moving to this image-based solution for individual plugins would require people with custom plugins(I'm not aware if this is a thing or not) to add containerization to their plugins.
  • Also, do all existing plugins in headlamp-k8s/plugins provide build artifacts images distributions? I did see some of them but am unsure if all are provided.

Overall, I'm a +1 for this approach, even if the above-mentioned concerns are real. I'd prefer implementing both an OCI Artifacts approach parallel to the tarballs approach in that case :)

Some maintainers' comments would be great.