Skip to content

Commit cdda8f7

Browse files
axolotlitebrunnels
axolotlite
and
brunnels
authored
Parameterize a few values and add @brunnels helm chart workflow
The initial goal of this PR was to allow the user to set a few parameters such as storage, requests/limits and gpus for the pods running the application + wolf, then a few changes to the way wolf works inside kubernetes (thanks to ABeltramo) and finally adding brunnels helm chart for ease of deployment on kubernetes. The following is the unsquashed commit log: * [operator] add ephemeral, volumeConfig option in app crd for wolf-data * [operator] allow user to define container resource requests/limits in user crd to configure gpus, generic-devices, etc... per user. * [fix] add RTSPFakeIP to client, more configs and more manifests * [operator] allow user defined volume mounts for sidecars * [security] comment out SecurityContext * [README] update readme and prepare for PR * [README] add crd installation step * [operator] use VolumeClaimTemplate, cleanup unused structs & update app manifests * [hotfix] prevent xml marshalling of volumeClaimTemplate * change hardcoded WOLF_RENDER_NODE * change hardcoded WOLF_RENDER_NODE * rename chart to direwolf add helm-chart action don't build arm64 for now * fix helm workflow * fix helm workflow * rename chart to direwolf-operator add crd schemas * switch to wolf-ui branch * add session_test.go to make it easier to see what the how the deployment will render. * expose more wolf configs in the crds * [operator] allow user to define securityContext for sidecars * [ci] remove arm64 build * uncomment the nvidia variables * uncomment more nvidia variables * [operator] add hostIPC flag to sidecars * back to wolf moonlight fixes * starting to combine workflows * combined workflows * fix digest file * fix yq stage * fix yq stage * fix yq stage * fix yq stage * cleanup * rework chart version * rework chart version * rework chart version * rework chart version * fix chart version switch to wolf:stable * fix chart version * fix chart version * fix chart version * fix identifiers in chart * just keep overwriting snapshot version * [sessions] acquire ClientIP dynamically from moonlight-operator * add schema generator script to update-codegen.sh update crds and schemas * switch to software renderer to see if it will work * merge with axolotl * dynamic image repos in values.yaml * fix escaping for github action * set the tag to the ref * bad digests? * bad path * implement Albetro's suggestion to use wolf only for wayland session * remove douplicate RuntimeWolfVariables and update apps * allow user the ability to define env vars for wolf * add nvidia examples * update chart home in preparation for merging --------- Co-authored-by: brunnels <kraven@kraven.org>
1 parent ae3ce58 commit cdda8f7

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+16420
-516
lines changed

‎.github/workflows/builder.yml‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
name: Build and Push Docker Images
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
release:
8+
types: [released]
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
permissions:
15+
packages: write
16+
17+
jobs:
18+
build_and_push_containers:
19+
runs-on: ubuntu-latest
20+
strategy:
21+
matrix:
22+
app:
23+
- wolf-agent
24+
- moonlight-proxy
25+
- operator
26+
steps:
27+
- name: Checkout
28+
uses: actions/checkout@v4
29+
with:
30+
fetch-depth: 0
31+
32+
- name: Set up QEMU
33+
uses: docker/setup-qemu-action@v3.6.0
34+
35+
- name: Set up Docker Buildx
36+
uses: docker/setup-buildx-action@v3.10.0
37+
38+
- name: Login to GitHub Container Registry
39+
uses: docker/login-action@v3.3.0
40+
if: ${{ !github.event.pull_request.head.repo.fork }}
41+
with:
42+
registry: ghcr.io
43+
username: ${{ github.repository_owner }}
44+
password: ${{ secrets.GITHUB_TOKEN }}
45+
46+
- name: Generate docker image tags
47+
id: metadata
48+
uses: docker/metadata-action@v5
49+
with:
50+
flavor: |
51+
# Disable latest tag
52+
latest=false
53+
images: |
54+
name=ghcr.io/${{ github.repository_owner }}/fenrir/${{ matrix.app == 'operator' && 'direwolf-operator' || matrix.app }}
55+
56+
- name: Build and push image
57+
id: build
58+
uses: docker/build-push-action@v6.15.0
59+
with:
60+
platforms: linux/amd64
61+
push: ${{ !github.event.pull_request.head.repo.fork }}
62+
cache-from: type=gha
63+
cache-to: type=gha,mode=max
64+
tags: ${{ steps.metadata.outputs.tags }}
65+
labels: ${{ steps.metadata.outputs.labels }}
66+
build-args: |
67+
APP_NAME=${{ matrix.app }}
68+
69+
- name: Write image digest file
70+
if: ${{ steps.build.outputs.digest }}
71+
run: |
72+
echo "${{ steps.build.outputs.digest }}" > "image-digest-${{ matrix.app }}.txt"
73+
74+
- name: Upload image digest artifact
75+
if: ${{ steps.build.outputs.digest }}
76+
uses: actions/upload-artifact@v4
77+
with:
78+
name: image-digest-${{ matrix.app }}
79+
path: image-digest-${{ matrix.app }}.txt
80+
81+
build_and_push_chart:
82+
runs-on: ubuntu-latest
83+
needs: build_and_push_containers
84+
steps:
85+
- name: Checkout
86+
uses: actions/checkout@v4
87+
with:
88+
fetch-depth: 0
89+
90+
- name: Set up Helm
91+
uses: azure/setup-helm@v3
92+
with:
93+
version: 'v3.12.0'
94+
95+
- name: Get chart version
96+
id: get_version
97+
uses: mikefarah/yq@master
98+
with:
99+
cmd: |
100+
set -euo pipefail
101+
102+
REF="${{ github.ref }}"
103+
CHART_PATH="charts/direwolf-operator"
104+
105+
if [[ "$REF" == refs/tags/* ]]; then
106+
FULL_VERSION="${{ github.ref_name }}"
107+
else
108+
if [ ! -f "$CHART_PATH/Chart.yaml" ]; then
109+
echo "Chart.yaml not found at $CHART_PATH" >&2
110+
exit 1
111+
fi
112+
113+
BASE_VERSION="$(yq e '.version' "${CHART_PATH}/Chart.yaml")"
114+
if [ -z "$BASE_VERSION" ]; then
115+
echo "Could not determine chart version" >&2
116+
exit 1
117+
fi
118+
RUN_ID="${{ github.run_id }}"
119+
FULL_VERSION="${BASE_VERSION}-${RUN_ID}-SNAPSHOT"
120+
#FULL_VERSION="${BASE_VERSION}-SNAPSHOT"
121+
fi
122+
123+
echo "Full Version: $FULL_VERSION"
124+
yq e -i ".version = \\"$FULL_VERSION\\"" "${CHART_PATH}/Chart.yaml"
125+
echo "version=$FULL_VERSION" >> "$GITHUB_OUTPUT"
126+
127+
- name: Copy CRDs into chart
128+
run: |
129+
# Ensure the chart CRDs directory exists and copy repository CRDs there. This makes them part of the packaged chart.
130+
mkdir -p charts/direwolf-operator/crds
131+
if compgen -G "crds/*.yaml" > /dev/null; then
132+
cp crds/*.yaml charts/direwolf-operator/crds/
133+
else
134+
echo "No CRDs found in repository crds/ - continuing"
135+
fi
136+
137+
- name: Download image digest artifact (operator)
138+
uses: actions/download-artifact@v4
139+
with:
140+
name: image-digest-operator
141+
path: operator-artifact
142+
143+
- name: Download image digest artifact (moonlight-proxy)
144+
uses: actions/download-artifact@v4
145+
with:
146+
name: image-digest-moonlight-proxy
147+
path: proxy-artifact
148+
149+
- name: Download image digest artifact (wolf-agent)
150+
uses: actions/download-artifact@v4
151+
with:
152+
name: image-digest-wolf-agent
153+
path: agent-artifact
154+
155+
- name: Update Helm chart image tags
156+
uses: mikefarah/yq@master
157+
with:
158+
cmd: |
159+
VALUES_FILE=charts/direwolf-operator/values.yaml
160+
161+
OP_DIGEST="$(cat operator-artifact/image-digest-operator.txt)"
162+
yq e -i ".controllers.direwolf_operator.containers.operator.image.repository = \\"ghcr.io/${{ github.repository_owner }}/fenrir/direwolf-operator\\"" "$VALUES_FILE"
163+
yq e -i ".controllers.direwolf_operator.containers.operator.image.tag = \\"${{ github.ref_name }}@${OP_DIGEST}\\"" "$VALUES_FILE"
164+
165+
AG_DIGEST="$(cat agent-artifact/image-digest-wolf-agent.txt)"
166+
yq e -i ".controllers.direwolf_operator.containers.operator.env.AGENT_IMAGE = \\"ghcr.io/${{ github.repository_owner }}/fenrir/wolf-agent:${{ github.ref_name }}@${AG_DIGEST}\\"" "$VALUES_FILE"
167+
168+
PX_DIGEST="$(cat proxy-artifact/image-digest-moonlight-proxy.txt)"
169+
yq e -i ".controllers.moonlight_proxy.containers.proxy.image.tag = \\"${{ github.ref_name }}@${PX_DIGEST}\\"" "$VALUES_FILE"
170+
yq e -i ".controllers.moonlight_proxy.containers.proxy.image.repository = \\"ghcr.io/${{ github.repository_owner }}/fenrir/moonlight-proxy\\"" "$VALUES_FILE"
171+
172+
echo "Updated $VALUES_FILE"
173+
174+
- name: Package Helm chart
175+
run: |
176+
mkdir -p packaged
177+
helm dependency update charts/direwolf-operator
178+
helm package charts/direwolf-operator -d packaged
179+
ls -lah packaged
180+
181+
- name: Login to GHCR for Helm OCI
182+
if: ${{ !github.event.pull_request.head.repo.fork }}
183+
run: |
184+
set -euo pipefail
185+
echo "Logging in to ghcr.io with helm registry login"
186+
echo "${{ secrets.GITHUB_TOKEN }}" | helm registry login ghcr.io -u "${{ github.repository_owner }}" --password-stdin
187+
188+
- name: Push packaged chart with Helm (OCI)
189+
env:
190+
CHART_VERSION: ${{ steps.get_version.outputs.version }}
191+
run: |
192+
set -euo pipefail
193+
chart_file=$(ls packaged/*.tgz | head -n1)
194+
if [ -z "$chart_file" ]; then
195+
echo "No packaged chart found" >&2
196+
exit 1
197+
fi
198+
chart_ref="oci://ghcr.io/${{ github.repository_owner }}/charts"
199+
echo "Pushing $chart_file to $chart_ref"
200+
helm push "$chart_file" "$chart_ref"
201+
202+
- name: Verify pushed artifact
203+
run: |
204+
chart_ref="oci://ghcr.io/${{ github.repository_owner }}/charts:${{ steps.get_version.outputs.version }}"
205+
echo "Pushed chart $chart_ref"
206+

‎.github/workflows/docker.yml‎

Lines changed: 0 additions & 67 deletions
This file was deleted.

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,11 @@ go.work.sum
2323

2424
# env file
2525
.env
26+
27+
.idea/
28+
29+
charts/*/Chart.lock
30+
charts/*/crds/
31+
charts/*/charts/
32+
33+
packaged/

‎README.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,3 +129,57 @@ The basic idea would be that we dont need to forward any ports if we implement
129129
the RTSP handshake, RTP PING in `moonlight-proxy`. Once direwolf gets the
130130
IP/PORT of the moonlight client it can have `wolf` just start uploading a stream,
131131
as long as it can fake the source IP of the packets from `wolf`.
132+
133+
## Getting started:
134+
Before you start you should install metallb and configure a loadbalancing ip pool.
135+
136+
137+
Then, install the operator to the your k8s cluster:
138+
`kubectl apply -f examples/install.yaml`
139+
140+
after which you'll need to install the CRDs:
141+
`kubectl create -f crds/`
142+
143+
### Using Generic Devices to pass the gpu to the pods
144+
In order for wolf to render / stream the images you need to pass it a device that can do it, I've an AMD gpu, so I opted to use generic devices to pass the `dri` device
145+
146+
first check the content of `/dev/dri`:
147+
```
148+
$ ls /dev/dri/
149+
>by-path card1 renderD128
150+
```
151+
then go to the `examples/generic-devices.yaml` to edit the dri device path to be the same as the desired card.
152+
```
153+
- path: /dev/dri/renderD128 |->|- path: /dev/dri/renderD128
154+
- path: /dev/dri/cardX |->|- path: /dev/dri/card1
155+
```
156+
157+
finally apply it:
158+
`kubectl apply -f examples/generic-devices.yaml`
159+
This should add the gpu as a sharable resource that can be used in the next step.
160+
161+
### Creating a user
162+
as of now, alex is the hardcoded user, don't change the name in the `examples/user.yaml` or `examples/user_generic_device_gpu.yaml`
163+
164+
However you can change the resources and volume mounts for the sidecars:
165+
- wolf: This one requires a gpu for encoding
166+
- wolf-agent
167+
- pulseaudio
168+
169+
170+
To create a user:
171+
`kubectl apply -f examples/user_generic_device_gpu.yaml`
172+
173+
### Adding an application and pairing with moonlight
174+
add the app the to cluster using:
175+
`kubectl apply -f examples/testball.yaml`
176+
177+
Next get the ip of the loadbalancer service to connect with moonlight:
178+
`kubectl get svc direwolf -n direwolf -o jsonpath='{.status.loadBalancer.ingress[0].ip}'`
179+
180+
open moonlight to pair with the acquired ip
181+
then get the moonlight-proxy pairing url through the logs:
182+
`kubectl logs -n direwolf deployments/direwolf-moonlight-proxy`
183+
184+
use it to pair and then connect with the app, it'll take a moment to pull the image, so the first pairing might fail.
185+

‎charts/direwolf-operator/Chart.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
apiVersion: v2
3+
appVersion: 0.0.1
4+
description: Streams graphic applications/games (retroarch, firefox, steam) running on Kubernetes
5+
name: direwolf-operator
6+
version: 0.0.1
7+
kubeVersion: ">=1.28.0-0"
8+
keywords:
9+
- games-on-whales
10+
- steam
11+
- X11
12+
- pulse
13+
- stream
14+
- retroarch
15+
home: https://github.com/users/games-on-whales/packages/container/package/charts%2Fdirewolf-operator
16+
icon: https://images.opencollective.com/games-on-whales/33a2797/logo/128.png
17+
sources:
18+
- https://github.com/games-on-whales/wolf
19+
maintainers:
20+
- name: brunnels
21+
email: kraven@kraven.org
22+
dependencies:
23+
- name: common
24+
repository: https://bjw-s-labs.github.io/helm-charts
25+
version: 4.4.0

‎charts/direwolf-operator/templates/common.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
{{- include "bjw-s.common.loader.init" . }}
3+
4+
{{- define "app-template.hardcodedValues" -}}
5+
# Set the nameOverride based on the release name if no override has been set
6+
{{ if not .Values.global.nameOverride }}
7+
global:
8+
nameOverride: "{{ .Release.Name }}"
9+
{{ end }}
10+
{{- end -}}
11+
{{- $_ := mergeOverwrite .Values (include "app-template.hardcodedValues" . | fromYaml) -}}
12+
13+
{{/* Render the templates */}}
14+
{{ include "bjw-s.common.loader.generate" . }}

0 commit comments

Comments
 (0)