Skip to content

Commit 43408d6

Browse files
docs(helm): Add recommended post-installation configuration guide (#556)
* docs(helm): Add recommended post-installation configuration guide Signed-off-by: John Duncan <30937261+roboav8r@users.noreply.github.com> Co-authored-by: Kevin DiMichel <56850465+kevin-dimichel@users.noreply.github.com>
1 parent ff2513b commit 43408d6

6 files changed

Lines changed: 346 additions & 0 deletions

File tree

.pre-commit-config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ repos:
7777
- '-'
7878
- -i
7979
- docker/README.md
80+
- helm/docs/post-install-recommended-configuration.md
8081
- helm/fiftyone-teams-app/README.md.gotmpl
8182
- helm/README.md
8283
- CONTRIBUTING.md

helm/README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
- [Set up http to https Forwarding](#set-up-http-to-https-forwarding)
2626
- [Install FiftyOne Enterprise App](#install-fiftyone-enterprise-app)
2727
- [Installation Complete](#installation-complete)
28+
- [Recommended Next Steps](#recommended-next-steps)
2829

2930
<!-- tocstop -->
3031

@@ -279,3 +280,19 @@ and add your storage credentials to access sample data.
279280
Congratulations! You should now be able to access your
280281
FiftyOne Enterprise installation at the DNS address you created
281282
[earlier](#obtain-a-global-static-ip-address-and-configure-a-dns-entry).
283+
284+
## Recommended Next Steps
285+
286+
The base `helm install` above starts FiftyOne Enterprise with
287+
**built-in only plugins** and **no delegated operator workers**.
288+
While sufficient to get started, we recommend enabling
289+
the following features for a production-ready deployment:
290+
291+
| Step | What it enables |
292+
| --- | --- |
293+
| **Dedicated Plugins** | Install and run custom plugins in an isolated `teams-plugins` pod, keeping plugin workloads separate from the main app |
294+
| **Delegated Operators** | Schedule compute-heavy tasks — embeddings, model evaluation, dataset import, annotation — from the UI and run them on dedicated background workers |
295+
| **On-Demand Orchestrator** | *(Optional)* Spin up Kubernetes pods on demand per job instead of maintaining always-on workers; more cost-efficient for infrequent or GPU-heavy workloads |
296+
297+
For step-by-step configuration instructions, see
298+
[Recommended Post-Installation Configuration](./docs/post-install-recommended-configuration.md).
Lines changed: 302 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,302 @@
1+
# Recommended Post-Installation Configuration
2+
3+
<!-- toc -->
4+
5+
- [Prerequisites: Shared Storage](#prerequisites-shared-storage)
6+
- [Dedicated Plugins Mode](#dedicated-plugins-mode)
7+
- [Delegated Operators](#delegated-operators)
8+
- [On-Demand Delegated Operators](#on-demand-delegated-operators)
9+
- [GPU Workloads (Optional)](#gpu-workloads-optional)
10+
- [Multiple Orchestrators](#multiple-orchestrators)
11+
- [Advanced: Custom Job Priorities](#advanced-custom-job-priorities)
12+
- [Verifying Your Setup](#verifying-your-setup)
13+
- [Getting Started with Plugins](#getting-started-with-plugins)
14+
15+
<!-- tocstop -->
16+
17+
After completing the base Helm installation, we recommend enabling the
18+
following features for a production-ready FiftyOne Enterprise deployment:
19+
20+
1. [Prerequisites](#prerequisites-shared-storage)
21+
1. [Dedicated Plugins Mode](#dedicated-plugins-mode)
22+
1. [Delegated Operators](#delegated-operators)
23+
1. [GPU Workloads (Optional)](#gpu-workloads-optional)
24+
25+
The default installation uses **built-in plugins** and
26+
has **no delegated operator workers**.
27+
While sufficient to get started, it limits the ability to install custom
28+
plugins and run long-running or compute-heavy tasks in the background.
29+
30+
---
31+
32+
## Prerequisites: Shared Storage
33+
34+
Dedicated plugins and delegated operators share a common plugin directory
35+
backed by a Kubernetes PersistentVolume (PV) and PersistentVolumeClaim (PVC).
36+
37+
If you do not already have shared storage configured, refer to
38+
[Adding Shared Storage for FiftyOne Enterprise Plugins](./plugins-storage.md)
39+
for steps on creating a PV and PVC.
40+
41+
The examples below assume:
42+
43+
- PVC name: `plugins-pvc`
44+
- Plugin directory: `/opt/plugins`
45+
46+
---
47+
48+
## Dedicated Plugins Mode
49+
50+
By default, plugins run inside the `fiftyone-app` pod (built-in only mode).
51+
We recommend enabling **dedicated plugins mode**, which runs plugins in their
52+
own dedicated `teams-plugins` pod. This provides:
53+
54+
- **Custom plugin support** — install plugins from the
55+
[FiftyOne Plugin Library](https://github.com/voxel51/fiftyone-plugins) or
56+
build your own
57+
- **Resource isolation** — plugin workloads do not affect `fiftyone-app`
58+
stability or performance
59+
- **Custom dependency support** — plugins with heavy ML dependencies (e.g.
60+
`transformers`) are isolated from the main app
61+
62+
There are three plugin modes available:
63+
64+
| Mode | Description |
65+
| --- | --- |
66+
| Built-in Only (default) | Only built-in plugins shipped with FiftyOne Enterprise |
67+
| Shared | Custom plugins run inside `fiftyone-app` — may starve the app |
68+
| **Dedicated (recommended)** | Custom plugins run in a dedicated `teams-plugins` pod |
69+
70+
To enable dedicated plugins, add the following to your `values.yaml`:
71+
72+
```yaml
73+
# Dedicated plugins pod
74+
pluginsSettings:
75+
enabled: true
76+
env:
77+
FIFTYONE_PLUGINS_DIR: /opt/plugins
78+
79+
# teams-api also requires access to the plugins directory
80+
apiSettings:
81+
env:
82+
FIFTYONE_PLUGINS_DIR: /opt/plugins
83+
```
84+
85+
Then apply the changes:
86+
87+
```bash
88+
helm upgrade fiftyone-teams-app voxel51/fiftyone-teams-app \
89+
--values values.yaml \
90+
--namespace <your-namespace>
91+
```
92+
93+
Verify the `teams-plugins` pod is running:
94+
95+
```bash
96+
kubectl get pods --namespace <your-namespace> | grep teams-plugins
97+
```
98+
99+
For more details, see
100+
[Configuring Plugins](./configuring-plugins.md).
101+
102+
---
103+
104+
## Delegated Operators
105+
106+
Delegated operators allow long-running or compute-heavy tasks (such as
107+
computing embeddings, running model evaluations, importing datasets, or
108+
annotation workflows) to be scheduled from the FiftyOne UI and executed in
109+
the background on dedicated compute workers.
110+
111+
To enable delegated operators with dedicated plugins, add the following to
112+
your `values.yaml`:
113+
114+
```yaml
115+
delegatedOperatorDeployments:
116+
deployments:
117+
teamsDo:
118+
replicaCount: 3 # adjust based on your concurrency needs
119+
env:
120+
FIFTYONE_PLUGINS_DIR: /opt/plugins
121+
volumes:
122+
- name: plugins-vol
123+
persistentVolumeClaim:
124+
claimName: plugins-pvc
125+
readOnly: true
126+
volumeMounts:
127+
- name: plugins-vol
128+
mountPath: /opt/plugins
129+
```
130+
131+
Then apply the changes:
132+
133+
```bash
134+
helm upgrade fiftyone-teams-app voxel51/fiftyone-teams-app \
135+
--values values.yaml \
136+
--namespace <your-namespace>
137+
```
138+
139+
Verify the `teams-do` pod is running:
140+
141+
```bash
142+
kubectl get pods --namespace <your-namespace> | grep teams-do
143+
```
144+
145+
For full configuration options, see
146+
[Configuring Delegated Operators](./configuring-delegated-operators.md).
147+
148+
### On-Demand Delegated Operators
149+
150+
As an alternative to always-on workers, FiftyOne Enterprise v2.11.0+ supports
151+
**on-demand delegated operators** that spin up compute pods only when a job is
152+
scheduled, and tear them down when complete.
153+
This is more cost-efficient for infrequent or GPU-intensive workloads.
154+
155+
See
156+
[Configuring On-Demand Orchestrator](../../docs/configuring-on-demand-orchestrator.md)
157+
for setup instructions.
158+
159+
---
160+
161+
## GPU Workloads (Optional)
162+
163+
If your team runs GPU-accelerated tasks (e.g. computing embeddings with
164+
`@voxel51/brain`, model evaluation, or custom ML operators), you can schedule
165+
`teams-do` pods on GPU-enabled nodes using `nodeSelector` and `tolerations`.
166+
167+
Add the following to your `delegatedOperatorDeployments.deployments.teamsDo`
168+
config in `values.yaml`:
169+
170+
```yaml
171+
delegatedOperatorDeployments:
172+
deployments:
173+
teamsDo:
174+
nodeSelector:
175+
<your-gpu-node-label-key>: <your-gpu-node-label-value>
176+
# e.g. node-type: gpu
177+
tolerations:
178+
- key: "nvidia.com/gpu"
179+
operator: "Exists"
180+
effect: "NoSchedule"
181+
resources:
182+
limits:
183+
nvidia.com/gpu: 1
184+
requests:
185+
nvidia.com/gpu: 1
186+
```
187+
188+
For full details, see
189+
[Configuring GPU Workloads](./configuring-gpu-workloads.md).
190+
191+
### Multiple Orchestrators
192+
193+
For deployments with mixed workloads, you may
194+
register multiple delegated operator orchestrators.
195+
For example, one targeting GPU nodes and one targeting CPU nodes.
196+
You may run specific operators to the appropriate
197+
orchestrator from the FiftyOne UI.
198+
199+
```yaml
200+
delegatedOperatorDeployments:
201+
deployments:
202+
teamsDoGpu:
203+
replicaCount: 1
204+
env:
205+
FIFTYONE_PLUGINS_DIR: /opt/plugins
206+
nodeSelector:
207+
node-type: gpu
208+
tolerations:
209+
- key: "nvidia.com/gpu"
210+
operator: "Exists"
211+
effect: "NoSchedule"
212+
resources:
213+
limits:
214+
nvidia.com/gpu: 1
215+
volumes:
216+
- name: plugins-vol
217+
persistentVolumeClaim:
218+
claimName: plugins-pvc
219+
readOnly: true
220+
volumeMounts:
221+
- name: plugins-vol
222+
mountPath: /opt/plugins
223+
224+
teamsDoCpu:
225+
replicaCount: 2
226+
env:
227+
FIFTYONE_PLUGINS_DIR: /opt/plugins
228+
volumes:
229+
- name: plugins-vol
230+
persistentVolumeClaim:
231+
claimName: plugins-pvc
232+
readOnly: true
233+
volumeMounts:
234+
- name: plugins-vol
235+
mountPath: /opt/plugins
236+
```
237+
238+
---
239+
240+
## Advanced: Custom Job Priorities
241+
242+
> **Note:** The Helm chart's `delegatedOperatorJobTemplates.jobs` does not
243+
> currently support `priorityClassName` natively.
244+
> To use Kubernetes
245+
> [PriorityClasses](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/)
246+
> with delegated operator jobs (for example, to prevent DO workloads from
247+
> contending with user-facing pods), define custom Jinja2 job templates via a
248+
> ConfigMap and register them using the
249+
> [FiftyOne Management SDK](https://docs.voxel51.com/enterprise/management_sdk.html).
250+
251+
See
252+
[Configuring On-Demand Orchestrator](../../docs/configuring-on-demand-orchestrator.md)
253+
for details on custom job templates.
254+
255+
---
256+
257+
## Verifying Your Setup
258+
259+
After applying all changes, verify that all expected pods are running:
260+
261+
```bash
262+
kubectl get pods --namespace <your-namespace>
263+
```
264+
265+
You should see:
266+
267+
- `teams-plugins-*` — dedicated plugins pod
268+
- `teams-do-*` (one or more) — delegated operator worker pod(s)
269+
270+
You can also verify delegated operator registration from the FiftyOne Python
271+
SDK:
272+
273+
```python
274+
import fiftyone.operators.orchestrator as foo
275+
276+
orc_svc = foo.OrchestratorService()
277+
for orc in orc_svc.list():
278+
print("{} \"{}\" {}".format(orc.instance_id, orc.description, orc.id))
279+
```
280+
281+
---
282+
283+
## Getting Started with Plugins
284+
285+
Once dedicated plugins are enabled, you can install community plugins from the
286+
[FiftyOne Plugin Library](https://github.com/voxel51/fiftyone-plugins)
287+
or the
288+
[FiftyOne Docs](https://docs.voxel51.com/plugins/index.html).
289+
290+
We recommend starting with these plugins:
291+
292+
- [`@voxel51/brain`](https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/brain)
293+
— compute embeddings and similarity indexes
294+
- [`@voxel51/annotation`](https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/annotation)
295+
— annotation workflows
296+
- [`@voxel51/evaluation`](https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/evaluation)
297+
— model evaluation panels
298+
- [`@voxel51/zoo`](https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/zoo)
299+
— access the FiftyOne Model Zoo
300+
301+
To use plugins with custom dependencies (e.g. `transformers`), build and use
302+
[Custom Plugin Images](https://github.com/voxel51/fiftyone-teams-app-deploy/blob/main/docs/custom-plugins.md).

helm/fiftyone-teams-app/README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ for steps on how to add your license file.
142142
- [Estimated Completion Time](#estimated-completion-time)
143143
- [Sizing](#sizing)
144144
- [Usage](#usage)
145+
- [Recommended Next Steps](#recommended-next-steps)
145146
- [Upgrades](#upgrades)
146147
- [Advanced Configuration](#advanced-configuration)
147148
- [Backup And Recovery](#backup-and-recovery)
@@ -340,6 +341,16 @@ helm install fiftyone-teams-app voxel51/fiftyone-teams-app \
340341
A minimal example `values.yaml` may be found
341342
[in the repository](https://github.com/voxel51/fiftyone-teams-app-deploy/blob/main/helm/values.yaml).
342343

344+
## Recommended Next Steps
345+
346+
The base installation enables FiftyOne Enterprise with
347+
**built-in only plugins** and **no delegated operator workers**.
348+
For a production-ready deployment, we recommend enabling **dedicated plugins**
349+
and **delegated operators** after completing the install above.
350+
351+
See [Recommended Post-Installation Configuration](../docs/post-install-recommended-configuration.md)
352+
for step-by-step instructions.
353+
343354
## Upgrades
344355

345356
When performing an upgrade, please review

helm/fiftyone-teams-app/README.md.gotmpl

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ for steps on how to add your license file.
142142
- [Estimated Completion Time](#estimated-completion-time)
143143
- [Sizing](#sizing)
144144
- [Usage](#usage)
145+
- [Recommended Next Steps](#recommended-next-steps)
145146
- [Upgrades](#upgrades)
146147
- [Advanced Configuration](#advanced-configuration)
147148
- [Backup And Recovery](#backup-and-recovery)
@@ -340,6 +341,16 @@ helm install fiftyone-teams-app voxel51/fiftyone-teams-app \
340341
A minimal example `values.yaml` may be found
341342
[in the repository](https://github.com/voxel51/fiftyone-teams-app-deploy/blob/main/helm/values.yaml).
342343

344+
## Recommended Next Steps
345+
346+
The base installation enables FiftyOne Enterprise with
347+
**built-in only plugins** and **no delegated operator workers**.
348+
For a production-ready deployment, we recommend enabling **dedicated plugins**
349+
and **delegated operators** after completing the install above.
350+
351+
See [Recommended Post-Installation Configuration](../docs/post-install-recommended-configuration.md)
352+
for step-by-step instructions.
353+
343354
## Upgrades
344355

345356
When performing an upgrade, please review

helm/values.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,10 @@ casSettings:
8383
# Set the name of the Mongo database for CAS if not using the default `cas`
8484
# CAS_DATABASE_NAME: your-cas-name
8585

86+
# For a production-ready deployment, we recommend enabling dedicated plugins
87+
# and delegated operators after your initial install. See:
88+
# https://github.com/voxel51/fiftyone-teams-app-deploy/blob/main/helm/docs/post-install-recommended-configuration.md
89+
8690
# delegatedOperatorDeployments:
8791
# deployments:
8892
# teamsDoCpuDefault:

0 commit comments

Comments
 (0)