Skip to content

fix: rm sys_ptrace from services, explicit user rather than root, set…#569

Closed
kaixi-wang wants to merge 49 commits into
release/v2.19.0from
kacey/telemetry-including-redis-without-root-or-persistence
Closed

fix: rm sys_ptrace from services, explicit user rather than root, set…#569
kaixi-wang wants to merge 49 commits into
release/v2.19.0from
kacey/telemetry-including-redis-without-root-or-persistence

Conversation

@kaixi-wang

@kaixi-wang kaixi-wang commented May 23, 2026

Copy link
Copy Markdown
Member

Rationale

Review Priority

  • high
  • medium
  • low

Changes

Checklist

  • This PR maintains parity between Docker Compose and Helm

Testing

kaixi-wang and others added 30 commits May 21, 2026 14:13
…v2.19.0-rc.5 api:v2.19.0rc6 app:v2.19.0rc6 cas:v2.19.0-rc.5
Add an opt-in escape hatch for operators running managed Redis
(ElastiCache, MemoryStore, etc.) or an existing shared Redis. When
`telemetry.redis.external.url` is set, the chart skips the bundled
Redis `Deployment`/`Service`/`PersistentVolumeClaim` and wires both
consumer workloads and auto-injected sidecars at the external URL.

Threads through a new `telemetry.redis.url` helper that returns either
the external URL or the in-cluster `redis://<release>-telemetry-redis`
URL, replacing the inline `printf` previously used in two places.

Tests in tests/unit/helm/telemetry-redis_test.go cover: bundled Redis
NOT rendered when external.url set; api deployment env var points at the
external URL on every container (workload + sidecar); the default
release-scoped URL still wires correctly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the single `telemetry.redis.storage` knob with a
`telemetry.redis.persistence` block:

- `persistence.enabled` (default `true`) — when `false`, skip the PVC
  and run Redis on an `emptyDir`. State is lost on pod restart; intended
  for dev/minikube clusters without a dynamic PV provisioner.
- `persistence.size` — replaces `storage`. Same default (`1Gi`).
- `persistence.storageClass` — passed through to the PVC. Leave unset
  to use the cluster's default `StorageClass`.

Tests added: PVC has the expected `storageClassName` when set;
`persistence.enabled=false` skips the PVC and the Deployment switches
the `redis-data` volume to `emptyDir`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The delegated-operator workload containers and DO Job templates were
still building `FIFTYONE_TELEMETRY_REDIS_URL` inline via
`printf "redis://%s:6379" (include "telemetry.redis.name" .ctx)`,
bypassing the `telemetry.redis.url` helper added in the external-Redis
commit. As a result, setting `telemetry.redis.external.url` correctly
pointed api/app/cas/plugins/teams-app at the external URL but left the
DO workloads pointing at the in-cluster Service.

Both sites now go through `telemetry.redis.url .ctx`.

Regression tests added to tests/unit/helm/telemetry-redis_test.go cover
both the DO deployment workload container and the DO Job ConfigMap.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…v2.19.0-rc.6 api:v2.19.0rc7 app:v2.19.0rc7 cas:v2.19.0-rc.6
- pin sidecar image default to v0.1.62 (matches helm
  `telemetry.sidecar.image`); update unit-test expectations in lockstep
- add `TELEMETRY_REDIS_MAXMEMORY_POLICY` env knob (was hardcoded to
  `allkeys-lru`; now mirrors helm `telemetry.redis.maxmemoryPolicy`)
- add `deploy.resources` cpu/memory limits + reservations to telemetry
  Redis and every sidecar service to match the helm defaults
  (Redis: 0.25/512M lim, 0.10/256M res; sidecars: 0.10/512M)
- rewrite env.template telemetry section: drop stale
  `compose.telemetry.yaml overlay` reference (bundled by default since
  the sidecar work landed), document the new policy knob and the v0.1.62
  pin, and clarify when to override
- expand `docs/configuring-telemetry.md` with the new env var and a
  resource-limits table cross-referencing the helm defaults

Applies symmetrically to internal-auth and legacy-auth, including the
plugins / dedicated-plugins / delegated-operators / delegated-operators
GPU overlay variants.
…s.yaml

The docker README documents that `compose.plugins.yaml` and
`compose.dedicated-plugins.yaml` are used *instead of* `compose.yaml`
(not layered on top of it), which previously forced each of the three
base files to redefine `telemetry-redis`, `fiftyone-app-telemetry`, and
`teams-api-telemetry` verbatim. Bumping image tags or resource limits
meant editing the same block three times — and missing one would
silently desync the deployments.

Move the telemetry services into `common-services.yaml` with `-common`
suffixes (mirroring the existing pattern for `fiftyone-app-common`,
`teams-api-common`, etc.), then have each compose file pull them in
via `extends:`. Each child redeclares `depends_on` because compose's
`extends` intentionally does not propagate it.

The rendered `docker compose config` output is byte-identical to the
pre-refactor version for every combo across both auth flavors; this is
a pure structural cleanup with no runtime behavior change.

`compose.delegated-operators.gpu.yaml` keeps its standalone sidecar
definition: it's only used in one file, and the GPU device reservation
diverges enough from the regular DO sidecar to make `extends` more
trouble than it saves.
The default RollingUpdate strategy spins up a new pod before the old
one terminates, which fails with a Multi-Attach error on the RWO PVC.
Recreate serializes pod replacement so the volume can detach first.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
voxelbuildbot and others added 17 commits May 21, 2026 14:17
…v2.19.0-rc.7 api:v2.19.0rc8 app:v2.19.0rc8 cas:v2.19.0-rc.7
… images

Tie the sidecar tag to the chart appVersion via the same `repository`/`tag`
split used by `appSettings.image` et al., and wire it into the existing
bump-fixtures helm script so releases bump it in lockstep with the rest.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nobs

These knobs were originally added as the consumer-supplied injection point
for the telemetry sidecar (commits c2c9025, b0b39bd). Once telemetry switched
to auto-injection (d35aa48) nothing in the chart populated them and the
defaults have been `[]` since. Remove the values, template plumbing, and
the now-orphaned native-sidecar merge logic in the DO Job ConfigMap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…v2.19.0-rc.9 api:v2.19.0rc10 app:v2.19.0rc10 cas:v2.19.0-rc.9
…v2.19.0-rc.10 api:v2.19.0rc11 app:v2.19.0rc11 cas:v2.19.0-rc.10
In configuring-kubernetes-orchestrator.md, fix the standalone K8s
orchestrator examples (pin sidecar image to v2.19.0, set TARGET_NAME
to "fiftyone delegated" to match chart+compose, drop runAsNonRoot
default, correct Redis service DNS) and drop "Optional" from the
section heading. Add a NOTE — repeated in the helm values.yaml comment
and the compose opt-out doc — explaining that disabling the sidecar
empties the FiftyOne UI's log viewer for delegated-operator runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The new compose.delegated-operators.gpu.yaml files for both auth modes
pinned voxel51/fiftyone-teams-cv-full at v2.17.1 — a copy-paste from an
older reference. The rest of the v2.19.0 branch (common-services.yaml,
configuring-gpu-workloads.md) uses v2.19.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The telemetry.redis.url helper returned redis://<release>-telemetry-redis:6379
— a short name that only resolves within the chart's namespace. Delegated-
operator Jobs (documented use-case in
docs/orchestrators/configuring-kubernetes-orchestrator.md) can be scheduled
into a different namespace and would silently fail to reach the bundled
Redis. Switch to a fully-qualified <svc>.<ns>.svc.cluster.local hostname.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
values.yaml said "chart version" — the helper actually falls back to
.Chart.AppVersion, which is distinct from .Chart.Version in some
charts. Make the comment unambiguous and regenerate the README +
values.schema.json.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The upstream redis:7-alpine image runs as root with no explicit
security posture, which clusters with PSA `restricted` or
opinionated admission policies reject. Default to:

- pod-level: runAsNonRoot, UID/GID 999 (the redis user in the
  upstream image), fsGroup 999 so the mounted /data PVC is
  group-writable
- container-level: allowPrivilegeEscalation: false,
  readOnlyRootFilesystem: true (writes go to the /data volume),
  drop all capabilities

Both blocks are exposed as values so operators can override on
clusters that mandate specific UIDs or policies.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three gaps in the telemetry test coverage:

- shareProcessNamespace on api/app/plugins/DO deployments was
  never asserted. Without it the sidecar's /proc/<pid>/fd/1
  access breaks silently. Cover both on (telemetry enabled) and
  off (telemetry disabled) states.
- The sidecar's securityContext (runAsUser: 0, runAsNonRoot:
  false, SYS_PTRACE) was never asserted. This is the posture
  that makes telemetry incompatible with PSA `restricted` —
  regressing it without an explicit posture change would
  silently break log capture on clusters that already accept it.
- api-role.yaml's pods/log GET rule wasn't pinned across the
  telemetry on/off boundary. Lock it in so future api-SA
  consolidation work doesn't accidentally lift the grant.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Telemetry is on by default and adds non-trivial resource and
cluster-policy requirements on upgrade. Document in both upgrade
guides:

- per-workload resource overhead (~+0.4 CPU, ~+2 GiB memory) plus
  bundled Redis
- helm: Pod Security Admission compatibility (sidecar needs
  SYS_PTRACE + runAsUser:0 + shareProcessNamespace), opt-out via
  `telemetry.enabled: false`, external Redis via
  `telemetry.redis.external.url`
- docker: Compose v2.17+ for `depends_on.restart: true`,
  PID-namespace + SYS_PTRACE host requirements, teams-do forced
  to replicas=1, opt-out via the override pattern documented in
  configuring-telemetry.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The default telemetry.role.subjects list bound both the main app SA
and the teams-api RBAC SA. But api-role.yaml already grants pods/log
GET to the teams-api SA — binding it a second time via the telemetry
Role is pure duplication.

Drop the teams-api SA from the default subjects. The main app SA still
covers sidecars on fiftyone-app, teams-plugins, and delegated-operator
workloads. When apiSettings.rbac.create is false, api-deployment falls
back to the main app SA anyway, which this binding still covers, so
the api sidecar retains pods/log access across both rbac modes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds telemetry.redis.persistence.existingClaim so operators on clusters
without a dynamic PV provisioner (or wanting to reuse a volume across
releases) can hand the chart a pre-created PVC instead of relying on
chart-managed provisioning. When set, the chart skips its PVC render and
the Deployment mounts the named claim; persistence.enabled=false still
wins and yields an emptyDir.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kaixi-wang kaixi-wang requested a review from a team as a code owner May 23, 2026 17:34
@coderabbitai

coderabbitai Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

This pull request introduces FiftyOne Enterprise 2.19.0 with a comprehensive telemetry system, updating version references across all deployment platforms (Docker Compose, Kubernetes, Helm), adding Helm telemetry templates and test coverage, and hardening pod security contexts with explicit non-root defaults.

Changes

Telemetry Feature & Security Configuration

Layer / File(s) Summary
Helm telemetry plumbing and helpers
helm/fiftyone-teams-app/templates/_telemetry.tpl, helm/fiftyone-teams-app/templates/_helpers.tpl, helm/fiftyone-teams-app/templates/_do_targets.tpl
New _telemetry.tpl defines Redis URL resolution (external vs. in-cluster), sidecar container specs (standard and native variants with security context), RBAC role helpers, and socket/volume mounts. Updates to _helpers.tpl and _do_targets.tpl conditionally inject telemetry environment variables (FIFTYONE_TELEMETRY_REDIS_URL, TELEMETRY_SOCKET) into multiple workload containers when telemetry is enabled.
Helm values documentation and NOTES
helm/fiftyone-teams-app/README.md, helm/fiftyone-teams-app/README.md.gotmpl, helm/fiftyone-teams-app/templates/NOTES.txt
Documents new telemetry values section with Redis, sidecar, and service account settings. Updates pod security context defaults (UID/GID 1000, runAsNonRoot) for apiSettings, appSettings, delegatedOperator, and pluginsSettings workloads. NOTES.txt now conditionally warns when telemetry Redis uses ephemeral emptyDir storage.
Helm unit tests for telemetry components
tests/unit/helm/telemetry-redis_test.go, tests/unit/helm/telemetry-rolebinding_test.go, tests/unit/helm/telemetry-sidecar_test.go, tests/unit/helm/yaml_helpers.go
Adds comprehensive test suites validating telemetry Redis Deployment/Service/PVC rendering (with persistence/external-Redis behavior), Role/RoleBinding RBAC configuration, and sidecar injection with security context alignment and PID namespace sharing. Includes helper functions for YAML document splitting and disableTelemetry() value overrides.
Test harness modernization for telemetry-aware testing
tests/unit/helm/api-deployment_test.go, tests/unit/helm/app-deployment_test.go, tests/unit/helm/plugins-deployment_test.go, tests/unit/helm/teams-app-deployment_test.go, tests/unit/helm/delegated-operator-instance-deployment_test.go, tests/unit/helm/delegated-operator-job-configmap_test.go
Updates all Helm deployment tests to disable telemetry by default via disableTelemetry() wrapper and revises pod security context assertions to expect concrete non-root values (FSGroup/RunAsGroup/RunAsUser=1000, RunAsNonRoot=true) instead of nil.
Docker Compose test updates for telemetry services
tests/unit/compose/docker-compose-internal-auth_test.go, tests/unit/compose/docker-compose-legacy-auth_test.go
Extends test expectations to include telemetry services (telemetry-redis, telemetry sidecars), volumes (telemetry-redis-data, telemetry-socket), environment variables (FIFTYONE_TELEMETRY_REDIS_URL), and GPU profile support for delegated operations.
RBAC and API role tests for telemetry
tests/unit/helm/api-role_test.go
Updates expected RBAC rules to include pods/log permission alongside existing rules and adds telemetry-disabled test case.
Test build tag configuration
tests/unit/helm/common_test.go
Extends Go build tags to include telemetry-related unit test matrices (unitTelemetryRedis, unitTelemetryRoleBinding, unitTelemetrySidecar).

Version Bump 2.18.1 → 2.19.0

Layer / File(s) Summary
Build and configuration version defaults
Makefile, docs/custom-plugins.md, docs/orchestrators/configuring-databricks-orchestrator.md, docker/docs/configuring-gpu-workloads.md
Updates Makefile VERSION variable and Docker image tag examples from 2.18.1 to 2.19.0 across documentation and configuration examples.
Integration test version fixtures
tests/fixtures/docker/integration_internal_auth.env, tests/fixtures/docker/integration_legacy_auth.env
Updates test environment version variables to 2.19.0rc11/2.19.0-rc.10 for integration test modes.
Helm upgrade and values documentation version updates
helm/docs/upgrading.md
Updates Helm documentation with 2.19.0 references, including upgrade paths, SDK compatibility notes, license requirements, and version-specific upgrade steps across multiple migration scenarios.
Docker upgrade and configuration documentation version updates
docker/README.md, docker/docs/upgrading.md
Updates Docker documentation to reference 2.19.0 in upgrade procedures, delegated operator examples, and SDK recommendations.

Telemetry Configuration Documentation

Layer / File(s) Summary
Docker Compose telemetry configuration guide
docker/docs/configuring-telemetry.md
Introduces new comprehensive guide covering default telemetry services, opt-out procedures via compose overrides, scaling guidance for delegated operators, environment variable configuration, resource limits, access control requirements, and Redis verification commands.
Docker environment template telemetry sections
docker/internal-auth/env.template, docker/legacy-auth/env.template
Adds documented telemetry configuration sections describing optional Redis URL overrides, bundled Redis image/memory settings, namespace labeling, and sidecar target-process name matching variables.
Docker README telemetry section
docker/README.md
Adds telemetry subsection under Advanced Configuration describing bundled sidecar + Redis backend, UI metrics visibility, and reference to full telemetry documentation.
Kubernetes telemetry sidecar configuration
docs/orchestrators/configuring-kubernetes-orchestrator.md
Introduces new Telemetry Sidecar section and extends production template example with PID namespace sharing, telemetry sidecar initContainer, environment variables, and volume/mount configuration.
Helm upgrade documentation telemetry section
helm/docs/upgrading.md
Adds v2.19+ Telemetry Sidecars upgrade section describing sidecar rollout scope, resource impact, cluster/POD security requirements (process namespace sharing, SYS_PTRACE), external Redis configuration, and opt-out via Helm values.
Fixture and validation script updates
utils/bump-fixtures-helm.sh, utils/validate-docker-pulls.sh
Updates utility scripts: bump-fixtures-helm.sh now conditionally updates telemetry sidecar image tags; validate-docker-pulls.sh extends expected image allowlist with redis:7-alpine and voxel51/telemetry-sidecar with proper tag-handling logic.

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

priority-low

Suggested reviewers

  • kevin-dimichel
  • mo-getter

Poem

🐰 Telemetry lights twinkle bright,
Redis sidecars stand through the night,
Two-point-one-nine brings security tight,
Non-root pods and RBAC—all right!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch kacey/telemetry-including-redis-without-root-or-persistence

@kaixi-wang kaixi-wang changed the base branch from alan/telemetry-including-redis to main May 23, 2026 17:38

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
utils/validate-docker-pulls.sh (1)

105-116: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Current tag derivation is causing deterministic CI failures for unreleased image tags.

Using Chart.appVersion as the implied tag for all untagged voxel51/* images is failing this pipeline (no such manifest ...:v2.19.0). Please decouple pull validation from unreleased chart versions (for example via explicit per-image tags from values or a gated “release-only” validation mode).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@utils/validate-docker-pulls.sh` around lines 105 - 116, The script currently
derives tags from Chart.yaml.appVersion (variable expected_tag) and appends it
to untagged voxel51/* images (setting img_with_tag), which causes CI failures
for unreleased chart versions; change the tag resolution to prefer explicit
per-image tags from Helm values (e.g., read a values mapping like
.images.<name>.tag or a VALUES_IMAGE_TAGS env var) and only fall back to
Chart.appVersion in a gated "release-only" mode controlled by an env flag (e.g.,
RELEASE_VALIDATION=true); update the loop that sets img_with_tag (and references
EXPECTED_IMAGES and expected_tag) to lookup per-image tags first and skip or
mark as non-fatal when no published tag exists unless RELEASE_VALIDATION is
enabled.
tests/unit/helm/plugins-deployment_test.go (1)

2520-2536: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Align podSecurityContext non-root defaults across helm deployment tests (or document exceptions).

plugins-deployment_test.go asserts podSecurityContext.FSGroup=1000, RunAsGroup=1000, RunAsUser=1000, and RunAsNonRoot=true. Other deployments are not consistently asserting the same pod-level non-root defaults—e.g., cas-deployment_test.go (and at least one teams-app-deployment_test.go case) asserts podSecurityContext.RunAsNonRoot is nil. If this is intentional for those workloads, add a short rationale; otherwise make the test expectations/defaults consistent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/helm/plugins-deployment_test.go` around lines 2520 - 2536, The
plugin deployment test case labeled "defaultValuesPluginsEnabled" asserts
podSecurityContext defaults (podSecurityContext.FSGroup, RunAsGroup, RunAsUser =
1000 and RunAsNonRoot = true) that are inconsistent with other tests (e.g.,
cases in cas-deployment_test.go and teams-app-deployment_test.go which expect
RunAsNonRoot == nil); either update those other tests to assert the same
pod-level non-root defaults or, if the difference is intentional, add a brief
rationale comment above the differing test cases explaining why
podSecurityContext.RunAsNonRoot (or other fields) are expected to be nil, and
ensure consistency by adjusting assertions for
podSecurityContext.FSGroup/RunAsGroup/RunAsUser/RunAsNonRoot accordingly.
🧹 Nitpick comments (1)
tests/unit/helm/app-deployment_test.go (1)

2094-2097: ⚡ Quick win

Guard pod security pointer fields before dereferencing.

Line 2094-Line 2097 dereference pointer fields directly; if rendering regresses to nil, this panics instead of producing a clear assertion failure.

Proposed hardening
 func(podSecurityContext *corev1.PodSecurityContext) {
-    s.Equal(int64(1000), *podSecurityContext.FSGroup, "fsGroup should be 1000 (image UID)")
-    s.Equal(int64(1000), *podSecurityContext.RunAsGroup, "runAsGroup should be 1000")
-    s.True(*podSecurityContext.RunAsNonRoot, "runAsNonRoot should be true")
-    s.Equal(int64(1000), *podSecurityContext.RunAsUser, "runAsUser should be 1000")
+    s.Require().NotNil(podSecurityContext, "pod security context should not be nil")
+    s.Require().NotNil(podSecurityContext.FSGroup, "fsGroup should be set")
+    s.Require().NotNil(podSecurityContext.RunAsGroup, "runAsGroup should be set")
+    s.Require().NotNil(podSecurityContext.RunAsNonRoot, "runAsNonRoot should be set")
+    s.Require().NotNil(podSecurityContext.RunAsUser, "runAsUser should be set")
+    s.Equal(int64(1000), *podSecurityContext.FSGroup, "fsGroup should be 1000 (image UID)")
+    s.Equal(int64(1000), *podSecurityContext.RunAsGroup, "runAsGroup should be 1000")
+    s.True(*podSecurityContext.RunAsNonRoot, "runAsNonRoot should be true")
+    s.Equal(int64(1000), *podSecurityContext.RunAsUser, "runAsUser should be 1000")
     s.Nil(podSecurityContext.FSGroupChangePolicy, "should be nil")
     s.Nil(podSecurityContext.SeccompProfile, "should be nil")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/helm/app-deployment_test.go` around lines 2094 - 2097, The test
dereferences podSecurityContext pointer fields (podSecurityContext.FSGroup,
.RunAsGroup, .RunAsNonRoot, .RunAsUser) directly which will panic if any are
nil; update the assertions to first guard each pointer with s.NotNil(...) (or
equivalent) referencing podSecurityContext and each field name, then dereference
them for the equality/true asserts so failures produce clear assertion errors
instead of panics.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docker/docs/configuring-telemetry.md`:
- Around line 70-71: Update the wording so it clarifies that the `SYS_PTRACE`
capability is required only for executor/delegated-operator telemetry sidecars
(the delegated-operator/executor path that uses py-spy) and not for service-mode
telemetry sidecars; modify the sentence around `SYS_PTRACE` and `py-spy` on the
line that currently implies every telemetry sidecar needs it, and ensure
`teams-plugins-telemetry` / `compose.dedicated-plugins.yaml` references remain
unchanged while adding a note that service-mode sidecars do not require
`SYS_PTRACE`.

In `@docker/docs/upgrading.md`:
- Around line 125-129: The docs overstate security requirements by implying
every sidecar needs the SYS_PTRACE capability; update the paragraph around
"PID-namespace sharing", "pid: \"service:<target>\"" and "SYS_PTRACE" to clarify
that only the telemetry-attaching sidecar/process that performs ptrace requires
SYS_PTRACE (or equivalent host PID namespace access), not all sidecars, and
provide a note that services removed from SYS_PTRACE per the PR should still
function when a dedicated attacher sidecar or host PID sharing is used as an
alternative to granting SYS_PTRACE broadly.

In `@helm/docs/upgrading.md`:
- Around line 170-181: The doc currently states the sidecar runs as root with
SYS_PTRACE, which conflicts with tests that validate non-root sidecars and only
grant SYS_PTRACE for executor/delegated-operator cases; update the paragraph to
reflect rendered manifests by stating that the telemetry sidecar runs as
non-root by default and that SYS_PTRACE is only added when using
executor/delegated-operator modes (or when explicitly enabled), and preserve the
guidance about PSA/admission by referencing the chart values telemetry.enabled
and namespace.name so users know to disable telemetry or relax policies when
they intentionally enable SYS_PTRACE.

In `@helm/fiftyone-teams-app/README.md`:
- Line 1044: Update the example pip install version in the README description to
match the documented recommended value: change the text that currently says "pip
install ... fiftyone==0.11.0" to use "pip install ... fiftyone==2.19.0" (or
interpolate the value referenced by
teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION) so the
displayed install command matches the recommended SDK version.

In `@tests/unit/helm/telemetry-redis_test.go`:
- Around line 121-129: The test TestBundledUrlWiresApiDeployment currently only
asserts the value when an env var named "FIFTYONE_TELEMETRY_REDIS_URL" is found,
so it can falsely pass if the variable is missing; update the test to assert the
env var is present for each container in
deployment.Spec.Template.Spec.Containers (e.g., set a found flag per container
while iterating container.Env and after the inner loop call s.True(found,
"container %s must set FIFTYONE_TELEMETRY_REDIS_URL") or use s.Require().True to
fail immediately), then keep the existing equality assertion of ev.Value ==
expectedURL when found. Ensure you reference the env variable name
"FIFTYONE_TELEMETRY_REDIS_URL" and the test function
TestBundledUrlWiresApiDeployment when making the change.

---

Outside diff comments:
In `@tests/unit/helm/plugins-deployment_test.go`:
- Around line 2520-2536: The plugin deployment test case labeled
"defaultValuesPluginsEnabled" asserts podSecurityContext defaults
(podSecurityContext.FSGroup, RunAsGroup, RunAsUser = 1000 and RunAsNonRoot =
true) that are inconsistent with other tests (e.g., cases in
cas-deployment_test.go and teams-app-deployment_test.go which expect
RunAsNonRoot == nil); either update those other tests to assert the same
pod-level non-root defaults or, if the difference is intentional, add a brief
rationale comment above the differing test cases explaining why
podSecurityContext.RunAsNonRoot (or other fields) are expected to be nil, and
ensure consistency by adjusting assertions for
podSecurityContext.FSGroup/RunAsGroup/RunAsUser/RunAsNonRoot accordingly.

In `@utils/validate-docker-pulls.sh`:
- Around line 105-116: The script currently derives tags from
Chart.yaml.appVersion (variable expected_tag) and appends it to untagged
voxel51/* images (setting img_with_tag), which causes CI failures for unreleased
chart versions; change the tag resolution to prefer explicit per-image tags from
Helm values (e.g., read a values mapping like .images.<name>.tag or a
VALUES_IMAGE_TAGS env var) and only fall back to Chart.appVersion in a gated
"release-only" mode controlled by an env flag (e.g., RELEASE_VALIDATION=true);
update the loop that sets img_with_tag (and references EXPECTED_IMAGES and
expected_tag) to lookup per-image tags first and skip or mark as non-fatal when
no published tag exists unless RELEASE_VALIDATION is enabled.

---

Nitpick comments:
In `@tests/unit/helm/app-deployment_test.go`:
- Around line 2094-2097: The test dereferences podSecurityContext pointer fields
(podSecurityContext.FSGroup, .RunAsGroup, .RunAsNonRoot, .RunAsUser) directly
which will panic if any are nil; update the assertions to first guard each
pointer with s.NotNil(...) (or equivalent) referencing podSecurityContext and
each field name, then dereference them for the equality/true asserts so failures
produce clear assertion errors instead of panics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0f8d243d-fa24-47bb-a7b9-0fbdc263f213

📥 Commits

Reviewing files that changed from the base of the PR and between 816edea and 23793fa.

⛔ Files ignored due to path filters (32)
  • docker/common-services.yaml is excluded by !**/*.yaml
  • docker/internal-auth/compose.dedicated-plugins.yaml is excluded by !**/*.yaml
  • docker/internal-auth/compose.delegated-operators.gpu.yaml is excluded by !**/*.yaml
  • docker/internal-auth/compose.delegated-operators.yaml is excluded by !**/*.yaml
  • docker/internal-auth/compose.plugins.yaml is excluded by !**/*.yaml
  • docker/internal-auth/compose.yaml is excluded by !**/*.yaml
  • docker/legacy-auth/compose.dedicated-plugins.yaml is excluded by !**/*.yaml
  • docker/legacy-auth/compose.delegated-operators.gpu.yaml is excluded by !**/*.yaml
  • docker/legacy-auth/compose.delegated-operators.yaml is excluded by !**/*.yaml
  • docker/legacy-auth/compose.plugins.yaml is excluded by !**/*.yaml
  • docker/legacy-auth/compose.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/Chart.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/api-deployment.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/api-role.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/app-deployment.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/delegated-operator-instance-deployment.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/delegated-operator-job-configmap.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/plugins-deployment.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/telemetry-redis.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/templates/telemetry-rolebinding.yaml is excluded by !**/*.yaml
  • helm/fiftyone-teams-app/values.schema.json is excluded by !**/*.json
  • helm/fiftyone-teams-app/values.yaml is excluded by !**/*.yaml
  • helm/values.yaml is excluded by !**/*.yaml
  • skaffold.yaml is excluded by !**/*.yaml
  • tests/fixtures/docker/compose.override.mongodb.yaml is excluded by !**/*.yaml
  • tests/fixtures/docker/compose.override.mongodb_do.yaml is excluded by !**/*.yaml
  • tests/fixtures/docker/compose.override.mongodb_plugins.yaml is excluded by !**/*.yaml
  • tests/fixtures/helm/integration_values.yaml is excluded by !**/*.yaml
  • tests/unit/helm/test_data/delegated-operator-job-configmap_test/expected-cpu-default-override-template-values.yaml is excluded by !**/*.yaml
  • tests/unit/helm/test_data/delegated-operator-job-configmap_test/expected-cpu-default.yaml is excluded by !**/*.yaml
  • tests/unit/helm/test_data/delegated-operator-job-configmap_test/expected-override-example-cascading-template.yaml is excluded by !**/*.yaml
  • tests/unit/helm/test_data/delegated-operator-job-configmap_test/expected-override-example-template.yaml is excluded by !**/*.yaml
📒 Files selected for processing (35)
  • Makefile
  • docker/README.md
  • docker/docs/configuring-gpu-workloads.md
  • docker/docs/configuring-telemetry.md
  • docker/docs/upgrading.md
  • docker/internal-auth/env.template
  • docker/legacy-auth/env.template
  • docs/custom-plugins.md
  • docs/orchestrators/configuring-databricks-orchestrator.md
  • docs/orchestrators/configuring-kubernetes-orchestrator.md
  • helm/docs/upgrading.md
  • helm/fiftyone-teams-app/README.md
  • helm/fiftyone-teams-app/README.md.gotmpl
  • helm/fiftyone-teams-app/templates/NOTES.txt
  • helm/fiftyone-teams-app/templates/_do_targets.tpl
  • helm/fiftyone-teams-app/templates/_helpers.tpl
  • helm/fiftyone-teams-app/templates/_telemetry.tpl
  • tests/fixtures/docker/integration_internal_auth.env
  • tests/fixtures/docker/integration_legacy_auth.env
  • tests/unit/compose/docker-compose-internal-auth_test.go
  • tests/unit/compose/docker-compose-legacy-auth_test.go
  • tests/unit/helm/api-deployment_test.go
  • tests/unit/helm/api-role_test.go
  • tests/unit/helm/app-deployment_test.go
  • tests/unit/helm/common_test.go
  • tests/unit/helm/delegated-operator-instance-deployment_test.go
  • tests/unit/helm/delegated-operator-job-configmap_test.go
  • tests/unit/helm/plugins-deployment_test.go
  • tests/unit/helm/teams-app-deployment_test.go
  • tests/unit/helm/telemetry-redis_test.go
  • tests/unit/helm/telemetry-rolebinding_test.go
  • tests/unit/helm/telemetry-sidecar_test.go
  • tests/unit/helm/yaml_helpers.go
  • utils/bump-fixtures-helm.sh
  • utils/validate-docker-pulls.sh

Comment on lines +70 to +71
with the `SYS_PTRACE` capability so py-spy can attach to the target.
- `teams-plugins-telemetry` (only with `compose.dedicated-plugins.yaml`) —

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Clarify SYS_PTRACE scope to delegated-operator sidecars only.

Line 70 currently reads as if every telemetry sidecar requires SYS_PTRACE. Based on this PR’s security direction and accompanying tests, that capability should be limited to executor/delegated-operator telemetry, not service-mode sidecars.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker/docs/configuring-telemetry.md` around lines 70 - 71, Update the
wording so it clarifies that the `SYS_PTRACE` capability is required only for
executor/delegated-operator telemetry sidecars (the delegated-operator/executor
path that uses py-spy) and not for service-mode telemetry sidecars; modify the
sentence around `SYS_PTRACE` and `py-spy` on the line that currently implies
every telemetry sidecar needs it, and ensure `teams-plugins-telemetry` /
`compose.dedicated-plugins.yaml` references remain unchanged while adding a note
that service-mode sidecars do not require `SYS_PTRACE`.

Comment thread docker/docs/upgrading.md
Comment on lines +125 to +129
1. **Linux host with PID-namespace sharing** (`pid: "service:<target>"`)
and **`SYS_PTRACE`** capability granted to each sidecar container.
Docker Desktop on macOS/Windows supports this, but some hardened
container runtimes (gVisor, Kata) do not — telemetry will fail to
attach to the target process there.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Security requirement is overstated for all sidecars.

This section implies every telemetry sidecar must run with SYS_PTRACE. That conflicts with the PR’s “remove SYS_PTRACE from services” direction and may push users toward unnecessarily permissive runtime policy.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker/docs/upgrading.md` around lines 125 - 129, The docs overstate security
requirements by implying every sidecar needs the SYS_PTRACE capability; update
the paragraph around "PID-namespace sharing", "pid: \"service:<target>\"" and
"SYS_PTRACE" to clarify that only the telemetry-attaching sidecar/process that
performs ptrace requires SYS_PTRACE (or equivalent host PID namespace access),
not all sidecars, and provide a note that services removed from SYS_PTRACE per
the PR should still function when a dedicated attacher sidecar or host PID
sharing is used as an alternative to granting SYS_PTRACE broadly.

Comment thread helm/docs/upgrading.md
Comment on lines +170 to +181
1. **The sidecar runs as root with `SYS_PTRACE`** (required for
`py-spy` and `/proc` access).
Clusters that enforce
[Pod Security Admission][psa]
`restricted`, or admission policies (OPA/Gatekeeper, Kyverno) that block
`runAsUser: 0` or capability adds, will reject these pods at
admission.
On such clusters, either:
- allow the chart's `namespace.name` namespace at PSA `baseline`
(or relax the relevant admission policy), or
- disable telemetry with `telemetry.enabled: false`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

This security guidance conflicts with the chart behavior validated by tests.

Line 170 says the sidecar runs as root with SYS_PTRACE, but the telemetry sidecar tests in this PR validate non-root sidecars and limit SYS_PTRACE to executor/delegated-operator cases. Please align this section with actual rendered manifests.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@helm/docs/upgrading.md` around lines 170 - 181, The doc currently states the
sidecar runs as root with SYS_PTRACE, which conflicts with tests that validate
non-root sidecars and only grant SYS_PTRACE for executor/delegated-operator
cases; update the paragraph to reflect rendered manifests by stating that the
telemetry sidecar runs as non-root by default and that SYS_PTRACE is only added
when using executor/delegated-operator modes (or when explicitly enabled), and
preserve the guidance about PSA/admission by referencing the chart values
telemetry.enabled and namespace.name so users know to disable telemetry or relax
policies when they intentionally enable SYS_PTRACE.

| teamsAppSettings.env.FIFTYONE_APP_ANONYMOUS_ANALYTICS_ENABLED | bool | `true` | Controls whether anonymous analytics are captured for the application. Set to false to opt-out of anonymous analytics. |
| teamsAppSettings.env.FIFTYONE_APP_DEPLOYMENT_CHARACTERISTICS | string | `"kubernetes"` | Deployment characteristics for the `teams-app`. `kubernetes`: Indicates the app is running in a Kubernetes environment. `docker`: Indicates the app is running in a Docker environment. `kubernetes,managed`: Indicates the app is running in a managed Kubernetes environment |
| teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION | string | `"2.18.1"` | The recommended fiftyone SDK version that will be displayed in the install modal (i.e. `pip install ... fiftyone==0.11.0`). |
| teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION | string | `"2.19.0"` | The recommended fiftyone SDK version that will be displayed in the install modal (i.e. `pip install ... fiftyone==0.11.0`). |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Update the SDK install example version in the description.

Line 1044 still says pip install ... fiftyone==0.11.0, which conflicts with the newly documented recommended version 2.19.0 and can mislead users.

✏️ Suggested doc fix
-| teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION | string | `"2.19.0"` | The recommended fiftyone SDK version that will be displayed in the install modal (i.e. `pip install ... fiftyone==0.11.0`). |
+| teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION | string | `"2.19.0"` | The recommended fiftyone SDK version that will be displayed in the install modal (i.e. `pip install ... fiftyone==2.19.0`). |
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION | string | `"2.19.0"` | The recommended fiftyone SDK version that will be displayed in the install modal (i.e. `pip install ... fiftyone==0.11.0`). |
| teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION | string | `"2.19.0"` | The recommended fiftyone SDK version that will be displayed in the install modal (i.e. `pip install ... fiftyone==2.19.0`). |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@helm/fiftyone-teams-app/README.md` at line 1044, Update the example pip
install version in the README description to match the documented recommended
value: change the text that currently says "pip install ... fiftyone==0.11.0" to
use "pip install ... fiftyone==2.19.0" (or interpolate the value referenced by
teamsAppSettings.env.FIFTYONE_APP_TEAMS_SDK_RECOMMENDED_VERSION) so the
displayed install command matches the recommended SDK version.

Comment on lines +121 to +129
for _, container := range deployment.Spec.Template.Spec.Containers {
for _, ev := range container.Env {
if ev.Name == "FIFTYONE_TELEMETRY_REDIS_URL" {
s.Equal(expectedURL, ev.Value,
"FIFTYONE_TELEMETRY_REDIS_URL on %s should be release-scoped in-cluster URL",
container.Name)
}
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

TestBundledUrlWiresApiDeployment can pass when the env var is missing.

Line 121-Line 129 only validates value if FIFTYONE_TELEMETRY_REDIS_URL exists, so the test silently passes when the variable is absent.

Proposed fix
 expectedURL := fmt.Sprintf("redis://%s-telemetry-redis.%s.svc.cluster.local:6379",
 	s.releaseName, "fiftyone-teams")
 for _, container := range deployment.Spec.Template.Spec.Containers {
-	for _, ev := range container.Env {
-		if ev.Name == "FIFTYONE_TELEMETRY_REDIS_URL" {
-			s.Equal(expectedURL, ev.Value,
-				"FIFTYONE_TELEMETRY_REDIS_URL on %s should be release-scoped in-cluster URL",
-				container.Name)
-		}
-	}
+	var found *corev1.EnvVar
+	for i, ev := range container.Env {
+		if ev.Name == "FIFTYONE_TELEMETRY_REDIS_URL" {
+			found = &container.Env[i]
+			break
+		}
+	}
+	s.Require().NotNil(found,
+		"FIFTYONE_TELEMETRY_REDIS_URL should be set on %s container", container.Name)
+	s.Equal(expectedURL, found.Value,
+		"FIFTYONE_TELEMETRY_REDIS_URL on %s should be release-scoped in-cluster URL",
+		container.Name)
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for _, container := range deployment.Spec.Template.Spec.Containers {
for _, ev := range container.Env {
if ev.Name == "FIFTYONE_TELEMETRY_REDIS_URL" {
s.Equal(expectedURL, ev.Value,
"FIFTYONE_TELEMETRY_REDIS_URL on %s should be release-scoped in-cluster URL",
container.Name)
}
}
}
for _, container := range deployment.Spec.Template.Spec.Containers {
var found *corev1.EnvVar
for i, ev := range container.Env {
if ev.Name == "FIFTYONE_TELEMETRY_REDIS_URL" {
found = &container.Env[i]
break
}
}
s.Require().NotNil(found,
"FIFTYONE_TELEMETRY_REDIS_URL should be set on %s container", container.Name)
s.Equal(expectedURL, found.Value,
"FIFTYONE_TELEMETRY_REDIS_URL on %s should be release-scoped in-cluster URL",
container.Name)
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/helm/telemetry-redis_test.go` around lines 121 - 129, The test
TestBundledUrlWiresApiDeployment currently only asserts the value when an env
var named "FIFTYONE_TELEMETRY_REDIS_URL" is found, so it can falsely pass if the
variable is missing; update the test to assert the env var is present for each
container in deployment.Spec.Template.Spec.Containers (e.g., set a found flag
per container while iterating container.Env and after the inner loop call
s.True(found, "container %s must set FIFTYONE_TELEMETRY_REDIS_URL") or use
s.Require().True to fail immediately), then keep the existing equality assertion
of ev.Value == expectedURL when found. Ensure you reference the env variable
name "FIFTYONE_TELEMETRY_REDIS_URL" and the test function
TestBundledUrlWiresApiDeployment when making the change.

@kaixi-wang kaixi-wang changed the base branch from main to release/v2.19.0 May 23, 2026 20:52
@kaixi-wang kaixi-wang self-assigned this May 24, 2026
@kaixi-wang

Copy link
Copy Markdown
Member Author

closing in favor of #566

@kaixi-wang kaixi-wang closed this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants