Feature request: virtual-hosted-style S3 addressing for the symbol object store

[cfregly]

Summary

The symdb symbol-bucket S3 client and the zymtrace-cli check-bucket preflight
(used by the zymtrace-bucket-check Helm hook) address the bucket using
path-style only. S3-compatible object stores that mandate virtual-hosted
(DNS-style) addressing and reject path-style cannot be used as the symbol store,
even with valid credentials and a pre-created bucket. Please add an option to use
virtual-hosted addressing (equivalently, force_path_style = false).

Environment

• zymtrace backend 26.5.0 (Helm chart), app image
  ghcr.io/zystem-io/zymtrace-pub-backend:26.4.5.
• Object storage: storage.mode: use_existing.
• Target: an internal S3-compatible object store that requires virtual-hosted
  addressing and returns a path-style-not-allowed error for s3api path-style
  requests. (Both AWS S3 and MinIO accept path-style, so this only surfaces on
  stores that disallow it.)

Observed behavior

With storage.use_existing pointed at the store, zymtrace-cli check-bucket
opens a connection to https://<endpoint-host>/ and places the bucket in the URL
path (path-style). The store rejects path-style; zymtrace surfaces the error
as Bucket does not exist, which fails the zymtrace-bucket-check
post-install/post-upgrade Helm hook and rolls the release back.

Reproduced (debug logging shows the connected host in every case):

Attempt	Flags	Client connects to	Result
1	--aws-endpoint https://<host> (+/- --aws-region)	https://<host>/ (path-style)	Bucket does not exist
2	--aws-endpoint https://<bucket>.<host> + region	https://<bucket>.<host>/ but the bucket name is still appended path-style	Bucket does not exist
3	--aws-region only (no endpoint)	real AWS endpoint	Bucket does not exist

Control: an independent S3 client (aws-cli with addressing_style = virtual)
performs HeadBucket / ListObjects / PutObject / GetObject / DeleteObject
successfully against the same endpoint, bucket, and credentials. This
isolates the cause to addressing style, not credentials, permissions, bucket
existence, or network reachability.

Impact

• helm install/helm upgrade fails: the zymtrace-bucket-check
  post-install/post-upgrade hook errors out, so the release rolls back. There is
  no supported way to complete an install against such a store.
• No durable symbol storage on these clusters: the only durable object store
  available is virtual-hosted-only, so operators are forced to fall back to an
  in-cluster MinIO. Where the cluster has no persistent block storage
  (no StorageClass/CSI), that MinIO is ephemeral, and symbolization breaks on
  every object-store pod reschedule.
• Net effect: flame graphs lose native symbolization until the bucket is
  manually recreated, with no first-class fix available.

Request

1. A config / Helm value (e.g. storage.use_existing.minio.forcePathStyle: false
   or storage.use_existing.<type>.addressingStyle: virtual) that switches the
   symbol-bucket S3 SDK client to virtual-hosted addressing. Default may remain
   path-style for backward compatibility.
2. The same option honored by zymtrace-cli check-bucket, so the preflight hook
   matches runtime addressing behavior.
3. (Optional) make check-bucket use an existence probe that does not depend on
   path-style addressing, and/or provide a value to disable the bucket-check hook
   for operators who validate the bucket out of band.

Why this is low-risk

force_path_style (default on / virtual-hosted off) is a standard, widely
supported S3 SDK setting. Exposing it is backward-compatible (path-style stays
the default) and unblocks any virtual-hosted-only S3-compatible object store,
not just one vendor.

References

• Chart storage.use_existing block (where the option would live):
  https://github.com/zystem-io/zymtrace-charts/blob/main/charts/backend/values.yaml#L488-L518
• Object-storage configuration docs:
  https://docs.zymtrace.com/databases/object-storage
• AWS S3 path-style deprecation (why virtual-hosted is the forward-compatible default):
  https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
• S3 virtual-hosted vs path-style addressing:
  https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
• The standard SDK toggle (example, Rust aws-sdk-s3 force_path_style):
  https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/config/struct.Builder.html#method.force_path_style

Appendix: candidate chart plumbing (contingent on backend support)

This is the Helm-side plumbing only. It is effective only if the backend
image honors the resulting setting -- the addressing style is chosen inside
the symdb binary and zymtrace-cli, not the chart. Default keeps path-style, so
this is non-breaking.

# values.yaml -- storage.use_existing
storage:
  use_existing:
    minio:
      forcePathStyle: false   # NEW: false => virtual-hosted addressing
    s3:
      forcePathStyle: false   # NEW

# symdb config (e.g. templates/databases/symdb-configmap.yaml)
+ SYMDB__SYMBOL_BUCKET__FORCE_PATH_STYLE: {{ .Values.storage.use_existing.minio.forcePathStyle | default false | quote }}

# bucket-check hook (templates/databases/bucket-check.yaml)
  zymtrace-cli check-bucket \
    --bucket-name "..." \
    --aws-endpoint "${SYMDB__SYMBOL_BUCKET__AWS_ENDPOINT}" \
+   --force-path-style "${SYMDB__SYMBOL_BUCKET__FORCE_PATH_STYLE}"