docs(deploy): VPS hosting gotchas and workarounds

[fpindej]

Summary

Lessons learned from deploying the demo to a budget VPS. These should be documented or fixed in the production overlay.

Issues Encountered

1. pids_limit conflicts with deploy.resources.limits.pids

Newer Docker Compose rejects having both the legacy top-level pids_limit (from the x-hardened anchor) and deploy.resources.limits.pids on the same service.

Fix: Remove pids_limit from x-hardened, set pids inside each service's deploy.resources.limits instead.

2. PostgreSQL / Redis / MinIO crash with cap_drop: ALL + no-new-privileges

The x-hardened anchor drops all capabilities and sets no-new-privileges. Infrastructure services (Postgres, Redis, MinIO) need CHOWN, SETUID, SETGID, FOWNER, DAC_OVERRIDE for their entrypoint user-switching and data directory permissions.

Fix: Override security_opt and cap_add on db, redis, and storage services to add back the minimum required capabilities.

3. MinIO requires x86-64-v2 CPU instruction set

MinIO's official images (both minio/minio from Docker Hub and quay.io/minio/minio) are compiled with x86-64-v2 requirement. Budget VPS hosts with older CPUs (pre-Haswell) fail with Fatal glibc error: CPU does not support x86-64-v2.

Workaround: Use SeaweedFS (chrislusf/seaweedfs) as a drop-in S3-compatible replacement. It's a Go binary with no glibc dependency. Requires a startup config for S3 credentials:

storage:
  image: chrislusf/seaweedfs:latest
  command:
    - sh
    - -c
    - |
      printf '{"identities":[{"name":"admin","credentials":[{"accessKey":"%s","secretKey":"%s"}],"actions":["Admin","Read","Write","List","Tagging"]}]}' "$$S3_ACCESS_KEY" "$$S3_SECRET_KEY" > /tmp/s3.json
      exec weed server -s3 -s3.port=9000 -s3.config=/tmp/s3.json -dir=/data

4. Redis health check SSL mismatch

StackExchange.Redis defaults to SSL in production. The containerized Redis runs plain TCP. Without explicitly setting Caching__Redis__UseSsl=false, the API fails to connect with SslStream.AuthenticateAsClient errors.

Fix: Set Caching__Redis__UseSsl: "false" in the API's environment block.

Affected Files

• deploy/docker-compose.production.yml
• deploy/envs/production-example/api.env (should document UseSsl=false)
• deploy/docker-compose.yml (base topology, storage service)

Labels

Production overlay, deployment, infrastructure

[fpindej]

Update: Additional SeaweedFS Issues Found & Fixed

Three more issues were discovered and fixed while getting SeaweedFS working end-to-end in both local and production Docker stacks.

---

5. SeaweedFS entrypoint vs command conflict

SeaweedFS's Docker image sets ENTRYPOINT ["weed"], so using command: [sh, -c, ...] results in weed sh -c ... which fails with unknown subcommand "sh".

Fix: Use entrypoint instead of command in the compose file:

storage:
  entrypoint:
    - sh
    - -c
    - |
      printf '...' "$$S3_ACCESS_KEY" "$$S3_SECRET_KEY" > /tmp/s3.json
      exec weed server -ip=0.0.0.0 -s3 -s3.port=9000 -s3.config=/tmp/s3.json -dir=/data

6. SeaweedFS binds to container IP only — cross-container networking fails

Without -ip=0.0.0.0, SeaweedFS binds its master/volume/S3 servers to the container's specific IP address, making it unreachable from other containers via the Docker DNS hostname.

Fix: Add -ip=0.0.0.0 to the weed server command.

7. Health check localhost resolves to IPv6 on Alpine

The wget health check using localhost resolves to ::1 (IPv6) on Alpine, but SeaweedFS only listens on IPv4 by default, causing connection refused.

Fix: Use 127.0.0.1 instead of localhost in the health check:

healthcheck:
  test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1:9333/cluster/status || exit 1"]

8. S3 bucket does not exist on fresh SeaweedFS start

Unlike MinIO, SeaweedFS starts with zero buckets. The S3 health check used GetBucketLocationAsync which fails immediately with AmazonS3Exception: The specified bucket does not exist. The bucket creation code (EnsureBucketExistsAsync) only ran on first upload, not during health checks — so the health check reported degraded and uploads failed.

Fix: The S3 health check now auto-creates the bucket via PutBucketAsync on its first run, with BucketAlreadyOwnedByYou/BucketAlreadyExists handling. Subsequent checks use the lightweight GetBucketLocationAsync.

9. SkiaSharp crashes on .NET 10 base image (FreeType symbol mismatch)

The .NET 10 aspnet:10.0 base image (Ubuntu Noble) ships FreeType 2.13.2 built without the BDF module. SkiaSharp's libSkiaSharp.so dynamically links against FT_Get_BDF_Property, causing:

dotnet: symbol lookup error: /app/runtimes/linux-arm64/native/libSkiaSharp.so: undefined symbol: FT_Get_BDF_Property

This crashes the container on any avatar upload attempt.

Fix: Switch from SkiaSharp.NativeAssets.Linux to SkiaSharp.NativeAssets.Linux.NoDependencies, which statically links FreeType (and all other native deps), eliminating the system library mismatch.

---

Commits

• 4f7c308 — fix(deploy): fix SeaweedFS startup in Docker (issues 5–7)
• b3b45d0 — fix(backend): auto-create S3 bucket in health check on first run (issue 8)
• 8da44d8 — fix(backend): use SkiaSharp NoDependencies to fix FreeType crash (issue 9)

All verified end-to-end: health checks green, avatar upload returns 200, avatar download returns processed WebP.