README.md.gotmpl

{{ template "chart.header" . }}
{{ template "chart.deprecationWarning" . }}

{{ template "chart.badgesSection" . }}

{{ template "chart.description" . }}

{{ template "chart.homepageLine" . }}

{{ template "chart.maintainersSection" . }}

{{ template "chart.sourcesSection" . }}

{{ template "chart.requirementsSection" . }}

## Prerequisites

- Kubernetes 1.19+
- Helm 3.2.0+
- PV provisioner support in the underlying infrastructure (if using filesystem backend with persistence)

## S3Proxy version compatibility

The chart tracks S3Proxy (`andrewgaul/s3proxy`) through `appVersion` (currently `{{ template "chart.appVersion" . }}`); override it with `image.tag`. Minimum supported version is `2.7.0`.

S3Proxy `3.0.0` deprecated the Apache jclouds storage backends (`s3`, `aws-s3`, `azureblob`, `filesystem`, `transient`) in favor of SDK / NIO2 providers. Upstream has announced that `3.3.0` is the last release to bundle jclouds and that future releases "will lack its Atmos and B2 storage backends" (no jclouds-free release has shipped yet; `3.3.0` remains the latest). The jclouds providers still work on 3.x, but are deprecated:

- `filesystem` / `transient`: already default to the non-deprecated `*-nio2` variants (`nio2: true`).
- `azureblob`: **defaults to `provider: azureblob-sdk`** (the Azure SDK provider, which signs correctly against custom endpoints such as Azurite). The legacy jclouds `azureblob` provider is deprecated and mis-signs against custom endpoints; set `provider: azureblob` only if you specifically need it. On real Azure, `azureblob-sdk` may require `config.backends.azureblob.regions` for bucket creation.
- `s3`, `googleCloudStorage`, `openstackSwift`: SDK providers exist upstream (`aws-s3-sdk`, `google-cloud-storage-sdk`, `openstack-swift-sdk`). `rackspaceCloudfiles` is OpenStack-Swift-compatible and may be served by `openstack-swift-sdk`. Migrating the chart defaults to the SDK providers is tracked separately.
- `b2` (and Atmos, if ever added) are jclouds-only with **no SDK successor**. These are the backends upstream has said future releases will drop.

### Our compatibility plan

- **Through S3Proxy `3.3.0` (chart `0.x`):** all current backends, including the jclouds-only `b2`, are supported. This is where the chart sits today.
- **Guardrail:** the chart **hard-fails at render time** if `b2` (or `atmos`) is enabled while the effective S3Proxy version (`image.tag`, else `appVersion`) is greater than `3.3.0`, so an upgrade past the jclouds sunset cannot silently ship a broken backend. Pin `image.tag` to `3.3.0` or earlier, or disable the backend.
- **When S3Proxy ships its first jclouds-free release:** the chart moves to `1.x.x` (major bump), migrates the remaining backends to their SDK providers, and drops `b2`/`atmos`. Tracked in the SDK-migration follow-up.

## Installation

### Install the chart

```bash
# Install with default values (filesystem backend)
helm install my-s3proxy oci://ghcr.io/comet-ml/{{ template "chart.name" . }}

# Install with custom values
helm install my-s3proxy oci://ghcr.io/comet-ml/{{ template "chart.name" . }} -f override-values.yaml
```

## Configuration

The following section lists the configurable parameters of the {{ template "chart.name" . }} chart and their default values.

{{- define "renderMarkdownLine" }}
  {{- $code := (mustRegexReplaceAll "\x60([^\x60]*)\x60" . "<code>${1}</code>") }}
  {{- $italStar := (mustRegexReplaceAll `([^\*]|\*\*)\*([^ \*].*[^ \*]|[^ \*])\*([^\*]|\*\*)` $code "${1}<i>${2}</i>${3}") }}
  {{- $italScore := (mustRegexReplaceAll "( |__)_([^ _].*[^ _]|[^ _])_( |__)" $italStar "${1}<i>${2}</i>${3}") }}
  {{- $boldStar := (mustRegexReplaceAll `\*\*([^ \*].*[^ \*]|[^ \*])\*\*` $italScore "<b>${1}</b>") }}
  {{- $boldScore := (mustRegexReplaceAll "__([^ _].*[^ _]|[^ _])__" $boldStar "<b>${1}</b>") }}
  {{- $boldScore }}
{{- end -}}

{{- define "chart.valuesTableHtml" }}
*Scroll sideways to see all columns.*

<table>
	<thead>
		<th>Key</th>
		<th>Description</th>
		<th>Type</th>
		<th>Default</th>
	</thead>
	<tbody>
	{{- range .Values }}
		<tr>
			<td><code>{{ .Key }}</code></td>
			<td>{{ template "renderMarkdownLine" (default .AutoDescription .Description) }}</td>
			<td><code>{{ .Type }}</code></td>
			<td>{{ template "renderMarkdownLine" (default .AutoDefault .Default) }}</td>
		</tr>
	{{- end }}
	</tbody>
</table>
{{- end }}

{{ template "chart.valuesSectionHtml" . }}

## Usage Examples

### Example 1: Filesystem Backend with Authentication

```yaml
# values-filesystem.yaml
config:
  auth:
    type: "aws-v4"
    identity: "myaccesskey"
    secret: "mysecretkey"
  backends:
    filesystem:
      enabled: true
      nio2: true
    filesystem:
      basedir: "/data/s3proxy"

persistence:
  enabled: true
  size: 20Gi
```

Install:
```bash
helm install s3proxy-fs ./{{ template "chart.name" . }} -f values-filesystem.yaml
```

### Example 2: AWS S3 Backend

```yaml
# values-aws-s3.yaml
config:
  auth:
    type: "aws-v4"
    identity: "proxy-access-key"  # For clients connecting to s3proxy
    secret: "proxy-secret-key"
  backend:
    provider: "aws-s3"
    awsS3:
      region: "us-west-2"
      accessKeyID: "aws-access-key-id"  # For s3proxy to connect to AWS
      secretAccessKey: "aws-secret-access-key"

persistence:
  enabled: false  # Not needed for S3 backend
```

Install:
```bash
helm install s3proxy-s3 ./{{ template "chart.name" . }} -f values-aws-s3.yaml
```

### Example 3: Azure Blob Storage Backend

```yaml
# values-azure.yaml
config:
  auth:
    type: "aws-v4"
    identity: "myaccesskey"
    secret: "mysecretkey"
  backend:
    provider: "azureblob"
    azureblob:
      account: "mystorageaccount"
      key: "storageaccountkey"

persistence:
  enabled: false  # Not needed for Azure backend
```

Install:
```bash
helm install s3proxy-azure ./{{ template "chart.name" . }} -f values-azure.yaml
```

### Example 4: Google Cloud Storage Backend

```yaml
# values-gcs.yaml
config:
  auth:
    type: "aws-v4"
    identity: "myaccesskey"
    secret: "mysecretkey"
  backend:
    provider: "google-cloud-storage"
    googleCloudStorage:
      projectID: "my-project"
      clientEmail: "service-account@my-project.iam.gserviceaccount.com"
      privateKey: |
        -----BEGIN RSA PRIVATE KEY-----
        ...
        -----END RSA PRIVATE KEY-----

persistence:
  enabled: false  # Not needed for GCS backend
```

### Example 5: Anonymous Access (No Authentication)

```yaml
# values-anonymous.yaml
config:
  auth:
    type: "none"
  backends:
    transient:
      enabled: true
      nio2: true  # In-memory storage

persistence:
  enabled: false
```

## Testing the Installation

Once deployed, you can test S3Proxy using the AWS CLI:

```bash
# Get the service endpoint
kubectl get svc

# Port-forward for local testing
kubectl port-forward svc/my-s3proxy 8080:8080

# Configure AWS CLI (if authentication is enabled)
export AWS_ACCESS_KEY_ID=myaccesskey
export AWS_SECRET_ACCESS_KEY=mysecretkey

# Test S3 operations
aws --endpoint-url http://localhost:8080 s3 ls
aws --endpoint-url http://localhost:8080 s3 mb s3://test-bucket
aws --endpoint-url http://localhost:8080 s3 cp test.txt s3://test-bucket/
aws --endpoint-url http://localhost:8080 s3 ls s3://test-bucket/
```

## TLS / HTTPS (native, in-pod)

By default S3Proxy binds plain HTTP and TLS is expected to be terminated at the
ingress. Set `config.tls.enabled=true` to have **S3Proxy itself terminate HTTPS in
the pod** (S3Proxy's `secure-endpoint`). When enabled:

- S3Proxy serves **HTTPS only** on `service.targetPort` (the plaintext endpoint is
  not bound). The container/Service port is named `https`.
- The health probes are `tcpSocket`, which only check the TCP accept (no TLS
  handshake), so they keep working unchanged against the TLS port.
- A **PKCS12** keystore is expected (Jetty's default keystore type; JKS also works
  if supplied). S3Proxy exposes only the keystore path and password, so the chart
  does not set a keystore type.
- The keystore **password is never written to the ConfigMap**; it is merged into
  the backend properties from the Secret by the config-merge initContainer.

### Option A: existing Secret (recommended; also the cert-manager path)

Reference a Secret that already holds the keystore file and the password. This is
also how a cert-manager `Certificate` with `spec.keystores.pkcs12` delivers a
keystore (point `keystore.existingSecret` at that Secret, and
`keystorePassword.existingSecret` at the password Secret it references):

```yaml
config:
  tls:
    enabled: true
    keystore:
      existingSecret: my-tls        # holds the PKCS12 archive
      secretKey: keystore.p12
    keystorePassword:
      existingSecret: my-tls        # holds the password (may be the same Secret)
      secretKey: keystore-password
```

### Option B: inline keystore + password

Provide the base64-encoded keystore and the password inline; both are stored in the
chart's own Secret. Convenient for testing; prefer Option A in production.

```yaml
config:
  tls:
    enabled: true
    keystore:
      value: "<base64-encoded PKCS12 keystore>"
    keystorePassword:
      value: "changeit"
```

Create a PKCS12 keystore, for example:

```bash
# From an existing cert + key:
openssl pkcs12 -export -inkey tls.key -in tls.crt \
  -out keystore.p12 -passout pass:changeit
# Inline value:
base64 -w0 keystore.p12
# Or as an existing Secret (Option A):
kubectl create secret generic my-tls \
  --from-file=keystore.p12=keystore.p12 \
  --from-literal=keystore-password=changeit
```

Testing over HTTPS (the CI keystore above is self-signed, so skip verification):

```bash
aws --endpoint-url https://localhost:9000 --no-verify-ssl s3 ls
```

## CORS Configuration

To enable CORS support:

```yaml
config:
  cors:
    enabled: true
    allowOrigins:
      - "https://example.com"
      - "https://.+\\.example\\.com"
    allowMethods:
      - "GET"
      - "PUT"
      - "POST"
      - "HEAD"
      - "DELETE"
    allowHeaders:
      - "Accept"
      - "Content-Type"
    allowCredential: true
```

## Middleware Configuration

S3Proxy supports various middlewares:

```yaml
config:
  middlewares:
    readOnly: false           # Make backend read-only
    eventualConsistency: true # Enable eventual consistency modeling
    shardedBackend: true      # Enable sharded backend containers
    largeObjectMocking: false # Enable large object mocking
```

## Bucket Configuration

### Bucket Aliasing

Map virtual bucket names to actual backend buckets:

```yaml
config:
  buckets:
    alias:
      virtual-bucket: "real-backend-bucket"
      another-bucket: "actual-bucket-name"
```

### Bucket Locator (routing buckets to backends)

When more than one backend is enabled, assign buckets to a specific backend with
that backend's own `bucketLocators` list. S3Proxy reads bucket-locators per
backend, so each list is emitted only into that backend's properties file. Glob
patterns are supported. A bucket that matches no backend's list falls through to
the first-enabled backend (the default).

```yaml
config:
  backends:
    s3:
      enabled: true
      # ... credentials ...
      bucketLocators:
        - "prod-*"
        - "customer-data"
    filesystem:
      enabled: true
      bucketLocators:
        - "scratch-*"   # everything else also lands here (first-enabled default)
```

## Monitoring

Check S3Proxy logs:
```bash
kubectl logs deployment/my-s3proxy
```

## Upgrading

```bash
helm upgrade my-s3proxy ./{{ template "chart.name" . }} -f my-values.yaml
```

## Uninstalling

```bash
helm uninstall my-s3proxy
```

This will remove all resources created by the chart. If using persistence, the PVC will be retained by default.

## Troubleshooting

### Common Issues

1. **Authentication failures**: Ensure `config.auth.identity` and `config.auth.secret` are set correctly for client authentication.

2. **Backend connection issues**: Verify backend credentials are correctly configured in the appropriate section (e.g., `config.backend.awsS3.*`).

3. **Persistence issues**: Check that your cluster has a default StorageClass or specify one explicitly.

4. **Port conflicts**: If port 8080 is already in use, change `service.port` and `service.targetPort`.

## References

- [S3Proxy GitHub Repository](https://github.com/gaul/s3proxy)
- [S3Proxy Docker Hub](https://hub.docker.com/r/andrewgaul/s3proxy/)
- [Storage Backend Examples](https://github.com/gaul/s3proxy/wiki/Storage-backend-examples)

## License

This Helm chart is provided as-is. S3Proxy itself is licensed under the Apache License 2.0.

{{ template "helm-docs.versionFooter" . }}