Support for s3 object storage

[mrsimonemms]

TL;DR

When are we likely to get support for s3 object storage in the API/SDK/Terraform? With it scheduled to move out of beta into GA this week, I'd like to start using it in my applications.

Expected behavior

S3 support

[jooola]

Hello all 👋

All our integrations rely on the Hetzner Cloud public API, which is available with a certain level of stability. Since the features you are requesting are not in the public API, we cannot implement them.

Therefore, for the time being, we do not plan to support:

• Generating credentials using the Public API.
• Creating buckets using the Public API: You can use an S3-compatible tool instead, for example using curl or the minio terraform provider.
• Embedding a S3 client in our integrations: You can use any S3-compatible tool instead.

Note that only a subset of the Amazon S3 features are currently supported.

We will leave this ticket open to increase its visibility. If you have questions, reach out to us using the Support Center.

[hakman]

@jooola A S3 compatible tool is a separate binary, not a library that you can use as part of your app.
Ideally, if Hetzner does not want to provide an S3 client lib as part of hcloud-go, it should at least suggest compatible clients and provide some examples.

[jooola]

@hakman You are correct, I'll do a bit of research and try to come up with a few examples.

[hakman]

@jooola Thanks. I was in a rush previously. Just for some background, someone reached out to us to ask about why he is getting an error when using Hetzner object storage with kOps. So there may be some incompatibilities with various clients.

error listing s3://the-bucket-name/the-cluster-name.k8s.local/instancegroup: operation error S3: ListObjectsV2, https response error StatusCode: 404, RequestID: the-request-id, HostID: the-host-id, api error NoSuchKey: UnknownError

[mrsimonemms]

Therefore, for the time being, we do not plan to support:

* Generating credentials using the Public API.

* Creating buckets using the Public API: You can use [an S3-compatible tool](https://docs.hetzner.com/storage/object-storage/getting-started/using-s3-api-tools) instead, for example using [curl](https://docs.hetzner.com/storage/object-storage/getting-started/using-curl) or the [minio terraform provider](https://docs.hetzner.com/storage/object-storage/getting-started/creating-a-bucket-minio-terraform).

* Embedding a S3 client in our integrations: You can use [any S3-compatible tool](https://docs.hetzner.com/storage/object-storage/getting-started/using-s3-api-tools) instead.

Is this plan a short or long-term thing? I understand this limitation in the short term for reasons of reliability/development speed, but being able to work on this via the API will be important for widespread adoption.

[LenzGr]

Just for some background, someone reached out to us to ask about why he is getting an error when using Hetzner object storage with kOps. So there may be some incompatibilities with various clients.

error listing s3://the-bucket-name/the-cluster-name.k8s.local/instancegroup: operation error S3: ListObjectsV2, https response error StatusCode: 404, RequestID: the-request-id, HostID: the-host-id, api error NoSuchKey: UnknownError

Could you please share a link to where this was reported? This would help us to take a look and check if it's a configuration issue or a genuine bug. Thanks!

[jooola]

Is this plan a short or long-term thing? I understand this limitation in the short term for reasons of reliability/development speed, but being able to work on this via the API will be important for widespread adoption.

Creating a bucket programmatically is already doable using existing tools or libraries, the only missing part is to generate credentials using the public API.

I agree that this is a limitation at the moment. Once we have made some necessary architectural changes, we should be able to work towards making this possible. However, we cannot give you a timeline at this point.

[jooola]

Here are a few libraries examples that can be used to interact with our object storage. Note that only a subset of the Amazon S3 features are currently supported.

Using github.com/aws/aws-sdk-go-v2/aws (Golang):

Details

package main

import (
	"context"
	"log"
	"strings"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/credentials"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
	ctx := context.Background()

	// Create a new s3 client
	client := s3.New(s3.Options{
		AppID: "my-application/0.0.1",

		Region:       "fsn1",
		BaseEndpoint: aws.String("https://fsn1.your-objectstorage.com"),

		Credentials: credentials.StaticCredentialsProvider{Value: aws.Credentials{
			AccessKeyID:     "YOUR-ACCESS-KEY",
			SecretAccessKey: "YOUR-SECRET-KEY",
		}},
	})

	// Create a new bucket
	_, err := client.CreateBucket(ctx, &s3.CreateBucketInput{
		Bucket: aws.String("my-bucket-2bc3a910"),
	})
	if err != nil {
		log.Fatal(err)
	}

	log.Printf("created bucket: my-bucket-2bc3a910")

	// Upload a small object
	body := strings.NewReader(strings.TrimSpace(`
# My object

This markdown document is the content of my object.
`))

	_, err = client.PutObject(ctx, &s3.PutObjectInput{
		Bucket:      aws.String("my-bucket-2bc3a910"),
		Key:         aws.String("my-object-e9be8f5c"),
		ContentType: aws.String("text/markdown"),

		Body: body,
	})
	if err != nil {
		log.Fatal(err)
	}

	log.Println("created object")
}

Using github.com/minio/minio-go/v7 (Golang):

Details

package main

import (
	"context"
	"log"
	"strings"

	"github.com/minio/minio-go/v7"
	"github.com/minio/minio-go/v7/pkg/credentials"
)

func main() {
	ctx := context.Background()

	// Create a new s3 client
	client, err := minio.New("fsn1.your-objectstorage.com", &minio.Options{
		Secure: true,
		Creds: credentials.NewStaticV4(
			"YOUR-ACCESS-KEY",
			"YOUR-SECRET-KEY",
			"",
		),
	})
	if err != nil {
		log.Fatal(err)
	}

	// Create a new bucket
	err = client.MakeBucket(ctx, "my-bucket-2bc3a910", minio.MakeBucketOptions{Region: "fsn1"})
	if err != nil {
		log.Fatal(err)
	}

	log.Printf("created bucket: my-bucket-2bc3a910")

	// Upload a small object
	body := strings.NewReader(strings.TrimSpace(`
# My object
	
This markdown document is the content of my object.
`))

	_, err = client.PutObject(ctx,
		"my-bucket-2bc3a910",
		"my-object",
		body,
		body.Size(),
		minio.PutObjectOptions{},
	)
	if err != nil {
		log.Fatal(err)
	}

	log.Println("created object")
}

Using boto3 (Python):

Details

import logging
from boto3 import client

# Create a new s3 client
s3 = client(
    "s3",
    region_name="fsn1",
    endpoint_url="https://fsn1.your-objectstorage.com",
    aws_access_key_id="YOUR-ACCESS-KEY",
    aws_secret_access_key="YOUR-SECRET-KEY",
)

# Create a new bucket
s3.create_bucket(Bucket="my-bucket-2bcxs3a0")

logging.info("created bucket: my-bucket-2bcxs3a0")

# Upload a small object
s3.put_object(
    Bucket="my-bucket-2bcxs3a0",
    Key="my-object",
    Body="""
# My object

This markdown document is the content of my object.
""",
)

logging.info("created object")

Let me know if you find a supported feature that is not working with one of the above SDKs.

EDIT: I misclicked the auto close button 😬

[hakman]

Just for some background, someone reached out to us to ask about why he is getting an error when using Hetzner object storage with kOps. So there may be some incompatibilities with various clients.

error listing s3://the-bucket-name/the-cluster-name.k8s.local/instancegroup: operation error S3: ListObjectsV2, https response error StatusCode: 404, RequestID: the-request-id, HostID: the-host-id, api error NoSuchKey: UnknownError

Could you please share a link to where this was reported? This would help us to take a look and check if it's a configuration issue or a genuine bug. Thanks!

@jooola Thank you for the examples and the list of supported features. I see that ListObjectsV2 is not on the list yet.

@LenzGr This was reported in the kubernetes/kops slack channel. I think Hetzner recommends using CAPI, but kOps does a pretty good job also.
https://kubernetes.slack.com/archives/C3QUFP0QM/p1730829144676499

[jooola]

The examples were published in the official documentation: https://docs.hetzner.com/storage/object-storage/getting-started/using-libraries

[hakman]

Thanks @jooola. Any plans to support ListObjectsV2?