Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 19 additions & 8 deletions documentation/topics/direct-uploads.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,16 @@ Direct uploads let clients upload files straight to your storage backend (S3, Mi

The flow has two steps:

1. **Prepare** — your server creates a blob record and returns a presigned URL
2. **Attach** — the client uploads to the presigned URL, then includes the `blob_id` in a normal create/update action
1. **Prepare** — your server creates a blob record and returns a signed upload target
(for example, an S3 presigned URL or an Azure SAS URL)
2. **Attach** — the client uploads to that signed target, then includes the `blob_id`
in a normal create/update action

## Setup

Direct uploads require a storage service that supports presigned URLs. `AshStorage.Service.S3` and `AshStorage.Service.AzureBlob` support this out of the box:
Direct uploads require a storage service that can delegate client uploads via signed URLs or forms.
`AshStorage.Service.S3` supports S3 presigned URLs/forms, and
`AshStorage.Service.AzureBlob` supports blob-level Azure SAS URLs:

```elixir
storage do
Expand Down Expand Up @@ -44,11 +48,18 @@ end

Azure browser uploads require CORS on the storage account and the `x-ms-blob-type: BlockBlob` request header. `AshStorage.Service.AzureBlob.direct_upload/2` includes that header in the returned upload info.

Azure terminology note: Azure SAS URLs can be handed to clients for a specific blob,
so they cover AshStorage's single-blob read and direct `Put Blob` upload flows. They
are not a perfect feature-for-feature match for S3 request presigning: this service
does not currently support block-level/per-part upload signing, resumable uploads,
or Azure AD/user-delegation SAS generation.

Azure-specific checklist:

- Create the blob container before uploads begin; the service creates blobs, not containers.
- Configure Blob service CORS for your browser origin, `PUT`/`OPTIONS`, and the request headers your client sends, including `x-ms-blob-type` and `content-type`.
- Prefer `:account_key_env` or `:sas_token_env` over literal secrets. Literal `:account_key` and `:sas_token` values work for immediate service requests but are not persisted on blob records. Use env-backed credentials for attachment flows that later operate from stored blobs, including purge, analysis, and variants.
- If shared-key access is disabled on the storage account, configure a pre-generated SAS with `:sas_token_env` for now; managed identity / Azure AD user-delegation SAS support is not implemented yet.
- If you provide a static SAS token via `:sas_token`/`:sas_token_env`, it is reused for every Azure operation. Use a container/account SAS with the needed permissions: read (`r`) for URLs/downloads/existence checks, create/write (`c`, `w`) for uploads/direct uploads, and delete (`d`) for purges.

Add the `AshStorage.Changes.AttachBlob` change to your create or update actions:
Expand Down Expand Up @@ -78,7 +89,7 @@ When `cover_image_blob_id` is `nil`, the change is skipped. For `has_one_attache

## Step 1: Prepare the upload

Call `prepare_direct_upload/3` to create a blob record and get a presigned URL:
Call `prepare_direct_upload/3` to create a blob record and get a signed upload URL:

```elixir
{:ok, %{blob: blob, url: url, method: method}} =
Expand All @@ -92,7 +103,7 @@ Call `prepare_direct_upload/3` to create a blob record and get a presigned URL:
The returned map contains:

- `:blob` — the created blob record (not yet attached to anything)
- `:url` — the presigned URL to upload to
- `:url` — the signed URL to upload to (an S3 presigned URL or Azure SAS URL)
- `:method` — `:put` or `:post` depending on configuration
- `:headers` — headers to include with the direct upload, when required by the service

Expand All @@ -112,7 +123,7 @@ The JS example below is for presigned PUT. For presigned POST, build a `FormData

## Step 2: Client uploads and attaches

Send the presigned URL and blob ID to your client. The client uploads directly to storage, then includes the blob ID in a normal create or update:
Send the signed URL and blob ID to your client. The client uploads directly to storage, then includes the blob ID in a normal create or update:

```javascript
// 1. Upload directly to storage
Expand Down Expand Up @@ -258,7 +269,7 @@ end
The client flow:

1. `POST /posts/prepare_cover_image_upload` with `{"filename": "photo.jpg"}` — returns `{"blob_id": "...", "url": "https://...", "method": "PUT", "headers": {}}`
2. `PUT` the file directly to the presigned storage URL with any returned headers
2. `PUT` the file directly to the signed storage URL with any returned headers
3. `POST /posts` with `{"title": "My Post", "cover_image_blob_id": "..."}` — creates the post with the image attached

## AshGraphql
Expand Down Expand Up @@ -306,7 +317,7 @@ mutation {
}
}

# 2. Upload directly to the presigned URL (outside GraphQL)
# 2. Upload directly to the signed URL (outside GraphQL)

# 3. Create with blob attached
mutation {
Expand Down
3 changes: 2 additions & 1 deletion lib/ash_storage/changes/attach_blob.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ defmodule AshStorage.Changes.AttachBlob do
An action change that attaches a pre-uploaded blob (by ID) to the record.

Used for direct uploads where the file was uploaded directly to storage
(e.g. via a presigned S3 URL) and the blob record already exists.
(e.g. via an S3 presigned URL or Azure Blob SAS URL) and the blob record already
exists.

create :create do
accept [:title]
Expand Down
6 changes: 4 additions & 2 deletions lib/ash_storage/operations.ex
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,11 @@ defmodule AshStorage.Operations do
end

@doc """
Prepare a direct upload: create a blob record and return a presigned URL.
Prepare a direct upload: create a blob record and return a signed upload target.

This stays as a standalone function since no parent record is involved yet.
The returned target is service-specific, such as an S3 presigned URL/form or an
Azure Blob SAS URL. This stays as a standalone function since no parent record
is involved yet.
"""
def prepare_direct_upload(resource, attachment_name, opts \\ []) do
with {:ok, attachment_def} <- Info.attachment(resource, attachment_name),
Expand Down
6 changes: 3 additions & 3 deletions lib/ash_storage/service.ex
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,11 @@ defmodule AshStorage.Service do
@callback delete_many([key()], Context.t()) :: :ok | {:error, term()}

@doc """
Generate a presigned URL or form for direct client-side upload.
Generate a signed URL or form for direct client-side upload.

Returns a map with at minimum a `:url` key. Depending on the service,
it may also include `:headers` (for presigned PUT) or `:fields` (for
presigned POST/form uploads).
it may also include `:headers` (for signed PUT URLs such as S3 presigned
URLs or Azure SAS URLs) or `:fields` (for S3 presigned POST/form uploads).
"""
@callback direct_upload(key(), Context.t()) ::
{:ok, map()} | {:error, term()}
Expand Down
8 changes: 7 additions & 1 deletion lib/ash_storage/service/azure_blob.ex
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,12 @@ if Code.ensure_loaded?(Req) do
read (`r`) for downloads/URLs/existence checks, create/write (`c`, `w`) for
uploads and direct uploads, and delete (`d`) for purge/delete.

Azure SAS URLs can be given to clients for a specific blob, which covers this
service's read URLs and single-request direct uploads. They are not identical
to S3 request presigning: this service does not currently support
block-level/per-part upload signing, resumable uploads, or Azure AD/user
delegation SAS generation.

For browser direct uploads, configure CORS on the storage account to allow your
application origin, the `PUT`/`OPTIONS` methods, and the request headers your
client sends, including `x-ms-blob-type` and `content-type`.
Expand Down Expand Up @@ -195,7 +201,7 @@ if Code.ensure_loaded?(Req) do
end

@doc """
Generate a presigned PUT URL for direct client-side upload.
Generate a SAS-signed PUT URL for direct client-side upload.

Azure Blob Storage direct uploads require the `x-ms-blob-type: BlockBlob`
header. The returned map includes this header.
Expand Down
11 changes: 6 additions & 5 deletions test/ash_storage/service/azure_blob_integration_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ defmodule AshStorage.Service.AzureBlobIntegrationTest do
"#{@endpoint_url}/#{@container}/folder/my%20file.txt"
end

test "generates a presigned URL that works" do
test "generates a SAS URL that works" do
key = unique_key()
AzureBlob.upload(key, "presigned content", ctx())

Expand All @@ -188,7 +188,7 @@ defmodule AshStorage.Service.AzureBlobIntegrationTest do
assert {:ok, %{status: 200, body: "presigned content"}} = Req.get(url)
end

test "presigned URL respects expires_in" do
test "SAS URL respects expires_in" do
key = unique_key()
AzureBlob.upload(key, "expiry test", ctx())

Expand Down Expand Up @@ -399,7 +399,7 @@ defmodule AshStorage.Service.AzureBlobIntegrationTest do
post = Ash.load!(post, :avatar_url)
assert post.avatar_url == "#{@endpoint_url}/#{@container}/#{blob.key}"

# Presigned URL calculation via config override.
# SAS URL calculation via config override.
Application.put_env(:ash_storage, AshStorage.Test.ConfigurablePost,
storage: [
service: {AshStorage.Service.AzureBlob, Keyword.put(@service_opts, :presigned, true)}
Expand All @@ -410,6 +410,7 @@ defmodule AshStorage.Service.AzureBlobIntegrationTest do
post = Ash.get!(AshStorage.Test.ConfigurablePost, post.id)
post = Ash.load!(post, :avatar_url)
assert post.avatar_url =~ "sig="
assert {:ok, %{status: 200, body: "azure file content"}} = Req.get(post.avatar_url)

# Purge should remove from Azure Blob Storage. The persisted service opts only
# contain the env var name, so this also verifies env-backed async credentials.
Expand Down Expand Up @@ -483,7 +484,7 @@ defmodule AshStorage.Service.AzureBlobIntegrationTest do
]
)

# Step 1: Prepare direct upload — creates blob, gets presigned URL and headers.
# Step 1: Prepare direct upload — creates blob, gets SAS URL and headers.
{:ok, %{blob: blob, url: upload_url, method: :put, headers: headers}} =
AshStorage.Operations.prepare_direct_upload(
AshStorage.Test.ConfigurablePost,
Expand All @@ -498,7 +499,7 @@ defmodule AshStorage.Service.AzureBlobIntegrationTest do
assert upload_url =~ "sig="
assert headers["x-ms-blob-type"] == "BlockBlob"

# Step 2: Client uploads directly to Azure using the presigned PUT URL.
# Step 2: Client uploads directly to Azure using the SAS-signed PUT URL.
assert {:ok, %{status: status}} =
Req.put(upload_url, body: "direct content", headers: headers)

Expand Down
Loading