Header on received data

[Jorckx]

Is there a way to add a custom header to te received data (response header)? I only seem to be able to add a custom header right after the query inside the composable itself (request header?), but I specifically need to control the header of the received data. I've tried by adding a rule to my ngnix-config (kirby cms) for each api-call '/api/', but that seems to be ignored. Is that due to the way nuxtKql proxies it?

[johannschopplich]

Hi there!
Actually, response headers are forwarded to the client. Can you share a minimum reproducible example in which case they are not working? What exactly is your usecase?

[Jorckx]

Usecase: https://staging.kasperdevos.com/

The backend (build with the kirby-headless starter) is deployed with Ploi on digitalocean. I've adde these to the site's Nginx config:

location ~ \.php$ {
   ...
   if ($request_uri ~ ^/api/) {
      add_header 'Cache-Control' 'public, max-age=0, immutable';
      add_header 'Cloudflare-CDN-Cache-Control' 'public, max-age=31536000, immutable';
   }
}

The idea is to catch the queries with a cache everything page-rule. The reason being that I've created a hook in kirby to purge my cache when updating the content. The cache-rule makes the pages load instantly, but when using hydration while navigating to other pages I want the queries to be available from the cache as well. I've tried the KV-storage (as suggested: https://nuxt-kirby.byjohann.dev/guides/caching-strategies.html#server-side-caching), and it works to store the queries. However, I don't seem to be able to revalidate them the same way as with a hook… And I don't like to rely on TTL. The purge-hook works fine though. So i've figured, since the queries are just plain json files, it should also be possible to cache them the same way with the cache-everything page-rule. Does that make sense?

[johannschopplich]

I get it, good idea! Where do you deploy your frontend? During hydration you don't have to worry about the cache, since the server-side query result is written to the payload and received from it during client-side hydration.

I don't know how Cloudflare-CDN-Cache-Control works. But does Cache-Control work with POST requests? I believe it's not as simple as that.

The headers from Kirby are forwarded to Nuxt, so this should not be the issue:

nuxt-kirby/src/runtime/server/handler.ts

Line 119 in eee5567

for (const [key, value] of response.headers) {

[Jorckx]

The frontend is deployed to Cloudflare pages ;) Interesting! I've read that POST requests are not cached by default with cloudflare… however, they offer an example code for caching: https://developers.cloudflare.com/workers/examples/cache-post-request/

But I haven't figured out how to implement it with the route as setup with your plugin. In hindsight… kv-storage might be a more solid cache strategy after all… but then I need to figure out how to 'purge' these by a hook instead of relying on TTL… Clients don't like to here that they have to wait for changes to appear.

[johannschopplich]

Interesting! I wasn't aware of the caches feature. That's actually great, since I have the exact same limits you ran into: TTL vs. cache invalidation.

Thanks for that article. When deployed to Cloudflare, I might add support for query caching on a worker level in the future. Love that idea.

[Jorckx]

Yes it is :) I got the idea from a befriended developer who works with sveltekit, but builds his queries as direct get-requests, so He didn't bumb into that post-cache issue. That got me searching when I found that article :)

In the meantime I have a follow-up question regarding the kv-storage method. Will it work out of the box when adding it in the config-file? Or should I include the cached key in the queries themself to look for the right queries in the storage? I bet it's already automated the same way the regular the client-side caching works? As a fallback solution I've made an additional file to bulk-delete my kv-storage:

<?php
function deleteCloudflareKvStorage()
{
    $cloudflareAccountId = ${ACCOUNT_ID};
    $cloudflareAPIKey = ${API_KEY};
    $kvNamespaceId = ${KV_NAMESPACE_ID};
    $cloudflareEmail = ${CLOUDFLARE_EMAIL};

    $allKeys = [];
    $cursor = null;

    // Step 1: Get ALL keys from the namespace (with pagination)
    do {
        $listUrl = "https://api.cloudflare.com/client/v4/accounts/{$cloudflareAccountId}/storage/kv/namespaces/{$kvNamespaceId}/keys?limit=1000";
        
        if ($cursor) {
            $listUrl .= "&cursor=" . urlencode($cursor);
        }
        
        $ch = curl_init($listUrl);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLOPT_HTTPHEADER, [
            'X-Auth-Email: ' . $cloudflareEmail,
            'X-Auth-Key: ' . $cloudflareAPIKey,
            'Content-Type: application/json',
        ]);

        $response = curl_exec($ch);
        $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        curl_close($ch);

        if ($response === false) {
            error_log('Failed to list KV keys: ' . curl_error($ch));
            return;
        }

        $result = json_decode($response, true);
        if ($httpCode !== 200 || !($result['success'] ?? false)) {
            error_log('Failed to list KV keys: ' . json_encode($result));
            return;
        }

        // Add keys from this page to our array
        $pageKeys = array_column($result['result'] ?? [], 'name');
        $allKeys = array_merge($allKeys, $pageKeys);
        
        // Get cursor for next page
        $cursor = $result['result_info']['cursor'] ?? null;
        
    } while ($cursor);

    // Check if there are any keys to delete
    if (empty($allKeys)) {
        error_log('No KV keys found in namespace');
        return;
    }

    error_log('Found ' . count($allKeys) . ' keys to delete');

    // Step 2: Delete ALL keys in bulk
    $deleteUrl = "https://api.cloudflare.com/client/v4/accounts/{$cloudflareAccountId}/storage/kv/namespaces/{$kvNamespaceId}/bulk";
    
    $ch = curl_init($deleteUrl);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, [
        'X-Auth-Email: ' . $cloudflareEmail,
        'X-Auth-Key: ' . $cloudflareAPIKey,
        'Content-Type: application/json',
    ]);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'DELETE');
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($allKeys));

    $response = curl_exec($ch);
    $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);

    if ($response === false) {
        error_log('Failed to delete KV storage: ' . curl_error($ch));
    } else {
        $result = json_decode($response, true);
        if ($httpCode === 200 && ($result['success'] ?? false)) {
            error_log('KV storage successfully cleared. Deleted ' . count($allKeys) . ' keys.');
        } else {
            error_log('Failed to delete KV storage: ' . json_encode($result));
        }
    }
}

I prefer a plain 'bulk deletes' and 'purge everything', since the website's I build don't have many pages and don't change that often. and after a first slow pageload (which is mostly done by the person who updates the cms anyway), the page is freshly cached again :)

[johannschopplich]

Once you have set up the KV cache, it will work out of the box. The cache key is built by the module based on the hashed query. You don't need to build it yourself.

[Jorckx]

Ah sweet! And since I'm prone to checking my network tab for feedback… how do I spot when it's cached over kv storage? I've set it up, and it actually creates the queries as kv's when routing. But then the cf-cache-status for these queries are still set to DYNAMIC instead of HIT (or does kv work differently?)

[johannschopplich]

You can't spot it in the network tab, KV storage readout is done in the worker itself that has access to the storage. However, you can check out the reads in the KV storage metrics in the Cf dashboard.

[Jorckx]

oh off-course! That's just great for now :) but being able to cache the POST… that would indeed be more elegant I think. If I find a way to do it, I'll try to put it in a pull-request. Just one more thing: I'd like to share a kv-storage with the staging and production site. will the hashed keys be the same? I don't know how random these are, not so familiar with how hash works.

[johannschopplich]

Go ahead, PRs welcome!

The hashed keys are deterministic based on the query. So if the queries sent to Kirby are identical in staging vs. production, the hashes are as well.