Network roundtrips doubled for API requests

[petipp]

We have a SPA that uses pop-ups to establish web sessions (tracked in first-party cookies) to other user-defined domains and then uses CORS XHR/Fetch POST requests to execute queries on those remote systems.

In trying to enable the Storage Access API, we have created an iframe that requests access. However, the access request is scoped only to that specific iframe.

Somehow we need the ability to request access for the many network requests we will be making.

Will Storage Access Headers solve this?
On the surface it does look like it would functionally allow things to work by returning the correct headers. However, the requirement to double every network request (each one being first sent without cookies and then with cookies) will have terrible performance consequences particularly when the latency between browser and data source can be higher than 300ms. (e.g. 5 sequential network request = 1.5 seconds of extra waiting).

Two possible changes

1. Automatic Storage Access for CORS

The best option would be to address this by considering CORS to automatically request storage access for all API calls if configured with Access-Control-Allow-Credentials? The single OPTIONS request made at the beginning of the CORS workflow should be sufficient to determine that the data source requires cookies, and every API call thereafter should automatically be granted storage access if there is already a pending grant (i.e. created via an iframe calling requestStorageAccess).

2. Storage Access Headers Once per Window for API calls

As a compromise, the storage access headers could be required only for the first request sent to the given target API - and all further requests to the same top-level domain would benefit from the request.

[johannhof]

Thanks for the feedback @petipp, I discussed this with @cfredric and we think that both suggested alternatives might be viable, but would like to explore the second option first as it gives 3p embed developers a little more control over consistency and security of their apps.

cc @bvandersloot who was previously arguing for a closer integration with CORS and @arturjanc who has opinions on security things

[arturjanc]

@johannhof Can you outline in a little more detail how the "second option" would work here? Would we store the fact that a particular URL opted into receiving credentialed requests by setting the Activate-Storage-Access header, and avoid the extra round trip on subsequent requests to that URL from the same document? (I think this would have to be scoped to a document rather than window because we don't want other cross-site content embedded in the same window to inherit the capability.)

Re: "automatic storage access for CORS", Fetch already has the credentials toggle that lets the caller control whether cookies will be attached to the request, and requires server opt-in by setting Access-Control-Allow-Credentials: true and Access-Control-Allow-Origin. So I imagine it wouldn't be difficult to figure out how to integrate with CORS if that's what we decided to pursue.

[johannhof]

@arturjanc Yeah we're coming up with a new proposal right now, in my mind this might be an explicit developer opt-in ala Activate-Storage-Access: Sticky, we can discuss the desired scope of it :)

Right, I don't think it's hard to limit this access to CORS requests, we're already doing that with RWS / rSAFor today. I don't think that CORS in itself is sufficient for opt-in though, given the lack of preflights for most practically used requests.

[petipp]

@johannhof That's fair. FWIW, CORS would be sufficient for us because our API is POST-based so there's always a preflight.
(edit: Actually, I lied. There are also some GET requests at the beginning of the workflow, so we would need cookies for those, too.)

[johannhof]

Are you sure? My understanding is that POST isn't CORS-preflighted by default, but I haven't used it in practice in a long while.

[petipp]

I see. Yes, we definitely do have the preflight:

But looking further at it, it seems to be because of custom headers, if I read this correctly:

[johannhof]

Ah, yeah, that makes sense!

[sandormajor]

Hi @petipp,

We're thinking of the scope of the "stickiness" discussed above.
Making the activation sticky for an entire site or origin would be too broad probably – that would mean that a single endpoint could downgrade security for all other endpoints, potentially maintained by completely different teams.
On the other extreme, if we key the activation for the exact URL, that could result in a lot of retries if you use a lot of unique query params.

Would it be reasonable for your usecase to maintain stickiness for the exact URLs? including the query params and/or URL fragment

[petipp]

Hi @sandormajor
In the typical (optimal) case, our initial load sends two requests to different URLs under a common path (a/b/C, a/b/E). User interaction (and less-optimized initial load cases) will make many more requests using the second path.
So exact matches would be okay over time - but initial load is a very critical workflow for the user's perception of performance.

Would it be possible to aggregate them by a common path? Would that then require an additional (optional) to be returned with Activate-Storage-Access? e.g. something like this:
Activate-Storage-Access: retry; allowed-origin="https://foo.bar"; allowed-scope="a/b"

[arturjanc]

@petipp Would you be willing to share a typical flow of requests for your use case with real(-ish) looking URLs?

We're trying to understand whether scoping should support wildcards (e.g. so that anything under /a/b would receive credentialed requests) or if you could, for example, explicitly list individual endpoints for which storage access would apply (e.g. allowed-scope=/a/b/endpoint1,/a/b/endpoint2,...). The latter is more cumbersome, but also reduces the potential for misconfigurations (say, a developer setting allowed-scope=/) which would expose the application to cross-site vulnerabilities.

[petipp]

@arturjanc
This would be what we would typically do:

GET https://host:port/a/b/service/v2/GetServerInfo
POST https://host:port/a/b/service/v2/GetResponse
POST https://host:port/a/b/service/v2/GetResponse
POST https://host:port/a/b/service/v2/GetResponse

So if we could make the initial GET handle the SAH permissions for the POST requests then this would be perfect. So for our case a list of specific other endpoints would also work fine. I would prefer if the paths could be relative (e.g. ./GetResponse) just in case a reverse proxy is used.

[sandormajor]

Hi @petipp

Please take a look at #22 if it would work for you