Proposal: Allow servers to take full responsibility for cross-origin access protection

[RubenVerborgh]

TL;DR: Servers that explicitly take full control of cross-origin access protection, do not want the browser to handle this. Unfortunately, fully and indefinitely opting out is currently impossible.

Following the WHATWG procedure for feature proposals, this issue describes the problem and use case requirements, not a solution (yet).

Problem description

By default, browsers are responsible for protecting cross-origin access to resources. This mechanism was created to avoid scripts running on one origin from having undesired access to personalized content on on another origin.

Currently, there is no sustainable way for a server to take full responsibility of cross-origin access protection. Servers can selectively opt out of blocking behavior, but no opt-out mechanism is guaranteed to work for all current and future applications. As an example, we recently witnessed breakage of several legitimate applications that relied on a widely used server-side configuration from enable-cors.org, because the fetch specification had changed in subtle ways. The only resort is reconfiguring all servers, without guarantee that this will be a permanent solution.

While useful as a default, the browser’s exclusive and changing control of cross-origin access creates an undesired obstacle in two common scenarios:

1. when the server does not provide any personalization for a resource (“open data” or “public APIs”);

2. when a resource’s personalized behavior is secured through other means such as API keys or authentication headers sent by the requesting script (“authenticated APIs”).

Use cases

(1) A Web server does not provide any personalization of certain resources, and wants to make those available to any Web application, now and forever.

Requirements:

• The server must be able to indicate that it wants to take full control of cross-origin access protection
• The server must be able to state this in a future-proof way
• The server must be able to state this for specific resources
• The server must be able to receive and send any headers, requests, and responses allowed by the HTTP specification, for those specific resources, from and to any origin
• The server must be able to not positively respond to requests whenever it so prefers
• The server must not require reconfiguration when the fetch specification is updated

(2) A Web server has its own cross-origin authorization mechanism for certain resources, and wants to make those available to any Web application, now and forever.

Requirements:

• The server must be able to indicate that it wants to take full control of cross-origin access protection
• The server must be able to state this in a future-proof way
• The server must be able to state this for specific resources
• The server must be able to receive and send any headers, requests, and responses allowed by the HTTP specification, for those specific resources, from and to any origin
• The server must be able to not positively respond to requests whenever it so prefers
• The server must not require reconfiguration when the fetch specification is updated

(3) A Web application that makes cross-origin requests to public resources on a certain server wants to keep working, now and forever (given no changes on the server).

Requirements:

• The application must be able to set any headers on the request
• The application must be able to read any response sent by the server
• The application’s ability to do the above must not change over time

(4) A Web application that makes authenticated cross-origin requests to resources on a certain server wants to keep working, now and forever (given no changes on the server).

Requirements:

• The application must be able to set any headers on the request
• The application must be able to read any response sent by the server
• The application’s ability to do the above must not change over time

(5) A browser wants to move the responsibility for granting access to cross-origin resources to the server, when requested.

Requirements:

• The browser must be able to send a cross-origin request to the server unconditionally, when the server indicates that it takes full responsibility for cross-origin protection on that resource
• The browser’s ability to do the above must not change over time

(6) A browser wants to maintain the possibility of providing granular cross-origin access protection to servers that do not explicitly opt out of this protection

Requirements:

• Existing browser-based cross-origin protections must continue working
• Existing granular CORS mechanisms must continue working

(7) A server developer wants a dedicated mechanism for taking server-side responsibility for cross-origin access control

Requirements:

• The developer must understand what the mechanism does, and what responsibilities come with it
• The developer must not rely on mechanisms that have a related, but more granular meaning
• The developer must not abuse existing mechanisms to achieve this effect
• The developer must be able to do this in a way that does not change over time

(8) A developer website (such as developer.mozilla.org) wants to document a future-proof way of taking server-side responsibility for cross-origin access protection.

Requirements:

• These instructions must not change over time, or when the fetch specification is updated

Shortcomings of current mechanisms

Currently, when trying to address the above use cases, servers must resort to multiple HTTP header settings that eliminate cross-origin request blocking by enabling Cross-Origin Resource Sharing (CORS). In contrast, their actual goal is to request full responsibility for this protection. Therefore, the fact that only fine-grained settings are available is problematic, because:

1. It is complex to indicate that a server requests responsibility for all current cross-origin requests, as this configuration involves an interplay of several HTTP headers with subtle edge cases.

2. It is impossible to indicate that a server requests responsibility for all future cross-origin requests, because of continuing changes to the fetch specification that tighten the mechanism. As such, legitimate Web applications relying on CORS can break at any time.

Clearly, this complexity and progressive tightening are beneficial for the protection of the user and servers in general. However, this proposal argues that there are many common cases where the server explicitly wants to take that protection in its own hands: public data, open APIs, authenticated APIs.

Recent changes in the fetch specification broke widely deployed instructions on how to disable CORS. For instance, Web applications using HTTP requests with long headers suddenly stopped working in 2018/2019 after browser updates, even though their servers followed configuration instructions with the explicit intention of this not happening. Fixing those applications requires changes on the server side. Getting all affected Web servers updated is expensive and will likely take several years, and there is no guarantee that such an update will not be obsoleted again. It is unsure whether troubled servers will be updated in timely manner or at all, because blocked requests do not show up in server logs, so servers have no way of knowing that applications have trouble accessing their resources.

Note that we are not arguing against past or future changes to CORS. For security reasons, it is beneficial and necessary that the fetch spec keeps on updating. Rather, we are arguing for the existence of cases in which the server a) does not need that security because it it an open API, or b) is already taking the burden of security by authenticating cross-API requests in different ways.

We thus argue that a considerable number of servers emitting the Access-Control-Allow-Origin: * header actually aim to express their wish to take control of cross-origin request protection themselves, and thus for the browser to fully delegate that responsibility, instead of the much more nuanced and limited meaning this header actually has. These servers thus need a proper way of expressing this, without having to rely on the misinterpretation of an existing header.

Current Web applications

The following Web applications seemingly have the intention of requesting full server-side control of cross-origin access protection. Instead, they resort to workarounds which, as argued above, can break at any point (and, in multiple cases, are currently broken):

• The Mapbox API: https://docs.mapbox.com/api/?language=JavaScript
• The Github API: https://developer.github.com/v3/#cross-origin-resource-sharing
• OpenDataSoft’s product explicitly publishes datasets allowing all origins for their clients: curl 'https://data.opendatasoft.com/api/records/1.0/search/?dataset=gare-de-train-et-tramway%40sarthe&facet=nom&facet=ligne&facet=typ_gare' -I
• The Solid project by Tim Berners-Lee https://solid.github.io/web-access-control-spec/Background#cross-origin-resource-sharing-cors
• All open data servers
  • data.gov
  • https://www.geonames.org/
  • See more at https://www.w3.org/wiki/CORS_Enabled
• public APIs
  • many of the APIs at https://github.com/toddmotto/public-apis
• dozens of additional examples available on request

Authors of this proposal: Ruben Verborgh (@RubenVerborgh) and Pieter Colpaert (@pietercolpaert).

[jakearchibald]

Is this a serious proposal for something new, or mostly just a complaint about CORS changing? (I'm not saying the complaint isn't valid)

[RubenVerborgh]

This is a very serious proposal. We are working with public APIs, and recently saw applications break. We want a sustainable mechanism for the server to take control of cross-origin protection. (For insight into our serious intentions, see the discussions in #865 and #862; the above proposal is the result of several weeks of drafting and rewriting).

[jakearchibald]

But the reason you don't see CORS as the solution is (1) multiple headers needed and (2) recent changes to CORS, or am I missing something?

[RubenVerborgh]

As described in Shortcomings of current mechanisms above, the problem is 1) the complexity (which is broader than the need for multiple headers; also the fact some of those headers need to be reactive, which is harder to configure in NGINX/Apache), but mainly 2) that no such configuration is a final solution. We understand that the fetch spec is in evolution, as it should be, but this currently means that all public APIs and authenticated APIs have to keep on updating to be able to function like before. And since the solution is on the server side, Web apps have no control over breakage.

[jakearchibald]

I don't think any solution can be final. If you invent yet another opt-in, say Allow-Superpowers-And-I-Really-Mean-It: honestly, and 5 years later a new capability is released that would create a vulnerability on 5% of those opt-in sites, you then need yet another opt-in.

The CORS change seems bad, but I don't see yet another opt-in making things easier. Another opt-in has the same server-updating problem.

[RubenVerborgh]

I don't think any solution can be final.

That would make stable public APIs and authenticated APIs an impossibility. I think we can do better.

If you invent yet another opt-in, say Allow-Superpowers-And-I-Really-Mean-It: honestly, and 5 years later a new capability is released that would create a vulnerability on 5% of those opt-in sites

The trick is in letting server operators understand exactly what they are opting in to. I challenge anyone to ask 10 server operators what they are opting in to when they are providing Access-Control-Allow-Origin: *. They will likely not know.

So this is not about opting in to certain features. It is a matter of saying Resource-Access-Protection-Responsibility: server or Resource-Personalization: none or Resource-Authentication-Responsibility: server. Providing a Web API that you want to be accessible from any Web app, regardless of what happens, seems like a very reasonable requirement.

The CORS change seems bad, but I don't see yet another opt-in making things easier. Another opt-in has the same server-updating problem.

So let's not have another opt-in then, but rather something sustainable.

[jakearchibald]

That would make stable public APIs and authenticated APIs an impossibility.

I don't think that's true.

So let's not have another opt-in then, but rather something sustainable.

What's your proposal, if not an opt-in?

[RubenVerborgh]

That would make stable public APIs and authenticated APIs an impossibility.

I don't think that's true.

It is: dozens of Web APIs are running that have followed https://enable-cors.org/ with the intention of working from any Web application. These instructions no longer hold, so those APIs need an update on the HTTP level (= not stable). Furthermore, any such changes can happen in the future, so APIs will have to make adjustments (= not stable). Hence, I conclude that stable public APIs are not a possibility if servers do not have the ability to explicitly take responsibility for cross-origin access protection.

What's your proposal, if not an opt-in?

A mechanism by which a server can say "I am taking care of all current and future cross-origin protections for this resource", and/or "this is a non-personalized resource". So not a matter of opting in or out from specific features (which CORS is designed for, and still useful for), but rather deciding who assumes that responsibility. As such, servers accessed from the browser would have the same protections as when accessed from the command line or a native application. So apps can do nothing from the browser which they wouldn't be able to do through other means anyway.

[jakearchibald]

dozens of Web APIs are running that have followed https://enable-cors.org/ with the intention of working from any Web application. These instructions no longer hold

I don't know the history behind the CORS change. It seems pretty bad that a large breaking change was made. But, if that change was justified, similar future issues would be a problem for your proposal too.

What's your proposal, if not an opt-in?

A mechanism by which a server can say "I am taking care of all current and future cross-origin protections for this resource"

You're describing an opt-in.

[RubenVerborgh]

But, if that change was justified, similar future issues would be a problem for your proposal too.

My proposal/request is exactly to be immune from such issues. Probably some security bug exists where long headers cause problem X or Y (@annevk couldn't publicly comment); hence the change. I'm describing a mechanism where the server says: you know what, if X or Y indeed are problems, then nothing additional is compromised compared to doing the same attack from the command line or a native app. Hence, I don't rely on the browser security mechanism. (This is very different from cases where, let's say, an API does cookie-based authentication, in which browser-based requests would have a privilege over other means.)

You're describing an opt-in.

Given that any mechanism could be characterized as opt-in (e.g., when I'm Accepting text/json, I'm opting in to JSON), I'll need better definitions to meaningfully address your concern with an opt-in. I do not share the view that opt-ins will necessarily need adjustments, if a server knows exactly what it is opting in or out of, and if that is broad enough to cover the above cases.

[jakearchibald]

My proposal/request is exactly to be immune from such issues

This guarantee cannot be made. If a vulnerability is discovered that puts real users at risk, browsers will fix that to protect users.

If the spec doesn't update, then the spec won't reflect reality.

If a browser doesn't update, users and competitors will rightly accuse that browser of being less secure than other browsers.

[jakearchibald]

Given that any mechanism could be characterized as opt-in

That isn't true. Only opt-ins can be characterised as opt-in. For instance, if you added this new behaviour by default, with a way for a server to say "I don't want this", then your proposal isn't opt-in (and an opt-out is provided).