Suggested improvement to biscuit / bwks specifications

[pcolmer]

At the moment, both specifications are unopinionated about how a consumer of biscuits should determine the public key used to encrypt a biscuit. The bwks specification goes some way to help solve that but limits itself with "Deciding which domain name to trust for a given token is still the responsibility of the relying party and has to happen out-of-band."

We are looking at building a series of micro-services that can pass around biscuits for consistent authorization. To handle that, there is a single source of truth for identity management, authentication and authorization.

Further down our roadmap, we envisage the possibility of whiteboxing this code, allowing customers to have the services labelled with their own identity and connected to their own URLs. That then means that a customer's version of the services would have its own "single source of truth". Not the end of the world - just configure the micro-services to pick up the bwks file from a URL tailored to that customer.

But what happens if we continue to have other micro-services that the whiteboxed platform consumes or interacts with? Then we're faced with the very real challenge of how our micro-services can determine who signed that biscuit.

And so to my suggestion...

1. A small change to the biscuit schema:

message Biscuit {
  optional uint32 rootKeyId = 1;
  required SignedBlock authority = 2;
  repeated SignedBlock blocks = 3;
  required Proof proof = 4;
  optional string domain = 5;
}

The optional domain string specifies the URL domain for the location of the bwks file, e.g. example.org

2. A variant of the existing function to create an authorizer. Instead of passing the known public key, we pass an array/list of acceptable domains. The function starts by checking that (a) there is a domain string in the biscuit and (b) that domain is in the list of acceptable domains. If those checks pass, the function attempts to retrieve the bwks from the specified domain. It then uses rootKeyId to retrieve the public key. Execution then continues as normal.

This suggestion is an extension to how things work today. I don't forsee any breaking changes to existing client code since this would be an optional alternative to the existing authorizing mechanism.

If it is undesirable to change the biscuit schema, an alternative would be to store a domain fact in the biscuit block and the bwks authorizer could look for that fact.

While this approach does make things a bit more opinionated (for example, there would need to be agreement that the protocol is HTTPS and that the bwks file lives at a known fixed location in the specified domain), making that specification avoids the problem of wheels being reinvented everytime someone wants to use biscuits and has to fill in the gaps.

[Geal]

there might be something I am missing here, so please tell me where I am turning the wrong way:

• the microservices can be started with a list of allowed domains, so they can be configured
• that means they can be configured with a list of bwks URLs, and they can be downloaded on startup (and refreshed regularly)
• try to verify the token signature using each of the bwks (using the root key id to select them if provided)
• if the microservice needs to know the token's origin for something else than signature verification (ex: to load some specific policies), now you know which one matched

Architectures where you would download the list of keys on demand are generally dangerous, because it is then easy to fall into the flaw of hammering on the identity provider, or worse, starting to download from random URLs during verification. Root keys should be loaded out of band, not in the hot path (which is why the RootKeyProvider in rust is not async)

[pcolmer]

I can identify one main philosophy difference here ... your comment seems to suggest an assumption that the microservices start up and then continue running. I'm inferring that from your suggestion that the bwks can be downloaded on startup and refreshed regularly.

Our code is running as AWS Lambdas, triggered by API Gateway. There isn't any persistence. Given the bwks specification, we were going to follow a similar approach to the one we use when verifying Auth0 JWTs ... you query the well known URL at the point of verifying the token.

In a stateless system, fetching bwks from multiple URLs potentially has a serious performance impact, hence the suggestion of using a domain mechanism.

I guess we could tackle it by using an out of band process for pulling the keys into, say, AWS Secrets Manager for the Lambda code to cycle through each public key in turn, but that just doesn't feel elegant. It feels almost brute force when one considers that the biscuit knows the root key ID - it just doesn't (currently) know who created it.

starting to download from random URLs during verification

That shouldn't happen given the proposal that the client knows which domains are acceptable and the authorizer would only fetch the bwks if the client's domain list includes the domain recorded in the biscuit.

Part of the reason for proposing this was because the Golang library doesn't support unauthenticated inspection of a biscuit. So we aren't able to retrieve the domain fact from the biscuit without creating an authorizer first, which requires the public key ...

[Geal]

I can identify one main philosophy difference here ... your comment seems to suggest an assumption that the microservices start up and then continue running. I'm inferring that from your suggestion that the bwks can be downloaded on startup and refreshed regularly.

ok, your question makes more sense in the context of lambdas. I would still caution against downloading the keys on demand. While providers like Auth0 can handle a reasonable traffic, most IdPs tend to rate limit JWKS downloads, because the IdP is not supposed to bear the same load as the application.

I guess we could tackle it by using an out of band process for pulling the keys into, say, AWS Secrets Manager for the Lambda code to cycle through each public key in turn, but that just doesn't feel elegant. It feels almost brute force when one considers that the biscuit knows the root key ID - it just doesn't (currently) know who created it.

honestly that's the kind of solution that would feel more reasonable to me. Even if it's not elegant, you don't run the risk of authorization failing if an easily cacheable file cannot be downloaded.

Part of the reason for proposing this was because the Golang library doesn't support unauthenticated inspection of a biscuit. So we aren't able to retrieve the domain fact from the biscuit without creating an authorizer first, which requires the public key ...

This is a more important issue. In the rust library, we have the UnverifiedBiscuit structure for that, and token blocks contain a context field that you could use to recognize token origin. The UnverifiedBiscuit API tends to be strict in preventing access to block data before verifying signature, but maybe that could be relaxed for the context field. Anyway, it should be added to the Go library

[pcolmer]

@Geal I understand your concerns about downloading the bwks too frequently.

How about taking your philosophy and using it to influence my suggestion? Rather than passing an array of domain strings, how about passing a dict of bwks blobs, with the keys being the domain strings? For example:

{
   "example.org":     [
      {
         "algorithm": "ed25519",
         "key_bytes": "edaabea9448310d5b54874a0ca5e431b4fed84f949d311c17945104f32afd77c",
         "key_id": "de76f169-22ed-4b29-88f4-3d98c0604067",
         "expires_at": 1704480249918
      },
      {
         "algorithm": "ed25519",
         "key_bytes": "1b4fed84f949d311c17945104f32afd77cedaabea9448310d5b54874a0ca5e43",
         "key_id": "3d98c0604067-4b29-88f4-de76f169-22ed",
         "expires_at": 1703370138818
      }
   ]
}

So code (call it the Client for reference later) wanting to creating an authorizer for an incoming biscuit will already have the bwks blobs for each of the domains they support. The code passes that dict to a new "authorize with bwks" function. That function tries to match the dictionary keys against the domain fact and, if no match, authorization fails.

If the domains match, the next test is the root key id. Again, if no match, authorization fails.

If the IDs match then the authorizer knows which public key to try ... and if that fails, authorization fails.

Otherwise, I'm struggling to see the value of the bwks file. Yes, it is a means of publishing the public key but the code wanting to create an authorizer needs to try each public key in turn which seems a waste of time when the biscuit (optionally) contains the root key ID which should match the key ID.

using the root key id to select them if provided

But reading the root key ID presumably requires the unverified access, along with the Client then needing to understand the internal structure of a biscuit ... something that then brings the added complexity of needing to update the Client when a new version of the biscuit structure is released. If the libraries instead are responsible for understanding the biscuit structure, getting them to process bwks blobs would seem to take that responsibility away from the Client in a clean way.