HTTP header categorization

[wbamberg]

This is related to mdn/content#39902.

The HTTP documentation includes various categorizations of headers, which are currently glossary entries. I'm going to omit the "forbidden" and "CORS safelisted" categories, because I think they're unproblematic. That leaves:

https://developer.mozilla.org/en-US/docs/Glossary/Request_header
https://developer.mozilla.org/en-US/docs/Glossary/Response_header
https://developer.mozilla.org/en-US/docs/Glossary/Representation_header
https://developer.mozilla.org/en-US/docs/Glossary/Content_header
https://developer.mozilla.org/en-US/docs/Glossary/Fetch_metadata_request_header
https://developer.mozilla.org/en-US/docs/Glossary/General_header (obsolete)
https://developer.mozilla.org/en-US/docs/Glossary/Entity_header (obsolete)

We also have "client hints", which doesn't have a glossary entry but appears in similar contexts to these.

As far as I know the main place this is surfaced is in the "Header type" row in the table in each reference page:

It's also used in the preamble sentence in the page:

I'd like to understand better which categories we expose here, and which we should expose.

What we do now

I went through all the header pages, and noted what we say for "header type": https://docs.google.com/spreadsheets/d/1llnxSjB9fu4TEMx5VrGOJFqh2PgIx7Wn_u5lD6DhCKU/edit?usp=sharing.

We can see a few things:

• many headers are just marked as request or response headers, or both.
• some headers are marked as only representation headers
• some headers are marked as request, response, and representation headers
• some headers are marked as request, response, and content headers
• some headers are marked as only fetch metadata request headers
• some headers are marked as request headers and client hints

Is this correct? For example, is it meaningful that some headers are representation headers only, and some are representation headers and request/response headers?

Is it helpful? Is the taxonomy we're exposing here useful to developers?

I think that to be helpful, a categorization should:

• be clearly defined: we need to know, for a given item, whether it is definitively in that category
• make a difference to developers

Request and Response headers

These seem pretty clear. Even I understand that some headers are request headers, some are response headers, and some can be both. But should every header fall into one of these categories? Naively I would have thought so, but our docs do not agree:

• Content-Type is listed only as a representation header, both in the table and in the intro sentence
• Repr-Digest is listed only as a representation header in the table, but as a request/response header in the intro sentence.

The glossary pages for request and response headers do touch on this:

Not all headers appearing in a response are categorized as response headers by the specification. For example, the Content-Type header is a representation header indicating the original type of data in the body of the response message (prior to the encoding in the Content-Encoding representation header being applied). However, "conversationally" all headers are usually referred to as response headers in a response message.

Is this a helpful distinction for our docs to make? Is it the case that some representation headers are also request/response headers, and some aren't? If so (and if our current docs are correct, it is), where in the spec is this indicated?

Representation headers

These are described in the glossary page. But for me at least it's quite hard to use this quite abstract definition to know whether a particular header is a representation header or not (and if it is only a representation header and not also a request/response header, or even whether that is a meaningful distinction).

It's also not clear from this page exactly which headers are representation headers. It has weasel words like:

Representation headers include:

and

Validators used in conditional requests, such as:

Section 8 of the spec talks about "representation metadata" but the headers listed there don't match up exactly with the list in the glossary page or the headers whose reference page calls them representation headers: for example, Content-Range is listed in the glossary page but not under section 8, and Repr-Digest is described in its reference page as a representation header, but is not in section 8.

What's the real answer here? Is any of this really useful to developers?

Content headers

These are also described in quite abstract terms in their glossary page. The description implies that they are different from representation headers:

Content headers are the group of HTTP headers that describe the content of the body of an HTTP message, after any message framing in the body has been removed.... They differ from the Representation headers, which describe the encoding, media type, language, and other characteristics of the resource, and allow the underlying data to be interpreted.

...but also that:

Note: There is some overlap between content headers and representation headers.

...and again it's not clear which headers exactly should live in this group, or what a developer is supposed to make of this information. The spec doesn't even describe this category any more.

Again, there is a list with weasel words:

These headers may be present in both HTTP request and response messages and include:

...but several of the items listed are not marked as content headers in the reference pages for those headers (e.g. Content-Encoding)

The glossary page is quite new actually, and I see @bsmth had this comment in the PR where it was added:

I'm not sure how useful this page is

...which makes me feel a bit better about the situation :).

Client hints

These seem like a more straightforward category. The set of client hint headers is clearly defined and we can see how a developer would want to consider them as a coherent part of HTTP. There's what seems like a good guide outlining them and it's surely worth consistently linking to it from the client-hint headers, and doing that in "header type" seems like a good approach.

Fetch metadata

Same goes for fetch metadata. The main difference between this group and client hints is that we don't have a standalone guide to fetch metadata, but we probably should.

Tentative proposal

So given all that, here's my tentative proposal.

• in the "header type" row in header reference pages, and the intro sentence, we note:

  • whether a header is a request header, a response header, or both
  • additionally, whether it's a fetch metadata header or a client hint.
  • We don't any more surface representation or content groups here.

• we keep something describing representation headers, although not as a glossary page, per Lists in forbidden req/resp header glossary entries should be in HTTP section content#39902.

• we remove the glossary page on content headers.

• eventually, we write a proper guide page on fetch metadata.

Should we also expose conditional headers?

[hamishwillee]

1. I can't find the spec info (so probably I am wrong), but I seem to remember that specs for representation header was such that this looked like a separate category from request and response. I don't see that separation looking at current specs, but it might explain why some docs omit the the request/response.
2. Yes, I think all docs should have request/response/both information.
3. There are different kinds of categories.
   • Something like Client hints or fetch metadata is a convenient grouping of common/related functionality.
     They are essentially an "API grouping" - you can see a whole bunch of them in https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers.
   • Request and response are about how where the headers are used. I see forbidden and CORS safelisted as kind of in this group too, because they control how the messages might be used.
   • Representation/content type are metadata about how the message should be interpreted/decoded. Its a different kind of grouping and not as useful to readers. Why it is useful, is that if you're in a representation header, it acts as a grouping for the similar headers.
4. Yes, IMO all of these glossary entries should be in HTTP section rather than glossary - they aren't that interesting outside of HTTP for a start, so it is an unnecessary context switch.

There is an a lot to go through in your post. I don't know if I'll be able to answer it all (or want to) but here are some comments.

---

https://developer.mozilla.org/en-US/docs/Glossary/Fetch_metadata_request_header

This will have appeared somewhere like this:

The HTTP Sec-Fetch-Site fetch metadata request header indicates the relationship between a request initiator's origin and the origin of the requested resource.

This is just a combo of an "Functional grouping" category "Fetch metadata" and the "request header" metadata group. It could have been.

The HTTP Sec-Fetch-Site [fetch metadata] [request header] indicates the relationship between a request initiator's origin and the origin of the requested resource.

To be consistent we might rename to [fetch metadata headers] as an "Functional group" like client hints.

---

We also have "client hints", which doesn't have a glossary entry but appears in similar contexts to these.

As above, I see this as an functional grouping that is no more special than any other. YOu'll see it linked mostly like this (which is IMO a good pattern - we should just adopt the same pattern everywhere).

The HTTP Save-Data request header is a network client hint which

---

As far as I know the main place this is surfaced is in the "Header type" row in the table in each reference page:

Hmmm. Yes, the information is duplicated. You could argue that the Header Type could be omitted here, though not sure if it harms. Perhaps we should just have these groupings in the table.

---

We can see a few things:

• many headers are just marked as request or response headers, or both.
• some headers are marked as only representation headers
• some headers are marked as request, response, and representation headers
• some headers are marked as request, response, and content headers
• some headers are marked as only fetch metadata request headers
• some headers are marked as request headers and client hints

Is this correct? For example, is it meaningful that some headers are representation headers only, and some are representation headers and request/response headers?

It is intended that users should know the context in which they expect to see a header - a request or a response, or both. It is also helpful when authoring to know this, because it reminds you to cover the use in both cases.

IMO it is largely irrelevant to most readers whether something is a representation (or entity header) or a content header. But it is a useful as a "functional group" for those headers. The request/response should be listed too.

For the fetch and client hint cases, yes, they should be request/response and the functional category. If not, then the request/response is missing or implied by the group name (for fetch metadata).

Is it helpful? Is the taxonomy we're exposing here useful to developers?

I think so. Perhaps the reasoning or omissions are now more explicit.

---

I think that to be helpful, a categorization should:

• be clearly defined: we need to know, for a given item, whether it is definitively in that category
• make a difference to developers

Agreed. I think generally they are.

The exceptions are things like Content and Representation, which have an annoying specification overlap. They are also really mostly useful as a functional group the related headers so they aren't floating around in "no group", with no explanation of what they are for.

---

Request and Response headers

These seem pretty clear. Even I understand that some headers are request headers, some are response headers, and some can be both. But should every header fall into one of these categories?

Yes.

---

Naively I would have thought so, but our docs do not agree:

• Content-Type is listed only as a representation header, both in the table and in the intro sentence
• Repr-Digest is listed only as a representation header in the table, but as a request/response header in the intro sentence.

Omissions.

See my point 1. I suspect that this may have been intentional originally, but now is an omission.

---

The glossary pages for request and response headers do touch on this:

Not all headers appearing in a response are categorized as response headers by the specification. For example, the Content-Type header is a representation header indicating the original type of data in the body of the response message (prior to the encoding in the Content-Encoding representation header being applied). However, "conversationally" all headers are usually referred to as response headers in a response message.

Is this a helpful distinction for our docs to make? Is it the case that some representation headers are also request/response headers, and some aren't? If so (and if our current docs are correct, it is), where in the spec is this indicated?

I think the above does not reflect the way the specs read now, and should be removed.

---

Representation headers

Content headers

OK, we should fix these up to match the spec. It may be that the headers/validator make sense - i..e that the spec doe list validators as representation headers.

They are conceptually different but the spec does overlap.
They very very confusing, in particular a representation, and because of this overlap.
The docs you see were the best I came up with when first tried to understand this, iterated over by a few others. I don't like them either.

The content headers are fairly clear. These allow you to restore your message or messages from whatever encoding was used to transmit it/them.

A representation is essentially a particular form of the data - such as an XML file or a JSON file, perhaps in a particular language. So you can request "a listing of all the different types of bees", and use these headers to get the form of the data you received - pdf, xml listing, french language version - or of course you can specify what you want and get the specific version that is useful for you from the "generic" endpoint.

The representation might also include the encoding I understand - for example, its a zipped XML.

---

Client hints

Fetch metadata

Yes. Functional groups. See also other categories in https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers

---

Tentative proposal

So given all that, here's my tentative proposal.

• in the "header type" row in header reference pages, and the intro sentence, we note:

  • whether a header is a request header, a response header, or both
  • additionally, whether it's a fetch metadata header or a client hint.

I think we should do the intro "XXX response header that is a network client hint"
Should we duplicate in the header (I am OK either way with this).

• We don't any more surface representation or content groups here.

IMO the representation header is find to include just for those headers that are of the kind. Its a functional group.

• we keep something describing representation headers, although not as a glossary page, per Lists in forbidden req/resp header glossary entries should be in HTTP section content#39902.

If we're ever going to mention/link to these things then having a document which explains them seems sensible to me.
I don't see the benefit of the list without the explanation.

Though I guess we have other cases where we just list related headers. If only the group is useful, then I guess so.

• we remove the glossary page on content headers.

As above.

• eventually, we write a proper guide page on fetch metadata.

Should we also expose conditional headers?

As above.

[bsmth]

Thanks a lot, Will.

Is it meaningful that some headers are representation headers only, and some are representation headers and request/response headers?

Is it helpful? Is the taxonomy we're exposing here useful to developers?

I think it's useful to know if something is a req/resp header first and foremost, for other types, some thoughts below:

Tentative proposal

So given all that, here's my tentative proposal.

• in the "header type" row in header reference pages, and the intro sentence, we note:

  • whether a header is a request header, a response header, or both
  • additionally, whether it's a fetch metadata header or a client hint.
  • We don't any more surface representation or content groups here.

For the intro sentence, I would keep 'whether a header is a request header, a response header, or both', and add that where we omit it.

One thing to consider is that showing when something is a representation header can help understand the purpose or how you can use it. For instance, Last-Modified is a good example of a representation header, so that will be the same for a given resource regardless of how it's delivered. Keeping that in the table, at least, could be useful.

• we keep something describing representation headers, although not as a glossary page, per Lists in forbidden req/resp header glossary entries should be in HTTP section content#39902.

Sounds good to me.

• we remove the glossary page on content headers.

I'm +1 on this, given the content header glossary page landed in mdn/content#35335, which you can see fell mostly out of recent HTTP revisions dropping "payload" terminology. IMO "payload header" is a lot more descriptive, and tells you about how something is delivered.
I wonder if a table would help so we could summarize and compare content & repr headers, and show when something can be both.

• eventually, we write a proper guide page on fetch metadata.

Also sounds good.

Should we also expose conditional headers?

In this case, there's an article on the topic, and each of the relevant headers are listed there. This is different, IMO, because there's a mechanism or feature you want to use and those are the headers to look at / use, whereas repr/content headers help understand protocol semantics.

I think we all agree at least that the intro sentence and the table should say if something appears in requests, responses, or both.

[hamishwillee]

I think we all agree at least that the intro sentence and the table should say if something appears in requests, responses, or both.

Yes

I think we all agree that the glossary pages should be part of HTTP but not glossary. I'm not certain about deleting these. A table for representation vs content headers probably won't help people understand the difference. The real question is whether it useful for them to understand at all.