Document fragment/full response render logic

[yawaramin]

Description

Document the exact logic needed by backend servers to decide whether to render an HTML fragment or a full HTML document.

Checklist

• I have read the contribution guidelines
• I have targeted this PR against the correct branch (master for website changes, dev for
  source changes)
• This is either a bugfix, a documentation update, or a new feature that has been explicitly
  approved via an issue
• I ran the test suite locally (npm run test) and verified that it succeeded

[MichaelWest22]

#3278 has already merged into dev branch and this will likely change how we do this documentation when the next release ships. I would suggest re-targeting these changes to dev and re-writing them to account for the new config item

[yawaramin]

The new config doesn't matter. The logic I documented will work either way whether the config is switched on or off.

[scrhartley]

I usually just go with hx-select, rather than doing conditional backend behavior. The additional logic on the backend seems like an optimization and I'm always unsure if it's worth it.

[yawaramin]

It might be expensive to render a full page in some cases. Maybe you're loading a bunch of stuff from some database or third-party system. hx-select is mentioned in the docs, but htmx's main mechanism for telling apart browser requests from htmx requests should be documented too.

[MichaelWest22]

## Requests & Responses {#requests}
Htmx expects responses to the AJAX requests it makes to be HTML, typically HTML fragments (although a full HTML
document, matched with a [hx-select](@/attributes/hx-select.md) tag can be useful too).  Htmx will then swap the returned
HTML into the document at the target specified and with the swap strategy specified.

Yeah there is already a section just above with this text talking about hx-select as an alternative instead of HTML fragments.

So maybe we could just reuse some the terminology at the start of your new documentation like:

### Rendering Response HTML {#response-html}

As mentioned above, htmx typically expects responses to be fragments of HTML (unless you use a
[hx-select](@/attributes/hx-select.md) tag). However, when you use htmx, not all requests from your
...

Just adding the word typically may clear up that it doesn't ALWAYS expect partial responses only. I don't know if we need to repeat the hx-select thing I added or not. Could also think about using 'often' instead of typically.

[yawaramin]

I added the word 'normally' and linked to the immediate above section which talks about hx-select. I think it would be hard to miss if anyone is reading through the 'Requests & Responses' section.

Also cross-linked against the section that talks about HTTP caching.

[MichaelWest22]

One interesting idea I had to resolve this properly is to add a new htmx response header.

dev...MichaelWest22:htmx:new-header

    if (target !== getDocument().body) {
      const selectOOB = getClosestAttributeValue(elt, 'hx-select-oob')
      const select = getClosestAttributeValue(elt, 'hx-select')
      if (select) {
        headers['HX-Response-Type'] = 'hx-select:' + select + (selectOOB ? ',' + selectOOB : '')
      } else {
        headers['HX-Response-Type'] = 'partial'
      }
    }

Here is a quick example of what could be added as another response header to fill in the gap we have in understanding the response we need. This HX-Response-Type header in my example is only present if the target is not body. So for history restore requests and boosted requests that have no target override or requests that explicitly target body will return nothing and otherwise it will return 'partial'. Then in your backend you can just check if this header == partial as the only simple check you would need. If you happen to use hx-select feature then instead you get a comma separated list of selectors that need to be sent but this should default to returning full page and not partials for the default use case. The great thing here is the info for knowing the response type needed is all in the client so we can easily expose it in this way.

Another cool feature is that users of hx-select could start out using hx-select for things while returning all simple full page responses and then over time update the backend routes to return partial responses based on the id selectors returned from this header to optimize performance without having to rework the templates to move from select pattern to normal partials.

It also gives htmx a new declarative descriptive power it didn't have before where you can have one full page template route url and multiple active htmx elements on the page with different hx-select tags defined can request from this one url different partial responses.

Anyway just an idea I had and it could easily be made into a JS htmx:configRequest event listener or a htmx extension but it would obviously work better if it was already there standard.

[yawaramin]

Why not use the existing HX-Request header for this? Ie send HX-Request only when a partial is wanted and not otherwise.

[scrhartley]

Why not use the existing HX-Request header for this? Ie send HX-Request only when a partial is wanted and not otherwise.

The job of HX-Request is very simple and it's to distinguish htmx vs. non-htmx requests. User will be depending upon this and subverting that expectation will lead to problems, even if it's just the way the user's application is broken is changed.

[yawaramin]

But that's not even the case any more. There's already a new config added that allows you to disable the HX-Request header for history restore requests. See above: #3311 (comment)

And I've already asked this elsewhere but what's even the use of being able to tell only that a request is from htmx or not? Who cares? What we will do with that information? Nothing 🤷 What we really care about is whether we should render a fragment or a full page and HX-Request was originally intended for that but just didn't work as designed imho. Why not just fix that instead of trying to work around it with a new header for every new case that pops up?

[scrhartley]

And I've already asked this elsewhere but what's even the use of being able to tell only that a request is from htmx or not?

I can imagine a backend which looks at HX-Request to determine whether it needs to look for any of the other htmx headers, for example to activate some type of request filter.

[gitblit]

My backend looks at HX-Request (and HX-Boost and HX-History-Restore-Request) to determine if it should send a fragment or a full page for a given request. Most to all of my pages can respond as both full renders and fragments. I do not use HX-Select.

[yawaramin]

@scrhartley then what's stopping it from directly looking for the headers it actually wants, instead of going through HX-Request? The ultimate effect is the same.

[scrhartley]

@yawaramin I'm just saying that the ship and sailed and we don't know what people have done and how optimal it might be. A change of behavior runs a risk of breaking something. I'm not thinking about some theoretical optimal system. Whatever the original intent was, how people have actually used it is the compatibility legacy.

[yawaramin]

That's why the PR I linked to flags the behaviour change behind a new config. Since it's already there, we might as well use it to mean 'HX-Request header is for fragments only, otherwise full pages'?

[yawaramin]

Anyway, this is all hypothetical to me. I don't believe in the general case that it's possible to have a header or combination of headers that allows us to determine with 100% certainty whether a fragment or a full page should be rendered, because of hx-select and hx-boost. In those cases it's just up to the developer's intention as to what should get rendered.

[scrhartley]

It doesn't seem necessary to have a dependency on the value of that config flag, it just seems an unneeded complication.

[scrhartley]

@MichaelWest22 Would it be useful to use a different joining character for select and selectOOB so that can know which ones are for which? They both can query for multiple elements with comma used to separate them.

[yawaramin]

I think it's time to move this discussion to a new issue. I'd like the discussion in this PR to reflect the actual behaviour as it stands today.

[MichaelWest22]

Yeah sorry for hijacking your PR! moved to a new issue #3319

[alexpetros]

Generally speaking, the htmx docs need to document htmx behavior. We have small examples that include references to server code—generally, in the examples section—but this sort of explainer is really out of scope, especially for the docs page.

If you wanted to add a small note to the hx-boost page saying "severs should check for both hx-request and the absence of hx-history-restore-request" I think that would work.