fix: prefer JSON in default Accept negotiation

Author: danielgtaylor

docs/design/003-content-types-and-encodings.md

@@ -212,9 +212,23 @@ text, structured values, and raw binary.
 Restish should generate a useful `Accept` header based on registered content
 types and their quality values.
 
-That ordering is user-visible and can affect what servers return. If CBOR is
-preferred over JSON, users may observe CBOR responses from APIs that support
-both.
+That ordering is user-visible and can affect what servers return. The default
+ordering should favor broadly interoperable, text-friendly structured formats
+before compact binary structured formats such as CBOR, MessagePack, and Ion.
+JSON and vendor JSON should be the strongest default preference; NDJSON and
+YAML should remain preferred over binary structured formats without tying JSON.
+Binary formats remain first-class decoding and output targets, but binary
+response negotiation should be opt-in through an explicit `Accept` header, an
+API profile, or an endpoint that only returns that format. Separately, users can
+still emit a binary structured document from any decoded response with an
+explicit output format such as `-o cbor`.
+
+This is deliberately independent of whether stdout is a TTY. Varying
+server-side negotiation based on redirection would make the same command ask
+for different representations depending on the shell plumbing around it. The
+more predictable model is: default negotiation asks for the representation most
+likely to work everywhere; output selection decides whether Restish preserves
+the original body bytes or decodes and re-renders them.
 
 The negotiation algorithm is:
 
@@ -235,10 +249,12 @@ exact list may grow as built-in aliases are added, but the quality order and
 deduplication rules remain stable:
 
 ```text
-application/cbor;q=0.9, application/msgpack;q=0.8, application/x-msgpack;q=0.8, application/vnd.msgpack;q=0.8, application/ion;q=0.8, text/ion;q=0.8, application/json;q=0.5, application/x-ndjson;q=0.5, application/ndjson;q=0.5, application/jsonl;q=0.5, application/jsonlines;q=0.5, application/yaml;q=0.5, application/x-yaml;q=0.5, text/yaml;q=0.5, text/x-yaml;q=0.5, application/x-www-form-urlencoded;q=0.3, multipart/form-data;q=0.3, text/event-stream;q=0.2, text/plain;q=0.2, text/*;q=0.2, application/octet-stream;q=0.1, */*;q=0.1
+application/json;q=0.9, application/x-ndjson;q=0.8, application/ndjson;q=0.8, application/jsonl;q=0.8, application/jsonlines;q=0.8, application/yaml;q=0.8, application/x-yaml;q=0.8, text/yaml;q=0.8, text/x-yaml;q=0.8, application/cbor;q=0.6, application/msgpack;q=0.6, application/x-msgpack;q=0.6, application/vnd.msgpack;q=0.6, application/ion;q=0.6, text/ion;q=0.6, application/x-www-form-urlencoded;q=0.3, multipart/form-data;q=0.3, application/xml;q=0.2, text/xml;q=0.2, text/event-stream;q=0.2, text/plain;q=0.2, text/*;q=0.2, application/octet-stream;q=0.1, */*;q=0.1
 ```
 
-Quality ordering should be stable and deliberate.
+Quality ordering should be stable and deliberate. It is part of the CLI's
+product contract because generated commands, generic requests, verbose output,
+and server behavior all expose it.
 
 Restish should not advertise suffix forms like `application/*+json` unless the
 runtime has an explicit reason to do so. Accept generation is based on concrete

docs/design/009-response-normalization-and-output.md

@@ -217,6 +217,14 @@ after HTTP content-encoding decompression. This keeps shell redirection aligned
 with the common file-saving mental model, even for structured binary formats
 such as CBOR.
 
+This raw-byte rule is separate from content negotiation. Restish's generated
+default `Accept` header should still prefer JSON and other text-friendly
+structured formats over CBOR, MessagePack, and Ion so the same command asks for
+the same broadly interoperable representation whether stdout is a terminal,
+pipe, or file. If a server returns CBOR because the user requested it, an API
+profile requested it, or the endpoint only supports it, redirected unfiltered
+output still preserves those response bytes.
+
 The practical rule is:
 
 - if Restish is still outputting the original payload unchanged, write body bytes

docs/design/028-document-and-record-output.md

@@ -136,16 +136,16 @@ The rule is:
   value without an explicit format, redirected output should default to pretty
   JSON; explicit `--rsh-print=b` selects compact rendered output for scripts
 
-Practically, saving a negotiated CBOR response keeps CBOR bytes:
+Practically, saving a CBOR response keeps CBOR bytes:
 
 ```bash
-restish api.rest.sh/content/cbor > response.cbor
+restish api.rest.sh/formats/cbor > response.cbor
 ```
 
 Converting that same response to JSON is explicit:
 
 ```bash
-restish api.rest.sh/content/cbor -o json > response.json
+restish api.rest.sh/formats/cbor -o json > response.json
 ```
 
 ### `-o json` Preserves Document Framing

internal/cli/generated_test.go

@@ -1902,7 +1902,7 @@ func TestGeneratedCommandAcceptHeaderIncludesDeclaredSupportedResponseTypes(t *t
 	if err := c.Run([]string{"restish", "tapi", "get-negotiated"}); err != nil {
 		t.Fatalf("get-negotiated: %v", err)
 	}
-	want := "application/cbor;q=0.9, application/vnd.example+json;q=0.5, application/json;q=0.5, text/plain;q=0.2"
+	want := "application/vnd.example+json;q=0.9, application/json;q=0.9, application/cbor;q=0.6, text/plain;q=0.2"
 	if gotAccept != want {
 		t.Fatalf("Accept = %q, want %q", gotAccept, want)
 	}

internal/content/defaults.go

@@ -36,7 +36,7 @@ func Default() *Registry {
 		Name:      "json",
 		MIMETypes: []string{"application/json"},
 		Suffixes:  []string{"+json"},
-		Quality:   0.5,
+		Quality:   0.9,
 		Marshal: func(v any) ([]byte, error) {
 			return json.Marshal(v)
 		},
@@ -55,7 +55,7 @@ func Default() *Registry {
 	r.AddContentType(&ContentType{
 		Name:      "ndjson",
 		MIMETypes: []string{"application/x-ndjson", "application/ndjson", "application/jsonl", "application/jsonlines"},
-		Quality:   0.5,
+		Quality:   0.8,
 		Marshal:   marshalNDJSON,
 		Unmarshal: func(data []byte) (any, error) {
 			lines := bytes.Split(data, []byte{'\n'})
@@ -90,7 +90,7 @@ func Default() *Registry {
 		Name:      "yaml",
 		MIMETypes: []string{"application/yaml", "application/x-yaml", "text/yaml", "text/x-yaml"},
 		Suffixes:  []string{"+yaml"},
-		Quality:   0.5,
+		Quality:   0.8,
 		Marshal: func(v any) ([]byte, error) {
 			return yaml.Marshal(v)
 		},
@@ -107,7 +107,7 @@ func Default() *Registry {
 		Name:      "cbor",
 		MIMETypes: []string{"application/cbor"},
 		Suffixes:  []string{"+cbor"},
-		Quality:   0.9,
+		Quality:   0.6,
 		Marshal: func(v any) ([]byte, error) {
 			return cbor.Marshal(v)
 		},
@@ -124,7 +124,7 @@ func Default() *Registry {
 		Name:      "msgpack",
 		MIMETypes: []string{"application/msgpack", "application/x-msgpack", "application/vnd.msgpack"},
 		Suffixes:  []string{"+msgpack"},
-		Quality:   0.8,
+		Quality:   0.6,
 		Marshal: func(v any) ([]byte, error) {
 			return msgpack.Marshal(v)
 		},
@@ -165,7 +165,7 @@ func Default() *Registry {
 		Name:      "ion",
 		MIMETypes: []string{"application/ion", "text/ion"},
 		Suffixes:  []string{"+ion"},
-		Quality:   0.8,
+		Quality:   0.6,
 		Marshal: func(v any) ([]byte, error) {
 			return ion.MarshalText(v)
 		},
@@ -178,8 +178,6 @@ func Default() *Registry {
 		},
 	})
 
-	// text/* is a catch-all for plain text responses — lowest quality so
-	// structured formats are always preferred.
 	r.AddContentType(&ContentType{
 		Name:      "form",
 		MIMETypes: []string{"application/x-www-form-urlencoded"},
@@ -238,6 +236,8 @@ func Default() *Registry {
 		},
 	})
 
+	// text/* is a catch-all for plain text responses. Keep it below explicit
+	// structured types so it does not win when a more specific format is known.
 	r.AddContentType(&ContentType{
 		Name:      "text",
 		MIMETypes: []string{"text/plain", "text/*"},

internal/content/registry_test.go

@@ -146,18 +146,32 @@ func TestMakeJSONSafeIntegerKeys(t *testing.T) {
 
 func TestAcceptHeader(t *testing.T) {
 	h := reg.AcceptHeader()
-	// cbor q=0.9 must appear before json q=0.5
+	// JSON is the least surprising default negotiation target. Binary
+	// structured formats remain supported, but should not win by default.
 	iCBOR := strings.Index(h, "application/cbor")
 	iJSON := strings.Index(h, "application/json")
-	if iCBOR == -1 || iJSON == -1 {
+	iYAML := strings.Index(h, "application/yaml")
+	if iCBOR == -1 || iJSON == -1 || iYAML == -1 {
 		t.Fatalf("Accept header missing expected types: %q", h)
 	}
-	if iCBOR > iJSON {
-		t.Errorf("cbor should appear before json in Accept header: %q", h)
+	if iJSON > iYAML {
+		t.Errorf("json should appear before yaml in Accept header: %q", h)
+	}
+	if iYAML > iCBOR {
+		t.Errorf("yaml should appear before cbor in Accept header: %q", h)
+	}
+	if iJSON > iCBOR {
+		t.Errorf("json should appear before cbor in Accept header: %q", h)
 	}
 	if !strings.Contains(h, "application/x-ndjson") {
 		t.Fatalf("Accept header missing application/x-ndjson: %q", h)
 	}
+	if !strings.Contains(h, "application/x-ndjson;q=0.8") {
+		t.Fatalf("Accept header should prefer NDJSON below JSON: %q", h)
+	}
+	if !strings.Contains(h, "application/yaml;q=0.8") {
+		t.Fatalf("Accept header should prefer YAML below JSON: %q", h)
+	}
 	iSSE := strings.Index(h, "text/event-stream")
 	if iSSE == -1 {
 		t.Fatalf("text/event-stream missing from Accept header: %q", h)
@@ -310,7 +324,7 @@ func TestAcceptHeaderForUsesRequestedSupportedMediaTypes(t *testing.T) {
 		"text/plain",
 		"application/json",
 	})
-	want := "application/cbor;q=0.9, application/vnd.example+json;q=0.5, application/json;q=0.5, text/plain;q=0.2"
+	want := "application/vnd.example+json;q=0.9, application/json;q=0.9, application/cbor;q=0.6, text/plain;q=0.2"
 	if got != want {
 		t.Fatalf("AcceptHeaderFor() = %q, want %q", got, want)
 	}

site/content/en/docs/guides/output.md

@@ -115,22 +115,24 @@ CBOR, YAML, images, octet streams, zip files, text, and unknown payloads:
 ```bash
 restish api.rest.sh/images/jpeg > dragonfly.jpg
 restish api.rest.sh/bytes/64 > sample.bin
-restish api.rest.sh/content/cbor > response.cbor
+restish api.rest.sh/formats/cbor > response.cbor
 ```
 
 Choose an output format when you want Restish to transform the decoded body:
 
 ```bash
-restish api.rest.sh/content/cbor -o json > response.json
+restish api.rest.sh/formats/cbor -o json > response.json
 ```
 
 Raw redirected output bypasses Restish's structured body decoding and
 formatting for presentation, but it is still based on the body after HTTP
-content-encoding decompression. `raw` is not an `-o` format. To save bytes
-unchanged, redirect stdout without choosing a filter, metadata shortcut,
-collection, or explicit output format. Response middleware plugins are skipped
-on this raw-download path; they run when Restish renders, filters, collects, or
-prints an interpreted response.
+content-encoding decompression. It also does not make Restish ask the server
+for a different representation: default `Accept` negotiation still prefers
+JSON and other text-friendly structured formats unless you set `Accept`
+yourself. `raw` is not an `-o` format. To save bytes unchanged, redirect stdout
+without choosing a filter, metadata shortcut, collection, or explicit output
+format. Response middleware plugins are skipped on this raw-download path; they
+run when Restish renders, filters, collects, or prints an interpreted response.
 
 Control exactly what stdout contains with `--rsh-print`:
 

site/content/en/docs/guides/troubleshooting.md

@@ -54,14 +54,17 @@ Add `--rsh-print=b` when the script needs compact rendered JSON.
 **Symptom:** A redirected response is CBOR, YAML, text, or another server format
 instead of JSON.
 
-**Likely cause:** Redirected unfiltered output saves response body bytes.
+**Likely cause:** Redirected unfiltered output saves response body bytes. This
+can happen when the endpoint only returns that format, a profile or command set
+an `Accept` header, or the server ignored Restish's JSON-first default
+negotiation.
 
 **How to confirm:** Run with an explicit format.
 
 **Fix:**
 
 ```bash
-restish api.rest.sh/content/cbor -o json > response.json
+restish api.rest.sh/formats/cbor -o json > response.json
 ```
 
 **Prevention:** Use `-o json` when a file or script needs JSON. Omit `-o` when
@@ -109,7 +112,9 @@ legacy location so they are not imported again.
 instead of JSON.
 
 **Likely cause:** Restish sends an `Accept` header listing all registered
-content types with quality values.
+content types with quality values. The default header prefers JSON and other
+text-friendly structured formats, but servers can still choose another
+supported response type or ignore client preferences.
 
 **How to confirm:** Inspect the request with `/headers` or verbose mode.
 

site/content/en/docs/recipes/save-a-response-unchanged.md

@@ -13,7 +13,7 @@ raw-download path, so installed plugins cannot silently alter saved files.
 
 ```bash
 restish api.rest.sh/bytes/64 > sample.bin
-restish api.rest.sh/content/cbor > response.cbor
+restish api.rest.sh/formats/cbor > response.cbor
 ```
 
 For an image:
@@ -28,5 +28,8 @@ Redirected byte output still uses the body that Go's HTTP client exposes after
 any HTTP content-encoding decompression; it is not a packet capture of the exact
 wire transfer. The distinction is part of Restish's
 [output defaults](/docs/reference/output-defaults/).
+Restish's default `Accept` header still prefers JSON and other text-friendly
+structured formats; set `Accept` yourself when you want the server to send a
+binary structured format such as CBOR.
 
 Related: [Output](/docs/guides/output/).

site/content/en/docs/reference/content-types.md

@@ -104,13 +104,20 @@ restish -H 'Accept: image/png' api.rest.sh/images/png
 ```
 
 Restish generates an `Accept` header from registered content types, ordered by
-quality and deduplicated by canonical MIME type. If a plugin registers the same
-MIME type later, that later registration is the effective one.
+quality and deduplicated by canonical MIME type. Defaults prefer JSON and
+vendor JSON first, then other text-friendly structured formats such as NDJSON
+and YAML, before binary structured formats such as CBOR, MessagePack, and Ion.
+That ordering is least surprising in terminals, scripts, logs, and tools.
+Binary formats are still supported; request one explicitly with `-H`, a
+profile, or an endpoint that only returns that format when the API and your
+workflow benefit from it. If a plugin registers the same MIME type later, that
+later registration is the effective one.
 
 Use `-o` when you want Restish to transform a decoded response after it arrives:
 
 ```bash
-restish api.rest.sh/content/cbor -o json > response.json
+restish api.rest.sh/formats/cbor > response.cbor
+restish api.rest.sh/formats/cbor -o json > response.json
 ```
 
 ## Compression