Merge pull request #360 from 01mf02/prepare-3.0-alpha

Prepare 3.0-alpha

Author: 01mf02

Cargo.lock

@@ -5,8 +5,14 @@
 [![Documentation](https://docs.rs/jaq-core/badge.svg)](https://docs.rs/jaq-core)
 [![Rust 1.69+](https://img.shields.io/badge/rust-1.69+-orange.svg)](https://www.rust-lang.org)
 
-jaq (pronounced /ʒaːk/, like *Jacques*[^jacques]) is a clone of the JSON data processing tool [jq].
-It is two things at a time:
+jaq (pronounced /ʒaːk/, like *Jacques*[^jacques]) is a clone of
+the JSON data processing tool [`jq`](https://jqlang.github.io/jq/).
+It has a few features not present in `jq`, such as
+support for the data formats YAML, CBOR, TOML, and XML.
+jaq has an own [manual](https://gedenkt.at/jaq/manual/).
+You can try jaq on the [playground](https://gedenkt.at/jaq/).
+
+jaq is two things at a time:
 
 - A command-line program, `jaq`, that can be used as drop-in replacement for `jq`.
 - A library, [`jaq-core`](https://docs.rs/jaq-core/),
@@ -15,10 +21,6 @@ It is two things at a time:
   can be safely used in multi-threaded environments and
   supports [arbitrary data types beyond JSON](https://docs.rs/jaq-core/latest/jaq_core/val/trait.ValT.html).
 
-jaq has an own [manual].
-You can try jaq online on the jaq [playground].
-Instructions for the playground can be found [here](jaq-play/).
-
 jaq focuses on three goals:
 
 * **Correctness**:
@@ -36,10 +38,6 @@ jaq focuses on three goals:
   reduce the potential for bugs and to
   facilitate contributions.
 
-[jq]: https://jqlang.github.io/jq/
-[manual]: https://gedenkt.at/jaq/manual/
-[playground]: https://gedenkt.at/jaq/
-
 [^jacques]: I wanted to create a tool that should be discreet and obliging, like a good waiter.
   And when I think of a typical name for a (French) waiter, to my mind comes "Jacques".
   Later, I found out about the old French word *jacquet*, meaning "squirrel",
@@ -200,7 +198,13 @@ Add your own testimonials via <https://github.com/01mf02/jaq/issues/355>.
 # Acknowledgements
 
 [This project](https://nlnet.nl/project/jaq/) was funded through the
-<a href="https://nlnet.nl/entrust">NGI0 Entrust</a> Fund, a fund established by
-<a href="https://nlnet.nl">NLnet</a> with financial support from the
-European Commission's <a href="https://ngi.eu">Next Generation Internet</a>
-programme, under the aegis of <a href="https://commission.europa.eu/about-european-commission/departments-and-executive-agencies/communications-networks-content-and-technology_en">DG Communications Networks, Content and Technology</a> under grant agreement N<sup>o</sup> 101069594.
+[NGI0 Entrust](https://nlnet.nl/entrust) and
+[NGI0 Commons](https://nlnet.nl/commonsfund) funds established by
+[NLnet](https://nlnet.nl) with financial support from the
+European Commission's [Next Generation Internet](https://ngi.eu)
+programme, under the aegis of [DG Communications Networks, Content and Technology](https://commission.europa.eu/about-european-commission/departments-and-executive-agencies/communications-networks-content-and-technology_en) under
+grant agreements
+№ [101069594](https://cordis.europa.eu/project/id/101069594) and
+№ [101135429](https://cordis.europa.eu/project/id/101135429).
+Additional funding is made available by the
+[Swiss State Secretariat for Education, Research and Innovation](https://www.sbfi.admin.ch/sbfi/en/home.html) (SERI).

README.md

@@ -19,12 +19,23 @@ I think that he did a great job creating jq.)
 
 jaq and `jq` pursue different approaches to execute assignments:
 
-- *Path-based*: In `jq`, an assignment `p |= f` first constructs paths to all values that match `p`.
-  *Only then*, it applies the filter `f` to these values.
-- *Pathless*: In jaq, an assignment `p |= f` applies `f` *immediately* to any value matching `p`.
-  Unlike in `jq`, assignment does not explicitly construct paths.
-
-Fortunately, in most cases, the result is the same.
+- *Path-based*: In `jq`, an assignment `p |= f`
+  constructs paths to all values that match `p` and
+  applies the filter `f` to these values.
+- *Pathless*: In jaq, an assignment `p |= f` is transformed to
+  a different filter that does not construct any paths.
+
+For example, consider the update
+`[1, 2, 3] | .[] |= .+1 --> [2, 3, 4]`:
+When `jq` executes this, it calculates
+`[1, 2, 3] | path(.[]) --> [0] [1] [2]` and applies
+`.+1` on each value at these paths.
+On the other hand, jaq transforms the update to
+`[1, 2, 3] | [.[] | .+1] --> [2, 3, 4]` ---
+the assignment does not involve any path construction.
+
+Fortunately, like in the example above, in most cases,
+the result of the both approaches is the same.
 The following sections explain the two approaches in more detail,
 and how to write updates that behave the same in both `jq` and jaq.
 
@@ -423,7 +434,7 @@ bind it to the variable `$name`, and
 make it accessible in the current module.
 
 For example, if `foo.json` in the current working directory contains `1 2 3`, then
-`jaq -L . -n 'import "foo.json" as $myfoo; $myfoo'` yields `1 2 3`.
+`jaq -L . -n 'import "foo.json" as $myfoo; $myfoo'` yields `[1, 2, 3]`.
 
 ### Search paths
 
@@ -476,6 +487,19 @@ This searches for `binary.json` at the following paths:
 2. `./bar/binary.json` (relative to the current working directory)
 3. `./binary.json` (relative to the parent directory of `decode.jq`)
 
+::: Compatibility
+If a file to load has been given without extension,
+such as `decode` and `binary` above, then
+jaq adds an extension (`.jq` for modules or `.json` for data).
+`jq` adds an extension _unconditionally_; that is,
+even if an extension has been given as part of the file name,
+`jq` adds an extension.
+
+jaq's behaviour is motivated by allowing instructions like
+`import "binary.cbor" as $binary` in the future.
+Here, unconditionally adding the `.json` extension would be counterproductive.
+:::
+
 
 ## Comments
 

docs/advanced.dj

@@ -12,44 +12,25 @@ performs the following steps:
     - For each input value in the file:
 
         - Run _FILTER_ on the input value and print its output values
+- If an uncaught error is encountered at any point, jaq stops.
 
 For example, `jaq '.name?' persons.json`
 parses the filter `.name?`, then
 reads all values in `persons.json` one-by-one.
 It then executes the filter `.name?` on each of the values and
 prints the outputs of the filter as JSON.
 
-There are a few rules:
-
-- If no _FILTER_ is given, jaq uses `.` (the [identity filter](#identity)) as filter.
-- If no _FILE_ is given, jaq reads from standard input.
-- jaq determines the format to parse a _FILE_ as follows:
 
-    - If [`--from`](#--from) _FORMAT_ is used, jaq uses that format.
-    - Otherwise, if _FILE_ has a file extension known by jaq, such as
-      `.json`, `.yaml`, `.cbor`, `.toml`, `.xml`,
-      jaq uses the corresponding format.
-    - Otherwise, jaq assumes JSON.
-- If an uncaught error is encountered at any point, jaq stops.
-
-When passing filters directly as _FILTER_ argument on the command-line,
-care has to be taken to properly escape the filter.
-How to do this depends from platform to platform, but on Unixoid systems,
-surrounding the filter with single quotes (`'`) and
-replacing occurrences of `'` in filters by `'\''` suffices.
-For example, to run the filter `"'"` that
-produces a string containing a single quote, you can use
-`jaq -n '"'\''"'`.
-
-Running filters that start with the [negation operator](#negation),
-such as `jaq '-1'`, fails because `-` is interpreted as
-start of a command-line switch rather than negation.
-You can remedy this by using
-`jaq -- '-1'` instead, or by surrounding the filter in parentheses, i.e.
-`jaq '(-1)'`.
+## Input
 
+If no _FILE_ is given, jaq reads from standard input.
+jaq determines the format to parse a _FILE_ as follows:
 
-## Input options
+- If [`--from`](#--from) _FORMAT_ is used, jaq uses that format.
+- Otherwise, if _FILE_ has a file extension known by jaq, such as
+  `.json`, `.yaml`, `.cbor`, `.toml`, `.xml`,
+  jaq uses the corresponding format.
+- Otherwise, jaq assumes JSON.
 
 {#--from}
 ### `--from` _FORMAT_
@@ -90,30 +71,28 @@ This can be useful to fold over all inputs with [`reduce` / `foreach`](#reduce-f
 Read lines of the input as sequence of strings.
 For example,
 `echo -e "Hello\nWorld" | jaq -R` yields two outputs; `"Hello"` and `"World"`.
-When combined with `-s` (`--slurp`), this yields the whole input as a single string.
+
+When combined with [`--slurp`](#--slurp),
+this yields the whole input as a single string.
 For example,
 `echo -e "Hello\nWorld" | jaq -Rs` yields `"Hello\nWorld\n"`.
+See [`--rawfile`](#--rawfile).
 
 This is equivalent to `--from raw`.
 
-::: Advanced
-When using `-Rs` to load a file (as opposed to standard input),
-jaq loads this file in constant time (if it can be memory-mapped).
-This is because unlike jq, jaq does not validate that strings are valid UTF-8.
-That permits loading arbitrary binary files;
-these can be processed as byte strings via [`tobytes`](#tobytes).
-:::
-
 {#--slurp}
 ### `-s`, `--slurp`
 
 Read (slurp) all input values into one array.
 For example,
 `jaq -s <<< "1 2 3"` yields a single output, namely the array `[1, 2, 3]`, whereas
 `jaq <<< "1 2 3"` yields three outputs, namely `1`, `2`, and `3`.
-When combining `--slurp` with [`--raw-input`](#--raw-input), jaq reads
-the full input as a single string; for example,
+
+When combined with [`--raw-input`](#--raw-input),
+jaq reads the full input as a single string.
+For example,
 `jaq -Rs <<< "1 2 3"` yields the single output `"1 2 3\n"`.
+See [`--rawfile`](#--rawfile).
 
 ::: Compatibility
 When multiple files are slurped in,
@@ -128,7 +107,7 @@ for example, to achieve the output of
 :::
 
 
-## Output options
+## Output
 
 {#--to}
 ### `--to` _FORMAT_
@@ -235,7 +214,25 @@ For example,
 Use _N_ spaces for indentation (default: 2).
 
 
-## Compilation options
+## Compilation
+
+If no _FILTER_ is given, jaq uses `.` (the [identity filter](#identity)) as filter.
+
+When passing filters directly as _FILTER_ argument on the command-line,
+care has to be taken to properly escape the filter.
+How to do this depends from platform to platform, but on Unixoid systems,
+surrounding the filter with single quotes (`'`) and
+replacing occurrences of `'` in filters by `'\''` suffices.
+For example, to run the filter `"'"` that
+produces a string containing a single quote, you can use
+`jaq -n '"'\''"'`.
+
+Running filters that start with the [negation operator](#negation),
+such as `jaq '-1'`, fails because `-` is interpreted as
+start of a command-line switch rather than negation.
+You can remedy this by using
+`jaq -- '-1'` instead, or by surrounding the filter in parentheses, i.e.
+`jaq '(-1)'`.
 
 {#--from-file}
 ### `-f`, `--from-file`
@@ -270,7 +267,7 @@ If `--library-path` is not given, the following global search paths are used:
 See the [modules](#modules) section for more details.
 
 
-## Variable options
+## Variables
 
 {#--arg}
 ### `--arg` _A_ _V_
@@ -305,6 +302,19 @@ For example, if `values.json` contains `1 2 3`, then
 
 Set variable `$A` to string containing the contents of file _F_.
 
+jaq tries to load the file via memory mapping,
+taking constant time and allowing to load files that do not fit into memory.
+If this fails, jaq loads the file regularly, taking linear time.
+This is also what happens when using
+`-Rs` ([--raw-input](#--raw-input) and [--slurp](#--slurp))
+to load a file (as opposed to standard input).
+
+::: Compatibility
+Unlike `jq`, jaq does not verify that the file is valid UTF-8.
+That permits loading arbitrary binary files;
+these can be processed as byte strings via [`tobytes`](#tobytes).
+:::
+
 ### `--args`
 
 Collect remaining positional arguments into `$ARGS.positional`.
@@ -322,3 +332,72 @@ the latter because it would not have been interpreted as input file.
 However, `-c` is collected into the array because it comes after `--`,
 which leads every argument after it to be interpreted as input file.
 
+
+## Miscellanea
+
+{#--exit-status}
+### `-e`, `--exit-status`
+
+Use the last output value as exit status code.
+
+This enables the use of the exit codes 1 and 4, which are not used otherwise.
+
+jaq uses the following exit codes:
+
+- 0: No errors.
+- 1: The last output value is `false` or `null`.
+- 2: I/O or CLI error, e.g. file not found or unknown CLI option.
+- 3: Filter parse/compilation error.
+- 4: The filter did not yield any output.
+- 5: Any other error, e.g. call to the filter [`error`](#error).
+
+The filters [`halt`](#halt) and [`halt_error`](#halt_error)
+can be used to exit jaq with arbitrary exit codes.
+
+For example:
+
+```
+$ jaq -n empty; echo $?
+0
+$ jaq -n false >/dev/null; echo $?
+0
+$ jaq -en false >/dev/null; echo $?
+1
+$ jaq . does_not_exit.json 2>/dev/null; echo $?
+2
+$ jaq --foo 2>/dev/null; echo $?
+2
+$ jaq '+' 2>/dev/null; echo $?
+3
+$ jaq -en empty; echo $?
+4
+$ jaq -n error 2>/dev/null; echo $?
+5
+$ jaq -n 'halt(9)'; echo $?
+9
+```
+
+
+{#--help}
+### `-h`, `--help`
+
+Print summary of CLI options.
+
+{#--version}
+### `-V`, `--version`
+
+Print jaq version.
+
+
+{#unsupported-cli}
+## Unsupported
+
+The following command-line options are supported by `jq`, but not by jaq:
+
+- `--ascii-output`, `-a`
+- `--raw-output0`
+- `--unbuffered`
+- `--stream`
+- `--stream-errors`
+- `--seq`
+- `--jsonargs`

docs/cli.dj

@@ -198,13 +198,10 @@ contain equivalent bytes are considered equal, e.g.
 `"Hello" | . == tobytes --> true`.
 
 ::: Compatibility
-
 Byte strings do not exist in `jq`; however, they exist in `fq`.
-
 :::
 
 ::: Advanced
-
 To find out whether a value is a byte string, we can use:
 
 ```
@@ -213,11 +210,8 @@ def isbytes: isstring and try (.[0] | true) catch false;
 ```
 
 This works because we can index byte strings, but not text strings.
-
 :::
 
-
-
 ### Arrays
 
 An array is a finite sequence of values.
@@ -278,7 +272,6 @@ Note that here, it is necessary to surround `1, 2` with parentheses,
 in order to clarify that `,` does not start a new key-value pair.
 
 ::: Compatibility
-
 In jq, keys must be strings, whereas
 in jaq, keys can be arbitrary values.
 Because object keys can be arbitrary values in jaq, you can write e.g.:
@@ -288,8 +281,10 @@ Because object keys can be arbitrary values in jaq, you can write e.g.:
 { 0 : 1, "2": 3,  [4] : 5,  {} : 6}
 ```
 
-This yields an error in `jq`.
+Note that in a jq filter, non-string keys must be surrounded with parentheses,
+whereas in XJON, parentheses must not be used.
 
+This yields an error in `jq`.
 :::