Merge remote-tracking branch 'origin/topic/robin/readme-polish'

* origin/topic/robin/readme-polish:
  Polish README a bit.

Author: rsmmr

CHANGES

@@ -1,38 +1,27 @@
-2.0.4-26 | 2022-03-23 11:19:46 +0100
+2.1.0 | 2022-03-29 12:45:55 +0200
 
-  * Add `files_columns` table that extracts selected columns from
-    on-disk files und sends them over to Zeek as a record. For
-    example: `SELECT columns from files_columns("/etc/passwd",
-    "$1:text,$3:count", ":")` splits `/etc/passwd` into its parts,
-    extracts the user name and ID for each line, and then sends Zeek a
-    record containing two fields: the name as a `string`, and the ID
-    as a `count`. See the `README` for an explanation on the
-    parameters that `files_columns` takes.
+  * GH-25: Add `files_columns` table that extracts selected columns
+    from on-disk files und sends them over to Zeek as a record. See
+    the `README` for documentation.
 
-  * Update SQLite to 3.38.1 to give us built-in JSON support.
-
-2.0.4-22 | 2022-03-22 17:09:08 +0100
-
-  * GH-7: Extend the type system for table columns.
-
-    We can now represent addresses, booleans, counts, intervals, ports,
-    records, sets, times, and vectors all end-to-end, so that they arrive
-    as such at Zeek. For those types that don't have a natural SQLite
-    representation we internally serialize them into JSON and store that
-    as BLOBS. For boolean values, we now store real bools instead of
-    turning them into integers, allowing us to render them more nicely
-    even without further type information. This requires a bit of compiler
-    voodoo because of C++ ickyness (and a GCC bug).
-
-    We also update our tables to use the new types where appropriate.
+  * GH-7: Extend the type system for table columns. We can now
+    represent addresses, booleans, counts, intervals, ports, records,
+    sets, times, and vectors end-to-end, so that they arrive as such
+    at Zeek. We also update our tables to use the new types where
+    appropriate.
 
-    Closes #7.
+  * GH-34: Disable communication with a Zeek instance if it's package
+    version is too old.
 
-  * Add a 2nd 'differences' mode where no initial snapshot is
-    provided. We now have (1) "snapshot-and-diffs", which sends an
+  * Add a 2nd 'differences' mode to queries where no initial snapshot
+    is provided. We now have (1) "snapshot-and-diffs", which sends an
     initial snapshot first, followed by diffs; and (2) "diffs", which
     sends an empty initial result and then just diffs.
 
+  * Move numerical version number computation from CMake to runtime.
+
+  * Update SQLite to 3.38.1 to give us built-in JSON support.
+
   * Add Broker version to agent handshake.
 
   * Support default values for table parameters.
@@ -58,8 +47,6 @@
 
   * Format Zeek scripts with current zeek-format.
 
-  * Update to current Zeek package.
-
 2.0.4 | 2022-03-04 16:55:11 +0100
 
   * Add new log options. In the configuration file we now provide more

README.md

@@ -179,8 +179,8 @@ table parameter, which is a string containing a comma-separated list
 of tuples `$<N>:<type>`, where `<N>` is a column number (`$1` being
 the 1st column, `$2` the 2nd, etc.); and `<type>` is the type as which
 the value in that column will be parsed. Types can be: `blob`,
-`count`, int`, `real`, `text`. (As a special case, the column `$0`
-refers to whole line, without any processing.)
+`count`, `int`, `real`, `text`. As a special case, the column `$0`
+refers to whole line, without any processing.
 
 The column separator is specified by the 3rd table parameter. It can
 be either left empty for splitting on white-space, or a string to
@@ -198,14 +198,18 @@ out into a Zeek `record`.
 
 Here's an example: `SELECT columns from files_columns("/etc/passwd",
 "$1:text,$3:count", ":")` splits `/etc/passwd` into its parts, and
-extracts the user name and ID for each line.
+extracts the user name and ID for each line. (As `passwd` files may
+include comments lines, you could add a 4th parameter `"^ *#"` to
+ignore these. However, comments starting with `#` are already covered
+by the pattern that the 4th parameter uses by default, so it's not
+necessary.)
 
 | Parameter | Type | Description | Default
 | --- | --- | --- | --- |
 | `pattern` | text | glob matching all files of interest |  |
 | `columns` | text | specification of columns to extract |  |
-| `separator` | text | separator string to split columns; empty for whitespace | `""` |
-| `ignore` | text | regular expression matching lines to ignore; empty to disable | `^[ \t]*([#;]|$)` |
+| `separator` | text | separator string to split columns; empty for whitespace | `<empty>` |
+| `ignore` | text | regular expression matching lines to ignore; empty to disable | `^[ \t]*([#;]\|$)` |
 
 | Column | Type | Description
 | --- | --- | --- |

VERSION

@@ -1 +1 @@
-2.0.4-26
+2.1.0

auxil/autodoc-to-md

@@ -58,9 +58,9 @@ def renderTable(name, meta):
             default = column["default"]
             if default != None:
                 if default == "":
-                    default = '""'
+                    default = '<empty>'
 
-                default = "`{}`".format(default)
+                default = "`{}`".format(default.replace("|", "\\|"))
             else:
                 default = ""
 

src/tables/files/files.h

@@ -99,9 +99,9 @@ class FilesColumnsCommon : public FilesBase {
                 comma-separated list of tuples `$<N>:<type>`, where `<N>` is a
                 column number (`$1` being the 1st column, `$2` the 2nd,
                 etc.); and `<type>` is the type as which the value in that
-                column will be parsed. Types can be: `blob`, `count`, int`,
-                `real`, `text`. (As a special case, the column `$0` refers to
-                whole line, without any processing.)
+                column will be parsed. Types can be: `blob`, `count`, `int`,
+                `real`, `text`. As a special case, the column `$0` refers to
+                whole line, without any processing.
 
                 The column separator is specified by the 3rd table parameter.
                 It can be either left empty for splitting on white-space, or a
@@ -119,9 +119,14 @@ class FilesColumnsCommon : public FilesBase {
                 the selected values for each line. On the Zeek-side, this array
                 will roll out into a Zeek `record`.
 
-                Here's an example: `SELECT columns from files_columns("/etc/passwd",
-                "$1:text,$3:count", ":")` splits `/etc/passwd` into its parts,
-                and extracts the user name and ID for each line.
+                Here's an example: `SELECT columns from
+                files_columns("/etc/passwd", "$1:text,$3:count", ":")` splits
+                `/etc/passwd` into its parts, and extracts the user name and ID
+                for each line. (As `passwd` files may include comments lines,
+                you could add a 4th parameter `"^ *#"` to ignore these.
+                However, comments starting with `#` are already covered by the
+                pattern that the 4th parameter uses by default, so it's not
+                necessary.)
                 )",
             .platforms = { Platform::Darwin, Platform::Linux },
             .columns = {