Release Notes: JC version 1.14.0

[kellyjonbrazil]

Happy New Year! I’m happy to announce the release of jc version 1.14.0 available on github and pypi.

jc now supports over 60 commands and file-types, including the new date, kv, hash, hashsum (md5, md5sum, shasum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum), cksum, and wc command parsers. The ls parser has been enhanced to work with vdir output and the env parser has been enhanced to work with printenv output. jc is now fully tested on python 3.9.

jc can be installed via pip or through several official OS package repositories, including Debian, Ubuntu, Fedora, openSUSE, Arch Linux, NixOS Linux, Guix System Linux, FreeBSD, and macOS. For more information on how to get jc, click here.

To upgrade with pip:

$ pip3 install --upgrade jc

New Features

• jc is now available on the official Debian and Ubuntu repository (apt-get install jc)
• Tested on python 3.9

New Parsers

jc now supports 61 parsers. New parsers include kv, date, hash, hashsum, cksum, and wc.

Documentation and schemas for all parsers can be found here.

kv key/value pair parser (added in v1.13.2)

Parses key/value pair files. Files can include comments prepended with # or ; and keys and values can be delimited by = or : with or without spaces. Quotation marks are stripped from quoted values, though they can be kept with the -r (raw output) jc argument.

These types of files can be found in many places, including configuration files in /etc. (e.g. /etc/sysconfig/network-scripts).

$ cat keyvalue.txt
# this file contains key/value pairs
name = John Doe
address=555 California Drive
age: 34
; comments can include # or ;
# delimiter can be = or :
# quoted values have quotation marks stripped by default
# but can be preserved with the -r argument
occupation:"Engineer"

$ cat keyvalue.txt | jc --kv -p
{
  "name": "John Doe",
  "address": "555 California Drive",
  "age": "34",
  "occupation": "Engineer"
}

date command parser (added in v1.13.2)

Linux, macOS, and FreeBSD support for the date command:

$ date | jc --date -p          # or:  jc -p date
{
  "year": 2020,
  "month_num": 7,
  "day": 31,
  "hour": 16,
  "minute": 48,
  "second": 11,
  "month": "Jul",
  "weekday": "Fri",
  "weekday_num": 6,
  "timezone": "PDT"
}

hash command parser

Linux, macOS, and FreeBSD support for the hash BASH shell builtin:

$ hash | jc --hash -p
[
  {
    "hits": 2,
    "command": "/bin/cat"
  },
  {
    "hits": 1,
    "command": "/bin/ls"
  }
]

hashsum command parser

Linux, macOS, and FreeBSD support for various MD5 and SHA hash commands, including md5, md5sum, shasum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum:

$ md5sum * | jc --hashsum -p          # or jc -p md5sum *
[
  {
    "filename": "devtoolset-3-gcc-4.9.2-6.el7.x86_64.rpm",
    "hash": "65fc958c1add637ec23c4b137aecf3d3"   
  },
  {
    "filename": "digout",
    "hash": "5b9312ee5aff080927753c63a347707d"
  },
  {
    "filename": "dmidecode.out",
    "hash": "716fd11c2ac00db109281f7110b8fb9d"
  },
  {
    "filename": "file with spaces in the name",
    "hash": "d41d8cd98f00b204e9800998ecf8427e"
  },
  {
    "filename": "id-centos.out",
    "hash": "4295be239a14ad77ef3253103de976d2"
  },
  {
    "filename": "ifcfg.json",
    "hash": "01fda0d9ba9a75618b072e64ff512b43"
  }
]

cksum command parser

Linux, macOS, and FreeBSD support for the cksum and sum commands:

$ cksum * | jc --cksum -p          # or jc -p cksum *
[
  {
    "filename": "__init__.py",
    "checksum": 4294967295,
    "blocks": 0
  },
  {
    "filename": "airport.py",
    "checksum": 2208551092,
    "blocks": 3745
  },
  {
    "filename": "airport_s.py",
    "checksum": 1113817598,
    "blocks": 4572
  }
]

wc command parser

Linux, macOS, and FreeBSD support for the wc command:

$ wc * | jc --wc -p          # or jc -p wc *
[
  {
    "filename": "airport-I.json",
    "lines": 1,
    "words": 30,
    "characters": 307
  },
  {
    "filename": "airport-I.out",
    "lines": 15,
    "words": 33,
    "characters": 348
  },
  {
    "filename": "airport-s.json",
    "lines": 1,
    "words": 202,
    "characters": 2152
  }
]

Updated Parsers

The env parser has been enhanced to work with printenv command output using the “magic” syntax. (e.g. jc printenv)

The ls parser has been enhanced to work with vdir command output using the “magic” syntax. (e.g. jc vdir)

Schema Changes

There are no schema changes in this release.

Full Parser List

• airport -I
• airport -s
• arp
• blkid
• cksum
• crontab
• crontab-u
• CSV
• date
• df
• dig
• dmidecode
• du
• env
• file
• free
• fstab
• /etc/group
• /etc/gshadow
• hash
• hashsum
• history
• /etc/hosts
• id
• ifconfig
• INI
• iptables
• jobs
• kv
• last and lastb
• ls
• lsblk
• lsmod
• lsof
• mount
• netstat
• ntpq
• /etc/passwd
• ping
• pip list
• pip show
• ps
• route
• /etc/shadow
• ss
• stat
• sysctl
• systemctl
• systemctl list-jobs
• systemctl list-sockets
• systemctl list-unit-files
• timedatectl
• tracepath
• traceroute
• uname -a
• uptime
• w
• wc
• who
• XML
• YAML

Happy parsing!

v1.14.1 Release Changes

• Add iw-scan parser tested on linux (beta)
• Update date parser for Ubuntu 20.04 support
• Update last parser for last -F support
• Update last parser to add convenience fields and augment data for easier parsing
• Update man page
• Minor documentation updates

Schema Changes:

date command parser

A new period field has been added to the schema to represent AM and PM which may appear depending on locale configuration on the host. If the locale does not print AM or PM then the value will be null.

{
  "year":         integer,
  "month_num":    integer,
  "day":          integer,
  "hour":         integer,
  "minute":       integer,
  "second":       integer,
  "period":       string,
  "month":        string,
  "weekday":      string,
  "weekday_num":  integer,
  "timezone":     string
 }

last command parser

The duration field calculation has changed to be more easily parsed and will display as total HOURS:MINUTES. Also, a few convenience calculated fields have been added and will display when the last -F option is used: login_epoch, logout_epoch, and duration_seconds.

[
  {
    "user":             string,
    "tty":              string,
    "hostname":         string,
    "login":            string,
    "logout":           string,
    "duration":         string,
    "login_epoch":      integer,   # available with last -F option
    "logout_epoch":     integer,   # available with last -F option
    "duration_seconds": integer    # available with last -F option
  }
]

[zerolagtime]

I just discovered this and WOW! I thought about writing a tool like this to parse just last and here you've got an entire toolbox ready to go. I can't thank you enough!
Minor note: last -F | jc --last messes the parser up since it pushes a whole new string format to jc that includes proper dates that could be further parsed by databases, etc.
I found you because of a post you made on unix.com.

[kellyjonbrazil]

Thanks! I love to hear how people use jc in their projects!

I'll take a look at updating the last parser for last -F in the next minor release. In the meantime, you can always copy the current last parser python file and place it under $HOME/.local/share/jc/jcparsers and modify it to override the production last parser for a quick hack.

[kellyjonbrazil]

@zerolagtime I have a fix for last -F you can test. I opened an issue where we can discuss in more detail, if needed:

#96

[zerolagtime]

Thanks! I’m not sure how to test it since I’m using the “pip install” version. Thanks, Zerolagtime

…

On Jan 5, 2021, at 4:05 PM, Kelly Brazil ***@***.***> wrote:  @zerolagtime I have a fix for last -F you can test. I opened an issue where we can discuss in more detail, if needed: #96 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kellyjonbrazil]

Yep, that works the same way - just copy/paste this file:
https://github.com/kellyjonbrazil/jc/blob/dev/jc/parsers/last.py

Into a file named .local/share/jc/jcparsers/last.py under your home directory.

jc looks under that .local/share/jc/jcparsers directory for custom parsers. In this case, the last.py parser will override the current parser so you can test: last -F | jc --last.

[kellyjonbrazil]

This update has been merged into v1.14.1 which is now available on pypi.