Tools for processing JSON

jq

The JSON processing tool. The syntax is akin to a functional language, so it takes some getting used to.

The exercism bash test runner uses jq to generate JSON:

The –arg and –argjson options to pass in individual values
the –args and –jsonargs options, and the $ARGS.positional variable, to pass in lists of values

Example:

$ jq -c -n \
     --arg foo bar \
     --argjson qux null \
     --jsonargs \
     '{$foo, baz: $ARGS.positional, $qux}' \
     1 2 3
{"foo":"bar","baz":[1,2,3],"qux":null}

I’ve been building an exercism jq track, so I’ve been learning it pretty intensely. I’ve written some notes about studying the manual.

ijq

An interactive terminal-based tool that displays then input on one side and the output on the other, filtered in real-time as you enter a jq expression. Uses jq.

jo

A small utility to create JSON objects from shell variables.

Example (take care with shell filename expansion)

$ jo foo=bar baz[]=1 baz[]=2 baz[]=3 qux=
{"foo":"bar","baz":[1,2,3],"qux":null}

Lists can be handled with tricky parameter expansions

$ values=(1 2 3)
$ jo foo=bar "${values[@]/#/baz[]=}" qux=
{"foo":"bar","baz":[1,2,3],"qux":null}

Nested objects involve multiple calls to jo

$ set -- alpha beta gamma
$ jo foo=bar items="$(jo "${@/#/baz[]=}")" qux=
{"foo":"bar","items":{"baz":["alpha","beta","gamma"]},"qux":null}

gron

“Flattens” JSON into a sequence of individual assignments, which makes it possible to use line-oriented tools (grep, etc) to process it.

Example:

$ echo '{"foo":"bar", "baz": [1, 2, 3], "qux": null}' | gron
json = {};
json.baz = [];
json.baz[0] = 1;
json.baz[1] = 2;
json.baz[2] = 3;
json.foo = "bar";
json.qux = null;

Using common tools to filter

$ echo '{"foo":"bar", "baz": [1, 2, 3], "qux": null}' | gron | grep '\.foo ='
json.foo = "bar";

$ echo '{"foo":"bar", "baz": [1, 2, 3], "qux": null}' \
  | gron \
  | awk -F ' = ' '$1 ~ /\.foo$/ {print $2}'
"bar";

and transform back to JSON with gron -u (“ungron”)

$ echo '{"foo":"bar", "baz": [1, 2, 3], "qux": null}' \
  | gron \
  | grep '\.foo =' \
  | gron -u
{"foo":"bar"}