Agent-First Data v0.37.0: The Glue That Was Left

The advice this crate gives to scripts is simple: read JSON, TOML, YAML, and dotenv files with afdata, not with a python3 -c one-liner or a jq pipeline. A one-liner is a second parser with its own opinion about numbers, missing keys, and quoting, and it is written fresh at every call site.

So we took our own advice and applied it to a whole repository of Bash scripts. Whenever the conversion could not be finished, because the script still needed a line of glue, we wrote the reason down. That list is the release. Every item is something a script could not say with afdata alone.

Reading something that might not be there

The most common glue was a two-step read: ask whether a subtree exists, then read it.

if [ "$(afdata has spec.json spores.nope.follows)" = true ]; then
  slice="$(afdata json spec.json spores.nope.follows)"
else
  slice='[]'
fi

That parses the file twice and spends three lines on “give me this, or an empty list”. value and values already had --default; json did not. It does now:

$ afdata json spec.json spores.nope.follows
{"error":{"code":"document_path_not_found", ...}}          # exit 1
$ afdata json spec.json spores.nope.follows --default '[]'
[]

The default is parsed and printed in the same compact form as any other result, not echoed back byte for byte, so a malformed default is json_parse_failed rather than something a downstream tool has to guess about. An ambiguous slug match still fails: a default is for “nothing there”, not for “more than one thing there”.

Counting had the same shape. To learn how many children an array has, a script listed them and piped the list through wc -l: three processes for one integer. afdata count FILE KEY prints the number, and --missing-ok makes an absent or null container count as zero.

A related rule changed underneath these. A path that runs through a scalar, such as follows.0.deeper where follows.0 is a string, used to be an error different from “not found”. For value, values, json, and has it now means the same thing as not found, so a --default covers it. A script that asks for something that is not there does not need to know why it is not there.

Building a path from names

A key that contains a dot, a backslash, or an asterisk has to be escaped to be addressed, and scripts did that with sed. afdata path takes the literal segments and prints the escaped path:

$ afdata path 'a.b' 'c*d' 'e\f'
a\.b.c\*d.e\\f

The same release adds afdata object and afdata array, which build escaped JSON from arguments, so a script that has to hand a record to another program does not assemble it with printf and hope the values contain no quotes.

Property lists

macOS configuration lives in property lists, and the scripts that read them shelled out to plutil or Python. afdata now reads XML .plist files like any other document, including the --input-format plist flag for stdin. The mapping is the usual one: dictionaries become objects, arrays stay arrays, integers and booleans keep their types, and a date is read as its ISO string.

$ afdata value Info.plist CFBundleIdentifier
com.example.app

It is read-only on purpose. The editing commands refuse a plist with document_unsupported_operation, because a rewrite would have to preserve comments and key order that this reader does not model. A binary plist is recognised by its magic bytes before any attempt to decode it as text, and the error points at plutil to convert it first. Without that check, the first thing you saw was a UTF-8 error about a file that was never text.

Two errors that named the wrong problem

A zero-byte file used to fail with a parse error from whichever format was guessed from the extension, which reads like a syntax mistake in a file that has no syntax. It is document_empty now, and says so. A path that is not a regular file hints at reading from -.

keys and paths on an array of records gained a clean split. --slug-field only addresses: it picks which element a key like servers.web.port means. Listing the elements by their slug value is a different question, so it is --slugs. Before, one flag answered both questions, and the answer depended on whether the key ended on the array or inside it.

Background processes that actually stop

The Bash kit has had helpers for starting a process and waiting for it to print a line. This release finishes the set, and fixes the one that was wrong.

afdata_timeout 30 some-command returned 124 after thirty seconds, but it stopped only the command it had started. A child that command had forked kept the pipe of a surrounding $(...) open, so the capture waited anyway and the timeout did nothing useful. The command now runs in its own process group, and the whole group gets TERM, then KILL after five seconds.

Around it:

• afdata_spawn starts a command with stdin closed, output to files, and its own process group.
• afdata_wait_line returns the first complete line of the output file. If the process dies before writing one, it returns 3 instead of waiting out the deadline.
• afdata_stop signals a pid or a whole group, escalates, and always reaps.
• afdata_mktemp, afdata_try, and afdata_into replace the three most common hand-written patterns: temporary files that get cleaned up, a command whose failure you want as a status without tripping set -e, and reading a list into an array in Bash 3.2.

Seconds are whole numbers everywhere. afdata_timeout 1.5 … is a usage error, since the deadlines count sleep 0.1 ticks and Bash 3.2 has no sub-second clock to read.

Scripts whose output is the value

Some scripts exist to print one thing for the caller to capture, such as a path or a version. Wrapping that in a record, as every afdata command does by default, is the wrong answer for them. A script now declares it with afdata_args_raw_output, between afdata_args_begin and afdata_args_parse. The parser then rejects --output and --output-to, and diagnostics go to stderr.

The first version of this was an environment variable, and a variable is inherited: a script that declared raw output would have changed the behaviour of every afdata script it called. The declaration is shell-internal state now. Calling it before afdata_args_begin, which resets state, or after afdata_args_parse, which has already read it, is a usage error and not a silently ignored line.

What this release was about

The usual way to find out what a tool lacks is to ask its users. The better one is to forbid yourself the workaround. We removed every python3 -c and jq call from a real set of scripts, and the places where that would not compile were precise: a read that cannot be absent, a count that costs three processes, a path that has to be escaped by hand, a timeout that leaves a child behind.

Each one is a script that now reads one way, and the glue it needed is gone.