Agent-First Data v0.32.0: Where the Value Comes From

A registry could say an argument was a string, that it was required, that it was a UUID or an integer in a range. It could not say the one thing every credential-taking flag needs to say: that the value does not have to be typed on the command line.

So the answer lived somewhere else. In afdata’s own CLI it was --secret-from stdin|prompt|fd:N|env:VAR — a flag with its own parser, its own capped reader, its own terminal-echo guard, its own /dev/tty prompt, all private to one binary. It worked. It was also 141 lines that said nothing to the registry, so --help could not describe it, validation could not reject a bad spelling before the command ran, and no other program could reuse a line of it.

That is the shape of the problem. Not that reading a secret from a file is hard, but that “where does this value come from” was being answered below the argument grammar, where nothing that reasons about the grammar can see it.

The source is part of the declaration

An argument now declares the sources it accepts:

ArgSpec::option("--password-secret", "SOURCE").sources(SourceSet::stream())

SourceSet::config() is the two sources a value can come from with neither a terminal nor a pipe — an environment variable, or an address inside a document. That is the set most arguments want. SourceSet::stream() adds stdin, fd:N, and prompt, for a value that may be piped in, handed over on an inherited descriptor, or typed by a person.

The grammar is small and closed:

env:NAME                    an environment variable
file:PATH#DOT_PATH          an address inside a JSON/TOML/YAML/dotenv/INI file
file+FORMAT:PATH#DOT_PATH   the same, naming the parser
stdin                       the whole of standard input
fd:N                        an inherited descriptor
prompt                      /dev/tty, echo suppressed
literal:VALUE               the escape hatch

A bare argument is still the literal value. literal: exists only for a value that would otherwise be read as one of the prefixes above — an escape, not a style.

Because the registry knows the set, three things follow without the application doing anything. Help prints the accepted syntax next to the argument. A recognized scheme outside the declared set is rejected as cli_invalid_argument_value — exit 2, a usage error, before config, filesystem, terminal, network, or domain I/O. And the whole thing serializes into cli-spec-v1, so a compiler in another language can implement the same argument from the spec alone.

One rule falls out of that and is worth stating: an argument with sources cannot declare a default. A default is a value. Making it an instruction to go read the host’s environment or filesystem would mean a program that reads a file nobody asked it to read.

Reading it is a separate act, with a type

Classification and reading are deliberately not the same step. The registry decides what the source is; the application reads it only when the command it selected actually needs the value.

// The registry has already proved this spelling is a source the argument
// accepts; classifying it again cannot fail on a value that got this far.
let source = sources.parse(invocation.required("password_secret").as_str().unwrap())?;
let password = source.read_secret()?;

read() returns a String. read_secret() returns a SecretString, whose Debug and Display are ***, and whose contents are reachable only through an explicit expose_secret(). The distinction is not cosmetic: the secret path refuses a non-string value rather than coercing it, and its errors carry nothing that could echo back what was read. On the plain path, quoting the file and the parser’s complaint is the whole point of the error. On the secret path it is a leak.

The Rust adapter adds one rule the core cannot: prompt on an argument whose id does not end in _secret is refused at build time. Prompting blocks on a terminal and suppresses echo. Both are correct for a password and wrong for anything else, and the argument’s own name is the only place that fact is already written down.

A host may add a scheme this crate does not implement — a secrets manager, a container lookup — by declaring a name and its syntax. That teaches help and validation the source exists; the host does the reading. The name is checked against the reserved env, file, stdin, fd, prompt, and literal, and against the host’s own other schemes, so a host source can never quietly shadow a built-in.

The file it would not read

file: was worth very little if it could not open the files that exist.

The commonest configuration-file shape in the world is a flat key=value list with no section header at all. /etc is full of them. So is the preamble of every php.ini. The INI editor answered all of them with root entries are not supported, because it had been built on one uniform depth: every value at section.key, no exceptions.

The uniformity was not worth the files it excluded. Entries before the first [section] are now the document root, addressed by bare key. That introduces exactly one ambiguity — a root key and a section of the same name would address the same top-level name — and it is refused rather than resolved, as a parse error beside the duplicate-section rule it resembles. Writing is where this gets subtle: save emits root entries first, and set inserts a new one before the first header. Not by preference. A key=value written after a header belongs to that header’s section, so anywhere else is a silent move of the author’s data.

The other half of the problem is that such a file is rarely named .ini. It is named .conf, or .cfg, or nothing at all. file+FORMAT:PATH#DOT_PATH lets the source say what the filename cannot:

$ tool auth --password-secret 'file+ini:/etc/app/app.conf#http-password'

The +FORMAT sits before the colon so that a Windows drive letter is never mistaken for it. And the CLI core never resolves the word: cli_spec decides argv grammar and is not permitted to know what a document format is, so it carries the caller’s spelling and the value-source layer turns it into a parser — and rejects an unknown one there, where the format table actually lives. That table, Format::from_cli_name, is now the single one; the afdata CLI’s --input-format reads from it instead of keeping a second copy that could drift.

The CLI eats its own cooking

--secret-from still exists, with the same spelling and the same behavior. It is now four lines over the shared library instead of a private implementation of it, and the two library messages that differed from the CLI’s shipped wording moved to the CLI’s, not the reverse — those strings are released, and its e2e pins them.

It kept its own flag rather than becoming .sources() on set’s VALUE positional, and the reason is a good illustration of what this facility is for. If a general value argument accepted sources, then

$ afdata set notes.json mood prompt

would block on a terminal waiting for someone to type a mood. literal: would stop being an escape hatch and start being a trap you have to remember. Per invocation, explicitly opted into, is what the flag is for. file: stays out of its set for the same reason: set is already pointed at a document, and a second document address in the same call reads as an operation on that file.

What this release was about

The last release was about failures routed somewhere they could not be answered. This one is about a fact routed somewhere it could not be read.

“This value may come from an environment variable” was always true of these arguments. It was written in a private parser, so --help could not say it, validation could not enforce it, the spec could not carry it to another language, and every program that needed it wrote it again. Moving it up into the declaration did not add a capability. It made an existing one visible to everything that had a reason to know.