Agent-First HTTP v0.16.0: A Host Only Its Owner Can Drive

by Agent-First Kit Contributors

A token-less host on loopback could be driven by any web page on the same machine, including the page being fetched. It now refuses foreign origins, rebinding and framing, and keeps tabs, browsers, profiles and containers on what their owner holds. Plus selector-stable waits for the part of a page you need, cookies that keep their scope through the browser, credentials that stay on their origin, and no wait a wedged browser can hold forever.

A local afhttp host started without a token trusts loopback: whatever can reach 127.0.0.1 is assumed to be its owner. A browser on the same machine can reach it too, and so can the page afhttp is fetching. This release is mostly about that assumption holding — the host answering only its owner, and everything the owner holds (a tab, a browser, a profile, a container) staying held until the owner lets go.

A host only its owner can drive

A token-less host now refuses a request that carries another site’s Origin or a non-loopback Host, so no web page can drive /cdp, and DNS rebinding is refused. It also refuses another site’s requests that are not top-level navigations, using Sec-Fetch-Site and Sec-Fetch-Dest, so a page cannot frame the takeover panel. A top-level navigation is still allowed, because a window opened from a local launch page arrives as exactly that. Unix-socket requests are exempt.

The listener itself refuses to serve without a token on a non-loopback address, rather than relying on the CLI to have checked. The takeover display socket accepts only the panel’s own origin, or none from a relay.

A --proxy containing @ is refused. Both a scheme-less user:pw@host and Chromium’s list form put the password on the browser’s command line, where any local process can read it.

What you hold stays held

A tab is orphaned only when the last connection holding it ends. Before, a one-off afhttp cdp --tab could get a live driver’s tab reaped, and the reaper ran only while the whole host was quiet, so one long-lived /cdp client kept it off every tab for good. The CDP proxy now keeps a ledger of the targets each connection creates and closes. Targets left behind when a connection ends are orphans whose idle clocks run while other drivers keep working, and a connection that attaches to one adopts it. The decision and the record happen under one lock, so an attach cannot land between them.

A profile switch is refused with 409 while CDP clients are connected, instead of killing the browser under them. A client is counted from the moment it picks a browser, so a switch cannot slip in before it attaches.

Takeover tabs are recycled only with affirmative evidence that the content was captured. A tab whose handoff could not be confirmed is kept, along with the partial result.

Waiting for the part of the page you need

--wait selector-stable:<css> waits until every matching region is visible, has text, is not aria-busy, and has stopped changing, and until the requests of the main frame and of the frames inside that region have finished. A pricing table inside a cross-origin iframe is required; an ad frame outside the region is not. There is no domain allowlist.

Browser auto and selector-stable fetches now report a readiness result: the main document, the target content and the ancillary frames, each ready, incomplete or unknown, with blocked for a detected verification wall. An embedded Turnstile widget is no longer mistaken for a wall over the whole page.

Cookies keep their scope through the browser

v0.15.0 kept host-only cookies host-only when they were read back from the browser. They could still widen on the way in: jar cookies were installed through CDP with a Domain even when host-only, and without Expires or SameSite, so the sync back stored them as session cookies that never expire. They keep their scope and lifetime both ways now, and a sync no longer overwrites changes a concurrent fetch persisted.

The jar’s rules follow RFC 6265 more closely:

  • Path=/adm no longer matches /admin;
  • domains compare without case;
  • Domain=com. no longer slips past the public-suffix check;
  • a plain-HTTP response can neither set nor replace a Secure cookie;
  • a caller’s own Cookie header no longer hides jar cookies, including the session a login response sets just before redirecting.

Credentials stay on their origin

Redirects no longer send Referer, which carried the previous URL’s query, and any ?token_secret in it, to the next origin. Caller credential headers reach the target origin only. A POST or PUT is never re-sent by an escalation or a retry, and a POST rewritten to GET drops Content-Encoding along with its body.

Redaction moved to where afhttp speaks. The SDK returns URLs and error details as fetched, and no longer links agent-first-data. The CLI and the host redact what they emit, including URLs quoted in warnings and recommended commands. Raw afhttp cdp results are page data and are emitted unaltered.

A wedged browser cannot hold afhttp

Every wait on a browser now has a bound: the launch handshake, each /json/version request, the CDP proxy’s upstream, raw afhttp cdp calls (30 seconds, ending in cdp_timeout) and takeover lease requests. On Windows, a browser child that would not die kept its stderr pipe open and stopped the process from exiting; that pipe is now read on a detached thread.

--wait idle subscribed to network-idle events only after enabling them, so a page that was already idle by then was never seen as idle and the fetch timed out. Subscriptions now come before the command that emits the events.

Profiles and containers stay on what afhttp owns

profile prune and profile delete no longer follow a symlink or Windows junction out of the profile root. A profile name may not end in ., which Windows strips. A lock guard removes its pidfile before unlocking, so it can no longer delete the next owner’s.

container install and uninstall refuse a --name held by a container that was not run from an afhttp image. Before, they would stop it, take over its volume and remove its image. The container entrypoint passes the token through the environment, never in arguments, and logged command lines hide proxy, token and environment values. uninstall --purge removes the image on Apple container too, and container install works from a native Windows shell that has no HOME.

scripts/deploy-local.sh replaces a local host with a private, verified backup of its data, and can resume from that backup if startup fails.

Smaller things

  • A network body’s known size no longer counts its response headers against --network-body-max-bytes.
  • When a host rejects a token taken from AFHTTP_TOKEN_SECRET, the error names the variable, so a stale value left in the environment is visible as such.
  • A CDP timeout during navigation reports navigation_timeout, as the wait stage already did.
  • cdp --tab --wait-event waits for that tab’s event, not any session’s.
  • IPv6 endpoints such as ws://[::1]:9222 connect.

Breaking changes

  • A token-less host refuses foreign origins and non-loopback addresses. Use a token to reach it from anywhere else.
  • --proxy with @ is refused, and so is --proxy with Camoufox.
  • A profile switch is refused while CDP clients are connected.
  • fetch --cookie-jar needs a host profile.
  • The SDK returns raw URLs and error details, and no longer depends on agent-first-data. Redact before you log them.
  • container install refuses a foreign container’s name.
  • Profile names ending in . are rejected.
  • The tab_crashed error code is gone; nothing ever produced it.
  • deploy-local.sh --reclaim-build-cache is gone.
  • Agent-First UI 0.9.0 and Agent-First Data 0.36.1 are the pinned dependencies, and Rust 1.99 is the minimum.

Getting it

$ brew install agentfirstkit/tap/afhttp
$ cargo install agent-first-http