Agent-First PDF: The Article, Not the Page

Ask any PDF library for the text of a Chinese regulation and you get all of it. Every character is there. It is also useless as an article: the third provision reads 第三条 国家广播电视总局（以下简称广电总局）负责 on one line and 全国广播电视节目传送业务的管理。 on the next, because that is where the typesetter wrapped it. Every page ends with — ５６ — in full-width digits. A table set in a math font has lost its rows. A two-column paper has been read across the gutter, so the first sentence of the left column runs straight into the first sentence of the right. The figure between two paragraphs is gone, or sits at the end of the page with no hint of where it belonged.

An agent that is handed this text will summarize it, index it, and quote it, and every one of those operations inherits the breakage. Agent-First PDF (afpdf) exists to hand the agent the article instead, and to do it without throwing away the page.

Two layers, so the evidence survives

The result has two layers, and the contract between them is the whole design.

Basic extraction is what the engine observed: the raw text of each page before anything was removed or joined, every glyph with its Unicode value, font name, size, color and bounds, which glyphs were drawn outside a clipping rectangle, which ones the font could not map to a character, and the position and pixel size of every image object. It is not a lossless dump of PDF operators. It is the evidence an organization step needs, serialized, so that step can run again later without the PDF.

Organization is a pure function over that evidence. It decides reading order, splits headings from paragraphs from numbered provisions from table rows from footnotes, joins lines into natural paragraphs, carries a paragraph across a page break when the geometry supports it, removes running headers, footers and page numbers, and places figures where they sat. Every one of those is an option a caller can switch off and rerun. The removed furniture is returned beside the body, not deleted. Every block carries the page and the rectangle it came from, and a paragraph that spans two pages carries two.

So a caller who disagrees with a heuristic is never stuck: turn it off, or read the evidence and apply a better one. And a text export is never the only thing you have.

What the organizer decides, and from what

Punctuation is not a paragraph boundary. A sentence can end mid-paragraph and a provision can run on for a page. The organizer uses what the page actually says: indentation, where lines end relative to the text edge, line spacing, centering, right alignment, and structural numbering. Lines are grouped by baseline and font size rather than glyph boxes, because a math-font glyph can be taller than the whole row and would otherwise swallow its neighbours. Chinese joins acquire no spaces. A table stays one row per line. Text that a report form clipped out of its cell is dropped and counted, not duplicated into the body.

Furniture is decided by repetition across pages and by position in the margin, with digit runs collapsed so — ５６ — and — ５７ — are the same template. Fewer than three pages is not enough evidence for repetition, and the result says so.

The first corpus was Chinese legislation and regulatory documents, and the checked-in fixtures are four of them. The rules are written to be general and the fixtures constrain regressions; they are not a claim about arbitrary papers, languages or scans. Table cells, caption association and formula semantics are explicitly not guaranteed.

Incomplete is reported, never hidden

A successful extraction states what it did not do. The document-level diagnostics are always in the result, whatever content was selected: pages processed and failed, whether the page cap cut the document short, how many glyphs were clipped away, how many the fonts could not map. One of the engine-selection fixtures loses twenty-two characters that way, turning 第 123 号 into 123; the count makes that visible instead of silent.

Each page carries a status: a usable text layer, no text layer at all, a text layer that decodes to nothing readable over a page-sized scan, empty, or failed. There is no OCR and no model is called behind the caller’s back. A scanned page is reported as such, and a full-page render is available on request so the caller can choose what to do about it.

One process per document

PDF engines are native code, and a malformed or hostile file can abort the process or never return. The default mode runs the afpdf executable as a child per document with a wall-clock timeout, a bounded result size, a page cap, and an address-space limit where the platform enforces one. An abort, a hang or a limit hit becomes a typed failure for that document; the caller is still standing. The in-process mode exists, explicitly, for trusted input.

The memory limit is honest about where it works: it is enforced on Linux and defaults to zero elsewhere, and asking for one on a platform that cannot enforce it fails every worker rather than running unlimited.

The engine is installed, not assumed

Decoding is PDFium, through pdfium-render. It was chosen against two Rust alternatives on the same documents, and the losers’ strengths became this project’s requirements: figures placed in reading order, clipped text dropped, a status per page, and unmapped glyphs surfaced. PDFium is a shared library, and a tool that silently requires one the user has to go and find is not agent-first. afpdf engine install downloads a pinned build for the platform, verifies it against a checksum compiled into the executable, and probes it; afpdf engine status reports which library would load, whether it does, and what this platform enforces.

What it looks like

$ afpdf engine status
{"kind":"result","result":{"available":true,"build":"8035","code":"engine_status",...,
 "memory_limit_supported":false,"pinned_build":"8035","source":"installed"},"trace":{}}

$ afpdf extract regulation.pdf --set extract.max_pages=2
{"kind":"result","result":{
  "article":{
    "blocks":[
      {"kind":"heading","text":"第一章 总 则","spans":[{"page_no":1,"rect":{...}}],...},
      {"kind":"provision","text":"第一条 为了加强广播电视节目传送业务管理，…","spans":[...],...},
      ...],
    "diagnostics":{"furniture_repetition_skipped":true,"legacy_pages":0},
    "furniture":[...],
    "pages":[{"page_no":1,"status":"text",...},{"page_no":2,"status":"text",...}],...},
  "diagnostics":{"clipped_chars":0,"image_failures":[],"page_limit_reached":true,
                 "pages_failed":0,"pages_processed":2,"unmapped_chars":0},
  "text":"广播电视节目传送业务管理办法\n…",
  "total_pages":7,
  ...},"trace":{}}

Two pages of seven, and the result says so. Organized blocks and continuous text by default; --set output.basic=true adds the per-page evidence, and --set extract.image_mode=embedded adds the image bytes, once, however the rest is configured.

Where it is

The first release is out: v0.1.0 ships the Rust SDK on crates.io and prebuilt afpdf binaries through Homebrew, Scoop and GitHub Releases. Its first consumer, a fetching service, runs it in isolated mode behind its own queue and transport, and the regression that gates every change is that service’s stored output on its own documents. The design principles, the result model and the limits are written down in the repository.