The Harness That Wrote the Book

The book I just put out argues that knowledge work is changing shape: you stop treating AI as a chat window and start operating it as a persistent, skill-equipped harness, a setup you direct rather than a session you visit. If I'd typed that book by hand over three years, it would have been arguing against itself. So I didn't. I built it through the same kind of harness it describes, and the production is the proof.

This is the engineering writeup: what the harness actually was, where it came from, the one phase that makes nonfiction harder than fiction, and the parts that broke. The whole thing is public at github.com/BioInfo/builder-leader-book, skills included, so you can read the machine instead of taking my word for it.

Where the loop came from

The architecture has a clear lineage, and it isn't a writing tool.

It starts with Karpathy's autoresearch⁴4Andrej Karpathy's own repo: an agent reads a program.md, modifies train.py, evaluates on val_bpb, keeps or discards, and loops roughly a dozen times an hour against a fixed five-minute training budget.: an agent reads a spec file, modifies one constrained surface, runs against a fixed budget, evaluates on a stable metric, keeps or discards, and loops. The discipline is in the constraints. The agent touches one file. The budget is fixed so experiments are comparable. The human writes the spec, not the code.

NousResearch's autonovel⁵5An autonomous fiction pipeline: foundation (world, characters, outline, canon), chapter drafts kept only above an eval threshold, adversarial revision loops, export to PDF, ePub, and audiobook. pointed that same loop at fiction: foundation, draft chapters, keep if the eval clears a threshold, adversarial revision, export. Two ideas there are worth stealing for any long document. A dual immune system: mechanical scanning (banned words, clichés, em-dashes) plus LLM judgment, because each catches what the other misses. And change propagation in both directions: a spec change flows down into chapters, and drafting surfaces gaps that flow back up into the spec.

The core pattern under both: modify, evaluate, keep or discard, repeat, against a stable spec. Point it at training runs and you get autoresearch. Point it at chapters and you get a book pipeline. But fiction has a property nonfiction doesn't, and that's where the build got interesting.

The stable spec as the program

Fiction generates its world from the model's own knowledge. Nonfiction can't. Every claim needs a citation, a primary source, or an explicit flag that it's opinion. That single requirement reshapes the whole harness.

The stable spec lives in canon/: thesis, outline, audience, and a book-specific voice file that extends my global voice rules. That directory is the program.md of the project. Every skill reads it before it runs, which is what keeps a 33,000-word argument coherent across dozens of separate agent invocations spread over days. The agent's surface area is constrained the same way autoresearch constrains it to train.py: a drafting run touches one chapter file, grounded in canon plus the prior chapters for continuity, and nothing else.

The skills that did the work

Everything is a project-local skill under .claude/skills/, not a global one, because each skill only makes sense against this book's canon and manuscript state. Six did the real work:

• book-platform-sweep runs before any chapter is drafted. It hits Perplexity, Hacker News with comment bodies, Reddit, Grok's X search, and direct source fetches, then consolidates raw pulls into a per-chapter findings.md. The hard rule in this project: no chapter gets drafted without its own completed sweep. Research grounding is a phase, not an afterthought.
• book-draft-chapter produces a draft from the outline node plus canon plus findings, checked against both voice files as it writes.
• book-fact-check extracts every factual claim into a tagged assertion registry, runs each through a verification loop, and flags anything unsourced.
• book-red-team runs an adversarial pass: weak arguments, unsupported claims, tonal drift, structural problems.
• book-adversarial-reader is a multi-persona pass, contrarian skeptic, domain expert, hostile reviewer, each producing a critique, looped until the critiques converge.
• book-export assembles the chapters into a typeset PDF and EPUB with front matter, TOC, and citations.

The rigor scaffold inside the review step is lifted, unchanged, from a grant-review pattern I'd already built: four numbered artifacts per chapter. An assertion registry, a data-verification pass, a structural critique, and a red-team. It works for any document where being wrong is expensive.side quest

The loop that separates nonfiction from a novel

Here is the divergence, and it's the whole reason this isn't just autonovel with a different prompt.

A fiction pipeline drafts and revises. A nonfiction pipeline drafts, then has to prove every line. Every chapter runs research sweep, then draft, then claim extraction into a queue, then a verification loop that either attaches a primary source to each claim or flags it for cutting, then a citation pass, then the adversarial readers. The claims-to-verify.md queue is the spine: drafts append to it, the fact-check drains it, and nothing ships with the queue dirty.

That's closer to peer review than to creative writing, and it's the part you can't skip, because the failure mode it catches is specific and nasty.

What broke

The failure modes, because they're the most useful thing here.

The research sweeps pull in synthetic supporting data that looks completely real. Twice, this project carried fabricated sources to the edge of shipping. The first was a single fabricated firm-and-figure. The second was worse: an entire section of one chapter rested on seven fabricated sources, all of which a verification pass confirmed do not exist. Plausible titles, plausible institutions, plausible numbers. Pure hallucination, dressed as a citation cluster.

Two rules came out of that, and they're now load-bearing in the harness:

Softening a fabricated source is not removing it. An earlier fact-check had "softened" some of those fabricated numbers to vague directional claims ("rose sharply" instead of a specific percent) but left the source attribution in the prose. The fabrication still shipped, because the vague version still pointed at a study that doesn't exist. You cut the attribution, not just the number.

The source index is authoritative over the research findings, and a stale registry lies. The findings.md files get written at research time, before any URL is verified, so they treat dead sources as real. A chapter's assertion registry describes the draft it was run against; after any reframe, it's stale and has to be re-run against the current text, across every surface, body and bullets and table cells and image captions, because fabrications hide in the bullets and tables, not just the prose.

So the pipeline now ends with a pre-ship gate: re-run the fact-check against the current drafts, drain the queue, and grep every chapter body against an enumerated list of known-fabricated sources before a build is allowed. The dual immune system from autonovel turned out to be the right instinct. The mechanical grep catches what the LLM judgment rationalizes away, and the LLM judgment catches what the grep can't read.side quest

The result

Kickoff to a typeset manuscript was eight calendar days. The output is around 33,000 words across eleven artifacts, fact-checked, voice-scanned, red-teamed twice, and exported to print-ready PDF and EPUB by the same book-export skill every run used.

The point isn't the eight days, though the compression is real. The point is that the structure that produced the book is the structure the book is about. A constrained surface, a stable spec, a draft-evaluate-keep loop, a dual immune system, and a verification phase that treats every claim as guilty until sourced. None of it required me to be a better writer. It required me to be a better operator of the thing doing the writing, which is the entire argument, demonstrated rather than asserted.

The repo is public. Clone it and read the skills. That was always the deal: the production is the proof.

Builder-Leader: The AI Exoskeleton That Crosses the Gap is out now at builder-leader.com. The companion repo is at github.com/BioInfo/builder-leader-book, and the nine-part field guide companion to this series is at ai.rundatarun.io/series/builder-leader-field-guide.