The bench
A lab notebook for building with AI
Every entry is an experiment, logged like bench work: the why, the shape, the hard part, and a reproduction prompt you can run yourself. Browse by topic, follow a reading path, or scan the archive by date.
Now on the bench
62.7 and 99.9 Are the Same Model
ARC Prize ran one model under two harnesses at all six reasoning-effort levels. The weaker harness swings 45 points across them; the better one never drops below 96.7. Four ARC results landed since late July saying the scaffold sets the score, and only one of them is a controlled comparison.
Open entrySpecial series
Build logs you can follow start to finish
Borrowed Iron
Standing up a retinal-AI research platform on a borrowed 8xH100 grant node, one session at a time.
Builder-Leader Field Guide
A nine-part, do-it-yourself companion to the book: one exercise per chapter for leaders crossing the AI build gap.
DGX Lab
Building a working AI research lab on a single small box, from intelligent gateway to RAG stack to honest benchmarks.
A Production ML Workspace
Five parts on the unglamorous scaffolding that makes ML work survive contact with a team: structure, docs, experiments, agents, collaboration.
Start here
Reading paths through the lab
Getting Started with AI Agents
Beginner to Intermediate · 6 entries · ~45 min
Cutting-Edge AI Research
Intermediate · 6 entries · ~45 min
GPU & ML Development
Beginner to Intermediate · 5 entries · ~51 min
A Production ML Workspace
Intermediate · 5 entries · ~55 min
Explore the archive
113 experiments, by topic and date
- AI Development & Agents9 min
62.7 and 99.9 Are the Same Model
ARC Prize ran one model under two harnesses at all six reasoning-effort levels. The weaker harness swings 45 points across them; the better one never drops below 96.7. Four ARC results landed since late July saying the scaffold sets the score, and only one of them is a controlled comparison.
- AI Development & Agents7 min
I checked the 90% token cut against 30 days of my own Claude Code. The hook would have fired on 3.5% of it.
Spotify's shunt plugin blocks big file reads and routes them to a cheap model. I ran its two hooks against 574 of my own sessions. Both numbers are true and they are not the same number.
- AI Systems & Architecture11 min
Everything In The Family Holds The Key
Researchers extracted hidden chain-of-thought from frontier APIs by replaying encrypted reasoning blocks through the cheapest sibling model, then found live credentials in agent traces already published on GitHub and Hugging Face.
- AI Development & Agents11 min
A Loop With Better Marketing
Twelve words on X produced a five-layer hierarchy, two invented Stanford studies, and a new discipline with no benchmark. The distinction underneath is one variable wide, and you can drag it.
- AI Development & Agents11 min
Four Slots Left
Ten days inside the ICML 2026 agent reproduction contest: 851 claim verdicts, 732 points, 17th of 370, and a full accounting of why 262 of those verdicts were worth nothing.
- Cutting-Edge AI11 min
Tabular foundation models got acquired before they got benchmarked
SAP, NVIDIA and Google all bought or shipped a tabular foundation model in one quarter. The independent evidence is thinner than that suggests, and the license on the best model forbids the benchmark you would use to decide.
- AI Development & Agents8 min
Leading From the Other Side (the Month-Three Checkpoint)
Part 9, the finish. The month-three checkpoint: name how your leadership looks different, audit your archetype, and check that builder-leader stuck.
- AI Development & Agents7 min
From One Operator to a Team (the Phase Everyone Skips)
Part 8. Solo operator to two: share a skill without drift, name the three failure modes, and test whether you're ready to add a third.
- Practical Applications9 min
Build a free Audible replacement in an afternoon with Claude Code
A public-domain audiobook pipeline you own: Standard Ebooks in, Kokoro renders a chaptered M4B, Audiobookshelf streams it to your phone. Plus the places an AI coding agent built the wrong thing with a clean exit code.
- AI Development & Agents8 min
Build the Harness Out (Weeks 2 to 6)
Part 7. Four moves for weeks two to six: turn what you do twice into a skill, split memory, spawn your first agent, ship one real thing.
- Cutting-Edge AI11 min
Bonsai 27B: Frontier Reasoning at 1.7 Bits a Weight
A 27B reasoning model that runs on a laptop and a phone, because the weights are natively binary and ternary, not quantized after the fact. First-hand testing plus an on-device agentic RAG loop.
- AI Development & Agents9 min
Your First Saturday With Claude Code
Part 6, the first one in a terminal. Five exercises for weeks one and two: install Claude Code, bootstrap a harness, rewrite real work, write a skill.
Same desk, other formats
The essay and the book
The bench is where things get built. The same work shows up in two other places, written for people who read the results and do not run the terminal.
The weekly essay
Run Data Run
What building with AI does to biopharma and the people running it. One essay a week on Substack, 1,000-plus readers, the argument without the code.
rundatarun.io →The book
Builder-Leader: The AI Exoskeleton That Crosses the Gap
The argument this notebook is the evidence for: senior people crossing from using AI to building with it, and the harness that carries them over.
Follow the lab
Get the next experiment
New entries land roughly weekly. No digest, no roundup. Just the next build log, when it ships.