The four asserts in eight lines

Every framework for building agents makes it easy to add a tool and hard to doubt one. I read twenty-five of them in their own documentation and published source: adding a tool is about four lines in nearly all of them, and not one asserts anything about the run that produced its results. A counting gate, a lock on the shared card, an artifact whose identity outlives its bytes, a judge’s reason that is checked rather than stored: none of the twenty-five ships any of them.

I did not set out to write a library. I ran a small fine-tuning harness on one consumer GPU for a fortnight and kept being wrong in the same four ways. These are those four, in the order the failures found me, with the incident behind each one and what it costs to add, measured rather than estimated. The code is public and installable; the harness it came out of is not.

1. The suite that reported fewer tests than it found

My suite printed Ran 3315 tests … OK on a tree where discovery found 3,822. No failure, no error, no skip line: a green tick over 87 percent of the suite, one command away from being pushed on. Two runs had been sharing one machine, and a runner reports what it ran and never what it was supposed to run. The case a count alone cannot see came later: a module that fails to import is replaced by one placeholder that runs like anything else, so the totals agree over a smaller tree. On a clean clone, 35 real cases became 2 placeholders while the count held.

assert_ran: the runner asserts that what ran is what was discovered, prints the denominator, names the modules that failed to import, and exits with its own code, checked before pass or fail is read.

2. The card taken twice

The lock existed and protected only the runs that remembered to take it: two full suites ran concurrently under two interpreters while nobody held it. Then a liveness check answered alive for a process id that belonged to the desktop shell, because every operating system reuses them. And within twenty minutes of the lock line gaining a process identity, a reader called a live lane a dead process, because two writers had filled the birth-time field in two notations and the comparison was a string.

hold: a lock file with a named holder, a purpose, the holder’s process id and its start time; a foreign lock refuses; a green run names the neighbours it can see; and a holder whose liveness cannot be decided is cannot decide, never free.

3. The artifact whose bytes are gone

A path is a location, not a version. A size-and-modification-time stamp is free and it is not identity: on one ordinary filesystem, two different files of the same length written back to back produced an identical stamp in 157 of 200 trials. The same shape cost me a picture in the other product: a chart persisted to a canvas file and wiped by the next write before anything saved it, and two run reports that a test rewrites on every push swept into commits by a blanket add.

witness: sha256 and size kept beside the path, so the record says read N of M and the identity outlives the bytes.

4. The judge reason nobody re-derived

A model asked to grade 72 rewrites that were not degradations kept 19, and every one of the 19 justified itself with a clause the rewrite still contained word for word. One cited a dropped condition where the only difference between the two texts was a removed full stop. Deciding the same question by rule refused 106 non-degradations at zero model calls and admitted 39 planted defects, of which the judge then caught 35.

void_unless: a verdict is void until its stated reason survives a rule re-run over the artifact, and the record distinguishes no rule was registered for this reason from the rule ran and the reason failed it.

The eight lines

from four_asserts import assert_ran, Hold, witness, void_unless

ran = assert_ran(discovered, ran, unimported=modules)     # exit_code 2 if it did not say
with Hold("gpu.hold", holder="lane-a", purpose="a suite"): ...   # waits while the holder lives
kept = witness("rows.jsonl")                              # sha256 + size, outlives the file
rulings = void_unless(verdicts)                           # void until a rule re-derives the reason

Measured on the MCP Python SDK, on a server with three tools and a runner, code lines only:

Against the survey’s own finding that adding a tool is about four lines, adding all four asserts to a server and its runner is eight. On a three-node graph in the other popular framework it was eight again.

What those lines bought on the same server in the same run: a tool that answered 120 rows now answers counted 120 rows; sha256 fd61574c… over all 1328 bytes; a grading tool returned void on a reason claiming a dropped clause where the two texts differ by a full stop, which is the exact fabrication the module was extracted for, caught through a live tool call; and a training tool waited for a planted foreign holder instead of running, and left no lock file behind, checked by planting one, because a context manager that never blocks is decoration.

What it does not do

• It does not decide whether your work is good. It decides whether the record of it says what it appears to say. A run where everything ran and everything failed is a valid record of a bad result.
• void_unless does not decide whether a verdict was right. A reason that survives its rule can still be a wrong call; it says only that the stated reason is about the text in front of it.
• hold is not a distributed lock. One file created atomically, one machine, named holders.
• assert_ran does not explain a mismatch. It is red whenever discovery and execution disagree, whatever the cause, because a guard that needs a diagnosis first fails on the next cause nobody diagnosed.
• The survey is a reading of documentation and published source, not of behaviour. Two projects’ documentation was unreachable and is recorded as not found rather than as absent.

One thing the package found in itself while being lifted out: the harness’s own classifier kept trailing punctuation on a word, so where two texts differed only by a removed full stop it reported a dropped clause for a dropped period. That is the very case the module exists for. Fixed in the package, left in the harness, and the changelog says which.

Since publishing: two real traces

Added 2026-09-06. The survey above is a reading of documentation and published source, and it is unchanged. What it could not tell me is what a working agent harness actually records, so I read two, from two different vendors, both already sitting on this machine: 186,546 records of one and 402,769 of the other, 589,315 in all, none unparseable.

The lock and the identity are absent from both. Across both traces, no key anywhere names a holder, a lock, a pid or a mutex, while one of those harnesses keeps a real lock file on this same disk, so no reader of its record can tell whether two of its threads ever collided. Not one content hash appears as a field value in either.

One of my four is ordinary practice, and I was wrong to imply otherwise. Both vendors pair every tool call to its result by id, and both survive the check from either direction. One of them does it while carrying thousands of duplicate call and result blocks across compaction and resume, because the id carries the meaning rather than the position. That is careful work and it is common, and a quarter of what I called discipline here is simply how traces are built.

And the fourth belongs to a harness, not to traces. One of the two records failure 2,455 times. The other reports success on all 507 of its patch applications, has no failure, abort or rejection event type in its schema at all, and records non-zero exit codes faithfully everywhere else. A gate that cannot fail, in somebody else’s system. That is a choice one team made, and the second trace is what stops it being a claim about agent traces in general.

Two traces are two traces, chosen for being on the disk. What they support is narrow and I will not stretch it: on the only two anyone here has read, the lock and the identity are absent from both.

Where it is

github.com/naidx0/four-asserts — MIT, no dependencies, Python 3.10 and up. Its own suite runs 38 of 38 discovered on both versions in CI, and the README’s usage lines are themselves a CI job, because a README that drifts from the code is worse than no README. Installing it on a machine with nothing but an interpreter and a connection is two commands, which I checked on a clean sandbox rather than assuming.

The harness these came out of is private. The library is the part that was worth handing to anyone else, and it is the part I would take with me if I lost the rest.