From logging to defense — and why a per-instance action can't be tested with two apps open
The honesty verifier stopped at recording verdicts; the founder pointed out that a log nobody reads is meaningless — the value is acting on the catch. So Dalkkak now feeds the correction back into the agent's pane. The hard part wasn't the code; it was a day lost discovering you cannot live-test a per-instance backend action while two app instances share data.
The verifier could catch a coding agent's false "tests pass" and write a failed verdict. The founder's
reaction was exact: "if it's just logging, it's meaningless — it has to actually defend." A log nobody
opens changes nothing. The value is in acting on the catch.
What "active defense" is
When Dalkkak catches a <dk-verify> claim that's false, it now does two things instead of one:
- Records the verdict (as before).
- Types the real result back into the agent's own pane —
[DALKKAK VERIFY] your claim is FALSE, real exit was 1, redo it with the same verify command— via the pane's PTY, submitted as a turn. The agent must respond to it. No human in the loop; the correction arrives as input the agent can't ignore. The message is built only from re-run facts, flattened to one TUI-safe line, and pinned to the same verify command so the agent can't swap in an easier check.
We also moved the trigger off a 20-second timer onto the existing 400 ms session-events watcher (a "Stop-hook-like" event) so the correction lands seconds after the turn, not up to 20 s later.
The actual lesson (a day, not the code)
The code compiled and unit-tested the first try. Proving it live is where the day went — and the lesson is worth more than the feature.
We tried to test by running the active-defense build as a second app instance, alongside the one the chat was in. Every attempt failed the same silent way: the verdict was recorded, but the correction never injected. The cause, eventually:
- Both instances share the same on-disk graph. The other instance (no injection code) recorded the
verdict first, so our instance saw
is_new = falseand skipped the injection — the dedup that's correct in production defeated the test. - Panes were registered in whichever instance created them, so the active-defense instance often didn't even process the pane the claim came from.
- And the honest agent kept self-correcting at the prompt layer anyway, hiding whether the backend fired.
The principle: a per-instance backend action — like writing to a pane — cannot be cleanly live-tested while two instances share state. The shared dedup and per-instance registration that are right for production tangle every signal. The fix for testing isn't more logging or a cleverer guard (though we did move the inject-once guard from the shared graph to a per-process set); it's one instance. We burned the day trying to be clever around that instead of accepting it.
Where it landed
The active defense, the event-driven trigger, the per-process guard, and a written live-test plan are merged into the integration line (compiles, 18 verify tests green). The real, clean test is now simple: build the app, quit everything else, run the one build, paste a false claim, watch the pane. The setup that fought us for a day was never the feature — it was running two of them.