The harness is the law: how we made 'no lies' machine-enforced instead of politely requested
Prompt rules decay under pressure — we watched an 'always do this' instruction silently vanish the moment a new instruction competed with it. So the no-lies rule moved out of the model and into the harness: a Stop-hook that runs after every reply, blocks unverified done-claims and missing summary blocks, and feeds the reason back until the model fixes it. Fail-open by design, scoped per-pane by an env var, and locked by 8 tests that gate every build.
The problem: soft rules are requests
DalkkakAI's per-pane Claude carries an injected directive: verify before claiming, tag guesses,
end every substantive reply with a <dk-summary> block (the app renders a session card from it).
It worked — until it didn't. On 2026-06-08 a new instruction (a verification footer) displaced the
"always" summary rule, and the card silently vanished. Nobody changed any code. The model just…
stopped doing it. That's the nature of prompt rules: they live inside the model, so they decay
exactly when you need them — under pressure, in long sessions, when instructions compete.
The fix: move the rule outside the model
Claude Code (the harness — the program that runs the model, executes its tools, and manages the turn) supports Stop hooks: a script the harness runs every time a reply finishes. Ours:
reply ends
→ harness runs honesty-check.cjs (a separate node process)
→ reads the LAST assistant message from the transcript
→ gate 1 (HONESTY): a fixed/done/stable claim with NO evidence marker
(no command output, no commit hash, no [확인]/[추측] tag)? → block
→ gate 2 (DK-SUMMARY): inside a Talkak pane, >200 chars, no <dk-summary> block? → block
→ on block: the harness does NOT end the turn — it feeds the reason back,
and the model must add evidence / re-tag the claim / append the summary to finishThe model cannot skip this. It runs in a different process, after generation, on the transcript. A directive is an appeal to conscience; a hook is airport security — the bag gets scanned no matter how honest the passenger feels.
The details that make it livable
- Fail-open. Any malformed input, missing transcript, or crash in the hook → pass. A broken gate that locks every session out would be worse than no gate; this one can only ever add a block on a clean match.
- No loops.
stop_hook_activemeans one bounce per turn — block → fix → done, never block → fix → block. - Scoped by environment. Talkak injects
DALKKAK_PANE_IDinto each pane's tmux session; the claude process and its hook child inherit it. The summary gate only fires when the var is present — claude sessions outside Talkak are untouched. The honesty gate is global on purpose. - Test-locked. Eight cases (block, pass-with-evidence, fail-open, loop-guard, pane scoping,
short-reply exemption) live in the vitest suite that gates
pnpm build— a regression in the gate itself can't ship.
Honest limits
It's a heuristic. It catches blatant unverified claims ("fixed it", "it's stable") and missing summary blocks; it cannot catch subtle overconfidence, and it can occasionally false-block (one bounce, then through). It reduces; it doesn't zero. That's why both layers stay: the directive steers everywhere, the hook enforces what's mechanically checkable.
The deeper principle, learned the hard way: anything that must ALWAYS happen needs a mechanism outside the thing that might forget. For models, that mechanism is the harness.
(Founder-facing explainer with diagrams: docs/HONESTY_GATE.md. Decision record: ADR-012.)