유대선
프로젝트로
·기술 회고·3

A deterministic anti-fake-completion guardrail, and the seam it isn't bolted to yet

Built a deterministic verification guardrail plus Mission Control completion hardening and a CI regression harness — and the honest part is that the guardrail reaches the app only through the result-inbox seam, never by a direct call.

The recurring failure in an agent platform is the confident "done" with nothing behind it: a worker claims a mutation it never executed, calls a mocked connector a live one, or treats prose in a result doc as passing evidence. So I built a deterministic guardrail to refuse those claims — not an LLM judge, just a pure model that classifies findings and decides what blocks.

packages/shared/src/verificationGuardrail.ts (868 lines, commit b896f81, 2026-06-18) emits findings across ten categories — fake_completion, underimplementation, weak_tests, scope_drift, prompt_literalism, unsafe_command, missing_evidence, unsupported_claim, external_action_without_approval, mock_claimed_as_real. Each finding carries id, severity, category, source, evidence, blocking, and required_fix. Placeholder signals are graded so test fixtures and documented future gaps stay allowed while runtime product placeholders block. The result-doc rules are strict: a bare DONE is rejected, DONE_CANDIDATE needs evidence, missing tests block, docs-only work cannot claim code implementation. On the Action SDK side, an external mutation claim fails without both approval and receipt, and approval and receipt stay distinct gates. Suite: 72 shared tests, 43 action-sdk tests, green.

Now the honest part, because the architecture is the story. The guardrail has zero direct importers in the app. Grep across apps/desktop for verificationGuardrail returns nothing in code — the only hits are string path constants in ownerManualData.ts and the product model, which are manifest text, not imports. The guardrail is consumed in exactly two places: packages/shared/src/missionControlResultInbox.ts (which calls evaluateResultDocGuardrail and folds blocking findings into merge-readiness reasons) and packages/action-sdk/src/index.ts. The app reaches it only transitively: missionControlProductModel.ts imports parseMissionControlResultInboxCard and evaluateMissionControlCompletionGate from @ddalkkak/shared, and the inbox is what calls the guardrail underneath. So the coupling is real but indirect — the product surface depends on the orchestration seam (inbox + completion gate), and the guardrail rides in through that seam rather than being a thing the app calls on its own. That's deliberate (one chokepoint, not ten call sites), but it means "the app enforces the guardrail" is only true the way "the car enforces the brake pads" is — through a part you don't see.

The same week's Mission Control hardening (MC-HMG05–HMG17) is more directly wired but pointedly conservative. A read-only State Doctor Tauri command checks for .lock/.tmp residue, registry/scanner divergence, missing-after-received result docs, and stale verification receipts — and explicitly does not delete, rewrite, or repair anything. HMG06/09/10 then route State Doctor health, worktree-cleanup state, and the real exit-code runner receipt into Final Gate, so divergence can block manual merge readiness. But every one of these stops at advisory: the cleanup plan is copyable text, there is still no durable receipt proving a human ran the cleanup, and repair stays a manual plan. The honest gap the result docs name themselves: rollback enforcement, provider rate-limit/backoff, network-pause, and persisted-schema migration are all out of scope.

The CI harness (.github/workflows/regression.yml, scripts/verify-local.sh, commit 2e9a892, 2026-06-16) is the unglamorous backstop — frozen-lockfile install, scoped typechecks, appsdesktop/server/web/action-sdk/MCP tests, cargo fmt --check, cargo test, scoped Biome. It is scoped, not repo-wide, because a shared worktree drags in parallel package work that isn't green yet, and the workflow says so in comments rather than pretending. Tellingly, verify-local.sh's fake_scan is still a standalone ripgrep — it even globs out verificationGuardrail.ts from its own scan and does not call the shared model. Two anti-fake systems, one deterministic-typed and one shell-regex, not yet unified.