Daeseon Yoo
Back to project
·Tech retro·2 min

The test gate: every bug that reached the user this week is now a test that blocks the build

This week's score was: 6 regressions, all 6 caught by the user — the founder WAS the safety net. Built the real one: a Vitest harness with 39 tests across 5 suites that lock the exact incidents (the shared-terminal corruption, the ultracode-banner copy bug, the Ctrl+3=ESC chord contract, dk-summary stream capture, the honesty hook), wired into the build command itself so a red test means no bundle and no release. Proven by a deliberately failing test that the pipeline refused to ship.

Why now

The regression tracker's first week reads: 6 regressions, 6 slipped to the user (100%). Every single one was found by the founder hitting it live — the gc session scare, the Ctrl+3 interrupt, the copy banner, the dropped summary card, and finally two startups rendering the same terminal. Measurement existed (REGRESSIONS.md + a weekly trend script) but had no teeth: nothing stood between a bad change and a shipped build.

What was built

A Vitest harness whose 39 tests are the literal incidents of this week, not abstract coverage:

  1. layoutSanitizer.test.ts — the shared-terminal bug, using the actual corrupted tree from the user's machine as a fixture: identical layouts → the later startup gets fresh panes, the earlier keeps its live ones; partial overlaps, 3-way shares, malformed JSON, global uniqueness.
  2. textClean.test.ts — the "──── ultracode ─" banner can never become a link or reach the clipboard, on BOTH copy paths (selection and click-a-colored-run).
  3. chordGuards.test.ts — the RULE #10 contract as executable truth: Ctrl+3 is claimed by the app (it's ESC if it leaks), Ctrl+D/Ctrl+W belong to the shell, Cmd variants belong to the app, and no chord matches two handlers.
  4. summaryStripper.test.ts — dk-summary capture across every possible chunk boundary (exhaustive split sweep), so the ⌘I card can't be lost to stream fragmentation.
  5. honestyHook.test.ts — the no-lies Stop-hook's four contracts: block unverified claims, pass evidenced ones, FAIL-OPEN on malformed input, never re-block in a loop.

To make the tests bind to reality, the copy-cleaning and chord logic were extracted into pure modules (textClean.ts, chordGuards.ts) and the production call sites now import them — the tests lock the real logic, not a lookalike copy that could drift.

The gate

"build": "vitest run && tsc && vite build"

Tauri's beforeBuildCommand is pnpm build, so the release pipeline runs the tests before every bundle. Proof, not promise: a deliberately failing test was added — pnpm build exited 1 and the bundle was not regenerated; removing it restored 39/39 green. The v0.2.23 release build's own output contains Tests 39 passed (39).

Honest limits

The rule going forward

A regression found = one line in REGRESSIONS.md + one test the same day. The weekly trend script tells us whether "caught by: user" actually moves toward "caught by: test" — the win condition stays the same: weeks with zero user-caught regressions.