AI Audit mode — train default-to-AI by auditing what the AI proposes
The owner's root anxiety is not being able to tell whether AI output is right — which is exactly the JD's default-to-AI grade. Built a voice mode that inverts the spar: the AI proposes a system design or code with planted flaws, and you audit it out loud (restate, flag the risk, ask a clarifying question); it scores understanding and catching, then teaches what you missed.
The owner named the real fear under all the prep: when AI explains something or builds something, he can't tell what's right or what it even did. That's not a side issue — it's the JD's "default to AI" grade stated from the inside. Using AI well isn't accepting its output; it's auditing it. The catch-the-fallible-AI idea already existed in /copilot (a deliberately buggy pair you correct), but it was keyboard-and-textarea, so on a treadmill it didn't exist.
So this mode inverts the sparring engine. In /sparring you explain and the AI attacks. Here the AI proposes and you attack. Two actions on one endpoint, both on gpt-5.4:
- propose: generate a system design (or Java code) for a topic, realistic but with one to three genuine, arguable things a senior should catch — a risky assumption, an ambiguity, a missing failure mode, a wrong-for-scale choice, or for code a subtle bug — plus a hidden answer key of those audit targets.
- grade: take the user's spoken audit and score two things separately — did they understand it (restate), and did they catch the real issues — then reveal the targets, teach every missed item plainly so it clicks immediately, show how a senior would audit it in a few sentences, and escalate with a follow-up.
The page reuses the proven voice loop (record, VAD auto-stop, transcribe, TTS) and prompts the 3-step audit reflex on screen: "So this does ___" / "the risky part is ___ / what if ___?" / "why ___ and not ___?". That reflex is the actual transferable skill; the tool just forces the reps hands-free.
Verified before shipping: a rate-limiter design came back with four real planted targets (cross-region async accuracy, local-cache staleness, TTL window choice, fail-open vs fail-closed). A deliberately weak audit — "looks fine, maybe Redis is slow" — scored understood 3, caught 1, with an 800-character teach of what was missed, a senior model-audit, and a follow-up that pushed on the multi-region scenario. Deploy confirmed by image-digest match and a live POST to the phone.
The build pattern worth keeping: to train auditing, make the model both produce the artifact (with arguable, planted flaws) and grade the critique against its own hidden answer key — and keep the flaws genuine and debatable, not cartoonish, or the user learns to spot toys instead of real risks.