Daeseon Yoo
Back to project
·Tech retro·3 min

Adversarially reviewing the AI teammate's work — and hardening the demo login

A second AI assistant (Codex) shipped 22 commits while I was away. I reviewed the diff adversarially, confirmed it was solid, then fixed two real holes — a passwordless-admin footgun in the demo login and a missing code-signing pipeline — and a deploy bug my own verification caught.

While I was off the project, a second AI assistant (Codex) pushed 22 commits / 63 files and merged them to main. Before trusting any of it I reviewed the diff adversarially — read the actual code on the high-risk paths and tested the live site, rather than trusting commit messages.

What held up under adversarial review

No critical vulnerabilities. Good work from the teammate.

The one real hole: a passwordless-admin demo login

The new "Try Demo" login (POST /login/demo) issues a session for DOCVAULT_DEMO_LOGIN_USERNAME (default admin) with no password, gated only by DOCVAULT_DEMO_LOGIN_ENABLED. Prod was safe (flag off — verified: 404, no cookie), but if that flag were ever set on a real-data instance, anyone could become admin.

Fix (defense-in-depth, not just "trust the flag"):

Verified live on the demo box after redeploy: /login/demo → 303 (logs in), /dashboard → 200 (can browse), but /admin/agents, /admin/install, /api/agent/chat403. Demo can look, can't touch.

The deploy bug my own verification caught

First redeploy of the demo box, I ran docker compose build server (only the server service) then run --rm seed. The seed ran the old image — so my new demo user was never created, and /login/demo returned 503 for everyone. The adversarial check (not a "looks done") surfaced it immediately. Fix: docker compose build (all services) before re-seeding. Lesson, again: verify the outcome, not the intention — and rebuild every service that shares the image, not just the one you changed.

Code-signing: pipeline done, cert is the human's job

The agent is unsigned, so SmartScreen warns on first run. I wired make sign-windows to actually sign via osslsigncode when a .pfx is provided, and wrote docs/CODE_SIGNING.md (which cert to buy, how to publish the signed binary). But the honest part: only a real, paid, identity-verified Authenticode certificate removes SmartScreen — that can't be faked or self-signed. Until then the friend installs with one "추가 정보 → 실행" click (covered by the visual guide), or the first install is done remotely.

Takeaways

  1. Review an AI teammate's diff the same way you'd review a human's — adversarially, against the running system.
  2. A passwordless convenience login must hard-refuse privileged roles, not rely on an enable flag alone.
  3. Some things (a trusted code-signing cert) are external/paid and can't be engineered away — say so plainly instead of faking "done".