749 swallowed errors, ~18 that mattered — auditing the silent failures
After a logging bug reported a dead service as healthy, audited the critical paths for error-swallows that hide real failures. The point wasn't the 749 raw `let _ =` sites — it was the ~18 where a genuine failure of a critical operation vanished with no trace.
Why
A launchd bug had just reported the AI brain "(re)loaded" while it was actually down — a green lie in the log. The founder's response was the right one: a product's logging is broken if an important failure doesn't show up in it. Audit the critical paths.
First, the honest size of the problem. Across the Rust source:
- ~749 sites handle a
Result/Optionwithout logging (let _ =,.ok(),unwrap_or*). - ~95 actual
tracinglog calls.
But 749 is not "749 bugs," and saying so would be its own kind of lie. Most are benign by design: an env var with a sane default, an expected-absent lookup, a cosmetic value. The real question is narrower and harder: where does a failure of a CRITICAL operation vanish with no trace?
The audit
Four read-only subagents, one per subsystem (memory store, terminal/session, connectors, agent push / remote), each told to flag ONLY swallows that hide a real failure of: persisting important state, PTY/terminal lifecycle, git, subprocess spawn, or network ingest — and to explicitly exclude the benign pattern. They came back with ~18 HIGH/MED findings. The worst:
- The whole memory could silently empty.
read_all_with_report(the authoritative graph read) didif let Ok(entries) = read_dir(dir)with noelse. If the store dir became unreadable, it returned an empty Vec — every query and answer would see no memory, indistinguishable from "no memory yet." Same for edges: lose those silently and superseded/corrected facts quietly resurface as current truth. - A dead tmux server looked like "no panes."
list_panes/list_sessionsswallowed a spawn failure or non-zero exit into an empty list. The orphan-GC even logged "killed" when the kill had failed — a zombie reported as reaped. - "Complete" syncs that dropped items. The live connector sync discarded per-item
appenderrors with.is_ok(); an ingested email/issue the store rejected just disappeared and the count under-reported. A CI log-fetch failure was passed off as an empty-but-valid diagnosis and cached forever. - Plus:
pty_killswallowing the local kill (leaked pty), a conflict edge dropped via.ok()?(stale fact keeps being cited), the embedding-cache write swallowed (permanent re-embed cliff), scan offsets / remote token / entropy reads.
Each got a warn! with a target: and structured fields, so it lands in
~/Library/Logs/DalkkakAI/runtime.log where it can actually be seen.
Verified
cargo check0.cargo test --lib— 210 passed, 0 failed, 4 ignored (including the restructuredread_all/read_edgesand the embedding-cache write path).parallel_mission.rs(142 swallow-ish sites, but a read-only cockpit) was deliberately DEFERRED for lower data-loss risk — noted, not silently skipped.
The lesson
let _ = has two faces. Discarding an expected-absent is fine. Discarding a failure of a
critical operation — a persist, a spawn, a network ingest, a lifecycle kill — is a silent
data-loss or zombie bug wearing the same syntax. You can't grep your way to the difference; you
audit by what the operation is, not by counting the pattern. 749 raw sites, ~18 that actually
mattered. A log you can trust is one where the important failures are guaranteed to appear.