유대선
프로젝트로
·기술 회고·3

A Korean query couldn't find an English memory — so the brain moved to Python (ADR-020 phase 1)

Our retrieval ranked with a hand-tuned keyword scorer that bypassed the embedding engine entirely — a Korean query scored 0 against an English memory. Phase 1 of ADR-020 routes graph_query through a local Python sentence-transformers sidecar, with the keyword path kept as a graceful fallback.

The bug under the feature

The whole operating-memory wedge rides on one Rust command: graph_query. It filters the founder's memory graph and ranks what's left. The ranking was graph_query_score — a hand-tuned keyword scorer: id match +30, title +24, domain match +14, all-terms-present +10.

Type "환불 처리" (Korean: "refund handling") and ask it to find the English memory "refund request from customer". Score: 0. No literal token overlaps, so the keyword scorer sees nothing. The most relevant memory in the graph ranks the same as noise.

Worse: an embedding engine (memory::EmbeddingEngine) was already loaded and managed in lib.rs. The ranker just never called it. We had the semantic machinery and were ranking with str::contains.

Why it was written that way

An old stack lock — "TypeScript + Rust only" — meant every piece of "intelligence" got written in Rust. But the mature ecosystem for embeddings, classification, and rerank is Python. So the semantic work came out as keyword and regex: the shape you reach for when the right library isn't in the language you're allowed to use. An audit (a 32-agent sweep cross-checked by an independent Codex run) found ~26 sites doing keyword/regex where embeddings belong. graph_query_score and context_score were the worst: keyword rankers bypassing the embedding engine, on the retrieval path that matters most.

Rust is still right for the body — PTY, git, the native window, the process tree. That's the product's identity. It was wrong as the forced default for the brain.

The fix: a brain behind a boundary

ADR-019 had already opened the door (polyglot by boundary — another language is allowed if it lives behind a clean process boundary with a language-neutral contract). ADR-020 walks through it:

Phase 1 is the first real cut. Inside graph_query, after the cheap keyword filter, the matched set is re-ranked by ai_brain::brain_rank() — a ureq POST to /rank that embeds the query and the candidates and sorts by cosine. If the brain isn't reachable, it falls straight back to graph_query_score. Recall stays keyword-gated for now (cheap, bounds the candidate set); only the ordering moved to embeddings. True semantic recall and an embedding cache are phase 2.

Verified, not claimed

POST /rank
{"query":"환불 처리","candidates":["refund request from customer","deploy failed on CI","weekly marketing report"],"top_k":3}
 
→ refund request from customer   0.53   (#1)
  weekly marketing report        0.33
  deploy failed on CI            0.088  (last)

A Korean query, zero literal overlap, ranks the English refund memory first and the unrelated deploy line last. The old scorer gave all three a flat 0.

What's not done: this isn't built into the shipped .app yet and isn't pushed to origin; workitem_context still uses the keyword context_score (next reroute target); the other ~24 sites and a before/after eval harness are still owed. The honest claim is one verified migration, not a finished brain.

The lesson

Write the semantic judgment in the language that has the ML ecosystem, behind a clean process boundary, and keep the old deterministic path as the fallback rather than ripping it out. A stack lock that forces ML into the wrong language doesn't prevent the ML — it ships keyword scorers wearing an "intelligence" label. Cross-lingual relevance is the cheapest litmus there is: if a Korean query can't find an English memory that means the same thing, you're doing string-match, not retrieval.