Don't classify against a label — classify against examples: 3/5 became 9/10
Earlier today the brain failed at classification — short messages collapsed onto the most generic label. The fix wasn't a bigger model; it was defining each class by a handful of real example messages and comparing to their centroid. Same model, 3/5 → 9/10.
The wall, this morning
The brain ranks memory well, but classification flunked: asked to sort a short customer message into intents (bug / B2B / docs / general), it sent almost everything to "general," at low confidence (0.18–0.29). Worse than the hand-tuned keyword rules. I deferred it and wrote down the likely fix — "few-shot exemplars, not bare labels" — as a hypothesis.
Why bare labels fail
To classify, I'd been embedding the label text — "지원/버그", "General" — and picking the closest.
But a label lives in abstract-label space, nowhere near how a real message reads. And the most generic
label is the closest thing to everything, so "general" swallowed the lot. The model wasn't bad; the
representation was.
Few-shot: classify against the centroid of real examples
So instead of one label per class, give each class a handful of real example messages and represent the class by their centroid:
support_bug ← ["the app crashes when I click export",
"I keep getting a 500 on login",
"nothing happens when I hit save, it's frozen", ...]Embed the incoming message and every example, average each class's example vectors into a centroid, and pick the nearest centroid. The class now lives in real-message space, right where the input is.
Live eval on keyword-miss English paraphrases (the cases keyword rules can't catch):
| message | want | got |
|---|---|---|
| "the screen just freezes the second I hit that button" | bug | bug 0.53 |
| "what would this run us for the whole company, ~50 people" | B2B | B2B 0.66 |
| "I can't figure out where to switch this feature on" | docs | docs 0.35 |
| "it keeps dying whenever I export" | bug | bug 0.57 |
| "circling back on the note I sent earlier" | general | wrong_project 0.43 ✗ |
9/10, correct hits at 0.35–0.66 with clear margins — versus 3/5 for bare labels and versus keyword missing every paraphrase. The single miss is a genuinely ambiguous follow-up; a score floor sends those to the keyword fallback rather than forcing a guess.
Shipped
POST /classify_fewshot on the brain (centroids), a Tauri command, ClassifyRule.examples on the pack
contract, classifyFewShot() with the keyword rules as the fallback, English exemplars on the email
pack, wired into the Sell inbox so each item's intent upgrades from keyword to few-shot in the
background. cargo 0, shared 131/131, desktop 330/330.
The lesson
When an embedding model "can't classify," check what you're comparing against. A label is a poor prototype; a few real examples are a great one. The upgrade from 3/5 to 9/10 cost no new model and no training — just the right representation of each class.