Active · May 25, 2026
Meta Smart Glass — English Assistant
A real-time English assistant for Koreans on working-holiday visas, targeting Meta Ray-Ban Display. iPhone app + Bun backend + Gemini vision; building against a Mock Device Kit while Display SDK access is pending.
- Role
- Solo
- Stack
- Swift / iOS · Bun · Node.js · Gemini 2.5 Flash (vision) · BLE · Mock Device Kit
Project log
Chronological record of troubleshooting, retros, and updates while building this.
Week of May 25, 20268 entries · 1 Update · 7 Tech retro
Look: name the objects you see in English — running on a real iPhone
Tech retroMay 31, 2026 · 2 min
Pivoted from text translation to the actual differentiated feature — point at a scene, get each object's English name + Korean meaning. Built on Gemini vision, working end-to-end on a physical iPhone.
Phase 1 feature: /api/suggest end-to-end (GPT-4o-mini → Claude Haiku fallback)
Tech retroMay 30, 2026 · 2 min
The first real product feature — a backend English-coach suggestion endpoint with provider fallback, plus a SwiftUI client to drive it. Verified up to the API-key boundary.
Gemini as the (free) primary provider — first real end-to-end responses
Tech retroMay 30, 2026 · 2 min
Added a Gemini provider on the free tier so suggest/translate finally return real output. The catch was Gemini 2.5 Flash's default thinking eating the token budget.
Live camera source for Scan — and wrestling Swift 6 strict concurrency
Tech retroMay 30, 2026 · 2 min
Added a live AVFoundation camera as a second Scan source (alongside PhotosPicker), feeding the same OCR→translate pipeline. The interesting part was getting AVFoundation to compile under Swift 6 strict concurrency.
Scan & Translate: Vision OCR → /api/translate, and a Meta-glasses reality check
Tech retroMay 30, 2026 · 3 min
Second Phase 1 feature — on-device OCR of a menu/sign → Korean translation — plus what I learned about whether the Meta glasses can actually run this yet.
Phase 1 prototype scaffold: Bun /health backend + SwiftUI mock glass HUD
Tech retroMay 28, 2026 · 2 min
First real scaffold — a Bun backend exposing only /health, and a SwiftUI app that polls it and renders a mock glass HUD. LLM/STT/Vision and the Meta SDK are scaffolded as plans, not yet wired.
Repository initialized
UpdateMay 26, 2026 · 1 min
First commit — the repo starts as a bare README; the real Phase 1 scaffold lands two days later.
Stack & scope decisions: Swift-native, iOS-first, cloud-LLM-first
Tech retroMay 25, 2026 · 1 min
Locked the foundational stack and scope for the smart-glass English assistant — Swift-native iOS, cloud LLM before local, and a Mock Device Kit instead of buying hardware.