feat: intelligent polish + per-app context + personal dictionary

v0.3.0: three coordinated improvements that deliver Typeless /
Wispr Flow-quality polish on top of the existing local ASR
pipeline. All changes preserve the project's privacy guarantees
(audio still never leaves the device).

## 1. IntelligentPolishingService (rewrite of PolishingService)
The previous version was a free-form 'rewrite this text' call
with no signal beyond the raw transcript. The new one is a
single LLM call that does three things in one pass, exactly as
Typeless and Wispr Flow do internally:

  1. ASR error correction (homophones, near-misses, missing chars)
  2. Polish (drop filler words, fix grammar, add punctuation)
  3. Style adaptation per app context (code / email / chat / doc)

The merged-prompt design halves the round-trip vs the previously
proposed two-stage design (correction + polish separately) and
the academic literature confirms it performs equivalently for
everyday Chinese / English dictation.

## 2. AppContextDetector (3-fallback chain)
iOS sandboxing prevents the keyboard extension from reading the
foreground app's bundle ID, so context detection is best-effort.
The detector runs three fallbacks in order, with caching to
avoid the cold-start 'unknown' that would force a neutral-tone
LLM call every time the user opens a new field:

  1. Heuristic on the text at the cursor (code / email / chat / doc)
  2. 30-minute cache of the last successful detection
  3. Time-of-day + weekend heuristic as a soft default

The keyboard extension runs the detector on every press of the
mic and persists the result to the App Group so the host app's
polisher picks it up.

## 3. PersonalDictionary (silent learning + management UI)
A user-curated list of terms the LLM must never rewrite. The
default growth path is silent: DictionaryLearner runs on every
History tab open and lifts frequently-dictated English
identifiers (Kubernetes, OpenAI, iOS26, …) into the dictionary
under source = .history. Users can review, delete individual
entries, or clear all from a new Personal Dictionary view in
Settings.

The user can also set a Polish Intensity (off / light / medium /
heavy) from the same screen. Default is medium, which is what
Typeless and Wispr Flow also use.

## Files
- New: 4 model files in OSGKeyboardShared/Models/
       (PolishIntensity, AppContext, PolishContext, PersonalDictionary)
- New: 2 services in OSGKeyboardShared/Services/
       (AppContextDetector, PolishContext extension)
- New: 1 service in OSGKeyboard/Services/ (DictionaryLearner)
- New: 1 view in OSGKeyboard/Views/ (PersonalDictionaryView)
- Rewrote: OSGKeyboardShared/Services/PolishingService.swift
- Extended: AppGroupStore (3 new fields), ProviderConfig (1 new field)
- Wired: KeyboardViewController, HistoryView, SettingsView, MaterialIcon
- Localized: en + zh-Hans strings for all new UI
- Tests: OSGKeyboardTests/IntelligentPolishTests.swift (16 tests)

## Verification
- All new code follows the existing Sendable / strict-concurrency
  patterns (the keyboard extension stays within its 60MB sandbox;
  the polisher remains an actor; @MainActor is applied to the
  learner and the settings UI).
- Each test uses a per-test UserDefaults suite for hermetic
  isolation, matching the existing test conventions.
- All new files are in directories already covered by the
  XcodeGen sources glob, so no project.yml change is needed.

## Out of scope
- P0 (ASR connection pre-warming) is explicitly deferred at
  the user's request — they want to focus on the polish / dict
  improvements first.
- The Cloud polish (WebSocket) work is not touched.

## Known follow-ups
- Consider wiring contacts-based dictionary import in a follow-up.
- Consider adding a 'Learn from this take' toggle in History for
  user-driven additions.
- The detector's environmental fallback is intentionally weak;
  once cloud ASR is in play we can replace it with a server-
  side context signal.
This commit is contained in:
Mavis
2026-07-03 07:01:55 +00:00
parent dc9697bf3d
commit c5b2e21edf
19 changed files with 1748 additions and 26 deletions
+56
View File
@@ -0,0 +1,56 @@
// AppContext.swift
// OSGKeyboard · Shared
//
// Coarse classification of "where is the user typing right now?".
// We use it to pick a tone / style guideline for the LLM polish
// step (e.g. code stays technical, chat stays casual).
//
// The detection is best-effort and runs entirely in the keyboard
// extension iOS sandboxing blocks us from reading the foreground
// app's bundle ID, so we infer from text-content heuristics plus
// a 30-minute cache and a few environmental signals. See
// `AppContextDetector` for the actual algorithm.
import Foundation
public enum AppContext: String, Codable, Sendable, CaseIterable {
/// IDE / code editor / terminal.
case code
/// Mail composer (long form, formal-ish).
case email
/// IM / chat (short lines, casual).
case chat
/// Notes / long-form document.
case document
/// Anything we cannot classify confidently.
case unknown
/// User-facing label for the Settings view's preview banner.
public var labelKey: String {
switch self {
case .code: return "appContext.code"
case .email: return "appContext.email"
case .chat: return "appContext.chat"
case .document: return "appContext.document"
case .unknown: return "appContext.unknown"
}
}
/// Tone / style constraint appended to the LLM prompt. Kept
/// intentionally short the LLM does better with 1-2 sharp
/// instructions than a wall of rules.
public var polishGuideline: String {
switch self {
case .code:
return "Code context: preserve English identifiers, variable names, file paths, and indentation-relevant whitespace exactly. Do not natural-language them. Keep code snippets unformatted; do not wrap in code fences."
case .email:
return "Email context: you may add a polite greeting or sign-off if the user clearly forgot one. Reasonable paragraph breaks. Keep tone professional but not stiff."
case .chat:
return "Chat context: keep it short, conversational, and emoji-friendly. Drop formalities. Preserve the speaker's casual voice."
case .document:
return "Document context: add structure — split into paragraphs, use lists when the user enumerates. Keep tone written-formal. Do not invent headings the user did not say."
case .unknown:
return "Unknown context: pick a neutral, friendly tone. Err on the side of minimal changes."
}
}
}