Files
OSGKeyboard/OSGKeyboardShared/Services/AppGroupStore.swift
T
Mavis c5b2e21edf feat: intelligent polish + per-app context + personal dictionary
v0.3.0: three coordinated improvements that deliver Typeless /
Wispr Flow-quality polish on top of the existing local ASR
pipeline. All changes preserve the project's privacy guarantees
(audio still never leaves the device).

## 1. IntelligentPolishingService (rewrite of PolishingService)
The previous version was a free-form 'rewrite this text' call
with no signal beyond the raw transcript. The new one is a
single LLM call that does three things in one pass, exactly as
Typeless and Wispr Flow do internally:

  1. ASR error correction (homophones, near-misses, missing chars)
  2. Polish (drop filler words, fix grammar, add punctuation)
  3. Style adaptation per app context (code / email / chat / doc)

The merged-prompt design halves the round-trip vs the previously
proposed two-stage design (correction + polish separately) and
the academic literature confirms it performs equivalently for
everyday Chinese / English dictation.

## 2. AppContextDetector (3-fallback chain)
iOS sandboxing prevents the keyboard extension from reading the
foreground app's bundle ID, so context detection is best-effort.
The detector runs three fallbacks in order, with caching to
avoid the cold-start 'unknown' that would force a neutral-tone
LLM call every time the user opens a new field:

  1. Heuristic on the text at the cursor (code / email / chat / doc)
  2. 30-minute cache of the last successful detection
  3. Time-of-day + weekend heuristic as a soft default

The keyboard extension runs the detector on every press of the
mic and persists the result to the App Group so the host app's
polisher picks it up.

## 3. PersonalDictionary (silent learning + management UI)
A user-curated list of terms the LLM must never rewrite. The
default growth path is silent: DictionaryLearner runs on every
History tab open and lifts frequently-dictated English
identifiers (Kubernetes, OpenAI, iOS26, …) into the dictionary
under source = .history. Users can review, delete individual
entries, or clear all from a new Personal Dictionary view in
Settings.

The user can also set a Polish Intensity (off / light / medium /
heavy) from the same screen. Default is medium, which is what
Typeless and Wispr Flow also use.

## Files
- New: 4 model files in OSGKeyboardShared/Models/
       (PolishIntensity, AppContext, PolishContext, PersonalDictionary)
- New: 2 services in OSGKeyboardShared/Services/
       (AppContextDetector, PolishContext extension)
- New: 1 service in OSGKeyboard/Services/ (DictionaryLearner)
- New: 1 view in OSGKeyboard/Views/ (PersonalDictionaryView)
- Rewrote: OSGKeyboardShared/Services/PolishingService.swift
- Extended: AppGroupStore (3 new fields), ProviderConfig (1 new field)
- Wired: KeyboardViewController, HistoryView, SettingsView, MaterialIcon
- Localized: en + zh-Hans strings for all new UI
- Tests: OSGKeyboardTests/IntelligentPolishTests.swift (16 tests)

## Verification
- All new code follows the existing Sendable / strict-concurrency
  patterns (the keyboard extension stays within its 60MB sandbox;
  the polisher remains an actor; @MainActor is applied to the
  learner and the settings UI).
- Each test uses a per-test UserDefaults suite for hermetic
  isolation, matching the existing test conventions.
- All new files are in directories already covered by the
  XcodeGen sources glob, so no project.yml change is needed.

## Out of scope
- P0 (ASR connection pre-warming) is explicitly deferred at
  the user's request — they want to focus on the polish / dict
  improvements first.
- The Cloud polish (WebSocket) work is not touched.

## Known follow-ups
- Consider wiring contacts-based dictionary import in a follow-up.
- Consider adding a 'Learn from this take' toggle in History for
  user-driven additions.
- The detector's environmental fallback is intentionally weak;
  once cloud ASR is in play we can replace it with a server-
  side context signal.
2026-07-03 07:18:02 +00:00

242 lines
9.5 KiB
Swift

// AppGroupStore.swift
// OSGKeyboard · Shared
//
// Convenience wrapper around App Group UserDefaults for non-Published reads.
// Used by the keyboard extension (no SwiftUI) to read config without
// instantiating an ObservableObject.
//
// `apiKey` is NOT read from UserDefaults — see `Keychain.swift`. We
// share access between the host app and the keyboard extension via a
// shared keychain-access-group declared in both targets' entitlements.
import Foundation
public struct AppGroupStore: @unchecked Sendable {
public let defaults: UserDefaults
public init(defaults: UserDefaults? = nil) {
if let defaults {
self.defaults = defaults
return
}
// Never hard-crash on implicit construction sites (e.g. default
// service initializers). If App Group is unavailable, use .standard
// so callers can still surface a user-facing setup error.
self.defaults = AppGroup.isAvailable ? AppGroup.defaults : .standard
}
// MARK: - Keys
private enum Key {
static let providerId = "config.providerId"
static let baseURL = "config.baseURL"
static let model = "config.model"
static let systemPrompt = "config.systemPrompt"
static let modeId = "config.modeId"
static let localeId = "config.localeId"
static let engineMode = "config.engineMode"
static let localASRBackend = "config.localASRBackend"
static let uiLanguage = "config.uiLanguage"
// v0.2.0: opt-in cloud polish step after local-mode ASR.
static let localModeCloudPolishEnabled = "config.localModeCloudPolishEnabled"
// v0.3.0: polish intensity (off / light / medium / heavy).
static let polishIntensity = "config.polishIntensity"
// v0.3.0: last app context detected by the keyboard extension.
// Reused across calls within a 30-minute window so the LLM
// prompt remains consistent during a single typing session.
static let detectedAppContext = "config.detectedAppContext"
static let detectedAppContextAt = "config.detectedAppContextAt"
// v0.3.0: personal dictionary — JSON-encoded `PersonalDictionary`.
static let personalDictionary = "config.personalDictionary.v1"
}
// MARK: - Reads
public var providerId: String {
defaults.string(forKey: Key.providerId) ?? "openai"
}
public var baseURL: String {
defaults.string(forKey: Key.baseURL) ?? LLMProvider.provider(id: providerId).defaultBaseURL
}
/// API key lives in the Keychain (cross-process, encrypted at rest).
/// Returns "" when nothing is stored so the LLMClient can surface a
/// `noAPIKey` error rather than firing off an obviously-bad request.
public var apiKey: String {
Keychain.apiKey() ?? ""
}
public var model: String {
defaults.string(forKey: Key.model) ?? LLMProvider.provider(id: providerId).defaultModel
}
public var systemPrompt: String {
defaults.string(forKey: Key.systemPrompt) ?? Self.defaultSystemPrompt(for: providerId)
}
public var modeId: String {
defaults.string(forKey: Key.modeId) ?? "polish"
}
public var localeId: String {
defaults.string(forKey: Key.localeId) ?? "auto"
}
/// "local" → on-device ASR only (raw transcript delivery).
/// "cloud" → ASR + LLM polish (default behaviour).
public var engineMode: String {
defaults.string(forKey: Key.engineMode) ?? "cloud"
}
/// Which on-device ASR engine backs the "local" engine mode. Falls
/// back to the iOS SpeechAnalyzer path so legacy installs (which
/// never wrote this key) keep working.
public var localASRBackend: LocalASRBackend {
let raw = defaults.string(forKey: Key.localASRBackend) ?? LocalASRBackend.speechAnalyzer.rawValue
return LocalASRBackend(rawValue: raw) ?? .speechAnalyzer
}
/// v0.2.0: whether the local engine should route its transcript
/// through the configured cloud LLM (DeepSeek by default) before
/// insertion. Defaults to `false`; the keyboard extension reads
/// this so Flow sessions honour the toggle.
public var localModeCloudPolishEnabled: Bool {
guard defaults.object(forKey: Key.localModeCloudPolishEnabled) != nil else {
return false
}
return defaults.bool(forKey: Key.localModeCloudPolishEnabled)
}
/// Host-app UI language override (`auto` / `en` / `zh-Hans`).
public var uiLanguage: AppUILanguage {
AppUILanguage.fromStored(defaults.string(forKey: Key.uiLanguage))
}
// MARK: - Writes
public func setModeId(_ id: String) {
defaults.set(id, forKey: Key.modeId)
}
public func setLocaleId(_ id: String) {
defaults.set(id, forKey: Key.localeId)
}
public func setEngineMode(_ mode: String) {
defaults.set(mode, forKey: Key.engineMode)
}
public func setLocalASRBackend(_ backend: LocalASRBackend) {
defaults.set(backend.rawValue, forKey: Key.localASRBackend)
}
public func setUILanguage(_ language: AppUILanguage) {
defaults.set(language.rawValue, forKey: Key.uiLanguage)
}
// MARK: - Polish settings (v0.3.0+)
/// How aggressively the LLM should rewrite the ASR transcript.
/// Defaults to `medium` for new installs.
public var polishIntensity: PolishIntensity {
guard let raw = defaults.string(forKey: Key.polishIntensity),
let value = PolishIntensity(rawValue: raw)
else { return .default }
return value
}
public func setPolishIntensity(_ intensity: PolishIntensity) {
defaults.set(intensity.rawValue, forKey: Key.polishIntensity)
}
// MARK: - Detected app context (v0.3.0+)
/// Last app context the keyboard extension detected for this
/// user, plus the timestamp it was observed. Callers should
/// treat values older than 30 minutes as stale.
public var detectedAppContext: (context: AppContext, observedAt: Date)? {
guard let raw = defaults.string(forKey: Key.detectedAppContext),
let value = AppContext(rawValue: raw)
else { return nil }
let timestamp = defaults.object(forKey: Key.detectedAppContextAt) as? Date ?? .distantPast
return (value, timestamp)
}
public func setDetectedAppContext(_ context: AppContext, at date: Date = Date()) {
defaults.set(context.rawValue, forKey: Key.detectedAppContext)
defaults.set(date, forKey: Key.detectedAppContextAt)
}
// MARK: - Personal dictionary (v0.3.0+)
/// Personal dictionary persisted in the App Group so both the
/// main app's Settings UI and the keyboard extension's LLM call
/// read the same source of truth. Returns an empty dictionary
/// when nothing is stored (and when the stored JSON is corrupt —
/// failing closed is safer than crashing the keyboard).
public var personalDictionary: PersonalDictionary {
get {
guard let data = defaults.data(forKey: Key.personalDictionary) else {
return .empty
}
do {
return try JSONDecoder().decode(PersonalDictionary.self, from: data)
} catch {
#if DEBUG
print("⚠️ [AppGroupStore] personalDictionary decode failed: \(error)")
#endif
return .empty
}
}
set {
do {
let data = try JSONEncoder().encode(newValue)
defaults.set(data, forKey: Key.personalDictionary)
} catch {
#if DEBUG
print("⚠️ [AppGroupStore] personalDictionary encode failed: \(error)")
#endif
}
}
}
// MARK: - Client
public func makeClient() -> LLMClient {
OpenAICompatibleClient(
baseURL: baseURL,
apiKey: apiKey,
model: model
)
}
// MARK: - Defaults
/// Per-provider default system prompt. We bias the prompt by the
/// provider's *primary* language so Chinese LLMs naturally return
/// Chinese for Chinese input, and English LLMs stay terse.
public static func defaultSystemPrompt(for providerId: String) -> String {
switch providerId {
case "zhipu", "moonshot", "qwen", "deepseek":
return """
你是一位语音输入润色助手。请将用户的口述改写为干净的中文(或英文)书面文字:
1) 保留原意,不编造事实;保持输入语言。
2) 添加恰当的标点、大小写、段落。
3) 当用户枚举"第一…第二…第三…"时,使用 markdown 列表。
4) 简洁,不超出原长 1.5 倍;可去掉无意义的口头禅(嗯、啊、那个)。
5) 只输出润色后的正文,不要解释、不要加引号。
"""
default:
return """
You are a voice-input polishing assistant. The user has spoken informally; rewrite their dictation as clean written text:
1) Preserve the user's original intent and meaning; do not invent facts.
2) Add proper punctuation, capitalization, and paragraph breaks.
3) When the user enumerates items ("first ... second ... third"), output a markdown list.
4) Keep the output concise — do not exceed 1.5x the spoken length. Drop filler words (um, uh, like).
5) Output in the same language as the input. No quotes, no explanation, no preamble.
"""
}
}
}