Files
OSGKeyboard/OSGKeyboardShared/Models/ProviderConfig.swift
T
Mavis c5b2e21edf feat: intelligent polish + per-app context + personal dictionary
v0.3.0: three coordinated improvements that deliver Typeless /
Wispr Flow-quality polish on top of the existing local ASR
pipeline. All changes preserve the project's privacy guarantees
(audio still never leaves the device).

## 1. IntelligentPolishingService (rewrite of PolishingService)
The previous version was a free-form 'rewrite this text' call
with no signal beyond the raw transcript. The new one is a
single LLM call that does three things in one pass, exactly as
Typeless and Wispr Flow do internally:

  1. ASR error correction (homophones, near-misses, missing chars)
  2. Polish (drop filler words, fix grammar, add punctuation)
  3. Style adaptation per app context (code / email / chat / doc)

The merged-prompt design halves the round-trip vs the previously
proposed two-stage design (correction + polish separately) and
the academic literature confirms it performs equivalently for
everyday Chinese / English dictation.

## 2. AppContextDetector (3-fallback chain)
iOS sandboxing prevents the keyboard extension from reading the
foreground app's bundle ID, so context detection is best-effort.
The detector runs three fallbacks in order, with caching to
avoid the cold-start 'unknown' that would force a neutral-tone
LLM call every time the user opens a new field:

  1. Heuristic on the text at the cursor (code / email / chat / doc)
  2. 30-minute cache of the last successful detection
  3. Time-of-day + weekend heuristic as a soft default

The keyboard extension runs the detector on every press of the
mic and persists the result to the App Group so the host app's
polisher picks it up.

## 3. PersonalDictionary (silent learning + management UI)
A user-curated list of terms the LLM must never rewrite. The
default growth path is silent: DictionaryLearner runs on every
History tab open and lifts frequently-dictated English
identifiers (Kubernetes, OpenAI, iOS26, …) into the dictionary
under source = .history. Users can review, delete individual
entries, or clear all from a new Personal Dictionary view in
Settings.

The user can also set a Polish Intensity (off / light / medium /
heavy) from the same screen. Default is medium, which is what
Typeless and Wispr Flow also use.

## Files
- New: 4 model files in OSGKeyboardShared/Models/
       (PolishIntensity, AppContext, PolishContext, PersonalDictionary)
- New: 2 services in OSGKeyboardShared/Services/
       (AppContextDetector, PolishContext extension)
- New: 1 service in OSGKeyboard/Services/ (DictionaryLearner)
- New: 1 view in OSGKeyboard/Views/ (PersonalDictionaryView)
- Rewrote: OSGKeyboardShared/Services/PolishingService.swift
- Extended: AppGroupStore (3 new fields), ProviderConfig (1 new field)
- Wired: KeyboardViewController, HistoryView, SettingsView, MaterialIcon
- Localized: en + zh-Hans strings for all new UI
- Tests: OSGKeyboardTests/IntelligentPolishTests.swift (16 tests)

## Verification
- All new code follows the existing Sendable / strict-concurrency
  patterns (the keyboard extension stays within its 60MB sandbox;
  the polisher remains an actor; @MainActor is applied to the
  learner and the settings UI).
- Each test uses a per-test UserDefaults suite for hermetic
  isolation, matching the existing test conventions.
- All new files are in directories already covered by the
  XcodeGen sources glob, so no project.yml change is needed.

## Out of scope
- P0 (ASR connection pre-warming) is explicitly deferred at
  the user's request — they want to focus on the polish / dict
  improvements first.
- The Cloud polish (WebSocket) work is not touched.

## Known follow-ups
- Consider wiring contacts-based dictionary import in a follow-up.
- Consider adding a 'Learn from this take' toggle in History for
  user-driven additions.
- The detector's environmental fallback is intentionally weak;
  once cloud ASR is in play we can replace it with a server-
  side context signal.
2026-07-03 07:18:02 +00:00

267 lines
12 KiB
Swift

// ProviderConfig.swift
// OSGKeyboard · Shared
//
// User's LLM configuration. Persisted in App Group UserDefaults so both
// the main app and keyboard extension read the same values.
//
// `apiKey` is the exception: it lives in the Keychain (see
// `Keychain.swift`) for at-rest encryption. The first time this struct
// inits after upgrade, a legacy plaintext value from UserDefaults is
// migrated to the Keychain and removed from UserDefaults.
import Foundation
import Combine
public final class ProviderConfig: ObservableObject, @unchecked Sendable {
public static let shared = ProviderConfig()
private enum Key {
static let providerId = "config.providerId"
static let baseURL = "config.baseURL"
// Legacy: apiKey used to live in UserDefaults before the
// migration. We still read it once (see init below) and then
// delete the entry, but no other code path touches this key.
static let apiKeyLegacy = "config.apiKey"
static let model = "config.model"
static let systemPrompt = "config.systemPrompt"
static let modeId = "config.modeId"
static let localeId = "config.localeId"
static let engineMode = "config.engineMode"
static let hasCompletedOnboarding = "config.hasCompletedOnboarding"
static let onboardingPage = "config.onboardingPage"
static let hasAcknowledgedCloudSharing = "config.hasAcknowledgedCloudSharing"
// Which on-device ASR engine to use when `engineMode == "local"`.
// Persisted in the App Group so the keyboard can read the
// selection even though it never instantiates the backend itself.
static let localASRBackend = "config.localASRBackend"
static let uiLanguage = "config.uiLanguage"
// v0.2.0: optional cloud polish step after on-device ASR finishes
// in the local engine. Default `false` — keeps the local engine
// truly local unless the user explicitly opts in.
static let localModeCloudPolishEnabled = "config.localModeCloudPolishEnabled"
// v0.3.0: how aggressively the LLM should rewrite transcripts.
static let polishIntensity = "config.polishIntensity"
}
@Published public var providerId: String {
didSet { defaults.set(providerId, forKey: Key.providerId) }
}
@Published public var baseURL: String {
didSet { defaults.set(baseURL, forKey: Key.baseURL) }
}
@Published public var apiKey: String {
didSet {
// Skip the round-trip on init — we read from Keychain and
// writing the same value back is wasteful.
guard oldValue != apiKey else { return }
do {
try Keychain.setAPIKey(apiKey)
} catch {
#if DEBUG
print("⚠️ [OSGKeyboard] Keychain write failed: \(error)")
#endif
}
}
}
@Published public var model: String {
didSet { defaults.set(model, forKey: Key.model) }
}
@Published public var systemPrompt: String {
didSet { defaults.set(systemPrompt, forKey: Key.systemPrompt) }
}
@Published public var modeId: String {
didSet { defaults.set(modeId, forKey: Key.modeId) }
}
@Published public var localeId: String {
didSet { defaults.set(localeId, forKey: Key.localeId) }
}
/// "local" → on-device ASR only (raw transcript delivery).
/// "cloud" → ASR + LLM polish (always on; modeId kept for compatibility).
@Published public var engineMode: String {
didSet { defaults.set(engineMode, forKey: Key.engineMode) }
}
@Published public var hasCompletedOnboarding: Bool {
didSet {
defaults.set(hasCompletedOnboarding, forKey: Key.hasCompletedOnboarding)
if hasCompletedOnboarding {
onboardingPage = 0
}
}
}
/// Persisted onboarding step so returning from Settings does not reset progress.
@Published public var onboardingPage: Int {
didSet { defaults.set(onboardingPage, forKey: Key.onboardingPage) }
}
/// User confirmed that Cloud polish sends transcripts to their configured third-party API.
@Published public var hasAcknowledgedCloudSharing: Bool {
didSet { defaults.set(hasAcknowledgedCloudSharing, forKey: Key.hasAcknowledgedCloudSharing) }
}
/// Which on-device ASR engine backs the "local" engine mode. Only
/// consulted when `isLocalEngine == true`; the cloud engine always
/// uses `SpeechAnalyzer`.
@Published public var localASRBackend: LocalASRBackend {
didSet { defaults.set(localASRBackend.rawValue, forKey: Key.localASRBackend) }
}
/// When `engineMode == "local"`, optionally route the ASR transcript
/// through the user's configured LLM (DeepSeek by default) before
/// inserting at the cursor. The polish step runs through the same
/// `LLMClient` + `PolishingService` stack the cloud engine uses.
///
/// Defaults to `false` — the local engine is ASR-only out of the
/// box. Users opt in from Settings when the iOS ASR output isn't
/// strong enough (noisy far-field audio, dialectal Chinese, etc.).
@Published public var localModeCloudPolishEnabled: Bool {
didSet { defaults.set(localModeCloudPolishEnabled, forKey: Key.localModeCloudPolishEnabled) }
}
/// Host-app UI language. Also mirrored to the App Group for the keyboard extension.
@Published public var uiLanguage: AppUILanguage {
didSet { defaults.set(uiLanguage.rawValue, forKey: Key.uiLanguage) }
}
/// v0.3.0: how aggressively the LLM should rewrite the ASR
/// transcript. Default is `medium` (Typeless-equivalent). The
/// `off` value never calls the LLM — equivalent to "transcribe
/// only" regardless of `engineMode`.
@Published public var polishIntensity: PolishIntensity {
didSet { defaults.set(polishIntensity.rawValue, forKey: Key.polishIntensity) }
}
public var isConfigured: Bool {
// Local engine (on-device ASR only) doesn't need an API key,
// base URL, or model — the LLM round-trip is skipped entirely.
// Treat it as always-configured so onboarding's "Next" button
// enables the moment the user picks the local path, instead
// of forcing them to fill in cloud fields they won't use.
if isLocalEngine { return true }
return !baseURL.isEmpty && !apiKey.isEmpty && !model.isEmpty
}
/// On-device ASR only; no cloud API required.
public var isLocalEngine: Bool { engineMode == "local" }
/// Whether a transcript produced by the local engine should be
/// sent through the cloud LLM polish step before insertion.
///
/// v0.2.0: the local engine defaults to ASR-only. When the user
/// enables "Cloud polish after ASR" (`localModeCloudPolishEnabled`)
/// we route the transcript through the configured LLM (DeepSeek by
/// default in local mode) — same `PolishingService` code path the
/// cloud engine uses.
///
/// If the user hasn't entered an API key we can't run the polish
/// step; callers should check `Keychain.apiKey()` before invoking.
public var shouldPolishLocalTranscript: Bool {
isLocalEngine && localModeCloudPolishEnabled
}
/// The system prompt the user *sees* in the editor — fall back to the
/// provider-aware default from `AppGroupStore` when nothing is set.
public var defaultSystemPrompt: String {
AppGroupStore.defaultSystemPrompt(for: providerId)
}
private let defaults: UserDefaults
public init(defaults: UserDefaults? = nil) {
let resolvedDefaults: UserDefaults = defaults ?? (AppGroup.isAvailable ? AppGroup.defaults : .standard)
self.defaults = resolvedDefaults
let pid = resolvedDefaults.string(forKey: Key.providerId) ?? "openai"
let preset = LLMProvider.provider(id: pid)
self.providerId = pid
self.baseURL = resolvedDefaults.string(forKey: Key.baseURL) ?? preset.defaultBaseURL
// Resolve the API key with a one-shot migration from the legacy
// UserDefaults slot. After this runs once, `Key.apiKeyLegacy`
// is empty in the suite and all subsequent reads go through the
// Keychain.
self.apiKey = ProviderConfig.resolveAPIKey(defaults: resolvedDefaults)
self.model = resolvedDefaults.string(forKey: Key.model) ?? preset.defaultModel
self.systemPrompt = resolvedDefaults.string(forKey: Key.systemPrompt)
?? AppGroupStore.defaultSystemPrompt(for: pid)
self.modeId = resolvedDefaults.string(forKey: Key.modeId) ?? "polish"
self.localeId = resolvedDefaults.string(forKey: Key.localeId) ?? "auto"
self.engineMode = resolvedDefaults.string(forKey: Key.engineMode) ?? "cloud"
self.hasCompletedOnboarding = resolvedDefaults.bool(forKey: Key.hasCompletedOnboarding)
let savedPage = resolvedDefaults.integer(forKey: Key.onboardingPage)
self.onboardingPage = savedPage > 0 ? savedPage : 0
self.hasAcknowledgedCloudSharing = resolvedDefaults.bool(forKey: Key.hasAcknowledgedCloudSharing)
// Tolerate missing / unknown raw values (e.g. an enum case that
// was renamed in a later build) by falling back to the default
// rather than crashing inside `RawRepresentable.init`.
let rawBackend = resolvedDefaults.string(forKey: Key.localASRBackend) ?? LocalASRBackend.speechAnalyzer.rawValue
self.localASRBackend = LocalASRBackend(rawValue: rawBackend) ?? .speechAnalyzer
// v0.2.0: local-mode cloud polish toggle. Defaults off; users
// opt in from Settings when iOS ASR is too lossy for their
// environment. `object(forKey:) == nil` covers fresh installs
// and upgrades from builds that never wrote the key.
if resolvedDefaults.object(forKey: Key.localModeCloudPolishEnabled) == nil {
self.localModeCloudPolishEnabled = false
} else {
self.localModeCloudPolishEnabled = resolvedDefaults.bool(forKey: Key.localModeCloudPolishEnabled)
}
self.uiLanguage = AppUILanguage.fromStored(
resolvedDefaults.string(forKey: Key.uiLanguage)
)
// v0.3.0: polish intensity. Default to `.medium` for new
// installs and upgrades; the existing `off` / `light` /
// `heavy` values are honored.
if let raw = resolvedDefaults.string(forKey: Key.polishIntensity),
let intensity = PolishIntensity(rawValue: raw) {
self.polishIntensity = intensity
} else {
self.polishIntensity = .default
}
// Cloud no longer exposes off/transcribe; migrate legacy values.
if self.engineMode == "cloud", self.modeId != "polish" {
self.modeId = "polish"
}
}
/// Read the API key from the Keychain, falling back to a one-time
/// migration from the legacy UserDefaults slot.
private static func resolveAPIKey(defaults: UserDefaults) -> String {
if let stored = Keychain.apiKey(), !stored.isEmpty {
return stored
}
if let legacy = defaults.string(forKey: Key.apiKeyLegacy),
!legacy.isEmpty {
try? Keychain.setAPIKey(legacy)
defaults.removeObject(forKey: Key.apiKeyLegacy)
return legacy
}
return ""
}
public func apply(preset: LLMProvider) {
// Capture the *previous* provider id BEFORE we mutate, so the
// system-prompt reset check below can compare against the actual
// prior default.
let oldId = providerId
providerId = preset.id
if !preset.defaultBaseURL.isEmpty {
baseURL = preset.defaultBaseURL
}
if !preset.defaultModel.isEmpty {
model = preset.defaultModel
}
// When switching providers, reset the system prompt to the new
// provider's default — otherwise the user is left editing a
// Chinese prompt on a US-English model.
if systemPrompt.isEmpty
|| systemPrompt == AppGroupStore.defaultSystemPrompt(for: oldId) {
systemPrompt = AppGroupStore.defaultSystemPrompt(for: preset.id)
}
}
public func reset() {
providerId = "openai"
let preset = LLMProvider.provider(id: "openai")
baseURL = preset.defaultBaseURL
apiKey = ""
model = preset.defaultModel
systemPrompt = AppGroupStore.defaultSystemPrompt(for: "openai")
hasAcknowledgedCloudSharing = false
}
}