Files
OSGKeyboard/OSGKeyboardShared/Models/PersonalDictionary.swift
T
Mavis c5b2e21edf feat: intelligent polish + per-app context + personal dictionary
v0.3.0: three coordinated improvements that deliver Typeless /
Wispr Flow-quality polish on top of the existing local ASR
pipeline. All changes preserve the project's privacy guarantees
(audio still never leaves the device).

## 1. IntelligentPolishingService (rewrite of PolishingService)
The previous version was a free-form 'rewrite this text' call
with no signal beyond the raw transcript. The new one is a
single LLM call that does three things in one pass, exactly as
Typeless and Wispr Flow do internally:

  1. ASR error correction (homophones, near-misses, missing chars)
  2. Polish (drop filler words, fix grammar, add punctuation)
  3. Style adaptation per app context (code / email / chat / doc)

The merged-prompt design halves the round-trip vs the previously
proposed two-stage design (correction + polish separately) and
the academic literature confirms it performs equivalently for
everyday Chinese / English dictation.

## 2. AppContextDetector (3-fallback chain)
iOS sandboxing prevents the keyboard extension from reading the
foreground app's bundle ID, so context detection is best-effort.
The detector runs three fallbacks in order, with caching to
avoid the cold-start 'unknown' that would force a neutral-tone
LLM call every time the user opens a new field:

  1. Heuristic on the text at the cursor (code / email / chat / doc)
  2. 30-minute cache of the last successful detection
  3. Time-of-day + weekend heuristic as a soft default

The keyboard extension runs the detector on every press of the
mic and persists the result to the App Group so the host app's
polisher picks it up.

## 3. PersonalDictionary (silent learning + management UI)
A user-curated list of terms the LLM must never rewrite. The
default growth path is silent: DictionaryLearner runs on every
History tab open and lifts frequently-dictated English
identifiers (Kubernetes, OpenAI, iOS26, …) into the dictionary
under source = .history. Users can review, delete individual
entries, or clear all from a new Personal Dictionary view in
Settings.

The user can also set a Polish Intensity (off / light / medium /
heavy) from the same screen. Default is medium, which is what
Typeless and Wispr Flow also use.

## Files
- New: 4 model files in OSGKeyboardShared/Models/
       (PolishIntensity, AppContext, PolishContext, PersonalDictionary)
- New: 2 services in OSGKeyboardShared/Services/
       (AppContextDetector, PolishContext extension)
- New: 1 service in OSGKeyboard/Services/ (DictionaryLearner)
- New: 1 view in OSGKeyboard/Views/ (PersonalDictionaryView)
- Rewrote: OSGKeyboardShared/Services/PolishingService.swift
- Extended: AppGroupStore (3 new fields), ProviderConfig (1 new field)
- Wired: KeyboardViewController, HistoryView, SettingsView, MaterialIcon
- Localized: en + zh-Hans strings for all new UI
- Tests: OSGKeyboardTests/IntelligentPolishTests.swift (16 tests)

## Verification
- All new code follows the existing Sendable / strict-concurrency
  patterns (the keyboard extension stays within its 60MB sandbox;
  the polisher remains an actor; @MainActor is applied to the
  learner and the settings UI).
- Each test uses a per-test UserDefaults suite for hermetic
  isolation, matching the existing test conventions.
- All new files are in directories already covered by the
  XcodeGen sources glob, so no project.yml change is needed.

## Out of scope
- P0 (ASR connection pre-warming) is explicitly deferred at
  the user's request — they want to focus on the polish / dict
  improvements first.
- The Cloud polish (WebSocket) work is not touched.

## Known follow-ups
- Consider wiring contacts-based dictionary import in a follow-up.
- Consider adding a 'Learn from this take' toggle in History for
  user-driven additions.
- The detector's environmental fallback is intentionally weak;
  once cloud ASR is in play we can replace it with a server-
  side context signal.
2026-07-03 07:18:02 +00:00

132 lines
4.7 KiB
Swift
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// PersonalDictionary.swift
// OSGKeyboard · Shared
//
// User-curated list of terms the LLM must never rewrite. Persisted
// in the App Group (JSON-encoded) so both the main app's Settings
// UI and the keyboard extension's LLM call read the same data.
//
// Sources (mutually exclusive per entry):
// - `.manual` user typed it in by hand
// - `.history` auto-extracted from the user's transcription
// history by `DictionaryLearner`
// - `.contacts` imported from the iOS Contacts framework
// - `.recentEdit` extracted from edits the user made to a
// polished transcript before sending
//
// The dictionary is intentionally read-mostly: writes only happen
// from the main app (or from a low-frequency background task). The
// keyboard extension never writes to it.
import Foundation
public struct PersonalDictionary: Codable, Sendable, Equatable {
public var entries: [Entry]
public var version: Int
public init(entries: [Entry] = [], version: 1) {
self.entries = entries
self.version = version
}
public struct Entry: Codable, Sendable, Equatable, Identifiable {
public let id: UUID
public var term: String
public var aliases: [String]
public var category: Category
public var source: Source
public var createdAt: Date
public var usageCount: Int
public init(
id: UUID = UUID(),
term: String,
aliases: [String] = [],
category: Category,
source: Source,
createdAt: Date = Date(),
usageCount: Int = 0
) {
self.id = id
self.term = term
self.aliases = aliases
self.category = category
self.source = source
self.createdAt = createdAt
self.usageCount = usageCount
}
public enum Category: String, Codable, Sendable, CaseIterable {
/// Person / place / brand / organization.
case properNoun
/// API, framework, library, language, file format.
case technical
/// Initialism like LLM, iOS, ML.
case acronym
/// Product name (Typeless, OSGKeyboard, ChatGPT).
case productName
/// Anything that does not fit the above.
case custom
public var labelKey: String {
switch self {
case .properNoun: return "dict.category.properNoun"
case .technical: return "dict.category.technical"
case .acronym: return "dict.category.acronym"
case .productName: return "dict.category.productName"
case .custom: return "dict.category.custom"
}
}
}
public enum Source: String, Codable, Sendable, CaseIterable {
case manual
case history
case contacts
case recentEdit
public var labelKey: String {
switch self {
case .manual: return "dict.source.manual"
case .history: return "dict.source.history"
case .contacts: return "dict.source.contacts"
case .recentEdit: return "dict.source.recentEdit"
}
}
}
/// Renders the entry for the LLM prompt. Includes aliases
/// in parentheses so the LLM recognizes voice variants
/// ("k8s" → "Kubernetes") without renaming.
public func promptFragment() -> String {
if aliases.isEmpty { return term }
return "\(term)\(aliases.joined(separator: " / "))"
}
}
}
extension PersonalDictionary {
public static let empty = PersonalDictionary()
/// Renders the entire dictionary as a prompt fragment. Entries
/// are grouped by category so the LLM can scan quickly. Empty
/// dictionary returns "" so the caller can blindly concatenate.
public func promptFragment() -> String {
guard !entries.isEmpty else { return "" }
let grouped = Dictionary(grouping: entries, by: { $0.category })
var lines: [String] = []
for category in Entry.Category.allCases {
guard let bucket = grouped[category], !bucket.isEmpty else { continue }
let terms = bucket
.sorted { $0.usageCount > $1.usageCount }
.map { $0.promptFragment() }
.joined(separator: "、")
lines.append("【\(category.rawValue)\(terms)")
}
guard !lines.isEmpty else { return "" }
return (
"以下为用户专有词汇,**必须**原样保留,**绝不**改写或翻译:" +
"\n" + lines.joined(separator: "\n")
)
}
}