feat: harden Flow cold-start/force-quit and polish macOS dictation UX

Fix cold-start overlay recursion that overflowed the main-thread stack when
recording began while the ready overlay was still up; also remove temporary
on-screen Flow DEBUG panels after the orange-mic investigation, and land the
macOS overlay/catalog/layout polish plus related Flow recovery hardening.
This commit is contained in:
Rocky
2026-07-10 12:39:41 +08:00
parent dcb66a9849
commit cdf833935a
104 changed files with 5794 additions and 853 deletions
+1 -1
View File
@@ -116,7 +116,7 @@
- **6.7"** (1290×2796) — 必需
- **6.1"** (1179×2556) — 必需(iPhone 17 Pro
- 5.5" 已被苹果官方文档降级为"可选"iPhone 8 Plus 等已停产机型)
- iPad 截图 — 项目声明 iPhone only无需提供
- iPad 截图 — **现已必需**:项目自 TARGETED_DEVICE_FAMILY "1,2" 起支持 iPadApp Store Connect 要求提供 13″ iPad Pro 截图套组(本条为后续更新覆盖原「iPhone only 无需提供」的结论)
- 每套至少 3 张、最多 10 张
**本审计交付**:
+14
View File
@@ -7,7 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **macOS dictation overlay**: a bottom-centered floating pill appears for any recording path (global hotkey, menu bar, or main window) — shows listening / transcribing state, a stronger live waveform, a one-line live transcript preview (partials when chunked ASR runs), front-app name, and a stop control, then briefly confirms success before fading out without stealing focus. / **macOS 听写浮层**:任意录音路径(全局热键、菜单栏或主窗口)都会在屏幕底部居中出现胶囊浮层——显示聆听 / 识别状态、更强的实时波形、单行转写预览(分块识别时显示 partial)、前台应用名与停止按钮,成功后短暂确认再淡出,且不抢前台焦点。
- **macOS Option-key picker**: Settings → Input lets you choose Left / Right / Either Option as the hold-to-talk key, so the shortcut can avoid conflicts with other apps. / **macOS Option 键选择**:设置 → 输入与快捷键可选择左 / 右 / 任一 Option 作为按住听写键,避免与其他应用冲突。
### Changed
- **Removed temporary Flow DEBUG panels**: the on-screen App Group / session debug text boxes on Home and the keyboard extension are gone now that the orange-mic investigation is closed. / **移除临时 Flow DEBUG 面板**:橙色麦克风排查结束后,首页与键盘扩展上的 App Group / 会话调试文本框已去掉。
- **macOS local ASR catalog**: removed offline Paraformer; SenseVoice / Qwen3 0.6B / Qwen3 1.7B now show Fastest / Most balanced / Best quality badges (users still on Paraformer migrate to Qwen3 0.6B). / **macOS 本地 ASR 目录**:移除 offline ParaformerSenseVoice / Qwen3 0.6B / Qwen3 1.7B 分别标注速度最快 / 最平衡 / 质量最好(仍选 Paraformer 的用户迁移到 Qwen3 0.6B)。
- **macOS visual system**: full-app redesign around the brand line “Speak it. Its typed.” / 「开口即文字。」— grouped sidebar, restrained accent selection, asymmetric Home stats (chars as hero), unified page headers, quieter status footer, and clearer dark-mode card elevation. / **macOS 视觉体系**:围绕品牌句「开口即文字。」/ “Speak it. Its typed.” 做全 App 设计升级——侧栏分组、克制的选中态、首页不对称统计(字数主卡)、统一页头、降权状态栏,以及更清晰的暗色卡片层次。
- **macOS Home stat cards**: the hero word-count card is now a full-width horizontal bar sized to its content (icon + label + big number) instead of a tall card stretched to match its neighbors, removing the large dead space below the number; the "Recent" list was removed from Home since it duplicated the History page. / **macOS 首页统计卡**:字数主卡改为按内容自适应高度的满宽横条(图标+标题+大数字),不再被拉伸到与相邻卡片等高、留出大片空白;首页底部与历史页重复的「最近」列表已移除。
- **macOS page margins**: Home / History / Dictionary / Settings share `pageHorizontalInset` on titles and scroll *content*; ScrollViews / Forms stay full-bleed so the scrollbar sits on the window edge, while cards stay aligned with the page title. Settings keeps native grouped Form for control layout. / **macOS 页边距**:首页 / 历史 / 词库 / 设置在标题与滚动*内容*上共用 `pageHorizontalInset`ScrollView / Form 通栏使滚动条贴窗口右缘,卡片仍与页标题对齐。设置保留原生分组 Form 以保证控件排版。
### Fixed
- **Cold-start overlay recursion crash**: dismissing the ready overlay while an utterance is already recording no longer recurses `refreshHostReady``reconcile``dismiss` on the main thread until stack overflow (`EXC_BAD_ACCESS`). Handoff flags are cleared before any refresh. / **冷启动浮层递归崩溃**:就绪浮层仍在时若已开始录音,不再在主线程上递归 `refreshHostReady``reconcile``dismiss` 直至栈溢出(`EXC_BAD_ACCESS`);交接标志会在任何 refresh 之前先清除。
- **macOS Qwen3 “language” garbage transcript**: Sherpa Qwen3 results that still include the model scaffold (`language Chinese<asr_text>…`) are now stripped to the spoken text; incomplete outputs that stop at the bare word `language` are treated as empty instead of being inserted. / **macOS Qwen3「language」乱码转写**Sherpa Qwen3 结果若仍带模型脚手架(`language Chinese<asr_text>…`)会剥到真实口语文案;不完整输出停在单词 `language` 时按空结果处理,不再插入。
- **macOS local ASR silence garbage output**: dictating with no speech (silence) on a Sherpa-backed local model (SenseVoice/Qwen3/Paraformer) no longer inserts the raw JSON result line (`{"lang": "", "emotion": "", ...}`) as the transcript — it now correctly reports "no speech recognized". / **macOS 本地识别静音乱码**:使用 Sherpa 本地模型(SenseVoice/Qwen3/Paraformer)听写时若未检测到语音,不再把原始 JSON 结果行(`{"lang": "", "emotion": "", ...}`)当作转写文本插入,现在会正确提示「没有识别到语音」。
- **Force-quit mic release**: on termination the host app now synchronously stops `AVAudioEngine`, deactivates `AVAudioSession`, and ends Live Activities (Dynamic Island + Lock Screen) in `applicationWillTerminate`, reducing “microphone in use” errors after reopening. / **强杀麦克风释放**:进程终止时在 `applicationWillTerminate` 内同步停止 `AVAudioEngine`、释放 `AVAudioSession` 并结束 Live Activity(灵动岛 + 锁屏),降低强杀后重开提示麦克风被占用的概率。
## [0.5.2] - 2026-07-09
+7
View File
@@ -117,5 +117,12 @@
<array>
<string>UIInterfaceOrientationPortrait</string>
</array>
<key>UISupportedInterfaceOrientations~ipad</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
<string>UIInterfaceOrientationPortraitUpsideDown</string>
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
</dict>
</plist>
+15
View File
@@ -8,6 +8,21 @@
<array/>
<key>NSPrivacyCollectedDataTypes</key>
<array>
<dict>
<!-- Voice recordings: processed on-device by default; uploaded to
the user's configured ASR provider only when the cloud engine
is explicitly enabled (opt-in with acknowledgment). -->
<key>NSPrivacyCollectedDataType</key>
<string>NSPrivacyCollectedDataTypeAudioData</string>
<key>NSPrivacyCollectedDataTypeLinked</key>
<false/>
<key>NSPrivacyCollectedDataTypeTracking</key>
<false/>
<key>NSPrivacyCollectedDataTypePurposes</key>
<array>
<string>NSPrivacyCollectedDataTypePurposeAppFunctionality</string>
</array>
</dict>
<dict>
<key>NSPrivacyCollectedDataType</key>
<string>NSPrivacyCollectedDataTypeOtherUserContent</string>
+5 -2
View File
@@ -97,8 +97,11 @@ final class AppSceneDelegate: NSObject, UIWindowSceneDelegate {
for item in items {
// `sourceApplication` is only non-nil when the caller belongs to
// the same Apple Developer Team (our own keyboard extension)
// exactly what the host-return whitelist relies on.
if let source = item.source {
// exactly what the host-return whitelist relies on. Record it
// ONLY for the `startflow` handoff: overwriting it for every
// deep link (e.g. `osgkeyboard://settings`) could point a later
// cold-start "return to host" at the wrong app.
if let source = item.source, item.url.host == "startflow" {
FlowSessionBridge.setPendingHostBundleId(source)
}
AppOpenURLRouter.shared.route(item.url)
@@ -15,11 +15,17 @@ enum FlowLiveActivityController {
nonisolated(unsafe) private static var currentPhase: FlowActivityAttributes.ContentState.Phase = .idle
/// If the host app is force-quit its `endSession()` never runs, orphaning
/// the Live Activity. A short `staleDate` lets the system grey it out and
/// reclaim it on its own within ~45s of the process dying. While the host is
/// alive the heartbeat calls `keepAlive()` well inside this window, so a
/// genuinely active session never looks stale.
private static let staleWindow: TimeInterval = 45
/// the Live Activity. `staleDate` semantics (verified against ActivityKit
/// behaviour, not folklore): the *Dynamic Island* presentation is reliably
/// removed shortly after the stale date passes, but the *lock-screen*
/// banner may linger greyed-out depending on the iOS version it is NOT
/// guaranteed to be dismissed. Treating staleDate as "auto-cleanup" is
/// therefore wrong on its own; the full zombie defence is this short
/// window + launch-time reconciliation (`clearOrphanedActivities`) + the
/// widget rendering an explicit "disconnected" state via
/// `context.isStale`. While the host is alive the heartbeat calls
/// `keepAlive()` every ~10 s, well inside this window.
private static let staleWindow: TimeInterval = 30
private static func freshContent(
phase: FlowActivityAttributes.ContentState.Phase
@@ -118,6 +124,8 @@ enum FlowLiveActivityController {
}
/// `applicationWillTerminate` `end` 退
/// ActivityKit `end` XPC watchdog
/// `wait()` ~5
nonisolated static func endAllSynchronouslyOnTerminate() {
let semaphore = DispatchSemaphore(value: 0)
Task.detached(priority: .userInitiated) {
@@ -129,7 +137,7 @@ enum FlowLiveActivityController {
FlowDiagnostics.log("Live Activity ended synchronously on terminate (count=\(count))")
semaphore.signal()
}
semaphore.wait()
_ = semaphore.wait(timeout: .now() + 2)
currentPhase = .idle
currentActivity = nil
}
+352 -79
View File
@@ -54,9 +54,10 @@ final class FlowSessionManager: ObservableObject {
private var currentUtteranceId: UUID?
private var currentCommandSeq: Int64 = 0
private var lastHandledCommandSeq: Int64 = 0
private var isUtteranceRecording = false
/// Published so Home / debug UI can show "recording" instead of a false "ready".
@Published private(set) var isUtteranceRecording = false
/// True from `stopped` until the result/error is written back to App Group.
private var isUtteranceProcessing = false
@Published private(set) var isUtteranceProcessing = false
private var finalizeTask: Task<Void, Never>?
private var asrTask: Task<Void, Never>?
private var utteranceSafetyTask: Task<Void, Never>?
@@ -77,17 +78,50 @@ final class FlowSessionManager: ObservableObject {
private var coldStartRecoveryTask: Task<Void, Never>?
/// Initial proof window cold mic sessions often need >2.5s after app switch.
private static let coldStartAudioProofTimeout: TimeInterval = 6
/// Extra window after the first timeout while the overlay shows a failure hint.
private static let coldStartRecoveryProofTimeout: TimeInterval = 12
/// Guards the once-per-process launch reconciliation (scene reconnects
/// recreate the `@StateObject`-owned manager within the same process).
private static var didRunLaunchReconciliation = false
init() {
// Sessions are (re)started explicitly on app foreground via
// `activateOnForeground()`. We deliberately do NOT silently reattach a
// stored session here after a force-quit that would resurrect capture
// (and keep a stale Live Activity alive) without the user re-opening.
//
// Launch reconciliation: a brand-new process can never own an
// in-flight session, so whatever the previous generation persisted
// (force-quit skips `applicationWillTerminate` entirely when the app
// was suspended) is void. Rotating the generation token also lets the
// keyboard invalidate stale ready snapshots instantly instead of
// waiting out the 60 s heartbeat-zombie window.
//
// Once per PROCESS, not per manager: iOS can disconnect and later
// reconnect the sole scene without killing the process, which
// recreates the `@StateObject` (and thus this init). Re-rotating then
// would wipe live state that belongs to this very process.
if AppGroup.isAvailable, !Self.didRunLaunchReconciliation {
Self.didRunLaunchReconciliation = true
let previous = FlowSessionBridge.rotateHostGeneration()
if previous != nil || FlowSessionBridge.isSessionActive() {
FlowSessionBridge.clearFlowStateOnHostLaunch()
FlowLiveActivityController.clearOrphanedActivities()
FlowSessionDarwin.postSessionChanged()
debug("launch reconciliation: voided previous-generation Flow state")
}
}
capture.onEngineLiveChanged = { [weak self] _ in
self?.refreshHostReady()
}
// A system interruption (call / Siri) stops audio frames mid-utterance;
// fail fast so the user is not silently recording into a gap.
capture.onInterruptionBegan = { [weak self] in
guard let self, self.isUtteranceRecording else { return }
self.failUtterance(
message: AppL10n.string("flow.error.recognitionInterrupted"),
kind: .recognitionInterrupted
)
}
FlowTerminationCoordinator.register(self)
}
@@ -126,6 +160,10 @@ final class FlowSessionManager: ObservableObject {
traceState("startSession.ignored", extra: "reason=alreadyStarting")
return
}
// Claim the flag synchronously: on a cold start the URL router and
// `activateOnForeground()` both fire in the same runloop turn, and
// setting it inside the async body let two start bodies interleave.
isStarting = true
startTask?.cancel()
startTask = Task { @MainActor [weak self] in
@@ -190,11 +228,15 @@ final class FlowSessionManager: ObservableObject {
func dismissColdStartOverlay() {
coldStartRecoveryTask?.cancel()
coldStartRecoveryTask = nil
// Clear handoff flags BEFORE any refreshHostReady call. Otherwise
// refresh reconcileColdStartOverlayIfRecovered dismiss refresh
// recurses until the main-thread stack overflows (EXC_BAD_ACCESS,
// "Thread stack size exceeded due to excessive recursion").
coldStartContext = nil
isColdStartHandoff = false
if isActive {
refreshHostReady()
}
coldStartContext = nil
isColdStartHandoff = false
}
func returnToPendingHostFromColdStart() {
@@ -204,6 +246,21 @@ final class FlowSessionManager: ObservableObject {
func retryColdStartReadiness() {
guard AppGroup.isAvailable else { return }
// A failed cold start leaves capture in a running-but-dead state on
// purpose (the recovery loop keeps probing it). A user-initiated
// retry must instead begin from a clean pipeline: tear down capture
// and the cached ASR instance so `startSession` rebuilds both
// otherwise the retry reuses the zombie engine and is guaranteed to
// hit the same audio-proof timeout.
coldStartRecoveryTask?.cancel()
coldStartRecoveryTask = nil
if capture.running {
capture.stop()
}
sessionASR?.cancel()
sessionASR = nil
sessionASREngineMode = nil
sessionASRWarmedLocaleID = nil
startSession(coldStart: true)
}
@@ -241,7 +298,10 @@ final class FlowSessionManager: ObservableObject {
if isUtteranceRecording || isUtteranceProcessing {
capture.cancelUtterance()
asr.cancel()
// `asr` is a computed property that ALLOCATES a fresh ASRService
// when `sessionASR` is nil never do that inside the ~5 s
// termination window; only cancel an instance that exists.
sessionASR?.cancel()
}
capture.cancelUtterance()
@@ -407,16 +467,42 @@ final class FlowSessionManager: ObservableObject {
private func reactivateCaptureIfNeeded() async {
guard isActive else { return }
// A system interruption (call / Siri) may be in progress. Probe it:
// `setActive(true)` inside `reassertIfRunning` fails while the
// interruption is live and succeeds once it ends which also covers
// the documented case where iOS never delivers `.ended` (the latch
// must not depend on that notification, or the session is dead until
// its TTL). While the probe fails we deliberately do NOT stop or
// rebuild: tearing the engine down would remove the observers the
// `.ended` rebuild relies on and churn the shared session mid-call.
if capture.isInterrupted {
guard capture.reassertIfRunning(), !capture.isInterrupted else { return }
}
if capture.running {
let reasserted = capture.reassertIfRunning()
if reasserted, capture.engineHasRecentAudio() {
if reasserted, await capture.awaitAudioFlowing(timeout: 2) {
sessionWarning = nil
} else if !reasserted {
sessionWarning = AppL10n.string("flow.error.audioUnavailable")
refreshHostReady()
return
}
refreshHostReady()
return
// The await above is a suspension point: the session may have
// ended (expiry, user, teardown) while we waited. Never restart
// the microphone for a session that no longer exists.
guard isActive, !Task.isCancelled else { return }
// Never tear capture down underneath a live utterance either a
// stalled route transition mid-recording must surface through the
// utterance pipeline (safety timer / empty-transcript error), not
// as a silent stop that truncates the take with no error at all.
guard !isUtteranceRecording, !isUtteranceProcessing else {
refreshHostReady()
return
}
// Reassert failed, or the engine reports running yet produces no
// frames (zombie after suspend / mediaserverd reset) fall
// through to a full rebuild instead of leaving it half-dead.
capture.stop()
debug("capture zombie after foreground — rebuilding")
}
do {
@@ -442,7 +528,8 @@ final class FlowSessionManager: ObservableObject {
reason: .noSession,
engineMode: store.engineMode,
localeId: store.localeId,
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt()
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt(),
hostGeneration: FlowSessionBridge.currentHostGeneration()
)
)
return
@@ -486,7 +573,8 @@ final class FlowSessionManager: ObservableObject {
engineMode: store.engineMode,
localeId: store.localeId,
busyUtteranceId: isUtteranceRecording || isUtteranceProcessing ? currentUtteranceId : nil,
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt()
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt(),
hostGeneration: FlowSessionBridge.currentHostGeneration()
)
)
let signature = [
@@ -509,9 +597,19 @@ final class FlowSessionManager: ObservableObject {
/// shows a stale preparing/failed snapshot, heal automatically.
private func reconcileColdStartOverlayIfRecovered() {
guard isColdStartHandoff, isActive else { return }
guard FlowSessionBridge.isHostReady() else { return }
guard let context = coldStartContext else { return }
// Mid-utterance is not "ready" dismiss the ready overlay so Home
// does not keep advertising "" while utt.rec=1.
if isUtteranceRecording || isUtteranceProcessing {
if case .ready = context.state {
dismissColdStartOverlay()
}
return
}
guard FlowSessionBridge.isHostReady() else { return }
switch context.state {
case .preparing:
presentColdStartReadyOverlay()
@@ -556,10 +654,11 @@ final class FlowSessionManager: ObservableObject {
// MARK: - Session start
private func startSessionAsync(duration: TimeInterval?) async {
traceState("startSessionAsync.begin")
isStarting = true
sessionWarning = nil
// `isStarting` was claimed synchronously in `startSession()`.
defer { isStarting = false }
guard !Task.isCancelled else { return }
traceState("startSessionAsync.begin")
sessionWarning = nil
guard AppPermissions.flowRequirementsMet else {
sessionWarning = permissionWarningMessage()
@@ -586,7 +685,9 @@ final class FlowSessionManager: ObservableObject {
return
}
guard await waitForAudioProof() else {
let audioProved = await waitForAudioProof()
guard !Task.isCancelled else { return }
guard audioProved else {
let message = AppL10n.string("flow.coldStart.error.audioTimeout")
sessionWarning = message
traceState("startSessionAsync.failed", extra: "reason=audioProofTimeout")
@@ -658,6 +759,16 @@ final class FlowSessionManager: ObservableObject {
refreshHostReady()
guard FlowSessionBridge.isHostReady() else {
// Busy broken: a startflow arriving mid-utterance (e.g. tapping
// the Live Activity while dictating) finds a healthy session that
// is simply recording/processing. Showing the audio-failure
// overlay here would be a lie and its recovery loop could even
// stop capture and kill the live utterance.
if isUtteranceRecording || isUtteranceProcessing {
dismissColdStartOverlay()
debug("cold-start handoff ignored: session busy with an utterance")
return
}
let message = AppL10n.string("flow.coldStart.error.audioTimeout")
sessionWarning = message
showColdStartAudioFailure(message: message)
@@ -687,18 +798,54 @@ final class FlowSessionManager: ObservableObject {
}
}
/// Keeps proving mic readiness after the first timeout instead of tearing
/// capture down many handoffs become ready a few seconds later.
/// Actively rebuilds the audio pipeline after a failed cold start instead
/// of passively waiting for frames that a dead engine will never produce.
/// Escalates per attempt: reassert the session full engine rebuild
/// bounce the audio session and rebuild. Force-quit relaunches routinely
/// inherit stale mediaserverd state that only a rebuild clears.
private func scheduleColdStartRecovery(duration: TimeInterval?) {
coldStartRecoveryTask?.cancel()
coldStartRecoveryTask = Task { @MainActor [weak self] in
guard let self else { return }
let recovered = await self.capture.awaitAudioFlowing(
timeout: Self.coldStartRecoveryProofTimeout
)
var recovered = false
for attempt in 1...3 {
guard !Task.isCancelled, self.isColdStartHandoff else { return }
switch attempt {
case 1:
_ = self.capture.reassertIfRunning()
case 2:
self.capture.stop()
try? self.capture.start()
default:
self.capture.stop()
try? await Task.sleep(nanoseconds: 300_000_000)
guard !Task.isCancelled else { return }
try? self.capture.start()
}
recovered = await self.capture.awaitAudioFlowing(
timeout: TimeInterval(attempt + 1)
)
self.traceState(
"coldStartRecovery.attempt",
extra: "attempt=\(attempt) recovered=\(recovered)"
)
if recovered { break }
}
guard !Task.isCancelled else { return }
guard self.isColdStartHandoff else { return }
guard recovered else { return }
guard recovered else {
// Out of attempts leave the failure overlay up; its retry
// button now performs a full teardown so the user always has
// a working escape hatch (no more force-quit loops). Only
// tear capture down when no session owns it: for an active
// session the 1 Hz heartbeat keeps self-healing, and a stop
// here would just fight it.
if !self.isActive {
self.capture.stop()
}
self.traceState("coldStartRecovery.exhausted")
return
}
self.sessionWarning = nil
self.traceState("coldStartRecovery.recovered")
@@ -1027,12 +1174,22 @@ final class FlowSessionManager: ObservableObject {
// Do NOT cancel `asrTask` or `asr` drain trailing PCM, then finalize.
// Capture ids now: a cancelled finalize must still clear *this*
// utterance's processing gate even if currentUtteranceId was cleared
// by a racing fail/abort path.
let drainingSessionId = activeSessionId
let drainingUtteranceId = currentUtteranceId
let drainingCommandSeq = currentCommandSeq
finalizeTask?.cancel()
finalizeTask = Task { @MainActor [weak self] in
guard let self else { return }
let drainReport = await self.capture.endUtteranceAndDrain()
FlowDiagnostics.logDrain(drainReport)
await self.finalizeUtterance()
await self.finalizeUtterance(
sessionId: drainingSessionId,
utteranceId: drainingUtteranceId,
commandSeq: drainingCommandSeq
)
}
debug("utterance stopped, draining tail")
}
@@ -1109,20 +1266,22 @@ final class FlowSessionManager: ObservableObject {
debug("utterance processing failed: \(message)")
}
private func finalizeUtterance() async {
let finalizeSessionId = activeSessionId
let finalizeUtteranceId = currentUtteranceId
private func finalizeUtterance(
sessionId finalizeSessionId: UUID?,
utteranceId finalizeUtteranceId: UUID?,
commandSeq finalizeCommandSeq: Int64
) async {
let pipelineStarted = Date()
// ALWAYS clear the processing gate for this utterance. The previous
// guard required currentUtteranceId to still match; a racing
// fail/abort/cancel path could nil the id (or leave processing stuck)
// and then skip refreshHostReady keyboard stayed white forever
// while host logs still said "utterance finalized".
defer {
if activeSessionId == finalizeSessionId,
currentUtteranceId == finalizeUtteranceId {
isUtteranceProcessing = false
FlowLiveActivityController.update(phase: .idle)
touchSessionActivity()
currentUtteranceId = nil
currentCommandSeq = 0
refreshHostReady()
}
completeFinalizeCleanup(
sessionId: finalizeSessionId,
utteranceId: finalizeUtteranceId
)
}
let asrWait = asrWaitTimeout()
@@ -1134,6 +1293,10 @@ final class FlowSessionManager: ObservableObject {
while Date() < asrDeadline {
if !lastFinal.isEmpty { break }
if asrTask?.isCancelled == true { break }
// Honour cooperative cancel so a replaced finalize exits promptly,
// but still run defer cleanup (unlike an early `return` mid-polish
// that used to leave processing=true when ids no longer matched).
if Task.isCancelled { break }
try? await Task.sleep(nanoseconds: 100_000_000)
}
@@ -1151,14 +1314,21 @@ final class FlowSessionManager: ObservableObject {
text = currentPartial.trimmingCharacters(in: .whitespacesAndNewlines)
}
guard !text.isEmpty else {
let key = (asrTask?.isCancelled == true)
let key = (asrTask?.isCancelled == true || Task.isCancelled)
? "flow.error.recognitionInterrupted"
: "flow.error.noSpeech"
let kind: FlowSessionKeys.TranscriptionErrorKind =
(asrTask?.isCancelled == true) ? .recognitionInterrupted : .noSpeech
(asrTask?.isCancelled == true || Task.isCancelled)
? .recognitionInterrupted : .noSpeech
FlowDiagnostics.log("finalize failed: empty transcript after \(String(format: "%.1f", asrElapsed))s")
utteranceRecordingStartedAt = nil
storeCurrentError(AppL10n.string(key), kind: kind)
storeFinalizedError(
AppL10n.string(key),
kind: kind,
sessionId: finalizeSessionId,
utteranceId: finalizeUtteranceId,
commandSeq: finalizeCommandSeq
)
return
}
@@ -1178,23 +1348,33 @@ final class FlowSessionManager: ObservableObject {
"translationTarget=\(pipelineStore.translationTargetLocaleId)"
)
do {
// If the finalize task was cancelled (cold-start churn / abort),
// skip the LLM round-trip and deliver the raw transcript so the
// keyboard is not left waiting on a result that never arrives.
if Task.isCancelled {
throw CancellationError()
}
let polished = try await polisher.polish(
text,
mode: polishMode,
providerIdOverride: pipelineStore.polishProviderIdOverride
)
delivered = polished
storeCurrentFinal(polished, warning: chunkNote)
storeFinalizedResult(
polished,
warning: chunkNote,
sessionId: finalizeSessionId,
utteranceId: finalizeUtteranceId,
commandSeq: finalizeCommandSeq
)
FlowDiagnostics.log(
"polish done in \(String(format: "%.1f", Date().timeIntervalSince(polishStarted)))s " +
"total=\(String(format: "%.1f", Date().timeIntervalSince(pipelineStarted)))s"
)
} catch {
// v0.2.0: local + cloud-polish-on + no API key surfaces
// `.missingAPIKey`. We translate it into a polishWarning
// so the keyboard can show the "fill in your key" hint
// inline rather than a generic failure message. The raw
// transcript is still delivered no data loss.
// CancellationError is common when the user jumps back via
// startflow mid-polish; still deliver raw text. Other errors
// keep the existing polish-warning fallback.
let fallback = Self.makeFallbackDelivery(
rawText: text,
error: error,
@@ -1206,7 +1386,13 @@ final class FlowSessionManager: ObservableObject {
"\(error.localizedDescription)"
)
delivered = fallback.text
storeCurrentFinal(fallback.text, warning: fallback.polishWarning)
storeFinalizedResult(
fallback.text,
warning: fallback.polishWarning,
sessionId: finalizeSessionId,
utteranceId: finalizeUtteranceId,
commandSeq: finalizeCommandSeq
)
}
SpeechHistoryStore.shared.recordUtterance(
@@ -1223,6 +1409,92 @@ final class FlowSessionManager: ObservableObject {
debug("utterance finalized length=\(text.count)")
}
/// Drop the processing gate and republish hostReady after finalize.
/// Must not depend on a perfect id match a racing fail/abort/cancel
/// used to skip this block and leave the keyboard stuck on white
/// even after "utterance finalized" was logged.
private func completeFinalizeCleanup(sessionId: UUID?, utteranceId: UUID?) {
// A newer utterance may have started; never clobber its gate.
if let current = currentUtteranceId,
let finished = utteranceId,
current != finished {
debug(
"finalize cleanup skipped — newer utterance live " +
"finished=\(finished.uuidString.prefix(8)) current=\(current.uuidString.prefix(8))"
)
return
}
let wasProcessing = isUtteranceProcessing
isUtteranceProcessing = false
FlowLiveActivityController.update(phase: .idle)
if isActive {
touchSessionActivity()
}
if currentUtteranceId == utteranceId || currentUtteranceId == nil {
currentUtteranceId = nil
currentCommandSeq = 0
}
refreshHostReady()
debug(
"finalize cleanup done wasProcessing=\(wasProcessing ? 1 : 0) " +
"utterance=\(utteranceId?.uuidString.prefix(8) ?? "nil") " +
"session=\(sessionId?.uuidString.prefix(8) ?? "nil")"
)
}
private func storeFinalizedResult(
_ text: String,
warning: String?,
sessionId: UUID?,
utteranceId: UUID?,
commandSeq: Int64
) {
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else {
storeFinalizedError(
AppL10n.string("flow.error.noSpeech"),
kind: .noSpeech,
sessionId: sessionId,
utteranceId: utteranceId,
commandSeq: commandSeq
)
return
}
guard let sessionId, let utteranceId else { return }
FlowSessionBridge.writeResult(
FlowResult(
sessionId: sessionId,
utteranceId: utteranceId,
commandSeq: commandSeq,
status: .final,
text: trimmed,
warning: warning
)
)
}
private func storeFinalizedError(
_ message: String,
kind: FlowSessionKeys.TranscriptionErrorKind,
sessionId: UUID?,
utteranceId: UUID?,
commandSeq: Int64,
status: FlowResult.Status = .error
) {
guard let sessionId, let utteranceId else { return }
FlowSessionBridge.writeResult(
FlowResult(
sessionId: sessionId,
utteranceId: utteranceId,
commandSeq: commandSeq,
status: status,
text: message,
errorKind: kind
)
)
}
private static func polishModeLogLabel(_ mode: PolishingService.PolishMode) -> String {
switch mode {
case .polish:
@@ -1252,35 +1524,12 @@ final class FlowSessionManager: ObservableObject {
engineMode: String,
chunkWarning: String?
) -> TranscriptionDelivery {
let fallbackText = TranscriptPostProcessor.cleanRawASRFallback(rawText)
let warning = warningFromPolishError(error, engineMode: engineMode)
?? polishDegradedWarning()
?? chunkWarning
return TranscriptionDelivery(text: fallbackText, polishWarning: warning)
}
private static func warningFromPolishError(_ error: Error, engineMode: String) -> String? {
if let polishError = error as? PolishingService.PolishError {
switch polishError {
case .missingAPIKey:
if engineMode == "local" {
return SharedL10n.string("flow.warning.localPolishUnavailable")
}
return SharedL10n.string("flow.warning.cloudPolishMissingKey")
case .timeout:
return polishDegradedWarning()
case .noTranscript:
return nil
}
}
if error is LLMError {
return polishDegradedWarning()
}
return nil
}
private static func polishDegradedWarning() -> String? {
SharedL10n.string("flow.warning.polishDegraded")
TranscriptionPolishFallback.makeDelivery(
rawText: rawText,
error: error,
engineMode: engineMode,
chunkWarning: chunkWarning
)
}
private func asrWaitTimeout() -> TimeInterval {
@@ -1318,8 +1567,8 @@ final class FlowSessionManager: ObservableObject {
// Refresh the Live Activity `staleDate` every N heartbeat ticks
// (1 Hz) well inside `FlowLiveActivityController.staleWindow` so a
// live session never looks stale, while a force-quit stops these
// refreshes and lets the system reclaim the orphaned island.
let liveActivityKeepAliveEveryTicks = 15
// refreshes and lets the island go stale within ~30 s.
let liveActivityKeepAliveEveryTicks = 10
var tick = 0
while !Task.isCancelled {
guard let self else { break }
@@ -1351,6 +1600,30 @@ final class FlowSessionManager: ObservableObject {
FlowDiagnostics.log(message)
}
// MARK: - Temporary Flow debug panel (remove after orange-mic investigation)
/// Snapshot for the on-screen debug panel. Safe to call from the main actor.
func makeDebugRows() -> [FlowDebugRow] {
let snapshot = FlowSessionBridge.readySnapshot()
let hostRows = FlowDebugAppGroupSnapshot.rows()
let memRows: [FlowDebugRow] = [
FlowDebugRow("isActive", isActive ? "1" : "0"),
FlowDebugRow("isStarting", isStarting ? "1" : "0"),
FlowDebugRow("coldStart", isColdStartHandoff ? "1" : "0"),
FlowDebugRow("engineLive", capture.engineIsLive ? "1" : "0"),
FlowDebugRow("audioFresh", capture.engineHasRecentAudio(maxAge: 2) ? "1" : "0"),
FlowDebugRow("mem.reason", snapshot?.reason.rawValue ?? "nil"),
FlowDebugRow("utt.rec", isUtteranceRecording ? "1" : "0"),
FlowDebugRow("utt.proc", isUtteranceProcessing ? "1" : "0"),
FlowDebugRow("sessionId", activeSessionId.map { String($0.uuidString.prefix(8)) } ?? "nil"),
FlowDebugRow("warning", sessionWarning == nil ? "0" : "1"),
FlowDebugRow("overlay", coldStartContext.map { String(describing: $0.state) } ?? "nil"),
FlowDebugRow("bridgeReady", FlowSessionBridge.isHostReady() ? "1" : "0")
]
// Prefer App Group snap.reason near the top of the shared block.
return memRows + hostRows
}
private func traceIgnoredCommand(reason: String, command: FlowCommand, detail: String) {
let signature = "\(reason)|\(command.action.rawValue)|\(command.commandSeq)|\(command.sessionId.uuidString)|\(command.utteranceId.uuidString)|\(detail)"
guard signature != lastIgnoredCommandSignature else { return }
+204
View File
@@ -0,0 +1,204 @@
// ASRSettingsCard.swift
// OSGKeyboard · Main App
//
// Cloud ASR credentials independent from the polish LLM card.
import SwiftUI
import OSGKeyboardShared
struct ASRSettingsCard: View {
@Environment(\.themePalette) private var palette: ThemePalette
@ObservedObject var config: ProviderConfig
@State private var showKey: Bool = false
@State private var testStatus: TestStatus = .idle
private enum TestStatus: Equatable {
case idle
case running
case success
case failure(String)
}
var body: some View {
VStack(spacing: 0) {
if CloudASRModelCatalog.strategy(for: config.asrProviderId) == .prompt {
field(
title: AppL10n.string("api.baseUrl"),
placeholder: "https://api.openai.com/v1",
text: $config.asrBaseURL,
autocap: false
)
Divider().background(palette.divider)
}
keyField
Divider().background(palette.divider)
field(
title: AppL10n.string("settings.asr.model"),
placeholder: CloudASRModelCatalog.defaultModel(for: config.asrProviderId),
text: $config.asrModel,
autocap: false
)
if let url = LLMProvider.provider(id: config.asrProviderId).apiKeyURL {
Divider().background(palette.divider)
Button {
UIApplication.shared.open(url)
} label: {
HStack {
Text("api.getKey")
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
Spacer()
Image(systemName: "arrow.up.right.square")
.foregroundStyle(palette.textSecondary)
}
.padding(.horizontal, Spacing.md)
.frame(minHeight: SettingsListMetrics.singleLineMinHeight)
.contentShape(Rectangle())
}
.buttonStyle(.plain)
}
Divider().background(palette.divider)
testConnectionRow
}
.background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
.overlay(
RoundedRectangle(cornerRadius: Radius.large, style: .continuous)
.stroke(palette.divider, lineWidth: 0.5)
)
}
private var keyField: some View {
VStack(alignment: .leading, spacing: 6) {
HStack {
Text("api.key")
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
Spacer()
Button(action: { showKey.toggle() }) {
Image(systemName: showKey ? "eye.slash.fill" : "eye.fill")
.foregroundStyle(palette.textSecondary)
}
.buttonStyle(.plain)
}
Group {
if showKey {
TextField("sk-…", text: $config.asrApiKey)
} else {
SecureField("sk-…", text: $config.asrApiKey)
}
}
.keyboardType(.asciiCapable)
.textInputAutocapitalization(.never)
.autocorrectionDisabled(true)
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
}
.padding(.horizontal, Spacing.md)
.frame(minHeight: SettingsListMetrics.doubleLineMinHeight, alignment: .center)
}
@ViewBuilder
private func field(
title: String,
placeholder: String,
text: Binding<String>,
autocap: Bool
) -> some View {
VStack(alignment: .leading, spacing: 6) {
Text(title)
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
TextField(placeholder, text: text)
.keyboardType(.asciiCapable)
.autocorrectionDisabled(true)
.textInputAutocapitalization(autocap ? .sentences : .never)
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
}
.padding(.horizontal, Spacing.md)
.frame(minHeight: SettingsListMetrics.doubleLineMinHeight, alignment: .center)
}
private var testConnectionRow: some View {
VStack(alignment: .leading, spacing: 6) {
HStack {
Text("settings.asr.testConnection")
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
Spacer()
Button(action: runTest) {
Group {
if testStatus == .running {
ProgressView().controlSize(.mini)
} else {
Text(testButtonLabel)
.font(TypeStyle.body)
.foregroundStyle(testTint)
}
}
}
.buttonStyle(.plain)
.disabled(testStatus == .running)
}
if let detail = testDetail {
Text(detail)
.font(TypeStyle.caption2)
.foregroundStyle(testTint)
.lineLimit(3)
}
}
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.xs)
.frame(minHeight: SettingsListMetrics.singleLineMinHeight, alignment: .center)
}
private var testButtonLabel: String {
switch testStatus {
case .idle: return AppL10n.string("api.test.idle")
case .running: return AppL10n.string("api.test.running")
case .success: return AppL10n.string("api.test.success")
case .failure: return AppL10n.string("api.test.failure")
}
}
private var testTint: Color {
switch testStatus {
case .idle, .running: return palette.accent
case .success: return palette.accent
case .failure: return palette.danger
}
}
private var testDetail: String? {
switch testStatus {
case .idle, .running, .success: return nil
case .failure(let message): return message
}
}
private func runTest() {
testStatus = .running
let store = AppGroupStore()
let client = CloudASRClientFactory.make(store: store)
Task {
do {
try await client.prepare(dictionary: store.personalDictionary)
let samples = [Float](repeating: 0.01, count: 16_000)
_ = try await client.transcribe(
samples: samples,
sampleRate: 16_000,
locale: Locale(identifier: store.localeId == "auto" ? "zh-CN" : store.localeId),
dictionary: store.personalDictionary
)
testStatus = .success
} catch CloudASRError.noAPIKey {
testStatus = .failure(AppL10n.string("api.test.missing"))
} catch let error as CloudASRError {
testStatus = .failure(error.localizedDescription ?? "\(error)")
} catch {
testStatus = .failure((error as? LocalizedError)?.errorDescription ?? "\(error)")
}
}
}
}
@@ -39,6 +39,18 @@ enum AppTab: Int, CaseIterable {
case .settings: return "tab.settings"
}
}
/// Sidebar label for iPad `NavigationSplitView` (SF Symbol + title).
var sidebarTitle: LocalizedStringKey { accessibilityKey }
var sidebarSystemImage: String {
switch self {
case .keyboard: return "house"
case .history: return "clock.arrow.circlepath"
case .dictionary: return "character.book.closed"
case .settings: return "gearshape"
}
}
}
struct MinimalTabBar: View {
@@ -0,0 +1,198 @@
// WideLayoutComponents.swift
// OSGKeyboard · Main App
//
// Reusable layout pieces for iPad / regular-width surfaces. Styled with the
// shared design tokens so the wide Home dashboard can mirror the macOS shell
// without pulling in AppKit-only types from OSGKeyboardMac.
import SwiftUI
import OSGKeyboardShared
// MARK: - Layout metrics
/// Fixed metrics that keep wide surfaces on the same grid as the macOS app.
enum WideLayoutMetrics {
static let sidebarWidth: CGFloat = 240
static let sidebarInset: CGFloat = Spacing.md
static let sidebarContentInset: CGFloat = sidebarInset + Spacing.sm
static let pageHorizontalInset: CGFloat = 40
static let dictationCanvasMinHeight: CGFloat = 120
}
// MARK: - Card container
/// Elevated surface used for stat tiles and the dictation canvas.
struct WideCard<Content: View>: View {
@Environment(\.themePalette) private var palette
var padding: CGFloat = Spacing.md
var cornerRadius: CGFloat = Radius.medium
@ViewBuilder var content: () -> Content
var body: some View {
let shape = RoundedRectangle(cornerRadius: cornerRadius, style: .continuous)
content()
.padding(padding)
.background(palette.surface, in: shape)
.overlay(
shape.stroke(palette.divider, lineWidth: 0.5)
)
}
}
// MARK: - Stat tile
struct WideStatCard: View {
@Environment(\.themePalette) private var palette
let title: String
let value: String
let caption: String
var systemImage: String?
var accent: Bool = false
/// Hero metric: wide horizontal layout for the primary word count.
var prominent: Bool = false
var body: some View {
WideCard(padding: Spacing.md) {
if prominent {
prominentBody
} else {
compactBody
}
}
}
private var compactBody: some View {
VStack(alignment: .leading, spacing: Spacing.xs) {
HStack {
Text(title.uppercased())
.font(TypeStyle.caption2)
.tracking(0.6)
.foregroundStyle(palette.textTertiary)
Spacer()
if let systemImage {
Image(systemName: systemImage)
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(accent ? palette.accent : palette.textTertiary)
.symbolRenderingMode(.hierarchical)
}
}
Text(value)
.font(TypeStyle.title2)
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
.lineLimit(1)
.minimumScaleFactor(0.7)
.contentTransition(.numericText())
.animation(Motion.soft, value: value)
Text(caption)
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
private var prominentBody: some View {
HStack(spacing: Spacing.md) {
if let systemImage {
ZStack {
Circle()
.fill(palette.accentMuted)
.frame(width: 44, height: 44)
Image(systemName: systemImage)
.font(.system(size: 17, weight: .semibold))
.foregroundStyle(palette.accent)
.symbolRenderingMode(.hierarchical)
}
}
VStack(alignment: .leading, spacing: 2) {
Text(title.uppercased())
.font(TypeStyle.caption2)
.tracking(0.6)
.foregroundStyle(palette.textTertiary)
Text(caption)
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
}
Spacer(minLength: Spacing.md)
Text(value)
.font(.system(size: 34, weight: .bold))
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
.lineLimit(1)
.minimumScaleFactor(0.6)
.contentTransition(.numericText())
.animation(Motion.soft, value: value)
}
}
}
// MARK: - Home stats cluster
/// Dashboard-style stat cluster for the wide Home layout.
struct WideHomeStatsCluster: View {
@ObservedObject private var stats = UsageStatisticsStore.shared
@ObservedObject private var config = ProviderConfig.shared
@State private var dictionaryCount = 0
private var language: AppUILanguage { config.uiLanguage }
var body: some View {
VStack(spacing: Spacing.md) {
WideStatCard(
title: AppL10n.string("home.stats.dictationCharacters", language: language),
value: UsageStatisticsStore.formatCount(
stats.dictationCharacterCount,
language: language
),
caption: AppL10n.string("home.wide.stat.transcribed", language: language),
systemImage: "text.alignleft",
accent: true,
prominent: true
)
HStack(spacing: Spacing.md) {
WideStatCard(
title: AppL10n.string("home.stats.dictationDuration", language: language),
value: UsageStatisticsStore.formatDuration(
stats.dictationDurationSeconds,
language: language
),
caption: AppL10n.string("home.wide.stat.cumulativeDuration", language: language),
systemImage: "waveform"
)
WideStatCard(
title: AppL10n.string("home.stats.translationCharacters", language: language),
value: UsageStatisticsStore.formatCount(
stats.translationCharacterCount,
language: language
),
caption: AppL10n.string("home.wide.stat.cumulativeTranslation", language: language),
systemImage: "character.bubble"
)
WideStatCard(
title: AppL10n.string("home.stats.dictionaryEntries", language: language),
value: UsageStatisticsStore.formatCount(
dictionaryCount,
language: language
),
caption: AppL10n.string("home.wide.stat.customTerms", language: language),
systemImage: "character.book.closed"
)
}
}
.onAppear(perform: refreshDictionaryCount)
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
refreshDictionaryCount()
}
.onReceive(NotificationCenter.default.publisher(for: .personalDictionaryDidSyncFromCloud)) { _ in
refreshDictionaryCount()
}
.onReceive(NotificationCenter.default.publisher(for: .usageStatisticsDidSyncFromCloud)) { _ in
stats.reloadFromDisk()
}
}
private func refreshDictionaryCount() {
dictionaryCount = AppGroupStore().personalDictionary.entries.count
}
}
+121 -19
View File
@@ -15,6 +15,7 @@ import UIKit
struct HomeView: View {
@Environment(\.themePalette) private var palette: ThemePalette
@Environment(\.scenePhase) private var scenePhase
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@ObservedObject private var config = ProviderConfig.shared
@EnvironmentObject private var flowManager: FlowSessionManager
@@ -24,6 +25,10 @@ struct HomeView: View {
@State private var micStatus = AppPermissions.micStatus
@State private var speechStatus = AppPermissions.speechStatus
private var usesWideLayout: Bool {
horizontalSizeClass == .regular
}
private var sessionIsLive: Bool {
flowManager.isActive || flowManager.isStarting
}
@@ -52,6 +57,32 @@ struct HomeView: View {
}
var body: some View {
Group {
if usesWideLayout {
wideBody
} else {
phoneBody
}
}
.onAppear {
refreshPermissionStatuses()
}
.onChange(of: scenePhase) { _, phase in
guard phase == .active else { return }
refreshPermissionStatuses()
}
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
refreshPermissionStatuses()
}
.onChange(of: previewFocused) { _, focused in
guard focused else { return }
Task { await flowManager.refreshForInlineKeyboardFocus() }
}
}
// MARK: - Phone layout
private var phoneBody: some View {
GeometryReader { geo in
let gradientHeight = geo.size.height * 0.30 + geo.safeAreaInsets.top
@@ -98,20 +129,63 @@ struct HomeView: View {
}
}
}
.onAppear {
refreshPermissionStatuses()
}
// MARK: - Wide layout (iPad / regular width)
private var wideBody: some View {
VStack(spacing: 0) {
VStack(alignment: .leading, spacing: Spacing.lg) {
wideHeroHeader
WideHomeStatsCluster()
if showsFlowSessionExtras {
flowSessionExtras
}
widePreviewStage
}
.padding(.horizontal, WideLayoutMetrics.pageHorizontalInset)
.padding(.top, Spacing.sm)
.padding(.bottom, Spacing.md)
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
}
.onChange(of: scenePhase) { _, phase in
guard phase == .active else { return }
refreshPermissionStatuses()
.background(palette.background)
.contentShape(Rectangle())
.onTapGesture {
if previewFocused {
previewFocused = false
}
}
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
refreshPermissionStatuses()
}
private var wideHeroHeader: some View {
VStack(alignment: .leading, spacing: Spacing.xs) {
Text("home.wide.tagline")
.font(.system(size: 30, weight: .semibold))
.foregroundStyle(palette.textPrimary)
.lineLimit(2)
.minimumScaleFactor(0.85)
Text("home.wide.tagline.subtitle")
.font(TypeStyle.footnote)
.foregroundStyle(palette.textTertiary)
}
.onChange(of: previewFocused) { _, focused in
guard focused else { return }
Task { await flowManager.refreshForInlineKeyboardFocus() }
.frame(maxWidth: .infinity, alignment: .leading)
}
private var widePreviewStage: some View {
WideCard(padding: Spacing.md, cornerRadius: Radius.large) {
previewFieldContent
.frame(
maxWidth: .infinity,
minHeight: WideLayoutMetrics.dictationCanvasMinHeight,
maxHeight: .infinity,
alignment: .topLeading
)
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top)
}
private func refreshPermissionStatuses() {
@@ -187,7 +261,18 @@ struct HomeView: View {
.foregroundStyle(palette.warning)
.lineLimit(1)
.minimumScaleFactor(0.85)
} else if flowManager.isUtteranceRecording {
Text("home.flow.recording")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textPrimary)
.lineLimit(1)
} else if flowManager.isUtteranceProcessing {
Text("home.flow.processing")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textPrimary)
.lineLimit(1)
} else if flowManager.isActive,
FlowSessionBridge.isHostReady(),
let expires = flowManager.sessionExpiresAt {
Text("home.flow.label")
.font(TypeStyle.caption2)
@@ -322,10 +407,14 @@ struct HomeView: View {
private var flowStatusColor: Color {
if needsCloudSetup { return palette.warning }
if flowManager.isActive { return palette.accent }
if flowManager.isUtteranceRecording { return palette.accent }
if flowManager.isUtteranceProcessing { return palette.accent }
if flowManager.isActive, FlowSessionBridge.isHostReady() { return palette.accent }
if flowManager.isStarting { return palette.accent }
if needsPermissionSetup { return palette.warning }
if flowManager.sessionWarning != nil { return palette.warning }
// Active but not host-ready (e.g. mid-utterance / audio proof) amber.
if flowManager.isActive { return palette.warning }
return palette.textTertiary
}
@@ -337,21 +426,26 @@ struct HomeView: View {
if flowManager.isStarting {
return AppL10n.string("home.flow.starting")
}
if flowManager.isActive {
if flowManager.isUtteranceRecording {
return AppL10n.string("home.flow.recording")
}
if flowManager.isUtteranceProcessing {
return AppL10n.string("home.flow.processing")
}
if flowManager.isActive, FlowSessionBridge.isHostReady() {
return AppL10n.string("home.flow.label")
}
if flowManager.isActive {
// Session flag is up but the ready contract is not do not lie.
return AppL10n.string("home.flow.notReady")
}
return AppL10n.string("home.flow.inactive")
}
// MARK: - Preview field
private var previewField: some View {
TextField("home.preview.placeholder", text: $previewText, axis: .vertical)
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
.tint(palette.accent)
.focused($previewFocused)
.lineLimit(1...100)
previewFieldContent
.frame(maxWidth: .infinity, minHeight: 180, maxHeight: .infinity, alignment: .topLeading)
.padding(Spacing.md)
.background(palette.surfaceElevated, in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
@@ -359,11 +453,19 @@ struct HomeView: View {
RoundedRectangle(cornerRadius: Radius.large, style: .continuous)
.stroke(previewFocused ? palette.dividerStrong : palette.dividerStrong.opacity(0.75), lineWidth: 1)
)
// TextField only hit-tests the text line(s); expand taps to the full card.
.contentShape(RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
.onTapGesture { previewFocused = true }
}
private var previewFieldContent: some View {
TextField("home.preview.placeholder", text: $previewText, axis: .vertical)
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
.tint(palette.accent)
.focused($previewFocused)
.lineLimit(1...100)
}
private var engineStatusLine: some View {
Text(
EngineServiceLabel.summary(
+9 -5
View File
@@ -39,14 +39,18 @@ struct MainAppRoot: View {
.animation(.easeInOut(duration: 0.2), value: flowManager.coldStartContext != nil)
.onAppear {
flowManager.setAppForeground(scenePhase == .active)
flowManager.activateOnForeground()
AppCloudSync.shared.startObservingExternalChanges()
// Registering here also flushes any URL buffered during a cold
// launch (the keyboard app `startflow` handoff arrives via the
// scene delegate before this view is on screen).
// Register the URL handler BEFORE the foreground auto-start.
// Registering flushes any URL buffered during a cold launch (the
// keyboard app `startflow` handoff arrives via the scene
// delegate before this view is on screen), so a cold start takes
// the cold-start path first and `activateOnForeground()`'s plain
// start then no-ops on the isStarting guard instead of two
// start bodies racing each other on the main actor.
AppOpenURLRouter.shared.register { url in
handleIncomingURL(url)
}
flowManager.activateOnForeground()
AppCloudSync.shared.startObservingExternalChanges()
Task {
await AppCloudSync.shared.pullAllIfEnabled()
}
+276
View File
@@ -0,0 +1,276 @@
// MainSplitView.swift
// OSGKeyboard · Main App
//
// iPad / regular-width shell: sidebar navigation + detail workspace.
// Mirrors the macOS `NavigationSplitView` structure while keeping iOS tabs
// and Flow session behaviour unchanged underneath.
import SwiftUI
import OSGKeyboardShared
import UIKit
struct MainSplitView: View {
@Binding var selection: AppTab
@Environment(\.themePalette) private var palette
@State private var columnVisibility: NavigationSplitViewVisibility = .all
var body: some View {
NavigationSplitView(columnVisibility: $columnVisibility) {
sidebar
.navigationSplitViewColumnWidth(WideLayoutMetrics.sidebarWidth)
} detail: {
detail
}
.navigationSplitViewStyle(.balanced)
.background(palette.background)
// No floating dock in split mode child scroll views should not
// reserve bottom clearance for the phone tab bar.
.environment(\.isTabBarVisible, false)
}
// MARK: - Sidebar
private var sidebar: some View {
VStack(spacing: 0) {
brandHeader
VStack(spacing: 4) {
ForEach(AppTab.allCases, id: \.rawValue) { tab in
WideSidebarRow(
tab: tab,
isSelected: selection == tab
) {
withAnimation(Motion.soft) { selection = tab }
}
}
}
.padding(.horizontal, WideLayoutMetrics.sidebarInset)
Spacer()
devicesFooter
}
.background(palette.background)
}
private var brandHeader: some View {
HStack {
Image("osglogo")
.resizable()
.scaledToFit()
.frame(height: 28)
.accessibilityLabel("OSGKeyboard")
Spacer()
}
.padding(.leading, WideLayoutMetrics.sidebarContentInset)
.padding(.trailing, WideLayoutMetrics.sidebarInset)
.padding(.top, Spacing.lg)
.padding(.bottom, Spacing.md)
}
private var devicesFooter: some View {
Label("home.wide.devices", systemImage: "ipad.and.iphone")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.horizontal, WideLayoutMetrics.sidebarInset + Spacing.sm)
.padding(.vertical, Spacing.sm)
}
// MARK: - Detail
private var detail: some View {
VStack(spacing: 0) {
MainTabContent(tab: selection)
.frame(maxWidth: .infinity, maxHeight: .infinity)
.id(selection)
.transition(.opacity)
WideStatusFooter()
}
.background(palette.background)
}
}
// MARK: - Sidebar row
private struct WideSidebarRow: View {
let tab: AppTab
let isSelected: Bool
let action: () -> Void
@Environment(\.themePalette) private var palette
var body: some View {
Button(action: action) {
Label(tab.sidebarTitle, systemImage: tab.sidebarSystemImage)
.font(.system(size: 13, weight: isSelected ? .semibold : .regular))
.foregroundStyle(isSelected ? palette.accent : palette.textPrimary)
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.horizontal, Spacing.sm)
.padding(.vertical, 7)
.background(
rowBackground,
in: RoundedRectangle(cornerRadius: 7, style: .continuous)
)
.contentShape(Rectangle())
}
.buttonStyle(.plain)
.animation(Motion.quick, value: isSelected)
.accessibilityAddTraits(isSelected ? .isSelected : [])
}
private var rowBackground: Color {
isSelected ? palette.accentMuted : .clear
}
}
// MARK: - Status footer
/// Quiet bottom strip: engine mode + translation target + Flow readiness.
private struct WideStatusFooter: View {
@Environment(\.themePalette) private var palette
@ObservedObject private var config = ProviderConfig.shared
@EnvironmentObject private var flowManager: FlowSessionManager
@State private var micStatus = AppPermissions.micStatus
@State private var speechStatus = AppPermissions.speechStatus
private var needsCloudSetup: Bool {
!config.isLocalEngine && !config.isConfigured
}
private var needsPermissionSetup: Bool {
micStatus != .granted || speechStatus != .granted
}
private var canManuallyStartSession: Bool {
!flowManager.isActive && !flowManager.isStarting && !needsPermissionSetup
}
var body: some View {
HStack(spacing: Spacing.sm) {
Spacer()
Label(
config.engineMode == "cloud" ? "home.wide.mode.cloud" : "home.wide.mode.local",
systemImage: config.engineMode == "cloud" ? "cloud" : "cpu"
)
.contentTransition(.opacity)
Text("·")
.foregroundStyle(palette.textTertiary.opacity(0.5))
Label(
translationLabel,
systemImage: "translate"
)
.contentTransition(.opacity)
Text("·")
.foregroundStyle(palette.textTertiary.opacity(0.5))
flowStatusControl
}
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.labelStyle(.titleAndIcon)
.padding(.horizontal, WideLayoutMetrics.pageHorizontalInset)
.padding(.vertical, Spacing.sm)
.animation(Motion.quick, value: config.engineMode)
.animation(Motion.quick, value: config.translationTargetLocaleId)
.animation(Motion.soft, value: flowManager.isActive)
.onAppear { refreshPermissionStatuses() }
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
refreshPermissionStatuses()
}
}
@ViewBuilder
private var flowStatusControl: some View {
HStack(spacing: Spacing.xs) {
Circle()
.fill(flowStatusColor)
.frame(width: 6, height: 6)
if needsCloudSetup {
Text("home.flow.notReady")
.foregroundStyle(palette.warning)
} else if flowManager.isUtteranceRecording {
Text("home.flow.recording")
} else if flowManager.isUtteranceProcessing {
Text("home.flow.processing")
} else if flowManager.isActive,
FlowSessionBridge.isHostReady(),
let expires = flowManager.sessionExpiresAt {
Text("home.flow.label")
Text(":")
Text(expires, style: .timer)
.monospacedDigit()
} else {
Text(flowStatusLabel)
}
if flowManager.isActive {
Button {
flowManager.endSession()
} label: {
Text("home.flow.endShort")
.foregroundStyle(palette.accent)
}
.buttonStyle(.plain)
} else if canManuallyStartSession && !needsCloudSetup {
Button {
flowManager.activateOnForeground()
} label: {
Text("home.flow.startShort")
.foregroundStyle(palette.accent)
}
.buttonStyle(.plain)
}
}
}
private func refreshPermissionStatuses() {
micStatus = AppPermissions.micStatus
speechStatus = AppPermissions.speechStatus
}
private var translationLabel: String {
let resolved = TranslationLanguageCatalog.resolve(config.translationTargetLocaleId)
if TranslationLanguageCatalog.isOff(resolved.id) {
return AppL10n.string("keyboard.translation.offMenu", language: config.uiLanguage)
}
return resolved.nativeName
}
private var flowStatusColor: Color {
if !config.isLocalEngine && !config.isConfigured { return palette.warning }
if flowManager.isUtteranceRecording || flowManager.isUtteranceProcessing {
return palette.accent
}
if flowManager.isActive, FlowSessionBridge.isHostReady() { return palette.accent }
if flowManager.isStarting { return palette.accent }
if flowManager.isActive { return palette.warning }
return palette.textTertiary
}
private var flowStatusLabel: LocalizedStringKey {
if !config.isLocalEngine && !config.isConfigured {
return "home.flow.notReady"
}
if flowManager.isStarting {
return "home.flow.starting"
}
if flowManager.isUtteranceRecording {
return "home.flow.recording"
}
if flowManager.isUtteranceProcessing {
return "home.flow.processing"
}
if flowManager.isActive, FlowSessionBridge.isHostReady() {
return "home.flow.label"
}
if flowManager.isActive {
return "home.flow.notReady"
}
return "home.flow.inactive"
}
}
+25
View File
@@ -0,0 +1,25 @@
// MainTabContent.swift
// OSGKeyboard · Main App
//
// Shared tab destination switcher used by both the phone dock and the iPad
// split-view detail column.
import SwiftUI
import OSGKeyboardShared
struct MainTabContent: View {
let tab: AppTab
var body: some View {
switch tab {
case .keyboard:
HomeView()
case .history:
HistoryView()
case .dictionary:
PersonalDictionaryView()
case .settings:
SettingsView(presentation: .tab)
}
}
}
+30 -25
View File
@@ -6,47 +6,52 @@ import OSGKeyboardShared
struct MainTabView: View {
@Environment(\.themePalette) private var palette: ThemePalette
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@EnvironmentObject private var flowManager: FlowSessionManager
@State private var tab: AppTab = .keyboard
@State private var isTabBarHidden = false
private var usesSplitLayout: Bool {
horizontalSizeClass == .regular
}
var body: some View {
Group {
if usesSplitLayout {
MainSplitView(selection: $tab)
} else {
phoneTabLayout
}
}
.background(palette.background)
.ignoresSafeArea(.keyboard, edges: .bottom)
}
// MARK: - Phone layout
private var phoneTabLayout: some View {
ZStack(alignment: .bottom) {
palette.background.ignoresSafeArea()
Group {
switch tab {
case .keyboard:
HomeView()
case .history:
HistoryView()
case .dictionary:
PersonalDictionaryView()
case .settings:
SettingsView(presentation: .tab)
MainTabContent(tab: tab)
.frame(maxWidth: .infinity, maxHeight: .infinity)
.environment(\.isTabBarVisible, !isTabBarHidden)
.safeAreaInset(edge: .bottom, spacing: 0) {
if !isTabBarHidden {
Color.clear.frame(height: 88)
}
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.environment(\.isTabBarVisible, !isTabBarHidden)
.safeAreaInset(edge: .bottom, spacing: 0) {
if !isTabBarHidden {
Color.clear.frame(height: 88)
.onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in
withAnimation(Motion.quick) {
isTabBarHidden = hidden
}
}
}
.onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in
withAnimation(Motion.quick) {
isTabBarHidden = hidden
}
}
if !isTabBarHidden {
MinimalTabBar(selection: $tab)
.transition(.move(edge: .bottom).combined(with: .opacity))
}
}
// Keep home card/input/tab layout fixed when system keyboard appears.
// Let the keyboard overlay the content instead of pushing it.
.ignoresSafeArea(.keyboard, edges: .bottom)
}
}
+15 -5
View File
@@ -8,17 +8,22 @@ struct ProviderPickerSection: View {
@Environment(\.themePalette) private var palette: ThemePalette
@ObservedObject var config: ProviderConfig
var role: CloudProviderRole = .polish
private var selectedProviderId: String {
role == .asr ? config.asrProviderId : config.providerId
}
var body: some View {
// v0.2.1 follow-up: filter out presets marked as
// `isUserSelectable == false` (DeepSeek is local-engine only).
let visiblePresets = LLMProvider.userSelectablePresets
let visiblePresets = role == .asr
? LLMProvider.asrSelectablePresets
: LLMProvider.userSelectablePresets
VStack(spacing: 0) {
ForEach(Array(visiblePresets.enumerated()), id: \.element.id) { index, provider in
Button {
select(provider)
} label: {
row(provider, selected: provider.id == config.providerId)
row(provider, selected: provider.id == selectedProviderId)
}
.buttonStyle(.plain)
if index < visiblePresets.count - 1 {
@@ -35,7 +40,12 @@ struct ProviderPickerSection: View {
private func select(_ provider: LLMProvider) {
withAnimation(Motion.quick) {
config.apply(preset: provider)
switch role {
case .polish:
config.apply(preset: provider)
case .asr:
config.applyAsr(preset: provider)
}
}
}
+31 -16
View File
@@ -43,16 +43,11 @@ struct SettingsView: View {
dictionaryAndPolishSection
flowSessionSection
engineSection
// v0.2.1: hide provider/api card when the
// local engine is active regardless of the
// cloud-polish toggle. Local mode is
// contractually ASR-only, so provider/model/
// base URL/API key controls have no use
// and exposing them invites the user to fill
// out a DeepSeek key they can't use.
polishProviderSection
polishApiSection
if config.engineMode == "cloud" {
providerSection
apiSection
asrProviderSection
asrApiSection
}
if config.engineMode == "local" {
localEngineSettingsSection
@@ -253,22 +248,42 @@ struct SettingsView: View {
}
}
private var providerSection: some View {
private var polishProviderSection: some View {
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
sectionHeader("settings.provider.title")
ProviderPickerSection(config: config)
sectionHeader("settings.polishProvider.title")
Text("settings.polishProvider.subtitle")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .leading)
ProviderPickerSection(config: config, role: .polish)
}
}
// MARK: - API
private var apiSection: some View {
private var asrProviderSection: some View {
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
sectionHeader("settings.api.title")
sectionHeader("settings.asrProvider.title")
Text("settings.asrProvider.subtitle")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .leading)
ProviderPickerSection(config: config, role: .asr)
}
}
private var polishApiSection: some View {
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
sectionHeader("settings.polishApi.title")
APISettingsCard(config: config)
}
}
private var asrApiSection: some View {
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
sectionHeader("settings.asrApi.title")
ASRSettingsCard(config: config)
}
}
// MARK: - Language helpers
/// Falls back to a static list while dynamic locales are loading.
+1 -1
View File
@@ -1,4 +1,4 @@
/* Permission prompts — English */
"NSMicrophoneUsageDescription" = "OSGKeyboard uses the microphone for voice dictation and keeps a background audio session active while a voice session is running.";
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard uses on-device speech recognition to transcribe your voice. Audio is processed on your device and is not uploaded for transcription.";
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard transcribes your voice with on-device speech recognition by default. If you explicitly switch to a cloud engine in Settings, recordings are sent to the ASR provider you configure.";
+22 -3
View File
@@ -98,11 +98,19 @@
"settings.engine.local.ios26" = "Always on-device, no network.";
"settings.engine.local.legacy" = "On-device ASR. Transcription only, no polish.";
"settings.engine.cloud.title" = "Cloud recognition & polish";
"settings.engine.cloud.subtitle" = "Cloud ASR (with your dictionary) + API polish. Audio is sent to your provider.";
"settings.engine.cloud.subtitle" = "Cloud ASR and polish LLM are configured separately. Audio goes to your ASR provider.";
"settings.engine.cloud.badge" = "Cloud engine";
"settings.provider.title" = "Provider";
"settings.provider.personalDictionaryBadge" = "Personal dictionary";
"settings.provider.subtitle" = "Pick the LLM that polishes your dictation.";
"settings.polishProvider.title" = "Text polish (LLM)";
"settings.polishProvider.subtitle" = "Cleans up the transcript after recognition. Independent from the ASR provider.";
"settings.polishApi.title" = "Polish API";
"settings.asrProvider.title" = "Speech recognition (ASR)";
"settings.asrProvider.subtitle" = "Transcribes your audio in cloud mode. Can differ from the polish provider.";
"settings.asrApi.title" = "ASR API";
"settings.asr.model" = "ASR model";
"settings.asr.testConnection" = "Test ASR";
"provider.openai" = "OpenAI";
"provider.deepseek" = "DeepSeek";
"provider.qwen" = "Qwen (DashScope)";
@@ -161,8 +169,8 @@
"settings.privacy.fullAccess.title" = "About Full Access";
"settings.privacy.fullAccess.body" = "Full Access is required for the microphone and to read your API key. OSGKeyboard does not record or upload what you type with the keyboard.";
"settings.privacy.cloud.body" = "In Cloud polish mode, transcribed text is sent to the API endpoint you configure (e.g. OpenAI or your own server). OSGKeyboard does not operate servers and does not store transcripts in the cloud.";
"settings.privacy.cloud.alert.title" = "Third-party API";
"settings.privacy.cloud.alert.message" = "Cloud polish sends transcribed text to the third-party API you configure. OSGKeyboard never stores data on our servers. Continue?";
"settings.privacy.cloud.alert.title" = "Audio leaves your device";
"settings.privacy.cloud.alert.message" = "The cloud engine uploads your voice recordings to the third-party ASR provider you configure, and sends the transcript to its API for polish. That provider's privacy policy applies. OSGKeyboard never stores data on our servers. Continue?";
"settings.link.support" = "Help & Feedback";
"settings.link.github" = "GitHub";
"settings.support.footer" = "OSGKeyboard is open source. Report bugs and share ideas on GitHub.";
@@ -287,6 +295,8 @@
"home.flow.label" = "Ready";
"home.flow.inactive" = "Voice session inactive";
"home.flow.notReady" = "Not ready";
"home.flow.recording" = "Recording…";
"home.flow.processing" = "Processing…";
"home.flow.hint" = "Switch to any app and tap the keyboard mic to dictate.";
"home.setup.permission.mic" = "Microphone access is off — voice input won't work.";
"home.setup.permission.speech" = "Speech recognition is off — voice input won't work.";
@@ -307,6 +317,15 @@
"home.stats.dictationCharacters" = "Dictation chars";
"home.stats.translationCharacters" = "Translation chars";
"home.stats.dictionaryEntries" = "Dictionary";
"home.wide.tagline" = "Voice dictation, anywhere.";
"home.wide.tagline.subtitle" = "Switch to any app and tap the keyboard mic to dictate.";
"home.wide.stat.transcribed" = "Total dictated";
"home.wide.stat.cumulativeDuration" = "Cumulative duration";
"home.wide.stat.cumulativeTranslation" = "Cumulative translation";
"home.wide.stat.customTerms" = "Custom terms";
"home.wide.mode.cloud" = "Cloud";
"home.wide.mode.local" = "On-device";
"home.wide.devices" = "iPhone & iPad";
"home.engine.unsupportedOS" = "Qwen3-ASR CoreML requires iOS 18+";
"home.engine.warming" = "Loading ASR model into memory…";
"home.engine.downloading" = "Downloading ASR model…";
+1 -1
View File
@@ -1,4 +1,4 @@
/* 权限说明 — 简体中文 */
"NSMicrophoneUsageDescription" = "OSGKeyboard 使用麦克风进行语音听写,并在语音会话运行期间保持后台音频会话。";
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard 使用设备端语音识别将你的语音转为文字。音频仅在设备上处理,不会上传用于转写。";
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard 默认使用设备端语音识别将你的语音转为文字。若你在设置中主动切换到云端引擎,录音会发送到你配置的识别服务商。";
+22 -3
View File
@@ -98,11 +98,19 @@
"settings.engine.local.ios26" = "全程在手机本地,不用联网";
"settings.engine.local.legacy" = "端侧 ASR,仅转录,无润色。";
"settings.engine.cloud.title" = "云端识别与润色";
"settings.engine.cloud.subtitle" = "云端 ASR(含个性词库)+ API 润色,音频将发往第三方服务";
"settings.engine.cloud.subtitle" = "云端 ASR 与润色 LLM 分开配置;音频发送至转写服务商。";
"settings.engine.cloud.badge" = "云端引擎";
"settings.provider.title" = "云端引擎";
"settings.provider.personalDictionaryBadge" = "个性词库";
"settings.provider.subtitle" = "选择 LLM 提供商。";
"settings.polishProvider.title" = "文本润色(LLM";
"settings.polishProvider.subtitle" = "识别完成后整理文字,可与 ASR 服务商不同。";
"settings.polishApi.title" = "润色接口";
"settings.asrProvider.title" = "语音转写(ASR";
"settings.asrProvider.subtitle" = "云端模式下负责听写转文字,可与润色模型分开配置。";
"settings.asrApi.title" = "转写接口";
"settings.asr.model" = "ASR 模型";
"settings.asr.testConnection" = "测试转写";
"provider.openai" = "OpenAI";
"provider.deepseek" = "DeepSeek";
"provider.qwen" = "通义千问";
@@ -161,8 +169,8 @@
"settings.privacy.fullAccess.title" = "关于完全访问";
"settings.privacy.fullAccess.body" = "用来调用麦克风和读取 API Key;不会读取或上传你的输入内容。";
"settings.privacy.cloud.body" = "云端润色模式下,转写文字会发送到你配置的 API(如 OpenAI 或自建服务)。OSGKeyboard 不运营服务器,也不会把转写内容存到云端。";
"settings.privacy.cloud.alert.title" = "第三方 API";
"settings.privacy.cloud.alert.message" = "云端润色会把转写文字发到你配置的第三方 API。OSGKeyboard 不会在自有服务器上存储数据。是否继续?";
"settings.privacy.cloud.alert.title" = "音频将离开你的设备";
"settings.privacy.cloud.alert.message" = "云端引擎会把你的语音录音上传到你配置的第三方识别服务,并把转写文字发送到其 API 进行润色,适用该服务商的隐私政策。OSGKeyboard 不会在自有服务器上存储数据。是否继续?";
"settings.link.support" = "帮助与反馈";
"settings.link.github" = "GitHub";
"settings.support.footer" = "OSGKeyboard 为开源项目,欢迎在 GitHub 提交问题与建议。";
@@ -286,6 +294,8 @@
"home.flow.label" = "就绪";
"home.flow.inactive" = "语音会话未启动";
"home.flow.notReady" = "未就绪";
"home.flow.recording" = "录音中…";
"home.flow.processing" = "处理中…";
"home.flow.hint" = "切到别的 App,点键盘麦克风就能说。";
"home.setup.permission.mic" = "麦克风还没授权,语音输入用不了。";
"home.setup.permission.speech" = "语音识别还没授权,语音输入用不了。";
@@ -306,6 +316,15 @@
"home.stats.dictationCharacters" = "听写字数";
"home.stats.translationCharacters" = "翻译字数";
"home.stats.dictionaryEntries" = "个性词库";
"home.wide.tagline" = "随处语音听写";
"home.wide.tagline.subtitle" = "切换到任意 App,点键盘麦克风即可听写。";
"home.wide.stat.transcribed" = "累计听写字数";
"home.wide.stat.cumulativeDuration" = "累计听写时长";
"home.wide.stat.cumulativeTranslation" = "累计翻译字数";
"home.wide.stat.customTerms" = "自定义词条";
"home.wide.mode.cloud" = "云端";
"home.wide.mode.local" = "本机";
"home.wide.devices" = "iPhone 与 iPad";
"home.engine.unsupportedOS" = "Qwen3-ASR CoreML 需要 iOS 18 或更高版本";
"home.engine.warming" = "正在加载语音识别模型…";
"home.engine.downloading" = "正在下载语音识别模型…";
+7 -1
View File
@@ -57,7 +57,11 @@ public final class KeyboardViewController: UIInputViewController {
// Voice-first keyboard hide the misleading "English" subtitle in Settings.
primaryLanguage = "mis"
OSGLog.keyboardExt.info("viewDidLoad — extension booted")
CustomLanguageModelManager.shared.prepareInBackgroundIfNeeded()
// Deliberately NO CustomLanguageModelManager prewarm here: the
// extension never runs ASR (the host app owns the microphone and
// the SpeechAnalyzer pipeline), and compiling/caching an LM inside
// the keyboard's ~60 MB jetsam budget risks the system killing the
// keyboard outright. The host app prewarms it on session start.
setNeedsUpdateOfScreenEdgesDeferringSystemGestures()
installKeyboardHeight()
configureDictationBehavior()
@@ -84,6 +88,7 @@ public final class KeyboardViewController: UIInputViewController {
setNeedsUpdateOfScreenEdgesDeferringSystemGestures()
configureDictationBehavior()
KeyboardSetupBridge.markExtensionAppearance(hasFullAccess: hasFullAccess)
state.debugHasFullAccess = hasFullAccess
flowCoordinator.refreshSessionState()
flowCoordinator.startSessionMonitor()
configSync.syncOnboardingStateFromAppGroup()
@@ -126,6 +131,7 @@ public final class KeyboardViewController: UIInputViewController {
textInserter = KeyboardTextInserter(
state: state,
insertText: { [weak self] text in self?.textDocumentProxy.insertText(text) },
contextBeforeInput: { [weak self] in self?.textDocumentProxy.documentContextBeforeInput },
scheduleAutoClearError: { [weak self] in self?.scheduleAutoClearError() }
)
+51 -10
View File
@@ -4,15 +4,17 @@
// Opens the host app from the keyboard extension.
//
// Reality check (verified against iOS 1826 behaviour):
// `extensionContext.open` is documented for Today widgets only; for a
// keyboard extension it resolves `false`, so we do not use it.
// The deprecated `openURL:` selector hack was disabled in iOS 18
// ("BUG IN CLIENT OF UIKIT migrate to open(_:options:completionHandler:)").
// The still-working path is: walk the responder chain to `UIApplication`
// and call the non-deprecated `open(_:options:completionHandler:)`. This
// requires Full Access and grows less reliable on newer iOS, so we report
// the *real* success from the completion handler instead of assuming it
// worked callers degrade to on-keyboard guidance when it returns false.
// Primary path: walk the responder chain to `UIApplication` and call the
// non-deprecated `open(_:options:completionHandler:)`. This requires Full
// Access and grows less reliable on newer iOS, so we report the *real*
// success from the completion handler instead of assuming it worked.
// Fallback: `extensionContext.open`. Historically documented for Today
// widgets only (and it used to resolve `false` for keyboards), but it is
// the Apple-documented API for extensions to open URLs and ships in
// production keyboards on current iOS worth trying before giving up.
// When both paths fail, callers degrade to on-keyboard guidance.
import UIKit
@@ -27,13 +29,52 @@ enum HostAppLauncher {
while let current = responder {
if let application = current as? UIApplication {
application.open(url, options: [:]) { success in
Task { @MainActor in completion(success) }
Task { @MainActor in
if success {
completion(true)
} else {
openViaExtensionContext(url: url, from: controller, completion: completion)
}
}
}
return
}
responder = current.next
}
// No `UIApplication` in the responder chain cannot open the host app.
completion(false)
// No `UIApplication` in the responder chain try the extension context.
openViaExtensionContext(url: url, from: controller, completion: completion)
}
@MainActor
private static func openViaExtensionContext(
url: URL,
from controller: KeyboardViewController,
completion: @escaping @MainActor (Bool) -> Void
) {
guard let context = controller.extensionContext else {
completion(false)
return
}
// `NSExtensionContext.open` from keyboards has historically been
// flaky about ever invoking its completion on some iOS versions.
// Callers rely on a real answer to fail fast (instead of spinning
// the 30 s start watchdog), so race the callback against a timeout
// and report the first result only.
var didComplete = false
let finish: @MainActor (Bool) -> Void = { success in
guard !didComplete else { return }
didComplete = true
completion(success)
}
Task { @MainActor in
// 1.5 s: long enough for a real open to call back, short enough
// that a dead completion degrades to on-keyboard guidance before
// the user gives up staring at nothing.
try? await Task.sleep(nanoseconds: 1_500_000_000)
finish(false)
}
context.open(url) { success in
Task { @MainActor in finish(success) }
}
}
}
@@ -44,6 +44,10 @@ final class KeyboardFlowCoordinator {
private var isAwaitingFlowResult = false
private var activeSessionId: UUID?
private var currentUtteranceId: UUID?
/// Utterance whose final result we already inserted (or failed). Prevents
/// `adoptHostBusyStateIfNeeded` from re-entering `.processing` after a
/// stale App Group snapshot still says `reason=processing`.
private var lastConsumedUtteranceId: UUID?
private var currentCommandSeq: Int64 = 0
private var lastAvailabilityTraceSignature = ""
@@ -139,6 +143,12 @@ final class KeyboardFlowCoordinator {
FlowSessionBridge.reloadFromDisk()
let readySnapshot = FlowSessionBridge.readySnapshot()
activeSessionId = readySnapshot?.sessionId ?? activeSessionId
// If the host is mid-utterance but this extension process lost local
// ownership (jetsam / recreate after app switch), re-adopt it so we
// show red/white instead of a fake orange "starting" state.
adoptHostBusyStateIfNeeded(snapshot: readySnapshot)
let hostReady = readySnapshot?.ready == true && FlowSessionBridge.isHostReady()
let now = Date().timeIntervalSince1970
if hostReady { lastHostReadyAt = now }
@@ -148,10 +158,20 @@ final class KeyboardFlowCoordinator {
// across cross-process read jitter and anchors this smoothing.
let withinReadyGrace = lastHostReadyAt > 0
&& (now - lastHostReadyAt) <= Self.hostReadyGrace
// Host busy (recording/processing) is NOT "still starting". Treating
// it as preparingSession was the orange-stuck bug after cold start:
// host utt.rec=1 ready=false keyboard forever "".
let hostBusy = readySnapshot?.reason == .recording
|| readySnapshot?.reason == .processing
let hostWarming = !hostReady
&& !hostBusy
&& FlowSessionBridge.isSessionActive()
&& (FlowSessionBridge.isHostReachable() || isPendingFlowStart || withinReadyGrace)
state.flowSessionActive = FlowSessionBridge.isSessionActive()
state.debugPendingFlowStart = isPendingFlowStart
state.debugFlowRecording = isFlowRecording
state.debugAwaitingFlowResult = isAwaitingFlowResult
state.debugHasFullAccess = hasFullAccess()
state.micVoiceAvailability = MicVoiceAvailabilityResolver.resolve(
phase: state.phase,
micDisabled: state.micDisabled,
@@ -176,6 +196,84 @@ final class KeyboardFlowCoordinator {
}
}
/// Re-attach to a host utterance this keyboard process no longer owns.
private func adoptHostBusyStateIfNeeded(snapshot: FlowReadySnapshot?) {
guard let snapshot, let sessionId = snapshot.sessionId else { return }
// Ignore snapshots from a dead host generation.
if let snapGen = snapshot.hostGeneration,
let liveGen = FlowSessionBridge.currentHostGeneration(),
snapGen != liveGen {
return
}
// Host already finished never re-adopt a consumed utterance, and
// clear sticky local processing left behind by a stale busy snapshot.
if snapshot.reason != .recording, snapshot.reason != .processing {
clearStickyProcessingIfNeeded(hostReady: snapshot.ready)
return
}
switch snapshot.reason {
case .recording:
guard !isFlowRecording else { return }
guard !isAwaitingFlowResult else { return }
// Require the host's utterance id inventing one makes matchingResult
// forever miss the real delivery and leaves the mic white forever.
guard let busyId = snapshot.busyUtteranceId else { return }
guard busyId != lastConsumedUtteranceId else { return }
activeSessionId = sessionId
currentUtteranceId = busyId
isPendingFlowStart = false
flowStartDeadline = 0
stopHostReadyWait()
isFlowRecording = true
state.phase = .recording
if state.lastTranscript.isEmpty {
state.lastTranscript = ""
}
if let view = wakeLockView() {
ExtensionScreenWakeLock.acquire(from: view)
}
startUtteranceCountdown()
startFlowLevelWatchdog()
traceState("adoptHostBusy.recording", extra: "session=\(sessionId)")
case .processing:
guard !isAwaitingFlowResult else { return }
guard let busyId = snapshot.busyUtteranceId else { return }
guard busyId != lastConsumedUtteranceId else { return }
activeSessionId = sessionId
currentUtteranceId = busyId
isPendingFlowStart = false
flowStartDeadline = 0
isFlowRecording = false
stopUtteranceCountdown()
ExtensionScreenWakeLock.release()
state.phase = .processing
if state.lastTranscript.isEmpty {
state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing")
}
startFlowResultWatchdog()
traceState("adoptHostBusy.processing", extra: "session=\(sessionId)")
default:
break
}
}
/// After insert, a stale `reason=processing` snapshot can bounce the mic
/// back to white loading. When the host is no longer busy, force idle.
private func clearStickyProcessingIfNeeded(hostReady: Bool) {
guard !isAwaitingFlowResult, !isFlowRecording else { return }
guard case .processing = state.phase else { return }
state.phase = .idle
state.lastTranscript = ""
stopFlowWatchdog()
currentUtteranceId = nil
traceState(
"stickyProcessing.cleared",
extra: hostReady ? "hostReady=1" : "hostReady=0"
)
}
/// Session is live but the ready contract has not landed yet poll
/// quickly instead of sticking on "session inactive" orange.
private func startHostReadyWaitIfNeeded() {
@@ -184,6 +282,12 @@ final class KeyboardFlowCoordinator {
stopHostReadyWait()
return
}
// Host busy waiting for ready. Do not spin the ready-wait poll.
if let reason = FlowSessionBridge.readySnapshot()?.reason,
reason == .recording || reason == .processing {
stopHostReadyWait()
return
}
guard !FlowSessionBridge.isHostReady() else {
stopHostReadyWait()
return
@@ -196,7 +300,9 @@ final class KeyboardFlowCoordinator {
guard let self, !Task.isCancelled else { return }
FlowSessionBridge.reloadFromDisk()
self.recomputeMicVoiceAvailability()
if self.state.micVoiceAvailability.isReady {
if self.state.micVoiceAvailability.isReady
|| self.state.micVoiceAvailability == .recording
|| self.state.micVoiceAvailability == .processing {
return
}
try? await Task.sleep(nanoseconds: 150_000_000)
@@ -365,6 +471,7 @@ final class KeyboardFlowCoordinator {
)
)
FlowSessionBridge.clearResult()
lastConsumedUtteranceId = result.utteranceId
currentUtteranceId = nil
textInserter.handleFlowTranscript(
TranscriptionDelivery(text: text, polishWarning: result.warning)
@@ -375,6 +482,7 @@ final class KeyboardFlowCoordinator {
isAwaitingFlowResult = false
stopFlowWatchdog()
FlowSessionBridge.clearResult()
lastConsumedUtteranceId = result.utteranceId
currentUtteranceId = nil
let error = FlowTranscriptionError(
message: result.text ?? ExtL10n.string("keyboard.flow.resultTimeout"),
@@ -626,6 +734,7 @@ final class KeyboardFlowCoordinator {
)
)
FlowSessionBridge.clearResult()
self.lastConsumedUtteranceId = result.utteranceId
self.currentUtteranceId = nil
self.debug("resultWatchdog consumed delivery len=\(text.count)")
self.textInserter.handleFlowTranscript(
@@ -637,6 +746,7 @@ final class KeyboardFlowCoordinator {
self.isAwaitingFlowResult = false
self.stopFlowWatchdog()
FlowSessionBridge.clearResult()
self.lastConsumedUtteranceId = result.utteranceId
self.currentUtteranceId = nil
let error = FlowTranscriptionError(
message: result.text ?? ExtL10n.string("keyboard.flow.resultTimeout"),
@@ -10,15 +10,18 @@ import OSGKeyboardShared
final class KeyboardTextInserter {
private let state: KeyboardState
private let insertText: (String) -> Void
private let contextBeforeInput: () -> String?
private let scheduleAutoClearError: () -> Void
init(
state: KeyboardState,
insertText: @escaping (String) -> Void,
contextBeforeInput: @escaping () -> String?,
scheduleAutoClearError: @escaping () -> Void
) {
self.state = state
self.insertText = insertText
self.contextBeforeInput = contextBeforeInput
self.scheduleAutoClearError = scheduleAutoClearError
}
@@ -30,7 +33,13 @@ final class KeyboardTextInserter {
return
}
// Host app already polished when configured; keyboard only inserts.
insertText(trimmed)
// Word-boundary hygiene: dictating "world" with the cursor right
// after "Hello" must yield "Hello world", not "Helloworld".
let separator = DictationTextComposer.insertionSeparator(
previousContext: contextBeforeInput(),
insertion: trimmed
)
insertText(separator + trimmed)
state.lastTranscript = ""
state.level = 0
if let warning = delivery.polishWarning {
@@ -33,6 +33,11 @@ private enum KeyboardLayoutMetrics {
static let topBarToTranscriptSpacing: CGFloat = Spacing.xs / 2
/// Outer inset for the bottom action row from screen edges (8 pt 24 pt, +200%).
static let sideActionHorizontalInset: CGFloat = Spacing.xs * 3
/// iPad: cap the content column. A full-width (~1180 pt) keyboard would
/// park delete/return at the far screen edges and turn each cursor-drag
/// pad into a ~450 pt runway capping keeps the reach ergonomics of the
/// phone layout. iPhone widths are all below this, so it is a no-op there.
static let contentMaxWidth: CGFloat = 700
// MARK: - Content-driven keyboard height (single source of truth)
static let outerPaddingTop: CGFloat = 2
@@ -109,6 +114,8 @@ public struct KeyboardRootView: View {
.padding(.bottom, KeyboardLayoutMetrics.outerPaddingBottom)
// chrome
.background(Color.clear)
.frame(maxWidth: KeyboardLayoutMetrics.contentMaxWidth)
.frame(maxWidth: .infinity)
.frame(height: Self.totalHeight)
// Feed the resolved palette to all nested chips/buttons.
.environment(\.themePalette, palette)
@@ -9,37 +9,50 @@ import SwiftUI
import WidgetKit
struct FlowLiveActivityWidget: Widget {
/// Deep link that restarts the Flow session. When the host process is
/// dead the activity goes stale; tapping it must take the *cold-start*
/// path (same as the keyboard's mic button), not just open the app.
private static let reconnectURL = URL(string: "osgkeyboard://startflow")
var body: some WidgetConfiguration {
ActivityConfiguration(for: FlowActivityAttributes.self) { context in
FlowLiveActivityLockScreenView(phase: context.state.phase)
.activityBackgroundTint(Color.black.opacity(0.82))
.activitySystemActionForegroundColor(.white)
FlowLiveActivityLockScreenView(
phase: context.state.phase,
isStale: context.isStale
)
.activityBackgroundTint(Color.black.opacity(0.82))
.activitySystemActionForegroundColor(.white)
// Deep-link to a session restart only when the host is dead
// tapping a HEALTHY activity should just open the app, not
// force a cold-start handoff into a running session.
.widgetURL(context.isStale ? Self.reconnectURL : nil)
} dynamicIsland: { context in
DynamicIsland {
DynamicIslandExpandedRegion(.leading) {
FlowLiveActivityBrandMark(height: 16)
}
DynamicIslandExpandedRegion(.trailing) {
FlowLiveActivityPhaseLabel(phase: context.state.phase)
FlowLiveActivityPhaseLabel(phase: context.state.phase, isStale: context.isStale)
}
DynamicIslandExpandedRegion(.center) {
Text("OSGKeyboard")
.font(.headline)
}
DynamicIslandExpandedRegion(.bottom) {
FlowLiveActivityPhaseCaption(phase: context.state.phase)
FlowLiveActivityPhaseCaption(phase: context.state.phase, isStale: context.isStale)
.font(.caption)
.foregroundStyle(.secondary)
}
} compactLeading: {
FlowLiveActivityBrandMark(height: 12)
} compactTrailing: {
FlowLiveActivityTrailingGlyph(phase: context.state.phase)
FlowLiveActivityTrailingGlyph(phase: context.state.phase, isStale: context.isStale)
} minimal: {
// The minimal slot is a tiny circle; a short wordmark keeps
// the natural ratio without overflowing its bounds.
FlowLiveActivityBrandMark(height: 6)
}
.widgetURL(context.isStale ? Self.reconnectURL : nil)
.keylineTint(Color(red: 0.35, green: 0.55, blue: 1.0))
}
}
@@ -49,6 +62,7 @@ struct FlowLiveActivityWidget: Widget {
private struct FlowLiveActivityLockScreenView: View {
let phase: FlowActivityAttributes.ContentState.Phase
let isStale: Bool
var body: some View {
HStack(spacing: 12) {
@@ -56,13 +70,16 @@ private struct FlowLiveActivityLockScreenView: View {
VStack(alignment: .leading, spacing: 4) {
Text("OSGKeyboard")
.font(.headline)
FlowLiveActivityPhaseCaption(phase: phase)
FlowLiveActivityPhaseCaption(phase: phase, isStale: isStale)
.font(.subheadline)
.foregroundStyle(.secondary)
}
Spacer(minLength: 0)
FlowLiveActivityTrailingGlyph(phase: phase)
FlowLiveActivityTrailingGlyph(phase: phase, isStale: isStale)
}
// A stale activity means the host process is gone never advertise
// "Voice session active" for a dead session; grey the card instead.
.opacity(isStale ? 0.55 : 1)
// iOS Live Activity lock-screen content needs margins so the leading
// logo and trailing glyph don't touch the card edges.
.padding(.horizontal, 16)
@@ -91,8 +108,19 @@ private struct FlowLiveActivityBrandMark: View {
private struct FlowLiveActivityTrailingGlyph: View {
let phase: FlowActivityAttributes.ContentState.Phase
var isStale: Bool = false
var body: some View {
if isStale {
Image(systemName: "bolt.slash.circle")
.foregroundStyle(.secondary)
} else {
phaseGlyph
}
}
@ViewBuilder
private var phaseGlyph: some View {
switch phase {
case .recording:
Image(systemName: "waveform")
@@ -114,8 +142,19 @@ private struct FlowLiveActivityTrailingGlyph: View {
private struct FlowLiveActivityPhaseLabel: View {
let phase: FlowActivityAttributes.ContentState.Phase
var isStale: Bool = false
var body: some View {
if isStale {
Image(systemName: "bolt.slash.circle")
.foregroundStyle(.secondary)
} else {
phaseLabel
}
}
@ViewBuilder
private var phaseLabel: some View {
switch phase {
case .recording:
Text("REC")
@@ -133,8 +172,20 @@ private struct FlowLiveActivityPhaseLabel: View {
private struct FlowLiveActivityPhaseCaption: View {
let phase: FlowActivityAttributes.ContentState.Phase
var isStale: Bool = false
var body: some View {
if isStale {
// Host process is gone be honest about it and turn the card
// into a recovery entry point (tap deep-links to startflow).
Text("Session disconnected · tap to reconnect")
} else {
phaseCaption
}
}
@ViewBuilder
private var phaseCaption: some View {
switch phase {
case .idle:
Text("Voice session active")
@@ -0,0 +1,7 @@
/* Live Activity captions. Keys are the English literals used in
FlowLiveActivityWidget — SwiftUI Text(_:) resolves string literals as
LocalizedStringKey against this table automatically. */
"Voice session active" = "Voice session active";
"Listening…" = "Listening…";
"Transcribing…" = "Transcribing…";
"Session disconnected · tap to reconnect" = "Session disconnected · tap to reconnect";
@@ -0,0 +1,6 @@
/* Live Activity 文案。键为 FlowLiveActivityWidget 中的英文字面量 —
SwiftUI Text(_:) 会把字符串字面量按 LocalizedStringKey 在本表解析。 */
"Voice session active" = "语音会话进行中";
"Listening…" = "正在聆听…";
"Transcribing…" = "正在转写…";
"Session disconnected · tap to reconnect" = "会话已断开 · 点按重连";
+109 -69
View File
@@ -1,7 +1,8 @@
// DashboardView.swift
// OSGKeyboard · Mac
//
// Primary workspace: session stats, dictation canvas, floating record bar.
// Primary workspace: brand voice, asymmetric stats, dictation stage, and
// the record bar. History lives on its own page no duplicate list here.
import SwiftUI
@@ -20,33 +21,24 @@ struct DashboardView: View {
self._stats = ObservedObject(wrappedValue: viewModel.usageStatistics)
}
// Four equal-width columns same metrics as the iOS home stats card.
private let columns = Array(
repeating: GridItem(.flexible(minimum: 120), spacing: Spacing.md),
count: 4
)
var body: some View {
VStack(spacing: 0) {
ScrollView {
VStack(alignment: .leading, spacing: Spacing.lg) {
if let appName = viewModel.foregroundAppName {
Text(MacL10n.format("mac.foregroundApp", language: lang, appName))
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
.transition(.opacity)
}
statGrid
dictationCanvas
}
.animation(Motion.soft, value: viewModel.foregroundAppName)
.padding(.horizontal, Spacing.lg)
.padding(.top, Spacing.sm)
.padding(.bottom, Spacing.lg)
VStack(alignment: .leading, spacing: Spacing.lg) {
heroHeader
statCluster
dictationStage
}
.padding(.horizontal, MacMetrics.pageHorizontalInset)
.padding(.top, Spacing.sm)
// Leftover window height splits evenly above / below the mic bar
// so spacing stays balanced at any window size.
Spacer(minLength: Spacing.xs)
BottomDictationBar(viewModel: viewModel)
.padding(.horizontal, Spacing.lg)
.padding(.bottom, Spacing.sm)
.padding(.horizontal, MacMetrics.pageHorizontalInset)
Spacer(minLength: Spacing.xs)
}
.onAppear { stats.reloadFromDisk() }
.onReceive(NotificationCenter.default.publisher(for: .usageStatisticsDidSyncFromCloud)) { _ in
@@ -54,18 +46,41 @@ struct DashboardView: View {
}
}
private var statGrid: some View {
LazyVGrid(columns: columns, spacing: Spacing.md) {
StatCard(
title: MacL10n.string("mac.stat.dictationTime", language: lang),
value: UsageStatisticsStore.formatDuration(
stats.dictationDurationSeconds,
language: lang
),
caption: MacL10n.string("mac.stat.cumulativeDuration", language: lang),
systemImage: "waveform",
accent: true
)
// MARK: - Hero
private var heroHeader: some View {
VStack(alignment: .leading, spacing: Spacing.xs) {
Text(MacL10n.string("mac.brand.tagline", language: lang))
.font(TypeStyle.pageTitle)
.foregroundStyle(palette.textPrimary)
.lineLimit(2)
.minimumScaleFactor(0.85)
HStack(spacing: Spacing.sm) {
Text(MacL10n.string("mac.brand.tagline.subtitle", language: lang))
.font(TypeStyle.footnote)
.foregroundStyle(palette.textTertiary)
if let appName = viewModel.foregroundAppName {
Text("·")
.foregroundStyle(palette.textTertiary.opacity(0.45))
Text(MacL10n.format("mac.foregroundApp", language: lang, appName))
.font(TypeStyle.footnote)
.foregroundStyle(palette.textTertiary)
.transition(.opacity)
.lineLimit(1)
}
}
.animation(Motion.soft, value: viewModel.foregroundAppName)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
// MARK: - Stats (hero word count, full width but content-height
// never stretched to match a taller sibling and left with dead air)
private var statCluster: some View {
VStack(spacing: Spacing.md) {
StatCard(
title: MacL10n.string("mac.stat.words", language: lang),
value: UsageStatisticsStore.formatCount(
@@ -73,28 +88,44 @@ struct DashboardView: View {
language: lang
),
caption: MacL10n.string("mac.stat.transcribed", language: lang),
systemImage: "text.alignleft"
)
StatCard(
title: MacL10n.string("mac.stat.translation", language: lang),
value: UsageStatisticsStore.formatCount(
stats.translationCharacterCount,
language: lang
),
caption: MacL10n.string("mac.stat.cumulativeTranslation", language: lang),
systemImage: "character.bubble"
)
StatCard(
title: MacL10n.string("mac.stat.dictionary", language: lang),
value: "\(viewModel.dictionaryTermCount)",
caption: MacL10n.string("mac.stat.customTerms", language: lang),
systemImage: "character.book.closed"
systemImage: "text.alignleft",
accent: true,
prominent: true
)
HStack(spacing: Spacing.md) {
StatCard(
title: MacL10n.string("mac.stat.dictationTime", language: lang),
value: UsageStatisticsStore.formatDuration(
stats.dictationDurationSeconds,
language: lang
),
caption: MacL10n.string("mac.stat.cumulativeDuration", language: lang),
systemImage: "waveform"
)
StatCard(
title: MacL10n.string("mac.stat.translation", language: lang),
value: UsageStatisticsStore.formatCount(
stats.translationCharacterCount,
language: lang
),
caption: MacL10n.string("mac.stat.cumulativeTranslation", language: lang),
systemImage: "character.bubble"
)
StatCard(
title: MacL10n.string("mac.stat.dictionary", language: lang),
value: "\(viewModel.dictionaryTermCount)",
caption: MacL10n.string("mac.stat.customTerms", language: lang),
systemImage: "character.book.closed"
)
}
}
}
private var dictationCanvas: some View {
MacCard(padding: Spacing.lg) {
// MARK: - Dictation stage
private var dictationStage: some View {
MacCard(padding: Spacing.md, cornerRadius: Radius.large) {
ZStack(alignment: .topLeading) {
if viewModel.transcript.isEmpty {
Text(
@@ -102,24 +133,37 @@ struct DashboardView: View {
? MacL10n.string("mac.status.listening", language: lang)
: MacL10n.string("mac.status.ready", language: lang)
)
.font(.system(size: 26, weight: .light))
.font(.system(size: 22, weight: .light))
.foregroundStyle(palette.textTertiary)
.contentTransition(.opacity)
.frame(maxWidth: .infinity, minHeight: 220, alignment: .topLeading)
.frame(
maxWidth: .infinity,
minHeight: MacMetrics.dictationCanvasMinHeight,
maxHeight: .infinity,
alignment: .topLeading
)
.transition(.opacity)
} else {
Text(viewModel.transcript)
.font(.system(size: 22, weight: .regular))
.font(.system(size: 20, weight: .regular))
.foregroundStyle(palette.textPrimary)
.lineSpacing(4)
.textSelection(.enabled)
.frame(maxWidth: .infinity, minHeight: 220, alignment: .topLeading)
.frame(
maxWidth: .infinity,
minHeight: MacMetrics.dictationCanvasMinHeight,
maxHeight: .infinity,
alignment: .topLeading
)
.transition(.opacity)
}
}
.frame(minHeight: MacMetrics.dictationCanvasMinHeight, maxHeight: 160)
}
.animation(Motion.soft, value: viewModel.transcript.isEmpty)
.animation(Motion.quick, value: viewModel.isRecording)
}
}
// MARK: - Floating record bar
@@ -142,12 +186,10 @@ struct BottomDictationBar: View {
recordControl
}
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.sm)
.macGlassSurface(in: RoundedRectangle(cornerRadius: Radius.xl, style: .continuous), fillOpacity: 1)
.overlay(
RoundedRectangle(cornerRadius: Radius.xl, style: .continuous)
.stroke(palette.dividerStrong, lineWidth: 0.5)
)
.padding(.vertical, Spacing.xs)
// No surface fill the mic bar sits on the page background so Home
// stays flat and the canvas above can stay shorter without a second
// floating card competing for height.
}
private var readinessChip: some View {
@@ -189,15 +231,14 @@ struct BottomDictationBar: View {
.foregroundStyle(palette.textSecondary)
.padding(.horizontal, Spacing.sm)
.padding(.vertical, 7)
.macGlassSurface(in: Capsule(), fillOpacity: 0.66)
.background(palette.surfaceElevated, in: Capsule())
}
.menuStyle(.borderlessButton)
.fixedSize()
}
//
//
// Option
// Mic stays geometrically centred: waveform lives inside the button,
// press stop floats above neither participates in layout.
private var recordControl: some View {
recordButton
.overlay(alignment: .top) {
@@ -225,7 +266,6 @@ struct BottomDictationBar: View {
)
Group {
if viewModel.isRecording {
// iOS
MiniWaveform(level: viewModel.audioLevel, barCount: 4, tint: palette.textOnAccent)
} else {
Image(systemName: "mic.fill")
+71 -2
View File
@@ -11,11 +11,16 @@
final class MacAudioRecorder: @unchecked Sendable {
enum RecorderError: Error, LocalizedError {
case converterUnavailable
case microphoneAccessDenied
var errorDescription: String? {
switch self {
case .converterUnavailable:
return "无法初始化音频转换器 / Failed to initialize audio converter"
case .microphoneAccessDenied:
return "麦克风权限被拒绝——请在「系统设置 → 隐私与安全性 → 麦克风」中启用"
+ " / Microphone access denied — enable it in System Settings"
+ " → Privacy & Security → Microphone"
}
}
}
@@ -30,7 +35,20 @@ final class MacAudioRecorder: @unchecked Sendable {
private var converter: AVAudioConverter?
private let lock = NSLock()
private var samples: [Float] = []
private var snapshotContinuation: AsyncStream<AudioBufferSnapshot>.Continuation?
private var isRunning = false
/// Hard cap on accumulated audio: 10 minutes @16 kHz 38 MB of Float32.
/// Recording is push-to-talk, but a stuck hotkey (or a latched Option
/// key) would otherwise grow this buffer without bound; past the cap we
/// keep the newest audio (drop from the front) so the take still ends
/// with what the user last said.
private static let maxSampleCount = 10 * 60 * 16_000
/// Trim hysteresis: dropping from the front is an O(n) memmove of the
/// whole ~38 MB buffer, done under the same lock the UI's level poll
/// takes doing it on EVERY tap callback once capped would stall the
/// render thread ~12×/s. Let the buffer overshoot by 30 s and trim the
/// whole excess in one move instead.
private static let trimHysteresisSamples = 30 * 16_000
private var smoothedLevel: Float = 0
/// One-shot flag for the converter pull block. Taps are serialized per
/// bus, so a plain instance property (not a captured local) is safe here.
@@ -42,8 +60,51 @@ final class MacAudioRecorder: @unchecked Sendable {
lock.withLock { smoothedLevel }
}
func start() throws {
lock.withLock { samples.removeAll(keepingCapacity: true) }
/// Resolves microphone authorization before capture. Prompts on first
/// use; throws `microphoneAccessDenied` once the user has declined so
/// failures surface as a permission problem, not an empty transcription.
private static func ensureMicrophoneAccess() async throws {
switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized:
return
case .notDetermined:
guard await AVCaptureDevice.requestAccess(for: .audio) else {
throw RecorderError.microphoneAccessDenied
}
case .denied, .restricted:
throw RecorderError.microphoneAccessDenied
@unknown default:
throw RecorderError.microphoneAccessDenied
}
}
func start() async throws {
try await Self.ensureMicrophoneAccess()
try startEngine()
}
/// Live 16 kHz mono snapshots for streaming ASR while the mic is open.
/// The stream is finished automatically in `stop()`.
func makeSnapshotStream() -> AsyncStream<AudioBufferSnapshot> {
AsyncStream { continuation in
lock.withLock {
snapshotContinuation?.finish()
snapshotContinuation = continuation
}
continuation.onTermination = { [weak self] _ in
self?.lock.withLock {
self?.snapshotContinuation = nil
}
}
}
}
private func startEngine() throws {
lock.withLock {
samples.removeAll(keepingCapacity: true)
snapshotContinuation?.finish()
snapshotContinuation = nil
}
let input = engine.inputNode
let inputFormat = input.outputFormat(forBus: 0)
@@ -67,6 +128,8 @@ final class MacAudioRecorder: @unchecked Sendable {
engine.stop()
isRunning = false
return lock.withLock {
snapshotContinuation?.finish()
snapshotContinuation = nil
let out = samples
samples.removeAll(keepingCapacity: false)
return out
@@ -105,8 +168,14 @@ final class MacAudioRecorder: @unchecked Sendable {
lock.withLock {
samples.append(contentsOf: chunk)
if samples.count > Self.maxSampleCount + Self.trimHysteresisSamples {
samples.removeFirst(samples.count - Self.maxSampleCount)
}
let factor: Float = normalized > smoothedLevel ? 0.5 : 0.15
smoothedLevel += (normalized - smoothedLevel) * factor
snapshotContinuation?.yield(
AudioBufferSnapshot(samples: chunk, sampleRate: 16_000)
)
}
}
}
@@ -0,0 +1,55 @@
// MacCloudASRChunkAdapter.swift
// OSGKeyboard · Mac
//
// Adapts configured cloud ASR clients to the shared chunked utterance pipeline.
import Foundation
import os
final class MacCloudASRChunkAdapter: ASRChunkTranscribing, @unchecked Sendable {
private let store: AppGroupStore
private let client: CloudASRTranscribing
private let cancelled = OSAllocatedUnfairLock(initialState: false)
init(store: AppGroupStore) throws {
let strategy = CloudASRModelCatalog.strategy(for: store.asrProviderId)
guard strategy != .localFallback else {
throw MacDictationError.providerHasNoCloudASR
}
self.store = store
self.client = CloudASRClientFactory.make(store: store)
}
func prepare() async throws {
try await client.prepare(dictionary: store.personalDictionary)
}
func resetForNewUtterance() {
cancelled.withLock { $0 = false }
}
func cancel() {
cancelled.withLock { $0 = true }
}
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult {
let isCancelled = cancelled.withLock { $0 }
if isCancelled || Task.isCancelled { return .cancelled }
guard !samples.isEmpty else { return .success("") }
do {
let text = try await client.transcribe(
samples: samples,
sampleRate: 16_000,
locale: locale,
dictionary: store.personalDictionary
)
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
return .success(trimmed)
} catch is CancellationError {
return .cancelled
} catch {
return .failure(error.localizedDescription)
}
}
}
+185 -38
View File
@@ -25,6 +25,22 @@ enum MacMetrics {
static let sidebarContentInset: CGFloat = sidebarInset + Spacing.sm
/// Reading width for single-column content.
static let contentMaxWidth: CGFloat = 720
/// Horizontal inset for page titles and scroll *content* (cards).
/// ScrollViews / Forms stay full-bleed so the scrollbar sits on the
/// window edge; only the content inside is inset.
/// Doubled from `Spacing.lg` so title + cards breathe from the edges.
static let pageHorizontalInset: CGFloat = Spacing.lg * 2
/// Built-in horizontal inset macOS grouped `Form` adds around its section
/// cards, on top of any padding we apply. Subtracted from
/// `pageHorizontalInset` on the Settings Form so its card outer edge lands
/// on `pageHorizontalInset` matching the History page and the page title.
static let groupedFormSectionInset: CGFloat = Spacing.lg
/// Default (= minimum) main-window size. Opening the app uses this size;
/// the window cannot shrink below it.
static let windowMinWidth: CGFloat = 860
static let windowMinHeight: CGFloat = 600
/// Compact dictation-canvas height so Home fits the min window without scrolling.
static let dictationCanvasMinHeight: CGFloat = 120
/// Top inset that clears the window traffic-light buttons now that the
/// title bar is hidden.
static let trafficLightInset: CGFloat = 28
@@ -81,16 +97,67 @@ extension View {
func macFieldStyle() -> some View { modifier(MacFieldStyleModifier()) }
}
// MARK: - Page header
/// Page title for History / Dictionary / Settings. Applies the shared
/// `pageHorizontalInset` so its left edge matches inset card content below.
/// Type size matches Home's brand line (`TypeStyle.pageTitle`).
struct MacPageHeader<Trailing: View>: View {
@Environment(\.themePalette) private var palette
let title: String
var subtitle: String?
@ViewBuilder var trailing: () -> Trailing
init(
title: String,
subtitle: String? = nil,
@ViewBuilder trailing: @escaping () -> Trailing
) {
self.title = title
self.subtitle = subtitle
self.trailing = trailing
}
var body: some View {
HStack(alignment: .firstTextBaseline, spacing: Spacing.md) {
VStack(alignment: .leading, spacing: Spacing.xxs) {
Text(title)
.font(TypeStyle.pageTitle)
.foregroundStyle(palette.textPrimary)
.lineLimit(1)
.minimumScaleFactor(0.85)
if let subtitle, !subtitle.isEmpty {
Text(subtitle)
.font(TypeStyle.footnote)
.foregroundStyle(palette.textTertiary)
}
}
Spacer(minLength: 0)
trailing()
}
.padding(.horizontal, MacMetrics.pageHorizontalInset)
.padding(.top, Spacing.sm)
.padding(.bottom, Spacing.sm)
}
}
extension MacPageHeader where Trailing == EmptyView {
init(title: String, subtitle: String? = nil) {
self.init(title: title, subtitle: subtitle) { EmptyView() }
}
}
// MARK: - Card container
/// Elevated surface used for stat tiles and the dictation canvas.
struct MacCard<Content: View>: View {
@Environment(\.themePalette) private var palette
var padding: CGFloat = Spacing.md
var cornerRadius: CGFloat = Radius.medium
@ViewBuilder var content: () -> Content
var body: some View {
let shape = RoundedRectangle(cornerRadius: Radius.medium, style: .continuous)
let shape = RoundedRectangle(cornerRadius: cornerRadius, style: .continuous)
content()
.padding(padding)
@@ -111,34 +178,82 @@ struct StatCard: View {
let caption: String
var systemImage: String?
var accent: Bool = false
/// Hero metric: wide horizontal layout that uses full-card width without
/// stretching to fill dead vertical space used for the primary word count.
var prominent: Bool = false
var body: some View {
MacCard {
VStack(alignment: .leading, spacing: Spacing.xs) {
HStack {
Text(title.uppercased())
.font(TypeStyle.caption2)
.tracking(0.6)
.foregroundStyle(palette.textTertiary)
Spacer()
if let systemImage {
Image(systemName: systemImage)
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(accent ? palette.accent : palette.textTertiary)
}
MacCard(padding: prominent ? Spacing.md : Spacing.md) {
if prominent {
prominentBody
} else {
compactBody
}
}
}
private var compactBody: some View {
VStack(alignment: .leading, spacing: Spacing.xs) {
HStack {
Text(title.uppercased())
.font(TypeStyle.caption2)
.tracking(0.6)
.foregroundStyle(palette.textTertiary)
Spacer()
if let systemImage {
Image(systemName: systemImage)
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(accent ? palette.accent : palette.textTertiary)
.symbolRenderingMode(.hierarchical)
}
Text(value)
.font(TypeStyle.title2)
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
.lineLimit(1)
.minimumScaleFactor(0.7)
.contentTransition(.numericText())
.animation(Motion.soft, value: value)
}
Text(value)
.font(TypeStyle.title2)
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
.lineLimit(1)
.minimumScaleFactor(0.7)
.contentTransition(.numericText())
.animation(Motion.soft, value: value)
Text(caption)
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
/// Wide "hero bar" layout: icon badge + title/caption on the left, the
/// big number anchored right fills the full card width edge-to-edge
/// instead of a tall card with empty space below a small number.
private var prominentBody: some View {
HStack(spacing: Spacing.md) {
if let systemImage {
ZStack {
Circle()
.fill(palette.accentMuted)
.frame(width: 44, height: 44)
Image(systemName: systemImage)
.font(.system(size: 17, weight: .semibold))
.foregroundStyle(palette.accent)
.symbolRenderingMode(.hierarchical)
}
}
VStack(alignment: .leading, spacing: 2) {
Text(title.uppercased())
.font(TypeStyle.caption2)
.tracking(0.6)
.foregroundStyle(palette.textTertiary)
Text(caption)
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
}
.frame(maxWidth: .infinity, alignment: .leading)
Spacer(minLength: Spacing.md)
Text(value)
.font(.system(size: 34, weight: .bold))
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
.lineLimit(1)
.minimumScaleFactor(0.6)
.contentTransition(.numericText())
.animation(Motion.soft, value: value)
}
}
}
@@ -152,30 +267,39 @@ struct MiniWaveform: View {
var barCount: Int = 5
/// Pass nil to inherit the palette accent automatically.
var tint: Color?
/// Peak bar height; overlay HUD uses a taller meter than the mic button.
var maxBarHeight: CGFloat = 22
var barWidth: CGFloat = 3
var barSpacing: CGFloat = 3
@State private var phase: CGFloat = 0
var body: some View {
HStack(spacing: 3) {
HStack(spacing: barSpacing) {
ForEach(0..<barCount, id: \.self) { index in
Capsule()
.fill(tint ?? palette.accent)
.frame(width: 3, height: barHeight(index))
.frame(width: barWidth, height: barHeight(index))
}
}
.frame(height: 22)
.frame(height: maxBarHeight)
.animation(Motion.instant, value: level)
.onAppear {
withAnimation(.linear(duration: 0.9).repeatForever(autoreverses: true)) {
withAnimation(.linear(duration: 0.75).repeatForever(autoreverses: true)) {
phase = 1
}
}
}
private func barHeight(_ index: Int) -> CGFloat {
let base = CGFloat(level) * 22
let wobble = sin((phase * .pi * 2) + CGFloat(index)) * 4 + 4
return max(4, min(22, base * (0.6 + CGFloat(index % 2) * 0.4) + wobble))
// Stronger level coupling + staggered phase so the meter reads as
// "alive" even at modest mic levels.
let boosted = min(1, CGFloat(level) * 1.35 + 0.08)
let base = boosted * maxBarHeight
let wobble = sin((phase * .pi * 2) + CGFloat(index) * 0.85) * (maxBarHeight * 0.22)
+ (maxBarHeight * 0.12)
let parity = 0.55 + CGFloat(index % 3) * 0.2
return max(maxBarHeight * 0.18, min(maxBarHeight, base * parity + wobble))
}
}
@@ -195,8 +319,8 @@ enum MacTranslationDisplay {
// MARK: - Status footer
/// Bottom status strip: engine mode (cloud/local), translation target, and
/// the connection state icons and wording mirror the dashboard record bar.
/// Bottom status strip: engine mode, translation target, connection
/// kept visually quiet so it never competes with the record bar.
struct MacStatusFooter: View {
@ObservedObject var viewModel: MacDictationViewModel
@Environment(\.themePalette) private var palette
@@ -204,7 +328,7 @@ struct MacStatusFooter: View {
private var lang: AppUILanguage { viewModel.config.uiLanguage }
var body: some View {
HStack(spacing: Spacing.md) {
HStack(spacing: Spacing.sm) {
Spacer()
Label(
viewModel.isCloudMode
@@ -212,24 +336,47 @@ struct MacStatusFooter: View {
: MacL10n.string("mac.mode.local", language: lang),
systemImage: viewModel.isCloudMode ? "cloud" : "cpu"
)
.foregroundStyle(palette.textSecondary)
.contentTransition(.opacity)
Text("·")
.foregroundStyle(palette.textTertiary.opacity(0.5))
Label(
MacTranslationDisplay.label(for: viewModel.config.translationTargetLocaleId, language: lang),
systemImage: "translate"
)
.foregroundStyle(palette.textSecondary)
.contentTransition(.opacity)
Text("·")
.foregroundStyle(palette.textTertiary.opacity(0.5))
Label(MacL10n.string("mac.connected", language: lang), systemImage: "link")
.foregroundStyle(palette.accent)
.foregroundStyle(palette.accent.opacity(0.85))
}
.font(TypeStyle.caption)
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.labelStyle(.titleAndIcon)
.padding(.horizontal, Spacing.lg)
.padding(.vertical, Spacing.xs)
.padding(.horizontal, MacMetrics.pageHorizontalInset)
.padding(.vertical, Spacing.sm)
.animation(Motion.quick, value: viewModel.isCloudMode)
.animation(Motion.quick, value: viewModel.config.translationTargetLocaleId)
}
}
// MARK: - Form alignment
private struct MacFormPageAlignModifier: ViewModifier {
func body(content: Content) -> some View {
// Form stays full-bleed (scrollbar on the window edge). Section
// cards are inset to match `MacPageHeader`.
content
.contentMargins(.horizontal, MacMetrics.pageHorizontalInset, for: .scrollContent)
}
}
extension View {
/// Insets grouped-`Form` section cards to `pageHorizontalInset`.
func macFormPageAligned() -> some View {
modifier(MacFormPageAlignModifier())
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ struct MacContentView: View {
VStack(alignment: .leading, spacing: Spacing.sm) {
header
recordButton
Text(MacL10n.string("mac.hint.holdOption", language: lang))
Text(MacL10n.string(viewModel.hotkeyTrigger.hintKey, language: lang))
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .center)
@@ -0,0 +1,222 @@
// MacDictationOverlayController.swift
// OSGKeyboard · Mac
//
// Owns a borderless, non-activating floating NSPanel that hosts the
// dictation HUD. Shown for any recording path (hotkey, menu bar, main
// window) and dismissed after a short success beat when processing ends.
import AppKit
import Combine
import SwiftUI
@MainActor
final class MacDictationOverlayController {
static let shared = MacDictationOverlayController()
private var panel: NSPanel?
private var hosting: NSHostingView<AnyView>?
private var cancellables = Set<AnyCancellable>()
private var hideWorkItem: DispatchWorkItem?
/// Keeps the pill visible briefly after a successful delivery.
private var showingCompletion = false
private var wasBusy = false
private let bottomMargin: CGFloat = 36
private let fallbackSize = NSSize(width: 400, height: 52)
private init() {}
func start(observing viewModel: MacDictationViewModel) {
guard cancellables.isEmpty else { return }
Publishers.CombineLatest3(
viewModel.$isRecording,
viewModel.$isPreparingToRecord,
viewModel.$isProcessing
)
.receive(on: RunLoop.main)
.sink { [weak self] recording, preparing, processing in
self?.handleBusyChange(
recording: recording,
preparing: preparing,
processing: processing,
viewModel: viewModel
)
}
.store(in: &cancellables)
// Keep waveform / app name / copy fresh while visible.
viewModel.objectWillChange
.receive(on: RunLoop.main)
.sink { [weak self] _ in
guard let self, self.panel?.isVisible == true else { return }
self.refreshContent(viewModel: viewModel)
self.resizeToFit()
}
.store(in: &cancellables)
NotificationCenter.default.publisher(for: NSApplication.didChangeScreenParametersNotification)
.receive(on: RunLoop.main)
.sink { [weak self] _ in self?.reposition() }
.store(in: &cancellables)
}
private func handleBusyChange(
recording: Bool,
preparing: Bool,
processing: Bool,
viewModel: MacDictationViewModel
) {
let busy = recording || preparing || processing
if busy {
hideWorkItem?.cancel()
hideWorkItem = nil
showingCompletion = false
wasBusy = true
present(viewModel: viewModel)
return
}
// Transition: busy idle. Flash a short "done" state, then hide.
if wasBusy {
wasBusy = false
showingCompletion = true
present(viewModel: viewModel)
scheduleHide()
return
}
if !showingCompletion {
hideImmediately()
}
}
private func present(viewModel: MacDictationViewModel) {
ensurePanel(viewModel: viewModel)
refreshContent(viewModel: viewModel)
resizeToFit()
reposition()
guard let panel else { return }
if panel.isVisible {
// Already up still bump to front in case another space stole it.
panel.orderFrontRegardless()
return
}
panel.alphaValue = 0
panel.orderFrontRegardless()
NSAnimationContext.runAnimationGroup { ctx in
ctx.duration = 0.22
panel.animator().alphaValue = 1
}
}
private func ensurePanel(viewModel: MacDictationViewModel) {
if panel != nil { return }
let host = NSHostingView(rootView: makeRoot(viewModel: viewModel))
host.frame = NSRect(origin: .zero, size: fallbackSize)
hosting = host
let panel = NSPanel(
contentRect: NSRect(origin: .zero, size: fallbackSize),
styleMask: [.borderless, .nonactivatingPanel],
backing: .buffered,
defer: false
)
panel.contentView = host
panel.isOpaque = false
panel.backgroundColor = .clear
panel.hasShadow = false
// Above normal floating windows so the HUD stays visible over browsers /
// full-screen apps, without going as high as the screen saver.
panel.level = NSWindow.Level(rawValue: Int(CGWindowLevelForKey(.floatingWindow)) + 1)
panel.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary, .stationary]
panel.isMovableByWindowBackground = false
panel.hidesOnDeactivate = false
panel.ignoresMouseEvents = false
panel.becomesKeyOnlyIfNeeded = true
self.panel = panel
}
private func refreshContent(viewModel: MacDictationViewModel) {
hosting?.rootView = makeRoot(viewModel: viewModel)
}
private func makeRoot(viewModel: MacDictationViewModel) -> AnyView {
AnyView(
MacDictationOverlayView(viewModel: viewModel)
.macSystemPalette()
.environment(\.locale, viewModel.config.uiLanguage.swiftUILocale)
.preferredColorScheme(MacAppearancePreference.current.colorScheme)
)
}
private func resizeToFit() {
guard let panel, let hosting else { return }
hosting.layoutSubtreeIfNeeded()
let fitting = hosting.fittingSize
let width = fitting.width.isFinite && fitting.width > 1
? min(max(fitting.width, 300), 520)
: fallbackSize.width
let height = fitting.height.isFinite && fitting.height > 1
? max(fitting.height, fallbackSize.height)
: fallbackSize.height
var frame = panel.frame
let midX = frame.midX
frame.size = NSSize(width: width, height: height)
if midX.isFinite {
frame.origin.x = midX - width / 2
}
panel.setFrame(frame, display: true)
hosting.frame = NSRect(origin: .zero, size: frame.size)
}
private func reposition() {
guard let panel else { return }
let screen = NSScreen.main ?? NSScreen.screens.first
guard let visible = screen?.visibleFrame else { return }
let size = panel.frame.size
let origin = NSPoint(
x: visible.midX - size.width / 2,
y: visible.minY + bottomMargin
)
panel.setFrameOrigin(origin)
}
private func scheduleHide() {
hideWorkItem?.cancel()
let work = DispatchWorkItem { [weak self] in
self?.showingCompletion = false
self?.hideAnimated()
}
hideWorkItem = work
DispatchQueue.main.asyncAfter(deadline: .now() + 1.15, execute: work)
}
private func hideImmediately() {
hideWorkItem?.cancel()
hideWorkItem = nil
showingCompletion = false
panel?.orderOut(nil)
panel?.alphaValue = 1
}
private func hideAnimated() {
guard let panel, panel.isVisible else {
hideImmediately()
return
}
NSAnimationContext.runAnimationGroup({ ctx in
ctx.duration = 0.2
panel.animator().alphaValue = 0
}, completionHandler: { [weak self] in
Task { @MainActor in
self?.panel?.orderOut(nil)
self?.panel?.alphaValue = 1
}
})
}
}
@@ -0,0 +1,170 @@
// MacDictationOverlayView.swift
// OSGKeyboard · Mac
//
// Compact bottom-of-screen HUD shown while dictating. Lives inside a
// non-activating NSPanel so it never steals focus from the front app.
// One-line layout: status / live transcript preview + waveform + stop.
import SwiftUI
struct MacDictationOverlayView: View {
@ObservedObject var viewModel: MacDictationViewModel
@Environment(\.themePalette) private var palette
private var lang: AppUILanguage { viewModel.config.uiLanguage }
private var isBusy: Bool {
viewModel.isRecording || viewModel.isPreparingToRecord || viewModel.isProcessing
}
/// Trimmed live / final transcript for the single-line preview.
private var previewText: String {
viewModel.transcript.trimmingCharacters(in: .whitespacesAndNewlines)
}
private var hasPreview: Bool { !previewText.isEmpty }
private var showsLiveBadge: Bool {
viewModel.isRecording && viewModel.isStreamingPartial
}
var body: some View {
HStack(spacing: Spacing.sm) {
statusDot
primaryLine
Spacer(minLength: Spacing.xs)
trailingControl
}
.padding(.horizontal, Spacing.md)
.padding(.vertical, 11)
.frame(minWidth: 300, idealWidth: 400, maxWidth: 520)
.fixedSize(horizontal: true, vertical: true)
.background(palette.surface, in: Capsule(style: .continuous))
.overlay(
Capsule(style: .continuous)
.stroke(palette.dividerStrong, lineWidth: 0.5)
)
.shadow(color: Color.black.opacity(0.22), radius: 14, y: 5)
.padding(2)
.animation(Motion.soft, value: hasPreview)
.animation(Motion.quick, value: viewModel.isRecording)
.animation(Motion.quick, value: viewModel.isStreamingPartial)
}
// MARK: - Primary line (status or one-line transcript)
@ViewBuilder
private var primaryLine: some View {
if hasPreview {
HStack(spacing: 6) {
if showsLiveBadge {
liveBadge
}
Text(previewText)
.font(TypeStyle.footnote)
.foregroundStyle(palette.textPrimary)
.lineLimit(1)
.truncationMode(.head)
.frame(maxWidth: showsLiveBadge ? 280 : 320, alignment: .leading)
.contentTransition(.opacity)
.animation(Motion.quick, value: previewText)
.accessibilityLabel(previewText)
}
} else {
HStack(spacing: 6) {
Text(statusText)
.font(TypeStyle.caption)
.foregroundStyle(palette.textPrimary)
.contentTransition(.opacity)
if let appName = viewModel.foregroundAppName, isBusy {
Text("·")
.foregroundStyle(palette.textTertiary.opacity(0.45))
Text(appName)
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
.lineLimit(1)
}
}
.animation(Motion.quick, value: statusText)
}
}
private var liveBadge: some View {
Text(MacL10n.string("mac.overlay.live", language: lang))
.font(.system(size: 9, weight: .semibold))
.foregroundStyle(palette.recordRed)
.padding(.horizontal, 5)
.padding(.vertical, 2)
.background(palette.recordRed.opacity(0.12), in: Capsule(style: .continuous))
.accessibilityHidden(true)
}
private var statusDot: some View {
Circle()
.fill(dotColor)
.frame(width: 8, height: 8)
.animation(Motion.quick, value: viewModel.isRecording)
.animation(Motion.quick, value: viewModel.isProcessing)
.animation(Motion.quick, value: viewModel.isPreparingToRecord)
.animation(Motion.quick, value: viewModel.isStreamingPartial)
}
private var dotColor: Color {
if viewModel.isRecording {
return viewModel.isStreamingPartial ? palette.accent : palette.recordRed
}
if viewModel.isPreparingToRecord || viewModel.isProcessing { return palette.warning }
return palette.accent
}
private var statusText: String {
if viewModel.isRecording {
return MacL10n.string("mac.overlay.listening", language: lang)
}
if viewModel.isPreparingToRecord {
return MacL10n.string("mac.overlay.preparing", language: lang)
}
if viewModel.isProcessing {
if viewModel.isStreamingPartial {
return MacL10n.string("mac.overlay.polishing", language: lang)
}
return MacL10n.string("mac.overlay.transcribing", language: lang)
}
return MacL10n.string("mac.overlay.done", language: lang)
}
@ViewBuilder
private var trailingControl: some View {
if viewModel.isRecording {
MiniWaveform(
level: viewModel.audioLevel,
barCount: 7,
tint: (viewModel.isStreamingPartial ? palette.accent : palette.recordRed)
.opacity(0.9),
maxBarHeight: 28,
barWidth: 3.5,
barSpacing: 2.5
)
stopButton
} else if viewModel.isPreparingToRecord || viewModel.isProcessing {
ProgressView()
.controlSize(.small)
} else {
Image(systemName: "checkmark.circle.fill")
.foregroundStyle(palette.accent)
.symbolRenderingMode(.hierarchical)
}
}
private var stopButton: some View {
Button(action: viewModel.toggleRecording) {
Image(systemName: "stop.fill")
.font(.system(size: 10, weight: .bold))
.foregroundStyle(palette.textOnAccent)
.frame(width: 28, height: 28)
.background(palette.recordRed, in: Circle())
}
.buttonStyle(.plain)
.accessibilityLabel(MacL10n.string("mac.record.stop", language: lang))
}
}
+245 -34
View File
@@ -1,9 +1,7 @@
// MacDictationPipeline.swift
// OSGKeyboard · Mac
//
// Dictation pipeline: samples ASR (cloud or local) polish.
// Cloud path reuses `CloudASRClientFactory`; local path uses Qwen3-ASR (MLX)
// with Apple Speech fallback when weights are missing.
// Dictation pipeline: samples ASR (cloud or local, chunked when long) polish.
import Foundation
@@ -24,39 +22,60 @@ enum MacDictationError: Error, LocalizedError {
}
}
/// Outcome of ASR that ran while the microphone was still open.
struct MacLiveASRCaptureResult: Sendable {
let raw: String
let chunkWarning: String?
let localBias: LocalASRBiasPayload?
/// When true, callers should fall back to batch ASR on the recorded samples.
let shouldFallbackToBatch: Bool
}
enum MacDictationPipeline {
/// Runs ASR then best-effort polish. Polish failures fall back to raw text.
static func run(samples: [Float], store: AppGroupStore) async throws -> String {
/// First-chunk threshold: longer local utterances use pipelined chunk ASR.
private static let chunkedLocalThresholdSamples = Int(
FlowUtteranceChunkConfig.flowDefault.maxChunkDurationSeconds(forChunkIndex: 0) * 16_000
)
/// Whether the active engine can surface `onPartial` text while recording.
static func supportsLivePartials(store: AppGroupStore) -> Bool {
if store.engineMode == "local" { return true }
return CloudASRModelCatalog.strategy(for: store.asrProviderId) != .localFallback
}
/// Runs ASR then polish. Polish failures return cleaned raw ASR plus a warning.
static func run(
samples: [Float],
store: AppGroupStore,
onPartial: (@Sendable (String) -> Void)? = nil
) async throws -> MacDictationResult {
guard !samples.isEmpty else { throw MacDictationError.noAudio }
let locale = Locale(identifier: store.localeId.isEmpty ? "zh-CN" : store.localeId)
let locale = resolvedLocale(store: store)
var chunkWarning: String?
let raw: String
var localBias: LocalASRBiasPayload?
if store.engineMode == "local" {
MacAppContextService.captureAndPersist(to: store)
let capabilities = MacLocalASRService.currentCapabilities()
let bias = LocalASRBiasAdapter.adapt(
LocalASRBiasRequest(
dictionary: store.personalDictionary,
localBias = resolveLocalBias(store: store, locale: locale)
if samples.count > chunkedLocalThresholdSamples {
let chunked = try await transcribeLocalChunked(
samples: samples,
locale: locale,
frontAppBundleId: MacAppContextService.frontmostBundleIdentifier(),
capabilities: capabilities
bias: localBias,
onPartial: onPartial
)
)
localBias = bias
LocalASRBiasDiagnosticsStore.save(
payload: bias,
modelId: MacLocalASRService.selectedModelDefinition()?.id,
backendLabel: MacLocalASRService.currentBackendLabel()
)
raw = try await MacLocalASRService.transcribe(
samples: samples,
locale: locale,
bias: bias
)
raw = chunked.text
chunkWarning = chunked.chunkWarning
} else {
raw = try await MacLocalASRService.transcribe(
samples: samples,
locale: locale,
bias: localBias
)
}
} else {
let strategy = CloudASRModelCatalog.strategy(for: store.providerId)
let strategy = CloudASRModelCatalog.strategy(for: store.asrProviderId)
guard strategy != .localFallback else { throw MacDictationError.providerHasNoCloudASR }
let client = CloudASRClientFactory.make(store: store)
@@ -69,6 +88,85 @@ enum MacDictationPipeline {
)
}
return try await polishCapturedASR(
raw: raw,
store: store,
localBias: localBias,
chunkWarning: chunkWarning
)
}
/// Consumes a live mic snapshot stream until finished; yields stitched partials.
static func captureLive(
stream: AsyncStream<AudioBufferSnapshot>,
store: AppGroupStore,
onPartial: @escaping @Sendable (String) -> Void
) async -> MacLiveASRCaptureResult {
let locale = resolvedLocale(store: store)
let localBias: LocalASRBiasPayload?
if store.engineMode == "local" {
localBias = resolveLocalBias(store: store, locale: locale)
} else {
localBias = nil
}
do {
let adapter = try makeChunkASRAdapter(
store: store,
locale: locale,
bias: localBias
)
if let cloudAdapter = adapter as? MacCloudASRChunkAdapter {
try? await cloudAdapter.prepare()
}
let pipeline = ChunkedUtterancePipeline(
asr: adapter,
locale: locale,
config: .flowDefault
)
let outcome = await pipeline.transcribe(stream: stream, onPartial: onPartial)
switch outcome {
case .success(let success):
return MacLiveASRCaptureResult(
raw: success.text,
chunkWarning: success.chunkWarnings.first,
localBias: localBias,
shouldFallbackToBatch: false
)
case .failure:
return MacLiveASRCaptureResult(
raw: "",
chunkWarning: nil,
localBias: localBias,
shouldFallbackToBatch: true
)
case .cancelled:
return MacLiveASRCaptureResult(
raw: "",
chunkWarning: nil,
localBias: localBias,
shouldFallbackToBatch: true
)
}
} catch {
return MacLiveASRCaptureResult(
raw: "",
chunkWarning: nil,
localBias: localBias,
shouldFallbackToBatch: true
)
}
}
/// Polish-only step after live or batch ASR has produced raw text.
static func polishCapturedASR(
raw: String,
store: AppGroupStore,
localBias: LocalASRBiasPayload?,
chunkWarning: String?
) async throws -> MacDictationResult {
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { throw MacDictationError.emptyTranscript }
@@ -91,14 +189,127 @@ enum MacDictationPipeline {
polishContext = nil
}
if let polished = try? await PolishingService(store: store).polish(
postASR,
mode: store.polishModeForPipeline,
context: polishContext
),
!polished.isEmpty {
return polished
do {
let polished = try await PolishingService(store: store).polish(
postASR,
mode: store.polishModeForPipeline,
context: polishContext
)
guard !polished.isEmpty else {
throw PolishingService.PolishError.noTranscript
}
return MacDictationResult(
text: polished,
polishWarning: nil,
chunkWarning: chunkWarning
)
} catch {
let delivery = TranscriptionPolishFallback.makeDelivery(
rawText: postASR,
error: error,
engineMode: store.engineMode,
chunkWarning: chunkWarning
)
return MacDictationResult(
text: delivery.text,
polishWarning: delivery.polishWarning,
chunkWarning: nil
)
}
}
// MARK: - Chunked local ASR
private struct ChunkedLocalResult {
let text: String
let chunkWarning: String?
}
private static func resolvedLocale(store: AppGroupStore) -> Locale {
Locale(identifier: store.localeId.isEmpty ? "zh-CN" : store.localeId)
}
private static func resolveLocalBias(
store: AppGroupStore,
locale: Locale
) -> LocalASRBiasPayload? {
MacAppContextService.captureAndPersist(to: store)
let capabilities = MacLocalASRService.currentCapabilities()
let bias = LocalASRBiasAdapter.adapt(
LocalASRBiasRequest(
dictionary: store.personalDictionary,
locale: locale,
frontAppBundleId: MacAppContextService.frontmostBundleIdentifier(),
capabilities: capabilities
)
)
LocalASRBiasDiagnosticsStore.save(
payload: bias,
modelId: MacLocalASRService.selectedModelDefinition()?.id,
backendLabel: MacLocalASRService.currentBackendLabel()
)
return bias
}
private static func makeChunkASRAdapter(
store: AppGroupStore,
locale: Locale,
bias: LocalASRBiasPayload?
) throws -> any ASRChunkTranscribing {
if store.engineMode == "local" {
return MacLocalASRChunkAdapter(locale: locale, bias: bias)
}
return try MacCloudASRChunkAdapter(store: store)
}
private static func transcribeLocalChunked(
samples: [Float],
locale: Locale,
bias: LocalASRBiasPayload?,
onPartial: (@Sendable (String) -> Void)?
) async throws -> ChunkedLocalResult {
let adapter = MacLocalASRChunkAdapter(locale: locale, bias: bias)
let pipeline = ChunkedUtterancePipeline(
asr: adapter,
locale: locale,
config: .flowDefault
)
let outcome = await pipeline.transcribe(
stream: audioStream(from: samples),
onPartial: { partial in
onPartial?(partial)
}
)
switch outcome {
case .success(let success):
let warning = success.chunkWarnings.first
return ChunkedLocalResult(text: success.text, chunkWarning: warning)
case .failure(let message):
throw MacLocalASRError.qwen3InferenceFailed(message)
case .cancelled:
throw MacLocalASRError.qwen3InferenceFailed("Cancelled")
}
}
/// Feeds recorded PCM into the chunker as if it arrived incrementally.
private static func audioStream(
from samples: [Float],
sliceSamples: Int = 8_000
) -> AsyncStream<AudioBufferSnapshot> {
AsyncStream { continuation in
var offset = 0
while offset < samples.count {
let end = min(offset + sliceSamples, samples.count)
continuation.yield(
AudioBufferSnapshot(
samples: Array(samples[offset..<end]),
sampleRate: 16_000
)
)
offset = end
}
continuation.finish()
}
return postASR
}
}
+12
View File
@@ -0,0 +1,12 @@
// MacDictationResult.swift
// OSGKeyboard · Mac
import Foundation
struct MacDictationResult: Sendable, Equatable {
let text: String
/// Shown when DeepSeek / cloud polish failed but raw ASR was delivered.
let polishWarning: String?
/// Non-fatal per-chunk ASR issues from long utterance chunking.
let chunkWarning: String?
}
+197 -25
View File
@@ -28,7 +28,7 @@ enum MacSection: String, CaseIterable, Identifiable {
var systemImage: String {
switch self {
case .dashboard: return "square.grid.2x2"
case .dashboard: return "house"
case .history: return "clock.arrow.circlepath"
case .dictionary: return "character.book.closed"
case .settings: return "gearshape"
@@ -45,7 +45,13 @@ final class MacDictationViewModel: ObservableObject {
@Published var selectedSection: MacSection = .dashboard
@Published var isRecording = false
/// True while microphone permission / engine start is in flight.
/// Drives the overlay so the HUD appears on Option-down immediately,
/// instead of waiting for the async `beginRecording` to finish.
@Published private(set) var isPreparingToRecord = false
@Published var isProcessing = false
/// True once live ASR has surfaced at least one partial during this take.
@Published private(set) var isStreamingPartial = false
@Published var transcript = ""
@Published var statusMessage = ""
@Published var audioLevel: Float = 0
@@ -55,6 +61,7 @@ final class MacDictationViewModel: ObservableObject {
@Published var autoPasteEnabled: Bool
@Published var hotkeyEnabled: Bool
@Published var hotkeyTrigger: MacHotkeyTrigger
@Published var config: ProviderConfig
@@ -64,6 +71,12 @@ final class MacDictationViewModel: ObservableObject {
private var levelTimer: Timer?
private var sessionTimer: Timer?
private var cancellables = Set<AnyCancellable>()
/// In-flight `beginRecording` started by the hotkey cancelled if the
/// key is released before the engine is ready (avoids a stuck session).
private var hotkeyBeginTask: Task<Void, Never>?
/// Live chunked ASR while recording (cloud / supported local paths).
/// Finished in `finishRecording` so partials can become the final draft.
private var liveCaptureTask: Task<MacLiveASRCaptureResult, Never>?
let usageStatistics: UsageStatisticsStore
let speechHistory = SpeechHistoryStore.shared
@@ -71,6 +84,7 @@ final class MacDictationViewModel: ObservableObject {
private enum StoredKeys {
static let autoPaste = "mac.autoPasteEnabled"
static let hotkey = "mac.hotkeyEnabled"
static let hotkeyTrigger = MacHotkeyTrigger.storageKey
}
init(defaults: UserDefaults = .standard) {
@@ -79,6 +93,9 @@ final class MacDictationViewModel: ObservableObject {
self.usageStatistics = UsageStatisticsStore(defaults: defaults)
self.autoPasteEnabled = defaults.object(forKey: StoredKeys.autoPaste) as? Bool ?? true
self.hotkeyEnabled = defaults.object(forKey: StoredKeys.hotkey) as? Bool ?? true
self.hotkeyTrigger = MacHotkeyTrigger(
rawValue: defaults.string(forKey: StoredKeys.hotkeyTrigger) ?? ""
) ?? .rightOption
MacICloudSyncBootstrap.configure(defaults: defaults)
statusMessage = MacL10n.string("mac.status.ready", language: config.uiLanguage)
@@ -114,10 +131,16 @@ final class MacDictationViewModel: ObservableObject {
// MARK: - Derived
var polishSelectableProviders: [LLMProvider] {
LLMProvider.userSelectablePresets
}
var asrSelectableProviders: [LLMProvider] {
LLMProvider.asrSelectablePresets
}
var selectableProviders: [LLMProvider] {
LLMProvider.presets.filter {
$0.isUserSelectable && $0.cloudASRStrategy != .localFallback
}
asrSelectableProviders
}
var dictionaryTermCount: Int {
@@ -185,6 +208,12 @@ final class MacDictationViewModel: ObservableObject {
if enabled { hotkeyService.start() } else { hotkeyService.stop() }
}
func setHotkeyTrigger(_ trigger: MacHotkeyTrigger) {
hotkeyTrigger = trigger
defaults.set(trigger.rawValue, forKey: StoredKeys.hotkeyTrigger)
hotkeyService.trigger = trigger
}
func setEngineMode(_ mode: String) {
config.engineMode = mode
}
@@ -192,23 +221,45 @@ final class MacDictationViewModel: ObservableObject {
// MARK: - Recording
func toggleRecording() {
if isRecording { finishRecording() } else { beginRecording() }
if isRecording || isPreparingToRecord {
cancelOrFinishRecording()
} else {
Task { await beginRecording() }
}
}
func beginRecording() {
guard !isProcessing else { return }
func beginRecording() async {
guard !isProcessing, !isRecording, !isPreparingToRecord else { return }
isPreparingToRecord = true
let store = AppGroupStore(defaults: defaults)
MacAppContextService.captureAndPersist(to: store)
refreshForegroundAppName()
do {
try recorder.start()
try await recorder.start()
// Hotkey may have been released while we awaited mic permission /
// engine start abandon cleanly instead of latching a stuck session.
isPreparingToRecord = false
if Task.isCancelled {
_ = recorder.stop()
return
}
isRecording = true
transcript = ""
isStreamingPartial = false
statusMessage = MacL10n.string("mac.status.listening", language: config.uiLanguage)
startTimers()
startLiveCaptureIfSupported(store: store)
// Tiny race: Option released between the cancel check and
// `isRecording = true`. Treat it as end-of-hold and finish.
if Task.isCancelled {
finishRecording()
}
} catch {
statusMessage = error.localizedDescription
isPreparingToRecord = false
if !Task.isCancelled {
statusMessage = error.localizedDescription
}
}
}
@@ -216,49 +267,166 @@ final class MacDictationViewModel: ObservableObject {
guard isRecording else { return }
isRecording = false
isProcessing = true
statusMessage = MacL10n.string("mac.status.transcribing", language: config.uiLanguage)
let hadLivePartial = isStreamingPartial
statusMessage = MacL10n.string(
hadLivePartial ? "mac.status.polishing" : "mac.status.transcribing",
language: config.uiLanguage
)
stopTimers()
audioLevel = 0
let samples = recorder.stop()
let store = AppGroupStore(defaults: defaults)
let liveTask = liveCaptureTask
liveCaptureTask = nil
Task { [weak self] in
guard let self else { return }
do {
let text = try await MacDictationPipeline.run(samples: samples, store: store)
self.transcript = text
let pasted = try self.deliver(text)
self.recordUsage(for: text)
self.speechHistory.append(text: text)
self.statusMessage = self.statusAfterDelivery(pasted: pasted)
let result: MacDictationResult
if let liveTask {
let capture = await liveTask.value
let trimmedLive = capture.raw.trimmingCharacters(in: .whitespacesAndNewlines)
if !capture.shouldFallbackToBatch, !trimmedLive.isEmpty {
if self.transcript.isEmpty {
self.transcript = trimmedLive
}
result = try await MacDictationPipeline.polishCapturedASR(
raw: capture.raw,
store: store,
localBias: capture.localBias,
chunkWarning: capture.chunkWarning
)
} else {
result = try await MacDictationPipeline.run(
samples: samples,
store: store,
onPartial: { [weak self] partial in
Task { @MainActor in
self?.transcript = partial
}
}
)
}
} else {
result = try await MacDictationPipeline.run(
samples: samples,
store: store,
onPartial: { [weak self] partial in
Task { @MainActor in
self?.transcript = partial
}
}
)
}
self.transcript = result.text
let pasted = try await self.deliver(result.text)
self.recordUsage(for: result.text)
self.speechHistory.append(text: result.text)
self.statusMessage = self.statusAfterDelivery(
pasted: pasted,
polishWarning: result.polishWarning,
chunkWarning: result.chunkWarning
)
} catch {
self.statusMessage = error.localizedDescription
}
self.isStreamingPartial = false
self.isProcessing = false
}
}
private func deliver(_ text: String) throws -> Bool {
try MacTextInsertionService.insert(text, autoPaste: autoPasteEnabled)
private func startLiveCaptureIfSupported(store: AppGroupStore) {
guard MacDictationPipeline.supportsLivePartials(store: store) else { return }
let stream = recorder.makeSnapshotStream()
liveCaptureTask = Task { [weak self] in
await MacDictationPipeline.captureLive(
stream: stream,
store: store,
onPartial: { [weak self] partial in
Task { @MainActor in
guard let self else { return }
guard self.isRecording || self.isProcessing else { return }
let trimmed = partial.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return }
self.transcript = partial
self.isStreamingPartial = true
}
}
)
}
}
private func statusAfterDelivery(pasted: Bool) -> String {
private func cancelLiveCapture() {
liveCaptureTask?.cancel()
liveCaptureTask = nil
isStreamingPartial = false
}
/// Stops an in-flight prepare, or finishes an active recording.
private func cancelOrFinishRecording() {
if isRecording {
finishRecording()
return
}
if isPreparingToRecord {
hotkeyBeginTask?.cancel()
hotkeyBeginTask = nil
// If the button-triggered prepare wasn't tracked by hotkeyBeginTask,
// still clear the preparing flag and stop any engine that raced in.
isPreparingToRecord = false
cancelLiveCapture()
_ = recorder.stop()
}
}
private func deliver(_ text: String) async throws -> Bool {
try await MacTextInsertionService.insert(text, autoPaste: autoPasteEnabled)
}
private func statusAfterDelivery(
pasted: Bool,
polishWarning: String? = nil,
chunkWarning: String? = nil
) -> String {
let lang = config.uiLanguage
let base: String
if autoPasteEnabled, pasted {
return MacL10n.string("mac.status.copiedAndPasted", language: lang)
base = MacL10n.string("mac.status.copiedAndPasted", language: lang)
} else if autoPasteEnabled, !pasted {
base = MacL10n.string("mac.status.copied", language: lang)
} else {
base = MacL10n.string("mac.status.copied", language: lang)
}
if autoPasteEnabled, !pasted {
return MacL10n.string("mac.status.copied", language: lang)
if let polishWarning, !polishWarning.isEmpty {
return MacL10n.format("mac.status.deliveryWithNote", language: lang, base, polishWarning)
}
return MacL10n.string("mac.status.copied", language: lang)
if let chunkWarning, !chunkWarning.isEmpty {
return MacL10n.format("mac.status.deliveryWithNote", language: lang, base, chunkWarning)
}
return base
}
private func wireHotkeyService() {
hotkeyService.trigger = hotkeyTrigger
hotkeyService.onPressBegan = { [weak self] in
self?.beginRecording()
guard let self else { return }
self.hotkeyBeginTask?.cancel()
self.hotkeyBeginTask = Task { [weak self] in
await self?.beginRecording()
}
}
hotkeyService.onPressEnded = { [weak self] in
self?.finishRecording()
guard let self else { return }
// Cancel a still-preparing start so a quick Option tap never
// latches recording. If recording already began, finish it.
if self.isRecording {
self.hotkeyBeginTask = nil
self.finishRecording()
} else {
self.hotkeyBeginTask?.cancel()
self.hotkeyBeginTask = nil
}
}
if hotkeyEnabled { hotkeyService.start() }
}
@@ -304,6 +472,10 @@ final class MacDictationViewModel: ObservableObject {
config.apply(preset: provider)
}
func selectAsrProvider(_ provider: LLMProvider) {
config.applyAsr(preset: provider)
}
func refreshForegroundAppName() {
foregroundAppName = MacAppContextService.frontmostApplicationName()
}
+88 -62
View File
@@ -1,9 +1,8 @@
// MacDictionaryView.swift
// OSGKeyboard · Mac
//
// Personal dictionary synced via iCloud KVS with the iOS app. Read-only on
// the desktop (words are authored on iPhone / iPad): grouped cards (native
// `Form`) with search, matching the Settings and History card style.
// Personal dictionary synced via iCloud KVS. ScrollView is full-bleed
// (scrollbar on the window edge); title + cards share `pageHorizontalInset`.
import SwiftUI
@@ -45,16 +44,27 @@ struct MacDictionaryView: View {
}
var body: some View {
Group {
if entries.isEmpty {
emptyState
.transition(.opacity)
} else {
form
.transition(.opacity)
VStack(spacing: 0) {
MacPageHeader(
title: MacL10n.string("mac.section.dictionary", language: lang),
subtitle: MacL10n.string("mac.page.dictionary.subtitle", language: lang)
) {
if !entries.isEmpty {
searchField
}
}
Group {
if entries.isEmpty {
emptyState
.transition(.opacity)
} else {
list
.transition(.opacity)
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.background(palette.background)
.animation(Motion.soft, value: entries.isEmpty)
.task {
@@ -64,33 +74,6 @@ struct MacDictionaryView: View {
.onReceive(NotificationCenter.default.publisher(for: .personalDictionaryDidSyncFromCloud)) { _ in
viewModel.refreshDictionaryFromCloud()
}
}
// MARK: - Grouped cards
private var form: some View {
Form {
if sections.isEmpty {
Section {
Text(MacL10n.string("mac.dict.noMatch", language: lang))
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .center)
}
} else {
ForEach(sections, id: \.category) { section in
Section(MacL10n.string(section.category.labelKey, language: lang)) {
ForEach(section.items) { entry in
row(entry)
}
}
}
}
}
.formStyle(.grouped)
.scrollContentBackground(.hidden)
.background(palette.background)
.animation(Motion.soft, value: query)
.safeAreaInset(edge: .top, spacing: 0) { centeredSearchField }
.confirmationDialog(
MacL10n.string("mac.dict.deleteTitle", language: lang),
isPresented: deletionDialogBinding,
@@ -110,6 +93,55 @@ struct MacDictionaryView: View {
}
}
// MARK: - List
private var list: some View {
// Full-bleed ScrollView scrollbar on the detail pane's right edge.
ScrollView {
LazyVStack(alignment: .leading, spacing: Spacing.md) {
if sections.isEmpty {
MacCard {
Text(MacL10n.string("mac.dict.noMatch", language: lang))
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .center)
}
} else {
ForEach(sections, id: \.category) { section in
categorySection(section)
}
}
}
.padding(.horizontal, MacMetrics.pageHorizontalInset)
.padding(.bottom, Spacing.md)
.animation(Motion.soft, value: query)
}
}
private func categorySection(
_ section: (category: PersonalDictionary.Entry.Category, items: [PersonalDictionary.Entry])
) -> some View {
VStack(alignment: .leading, spacing: Spacing.xs) {
Text(MacL10n.string(section.category.labelKey, language: lang))
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
MacCard(padding: 0) {
VStack(spacing: 0) {
ForEach(Array(section.items.enumerated()), id: \.element.id) { index, entry in
row(entry)
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.sm)
if index < section.items.count - 1 {
// Full-bleed like macOS list rows (not iOS inset separators).
Divider()
.overlay(palette.divider)
}
}
}
}
}
}
private var deletionDialogBinding: Binding<Bool> {
Binding(
get: { entryPendingDeletion != nil },
@@ -117,25 +149,19 @@ struct MacDictionaryView: View {
)
}
private var centeredSearchField: some View {
HStack {
Spacer()
HStack(spacing: Spacing.xs) {
Image(systemName: "magnifyingglass")
.foregroundStyle(palette.textTertiary)
TextField(MacL10n.string("mac.dict.search", language: lang), text: $query)
.textFieldStyle(.plain)
}
.padding(.horizontal, Spacing.sm)
.padding(.vertical, 7)
.frame(width: 240)
.macGlassSurface(in: Capsule(), fillOpacity: 0.72)
.overlay(Capsule().stroke(palette.divider, lineWidth: 0.5))
Spacer()
private var searchField: some View {
HStack(spacing: Spacing.xs) {
Image(systemName: "magnifyingglass")
.foregroundStyle(palette.textTertiary)
TextField(MacL10n.string("mac.dict.search", language: lang), text: $query)
.textFieldStyle(.plain)
.font(TypeStyle.footnote)
}
.padding(.horizontal, Spacing.lg)
.padding(.vertical, Spacing.xs)
.background(palette.background)
.padding(.horizontal, Spacing.sm)
.padding(.vertical, 6)
.frame(width: 220)
.background(palette.surface, in: Capsule())
.overlay(Capsule().stroke(palette.divider, lineWidth: 0.5))
}
private func row(_ entry: PersonalDictionary.Entry) -> some View {
@@ -165,19 +191,19 @@ struct MacDictionaryView: View {
private var emptyState: some View {
VStack(spacing: Spacing.sm) {
Image(systemName: "character.book.closed")
.font(.system(size: 34))
.foregroundStyle(palette.textTertiary.opacity(0.6))
.font(.system(size: 34, weight: .light))
.foregroundStyle(palette.textTertiary.opacity(0.55))
.symbolRenderingMode(.hierarchical)
Text(MacL10n.string("mac.dict.empty", language: lang))
.font(TypeStyle.body)
.font(TypeStyle.headline)
.foregroundStyle(palette.textSecondary)
Text(MacL10n.string("mac.dict.emptyBody", language: lang))
.font(TypeStyle.caption)
.font(TypeStyle.footnote)
.foregroundStyle(palette.textTertiary)
.multilineTextAlignment(.center)
.frame(maxWidth: 360)
.frame(maxWidth: 320)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.padding(.horizontal, Spacing.xl)
}
private func delete(_ entry: PersonalDictionary.Entry) {
+74 -44
View File
@@ -1,9 +1,8 @@
// MacHistoryView.swift
// OSGKeyboard · Mac
//
// Single-column, day-grouped transcript log rendered as grouped cards (the
// same native `Form` container as Settings). Every entry shows its full text
// inline no master/detail split, so content never pushes the sidebar out.
// Day-grouped transcript log. ScrollView is full-bleed (scrollbar on the
// window edge); title + cards share `pageHorizontalInset` on their content.
import SwiftUI
@@ -31,36 +30,39 @@ struct MacHistoryView: View {
}()
var body: some View {
Group {
if historyStore.entries.isEmpty {
emptyState
.transition(.opacity)
} else {
form
.transition(.opacity)
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
.background(palette.background)
.animation(Motion.soft, value: historyStore.entries.isEmpty)
}
// MARK: - Grouped cards
private var form: some View {
Form {
ForEach(historyStore.groupedByDay, id: \.day) { group in
Section(Self.dayFormatter.string(from: group.day)) {
ForEach(group.items) { entry in
row(entry)
VStack(spacing: 0) {
MacPageHeader(
title: MacL10n.string("mac.section.history", language: lang),
subtitle: MacL10n.string("mac.page.history.subtitle", language: lang)
) {
if !historyStore.entries.isEmpty {
Button {
showClearConfirmation = true
} label: {
Label(
MacL10n.string("mac.history.clearConfirm", language: lang),
systemImage: "trash"
)
.font(TypeStyle.caption)
.foregroundStyle(palette.textSecondary)
}
.buttonStyle(.plain)
}
}
Group {
if historyStore.entries.isEmpty {
emptyState
.transition(.opacity)
} else {
list
.transition(.opacity)
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
.formStyle(.grouped)
.scrollContentBackground(.hidden)
.background(palette.background)
.safeAreaInset(edge: .top, spacing: 0) { toolbar }
.animation(Motion.soft, value: historyStore.entries.isEmpty)
.confirmationDialog(
MacL10n.string("mac.history.clearTitle", language: lang),
isPresented: $showClearConfirmation,
@@ -75,21 +77,43 @@ struct MacHistoryView: View {
}
}
private var toolbar: some View {
HStack {
Spacer()
Button {
showClearConfirmation = true
} label: {
Label(MacL10n.string("mac.history.clearConfirm", language: lang), systemImage: "trash")
.font(TypeStyle.caption)
// MARK: - List
private var list: some View {
// Full-bleed ScrollView scrollbar on the detail pane's right edge.
// Horizontal inset lives on the content so cards align with the title.
ScrollView {
LazyVStack(alignment: .leading, spacing: Spacing.md) {
ForEach(historyStore.groupedByDay, id: \.day) { group in
daySection(group)
}
}
.padding(.horizontal, MacMetrics.pageHorizontalInset)
.padding(.bottom, Spacing.md)
}
}
private func daySection(_ group: (day: Date, items: [SpeechHistoryEntry])) -> some View {
VStack(alignment: .leading, spacing: Spacing.xs) {
Text(Self.dayFormatter.string(from: group.day))
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
MacCard(padding: 0) {
VStack(spacing: 0) {
ForEach(Array(group.items.enumerated()), id: \.element.id) { index, entry in
row(entry)
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.sm)
if index < group.items.count - 1 {
// Full-bleed like macOS list rows (not iOS inset separators).
Divider()
.overlay(palette.divider)
}
}
}
}
.buttonStyle(.borderless)
.foregroundStyle(palette.textSecondary)
}
.padding(.horizontal, Spacing.lg)
.padding(.vertical, Spacing.xs)
.background(palette.background)
}
private func row(_ entry: SpeechHistoryEntry) -> some View {
@@ -107,11 +131,17 @@ struct MacHistoryView: View {
private var emptyState: some View {
VStack(spacing: Spacing.sm) {
Image(systemName: "text.bubble")
.font(.system(size: 34))
.foregroundStyle(palette.textTertiary.opacity(0.6))
.font(.system(size: 34, weight: .light))
.foregroundStyle(palette.textTertiary.opacity(0.55))
.symbolRenderingMode(.hierarchical)
Text(MacL10n.string("mac.history.empty", language: lang))
.font(TypeStyle.body)
.font(TypeStyle.headline)
.foregroundStyle(palette.textSecondary)
Text(MacL10n.string("mac.history.emptyBody", language: lang))
.font(TypeStyle.footnote)
.foregroundStyle(palette.textTertiary)
.multilineTextAlignment(.center)
.frame(maxWidth: 320)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
+112 -19
View File
@@ -1,33 +1,93 @@
// MacHotkeyService.swift
// OSGKeyboard · Mac
//
// Global hold-to-talk: while Option () is held, dictation runs. Mirrors
// Typeless / SayIt push-to-talk from any foreground app.
// Global hold-to-talk: while the configured Option () key is held, dictation
// runs. Mirrors Typeless / SayIt push-to-talk from any foreground app.
import AppKit
import Foundation
/// Which physical Option () key triggers global hold-to-talk.
///
/// Right Option is the default: the left key is a routine typing modifier
/// (special characters, app shortcuts), so firing on any Option press
/// constantly misfires during normal typing.
enum MacHotkeyTrigger: String, CaseIterable, Identifiable {
case rightOption
case leftOption
case eitherOption
var id: String { rawValue }
var labelKey: String {
switch self {
case .rightOption: return "mac.hotkeyTrigger.rightOption"
case .leftOption: return "mac.hotkeyTrigger.leftOption"
case .eitherOption: return "mac.hotkeyTrigger.eitherOption"
}
}
/// Main-window hint under the record button must follow the picker,
/// or the UI tells left-Option users to hold the right key.
var hintKey: String {
switch self {
case .rightOption: return "mac.hint.hold.rightOption"
case .leftOption: return "mac.hint.hold.leftOption"
case .eitherOption: return "mac.hint.hold.eitherOption"
}
}
/// `@AppStorage`-compatible key; persisted via the view model's defaults.
static let storageKey = "mac.hotkeyTrigger"
/// Device-dependent modifier bits (IOKit `NX_DEVICELALTKEYMASK` /
/// `NX_DEVICERALTKEYMASK`) that `.flagsChanged` events carry alongside the
/// device-independent `.option` flag, telling left and right apart.
private static let leftOptionMask: UInt = 0x20
private static let rightOptionMask: UInt = 0x40
/// Whether this trigger's key is currently down in a `.flagsChanged` event.
func isPressed(in event: NSEvent) -> Bool {
guard event.modifierFlags.contains(.option) else { return false }
let raw = event.modifierFlags.rawValue
switch self {
case .rightOption: return raw & Self.rightOptionMask != 0
case .leftOption: return raw & Self.leftOptionMask != 0
case .eitherOption: return true
}
}
}
@MainActor
final class MacHotkeyService {
/// How long the trigger key must stay held before recording begins.
/// Filters out quick -taps and +key combos (special characters, app
/// shortcuts) that would otherwise start and immediately abort dictation.
private static let holdDebounce: Duration = .milliseconds(150)
var onPressBegan: (() -> Void)?
var onPressEnded: (() -> Void)?
var trigger: MacHotkeyTrigger = .rightOption
private var globalFlagsMonitor: Any?
private var localFlagsMonitor: Any?
private var optionHeld = false
/// The trigger key is physically down (debounce may still be pending).
private var triggerKeyDown = false
/// `onPressBegan` has fired and `onPressEnded` is owed.
private var pressActive = false
private var pendingBegin: Task<Void, Never>?
private var isEnabled = true
func setEnabled(_ enabled: Bool) {
isEnabled = enabled
if !enabled, optionHeld {
optionHeld = false
onPressEnded?()
}
if !enabled { cancelPress() }
}
func start() {
guard globalFlagsMonitor == nil else { return }
_ = MacTextInsertionService.requestAccessibilityIfNeeded()
guard globalFlagsMonitor == nil, localFlagsMonitor == nil else { return }
// Global monitors require Accessibility; without it the call returns
// nil and Option-hold never fires outside our own windows.
let trusted = MacTextInsertionService.requestAccessibilityIfNeeded()
globalFlagsMonitor = NSEvent.addGlobalMonitorForEvents(matching: .flagsChanged) { [weak self] event in
Task { @MainActor in self?.handleFlagsChanged(event) }
@@ -36,6 +96,12 @@ final class MacHotkeyService {
Task { @MainActor in self?.handleFlagsChanged(event) }
return event
}
#if DEBUG
if !trusted || globalFlagsMonitor == nil {
NSLog("[OSGKeyboard] Hotkey global monitor unavailable — grant Accessibility in System Settings")
}
#endif
}
func stop() {
@@ -47,20 +113,47 @@ final class MacHotkeyService {
NSEvent.removeMonitor(localFlagsMonitor)
self.localFlagsMonitor = nil
}
if optionHeld {
optionHeld = false
onPressEnded?()
}
cancelPress()
}
private func handleFlagsChanged(_ event: NSEvent) {
guard isEnabled else { return }
let optionDown = event.modifierFlags.contains(.option)
if optionDown, !optionHeld {
optionHeld = true
onPressBegan?()
} else if !optionDown, optionHeld {
optionHeld = false
let triggerDown = trigger.isPressed(in: event)
if triggerDown, !triggerKeyDown {
triggerKeyDown = true
scheduleBegin()
} else if !triggerDown, triggerKeyDown {
triggerKeyDown = false
pendingBegin?.cancel()
pendingBegin = nil
if pressActive {
pressActive = false
onPressEnded?()
}
}
}
/// Debounce: begin only after the key has stayed held for `holdDebounce`.
/// Releasing the key first cancels the pending start, so a quick
/// Option+key combo never triggers recording.
private func scheduleBegin() {
pendingBegin?.cancel()
pendingBegin = Task { [weak self] in
try? await Task.sleep(for: Self.holdDebounce)
guard let self, !Task.isCancelled else { return }
self.pendingBegin = nil
guard self.isEnabled, self.triggerKeyDown, !self.pressActive else { return }
self.pressActive = true
self.onPressBegan?()
}
}
private func cancelPress() {
pendingBegin?.cancel()
pendingBegin = nil
triggerKeyDown = false
if pressActive {
pressActive = false
onPressEnded?()
}
}
@@ -0,0 +1,43 @@
// MacLocalASRChunkAdapter.swift
// OSGKeyboard · Mac
//
// Adapts macOS local ASR to the shared chunked utterance pipeline.
import Foundation
import os
final class MacLocalASRChunkAdapter: ASRChunkTranscribing, @unchecked Sendable {
private let locale: Locale
private let bias: LocalASRBiasPayload?
private let cancelled = OSAllocatedUnfairLock(initialState: false)
init(locale: Locale, bias: LocalASRBiasPayload?) {
self.locale = locale
self.bias = bias
}
func resetForNewUtterance() {
cancelled.withLock { $0 = false }
}
func cancel() {
cancelled.withLock { $0 = true }
}
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult {
let isCancelled = cancelled.withLock { $0 }
if isCancelled || Task.isCancelled { return .cancelled }
guard !samples.isEmpty else { return .success("") }
do {
let text = try await MacLocalASRService.transcribe(
samples: samples,
locale: locale,
bias: bias
)
return .success(text)
} catch {
return .failure(error.localizedDescription)
}
}
}
@@ -276,14 +276,17 @@ struct MacLocalASRModelSettingsView: View {
HStack(spacing: Spacing.xs) {
Text(model.displayName)
.foregroundStyle(palette.textPrimary)
if let badgeKey = model.badgeKey {
modelBadge(
MacL10n.string(badgeKey, language: lang),
emphasized: true
)
}
if model.supportsHotwords {
Text(MacL10n.string("mac.localASR.personalDictionaryTag", language: lang))
.font(TypeStyle.caption2)
.padding(.horizontal, 6)
.padding(.vertical, 2)
.background(palette.accent.opacity(0.15))
.foregroundStyle(palette.accent)
.clipShape(Capsule())
modelBadge(
MacL10n.string("mac.localASR.personalDictionaryTag", language: lang),
emphasized: false
)
}
}
Text(modelSubtitle(model, installed: installed))
@@ -305,6 +308,20 @@ struct MacLocalASRModelSettingsView: View {
.padding(.vertical, 2)
}
private func modelBadge(_ title: String, emphasized: Bool) -> some View {
Text(title)
.font(TypeStyle.caption2)
.padding(.horizontal, 6)
.padding(.vertical, 2)
.background(
emphasized
? palette.accent.opacity(0.18)
: palette.textTertiary.opacity(0.12)
)
.foregroundStyle(emphasized ? palette.accent : palette.textSecondary)
.clipShape(Capsule())
}
@ViewBuilder
private func modelRowActions(
model: LocalASRModelDefinition,
+3 -3
View File
@@ -58,7 +58,7 @@ enum MacLocalASRPreferences {
/// Maps removed catalog entries to the current default Sherpa model.
static func migratedModelId(_ id: String) -> String {
switch id {
case "qwen3-mlx-1.7b":
case "qwen3-mlx-1.7b", "sherpa-paraformer-zh-int8":
return "sherpa-qwen3-0.6b-int8"
default:
return id
@@ -120,7 +120,7 @@ enum MacLocalASRService {
return try await transcribeWithModel(model, samples: samples, locale: locale, bias: bias)
}
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale)
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale, bias: bias)
}
private static func transcribeWithModel(
@@ -141,7 +141,7 @@ enum MacLocalASRService {
bias: bias
)
case .appleSpeech:
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale)
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale, bias: bias)
}
}
}
+1 -1
View File
@@ -198,7 +198,7 @@ struct MacOnboardingView: View {
.frame(maxWidth: .infinity)
}
}
.frame(minWidth: 860, minHeight: 600)
.frame(minWidth: MacMetrics.windowMinWidth, minHeight: MacMetrics.windowMinHeight)
.onAppear {
applyDefaults()
model.reload()
+11 -20
View File
@@ -18,15 +18,6 @@ struct MacRootView: View {
private var uiLanguage: AppUILanguage { viewModel.config.uiLanguage }
/// `List` selection is optional; keep the view model's non-optional section
/// in sync without letting a nil selection blank the detail pane.
private var selection: Binding<MacSection?> {
Binding(
get: { viewModel.selectedSection },
set: { if let new = $0 { viewModel.selectedSection = new } }
)
}
var body: some View {
NavigationSplitView(columnVisibility: $columnVisibility) {
sidebar
@@ -35,7 +26,7 @@ struct MacRootView: View {
detail
}
.navigationSplitViewStyle(.balanced)
.frame(minWidth: 860, minHeight: 600)
.frame(minWidth: MacMetrics.windowMinWidth, minHeight: MacMetrics.windowMinHeight)
.onAppear {
// Let the AppKit status-bar popover reopen this window on demand.
MacWindowBridge.shared.open = { openWindow(id: "main") }
@@ -72,7 +63,7 @@ struct MacRootView: View {
.renderingMode(.template)
.resizable()
.scaledToFit()
.frame(height: 30)
.frame(height: 28)
.foregroundStyle(palette.accent)
.accessibilityLabel("OSGKeyboard")
Spacer()
@@ -80,7 +71,7 @@ struct MacRootView: View {
.padding(.leading, MacMetrics.sidebarContentInset)
.padding(.trailing, MacMetrics.sidebarInset)
.padding(.top, Spacing.lg)
.padding(.bottom, Spacing.lg)
.padding(.bottom, Spacing.md)
}
private var devicesFooter: some View {
@@ -88,10 +79,10 @@ struct MacRootView: View {
MacL10n.string("mac.devices", language: uiLanguage),
systemImage: "laptopcomputer.and.iphone"
)
.font(TypeStyle.caption)
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.horizontal, MacMetrics.sidebarInset)
.padding(.horizontal, MacMetrics.sidebarInset + Spacing.sm)
.padding(.vertical, Spacing.sm)
}
@@ -118,8 +109,8 @@ struct MacRootView: View {
// MARK: - Sidebar row
/// A navigation row with an animated hover highlight and selection state,
/// matching the macOS System Settings feel.
/// Navigation row with a restrained selected state: muted accent fill +
/// accent label (not a solid green pill), matching System Settings polish.
private struct MacSidebarRow: View {
let section: MacSection
let isSelected: Bool
@@ -132,8 +123,8 @@ private struct MacSidebarRow: View {
var body: some View {
Button(action: action) {
Label(section.title(language: language), systemImage: section.systemImage)
.font(.system(size: 13))
.foregroundStyle(isSelected ? palette.textOnAccent : palette.textPrimary)
.font(.system(size: 13, weight: isSelected ? .semibold : .regular))
.foregroundStyle(isSelected ? palette.accent : palette.textPrimary)
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.horizontal, Spacing.sm)
.padding(.vertical, 7)
@@ -150,7 +141,7 @@ private struct MacSidebarRow: View {
}
private var rowBackground: Color {
if isSelected { return palette.accent }
return isHovering ? palette.textPrimary.opacity(0.06) : .clear
if isSelected { return palette.accentMuted }
return isHovering ? palette.textPrimary.opacity(0.05) : .clear
}
}
+182 -49
View File
@@ -1,9 +1,10 @@
// MacSettingsView.swift
// OSGKeyboard · Mac
//
// Settings built on the native grouped `Form` the same container macOS
// System Settings uses. This gives system-accurate cards, dividers, insets
// and right-aligned controls for free, on both light and dark.
// Settings uses native grouped `Form` for correct control layout (Picker /
// Toggle / LabeledContent). Title and Form share the same plain
// `pageHorizontalInset` padding (Form scroll margins are zeroed first) so
// card chrome lines up with the page title.
import SwiftUI
#if os(macOS)
@@ -20,6 +21,7 @@ struct MacSettingsView: View {
private var hasCompletedMacOnboarding = true
@State private var accessibilityTrusted = MacTextInsertionService.isAccessibilityTrusted
@State private var showProviderPicker = false
@State private var showAsrProviderPicker = false
private var lang: AppUILanguage { viewModel.config.uiLanguage }
private let recognitionLocales: [(id: String, key: String, fallback: String)] = [
@@ -33,23 +35,43 @@ struct MacSettingsView: View {
var body: some View {
NavigationStack {
Form {
generalSection
recognitionSection
if viewModel.config.engineMode == "cloud" {
providerSection
.transition(.opacity)
VStack(spacing: 0) {
MacPageHeader(
title: MacL10n.string("mac.section.settings", language: lang),
subtitle: MacL10n.string("mac.page.settings.subtitle", language: lang)
)
Form {
generalSection
recognitionSection
polishProviderSection
if viewModel.config.engineMode == "cloud" {
asrProviderSection
.transition(.opacity)
}
if viewModel.config.engineMode == "local" {
MacLocalASRModelSettingsView(viewModel: viewModel)
.transition(.opacity)
}
inputSection
legalSection
}
if viewModel.config.engineMode == "local" {
MacLocalASRModelSettingsView(viewModel: viewModel)
.transition(.opacity)
}
inputSection
legalSection
.formStyle(.grouped)
// Zero Form's own scroll margins, then inset via padding so the
// section cards line up with MacPageHeader (contentMargins alone
// does not match plain padding on macOS).
//
// grouped Form adds its own built-in section inset on top of our
// padding, so cards sat ~`groupedFormSectionInset` wider than the
// History page. Subtract that inset here so the card OUTER edge
// lands on `pageHorizontalInset` (40pt), matching History and the
// page title's left edge.
.contentMargins(.horizontal, 0, for: .scrollContent)
.padding(.horizontal, MacMetrics.pageHorizontalInset - MacMetrics.groupedFormSectionInset)
.tint(palette.accent)
.scrollContentBackground(.hidden)
.background(palette.background)
}
.formStyle(.grouped)
.tint(palette.accent)
.scrollContentBackground(.hidden)
.background(palette.background)
}
.onAppear { refreshAccessibilityState() }
@@ -82,27 +104,21 @@ struct MacSettingsView: View {
}
}
// MARK: - Cloud provider
// MARK: - Polish LLM
private var providerSection: some View {
Section(MacL10n.string("mac.settings.cloudProvider", language: lang)) {
LabeledContent(MacL10n.string("mac.settings.service", language: lang)) {
Button {
showProviderPicker = true
} label: {
HStack(spacing: 6) {
providerLogo(currentProvider.id)
Text(currentProvider.name)
.foregroundStyle(palette.textPrimary)
Image(systemName: "chevron.up.chevron.down")
.font(.system(size: 9, weight: .semibold))
.foregroundStyle(palette.textTertiary)
}
.contentShape(Rectangle())
}
.buttonStyle(.plain)
.popover(isPresented: $showProviderPicker, arrowEdge: .bottom) {
providerPickerList
private var polishProviderSection: some View {
Section(MacL10n.string("mac.settings.polishProvider", language: lang)) {
providerPickerRow(
title: MacL10n.string("mac.settings.service", language: lang),
provider: currentPolishProvider,
isPresented: $showProviderPicker
) {
providerPickerList(
providers: viewModel.polishSelectableProviders,
selectedId: viewModel.config.providerId
) { provider in
viewModel.selectProvider(provider)
showProviderPicker = false
}
}
@@ -117,6 +133,17 @@ struct MacSettingsView: View {
Text(MacL10n.string("mac.settings.apiKey", language: lang))
}
LabeledContent {
TextField(text: $viewModel.config.baseURL, prompt: Text(verbatim: "")) {
Text(MacL10n.string("mac.settings.baseURL", language: lang))
}
.labelsHidden()
.macFieldStyle()
.frame(maxWidth: MacMetrics.controlWidth)
} label: {
Text(MacL10n.string("mac.settings.baseURL", language: lang))
}
LabeledContent {
TextField(text: $viewModel.config.model, prompt: Text(verbatim: "")) {
Text(MacL10n.string("mac.settings.model", language: lang))
@@ -130,6 +157,86 @@ struct MacSettingsView: View {
}
}
// MARK: - Cloud ASR
private var asrProviderSection: some View {
Section(MacL10n.string("mac.settings.asrProvider", language: lang)) {
providerPickerRow(
title: MacL10n.string("mac.settings.asrService", language: lang),
provider: currentAsrProvider,
isPresented: $showAsrProviderPicker
) {
providerPickerList(
providers: viewModel.asrSelectableProviders,
selectedId: viewModel.config.asrProviderId
) { provider in
viewModel.selectAsrProvider(provider)
showAsrProviderPicker = false
}
}
LabeledContent {
SecureField(text: $viewModel.config.asrApiKey, prompt: Text(verbatim: "sk-…")) {
Text(MacL10n.string("mac.settings.apiKey", language: lang))
}
.labelsHidden()
.macFieldStyle()
.frame(maxWidth: MacMetrics.controlWidth)
} label: {
Text(MacL10n.string("mac.settings.asrApiKey", language: lang))
}
if CloudASRModelCatalog.strategy(for: viewModel.config.asrProviderId) == .prompt {
LabeledContent {
TextField(text: $viewModel.config.asrBaseURL, prompt: Text(verbatim: "")) {
Text(MacL10n.string("mac.settings.baseURL", language: lang))
}
.labelsHidden()
.macFieldStyle()
.frame(maxWidth: MacMetrics.controlWidth)
} label: {
Text(MacL10n.string("mac.settings.baseURL", language: lang))
}
}
LabeledContent {
TextField(text: $viewModel.config.asrModel, prompt: Text(verbatim: "")) {
Text(MacL10n.string("mac.settings.asrModel", language: lang))
}
.labelsHidden()
.macFieldStyle()
.frame(maxWidth: MacMetrics.controlWidth)
} label: {
Text(MacL10n.string("mac.settings.asrModel", language: lang))
}
}
}
private func providerPickerRow<Content: View>(
title: String,
provider: LLMProvider,
isPresented: Binding<Bool>,
@ViewBuilder picker: @escaping () -> Content
) -> some View {
LabeledContent(title) {
Button {
isPresented.wrappedValue = true
} label: {
HStack(spacing: 6) {
providerLogo(provider.id)
Text(provider.name)
.foregroundStyle(palette.textPrimary)
Image(systemName: "chevron.up.chevron.down")
.font(.system(size: 9, weight: .semibold))
.foregroundStyle(palette.textTertiary)
}
.contentShape(Rectangle())
}
.buttonStyle(.plain)
.popover(isPresented: isPresented, arrowEdge: .bottom, content: picker)
}
}
// MARK: - Recognition method
private var recognitionSection: some View {
@@ -161,6 +268,19 @@ struct MacSettingsView: View {
)
}
Picker(selection: hotkeyTriggerBinding) {
ForEach(MacHotkeyTrigger.allCases) { trigger in
Text(MacL10n.string(trigger.labelKey, language: lang))
.tag(trigger.rawValue)
}
} label: {
rowLabel(
MacL10n.string("mac.settings.hotkeyTrigger", language: lang),
subtitle: MacL10n.string("mac.settings.hotkeyTriggerDesc", language: lang)
)
}
.disabled(!viewModel.hotkeyEnabled)
Toggle(isOn: autoPasteBinding) {
rowLabel(
MacL10n.string("mac.settings.autoPaste", language: lang),
@@ -258,28 +378,34 @@ struct MacSettingsView: View {
.buttonStyle(.plain)
}
private var currentProvider: LLMProvider {
viewModel.selectableProviders.first { $0.id == viewModel.config.providerId }
?? viewModel.selectableProviders.first
private var currentPolishProvider: LLMProvider {
viewModel.polishSelectableProviders.first { $0.id == viewModel.config.providerId }
?? viewModel.polishSelectableProviders.first
?? LLMProvider.presets[0]
}
/// Custom dropdown list shown in a popover. SwiftUI's `Menu` label / items
/// silently drop bundled (non-SF-Symbol) images on macOS, so we render the
/// brand marks in a plain view stack instead.
private var providerPickerList: some View {
private var currentAsrProvider: LLMProvider {
viewModel.asrSelectableProviders.first { $0.id == viewModel.config.asrProviderId }
?? viewModel.asrSelectableProviders.first
?? LLMProvider.presets[0]
}
private func providerPickerList(
providers: [LLMProvider],
selectedId: String,
onSelect: @escaping (LLMProvider) -> Void
) -> some View {
VStack(spacing: 0) {
ForEach(viewModel.selectableProviders) { provider in
ForEach(providers) { provider in
Button {
viewModel.selectProvider(provider)
showProviderPicker = false
onSelect(provider)
} label: {
HStack(spacing: Spacing.sm) {
providerLogo(provider.id)
Text(provider.name)
.foregroundStyle(palette.textPrimary)
Spacer(minLength: Spacing.md)
if provider.id == currentProvider.id {
if provider.id == selectedId {
Image(systemName: "checkmark")
.font(.system(size: 12, weight: .semibold))
.foregroundStyle(palette.accent)
@@ -359,6 +485,13 @@ struct MacSettingsView: View {
)
}
private var hotkeyTriggerBinding: Binding<String> {
Binding(
get: { viewModel.hotkeyTrigger.rawValue },
set: { viewModel.setHotkeyTrigger(MacHotkeyTrigger(rawValue: $0) ?? .rightOption) }
)
}
private var autoPasteBinding: Binding<Bool> {
Binding(
get: { viewModel.autoPasteEnabled },
+59 -6
View File
@@ -165,16 +165,69 @@ enum MacSherpaONNXRunner {
.filter { !$0.isEmpty }
for line in lines.reversed() {
if line.hasPrefix("{"), let data = line.data(using: .utf8),
let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
let text = object["text"] as? String {
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty { return trimmed }
if line.hasPrefix("{") {
// Sherpa's JSON result line (`{"text": ..., "lang": ..., ...}`).
// Trust only its `text` field including when it's empty
// (silence/no-speech) and never fall through to the raw
// JSON below, or the JSON blob itself gets inserted as text.
if let data = line.data(using: .utf8),
let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
let text = object["text"] as? String {
return sanitizeTranscript(text)
}
continue
}
if isMetadataNoiseLine(line) { continue }
if !line.hasPrefix("/"), !line.hasPrefix("--"), line.count > 1 {
return line
let cleaned = sanitizeTranscript(line)
if !cleaned.isEmpty { return cleaned }
}
}
return ""
}
/// Qwen3-ASR (via sherpa-onnx) often prefixes the transcript with a
/// scaffold such as `language Chinese<asr_text>`. Older runtimes leave
/// that intact in `result.text`; incomplete generations can even stop at
/// the bare word `language`. Strip the scaffold so only spoken text remains.
private static func sanitizeTranscript(_ raw: String) -> String {
var text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
if text.isEmpty { return "" }
// Prefer the payload after the last `<asr_text>` marker.
if let marker = text.range(of: "<asr_text>", options: .backwards) {
text = String(text[marker.upperBound...])
.trimmingCharacters(in: .whitespacesAndNewlines)
} else if let match = text.range(
of: #"^language\s+\S+\s*"#,
options: [.regularExpression, .caseInsensitive]
) {
// Fallback when the marker token was lost but the language prefix remains.
text = String(text[match.upperBound...])
.trimmingCharacters(in: .whitespacesAndNewlines)
}
// Drop leftover control tokens / bare scaffold words.
if isMetadataNoiseLine(text) { return "" }
return text
}
/// Lines that are sherpa/Qwen metadata rather than spoken content.
private static func isMetadataNoiseLine(_ line: String) -> Bool {
let lowered = line.lowercased()
switch lowered {
case "language", "emotion", "event", "text",
"<asr_text>", "</asr_text>", "<|im_end|>":
return true
default:
// Exact scaffold with no spoken payload, e.g. "language Chinese".
if lowered.range(
of: #"^language(\s+\S+)?$"#,
options: .regularExpression
) != nil {
return true
}
return false
}
}
}
+99 -5
View File
@@ -9,10 +9,65 @@ import Foundation
import Speech
enum MacSpeechLocalASR {
static func transcribe(samples: [Float], locale: Locale) async throws -> String {
/// Shared resume-once state for one recognition run. The recognizer
/// callback (delivered on an arbitrary Speech queue) and the timeout task
/// race to finish, and a `CheckedContinuation` must resume exactly once,
/// so both go through this lock-guarded gate. It also retains the
/// `SFSpeechRecognitionTask` so the losing/failing path can cancel it.
private final class RecognitionSession: @unchecked Sendable {
private let lock = NSLock()
private var isResumed = false
private var task: SFSpeechRecognitionTask?
private var timeoutTask: Task<Void, Never>?
func retain(_ task: SFSpeechRecognitionTask) {
lock.lock()
self.task = task
let alreadyResumed = isResumed
lock.unlock()
// Timeout won the race before the task handle was stored.
if alreadyResumed { task.cancel() }
}
func retainTimeout(_ task: Task<Void, Never>) {
lock.lock()
timeoutTask = task
let alreadyResumed = isResumed
lock.unlock()
// Recognition finished before the handle landed stop the timer.
if alreadyResumed { task.cancel() }
}
/// Returns `true` exactly once across all callers; the winner may
/// resume the continuation. Pass `cancellingTask: true` on failure
/// paths so the in-flight recognition stops doing work. The winner
/// also cancels the timeout task so it doesn't keep the session (and
/// continuation captures) alive for the rest of its sleep.
func claimResume(cancellingTask: Bool) -> Bool {
lock.lock()
guard !isResumed else {
lock.unlock()
return false
}
isResumed = true
let task = self.task
let timeout = timeoutTask
lock.unlock()
if cancellingTask { task?.cancel() }
timeout?.cancel()
return true
}
}
static func transcribe(samples: [Float], locale: Locale, bias: LocalASRBiasPayload? = nil) async throws -> String {
let auth = await requestAuthorization()
guard auth == .authorized else { throw MacLocalASRError.speechDenied }
if Self.isChineseLocale(locale) {
CustomLanguageModelManager.shared.prepareInBackgroundIfNeeded()
_ = try? await CustomLanguageModelManager.shared.prepareIfNeeded()
}
let wavURL = try writeTemporaryWAV(samples: samples, sampleRate: 16_000)
defer { try? FileManager.default.removeItem(at: wavURL) }
@@ -20,18 +75,43 @@ enum MacSpeechLocalASR {
guard let recognizer, recognizer.isAvailable else {
throw MacLocalASRError.speechFailed("Speech recognizer unavailable")
}
// The request below sets `requiresOnDeviceRecognition = true`, which
// fails (or worse, never produces a final result) when the on-device
// model for the locale is missing fail fast with a clear error.
guard recognizer.supportsOnDeviceRecognition else {
throw MacLocalASRError.speechFailed(
"On-device speech recognition is not available for \(recognizer.locale.identifier). Download the language in System Settings → Keyboard → Dictation."
)
}
return try await withCheckedThrowingContinuation { continuation in
// Overall deadline: recognition of a file is normally much faster than
// realtime, so 2× audio length with a 30 s floor is generous. Without
// it, empty audio / cancellation / a missing model can leave the
// callback silent forever and the continuation never resumes.
let audioSeconds = Double(samples.count) / 16_000
let timeoutSeconds = max(30.0, audioSeconds * 2)
let session = RecognitionSession()
return try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<String, Error>) in
let request = SFSpeechURLRecognitionRequest(url: wavURL)
request.shouldReportPartialResults = false
request.requiresOnDeviceRecognition = true
CustomLanguageModelManager.applyCustomLanguageModel(
to: request,
locale: locale,
bias: bias
)
recognizer.recognitionTask(with: request) { result, error in
let task = recognizer.recognitionTask(with: request) { result, error in
if let error {
continuation.resume(throwing: MacLocalASRError.speechFailed(error.localizedDescription))
if session.claimResume(cancellingTask: true) {
continuation.resume(throwing: MacLocalASRError.speechFailed(error.localizedDescription))
}
return
}
// Non-final callbacks carry no usable transcript yet; if a
// final result never arrives, the timeout below resumes us.
guard let result, result.isFinal else { return }
guard session.claimResume(cancellingTask: false) else { return }
let text = result.bestTranscription.formattedString
.trimmingCharacters(in: .whitespacesAndNewlines)
if text.isEmpty {
@@ -40,6 +120,16 @@ enum MacSpeechLocalASR {
continuation.resume(returning: text)
}
}
session.retain(task)
let timeout = Task {
try? await Task.sleep(for: .seconds(timeoutSeconds))
guard !Task.isCancelled else { return }
if session.claimResume(cancellingTask: true) {
continuation.resume(throwing: MacLocalASRError.speechFailed("Speech recognition timed out"))
}
}
session.retainTimeout(timeout)
}
}
@@ -58,4 +148,8 @@ enum MacSpeechLocalASR {
try wav.write(to: url)
return url
}
private static func isChineseLocale(_ locale: Locale) -> Bool {
locale.identifier(.bcp47).lowercased().hasPrefix("zh")
}
}
+141 -13
View File
@@ -1,8 +1,10 @@
// MacTextInsertionService.swift
// OSGKeyboard · Mac
//
// Inserts transcribed text into the frontmost app: clipboard first, then
// Inserts transcribed text into the target app: clipboard first, then
// a synthetic V (SayIt / Typeless-style). Requires Accessibility trust.
// Re-activates the app the user was dictating into (the popover steals
// focus) and restores the original clipboard once the paste has landed.
import AppKit
@preconcurrency import ApplicationServices
@@ -30,32 +32,158 @@ enum MacTextInsertionService {
return AXIsProcessTrustedWithOptions(options)
}
/// Copy to pasteboard and optionally simulate V in the front app.
// MARK: - Paste-target tracking
/// Start observing app activations early (app launch) so a paste target
/// can still be recovered while OSGKeyboard itself is frontmost e.g.
/// when a recording is started from the menu-bar popover.
@MainActor
static func beginTrackingFrontmostApp() {
_ = FrontmostAppTracker.shared
}
/// The app a synthesized V should land in: the current frontmost app,
/// or when OSGKeyboard is frontmost because the popover has key
/// focus the app that was active immediately before it.
@MainActor
static func captureTargetApplication() -> NSRunningApplication? {
let selfPid = NSRunningApplication.current.processIdentifier
if let front = NSWorkspace.shared.frontmostApplication,
front.processIdentifier != selfPid {
return front
}
return FrontmostAppTracker.shared.lastExternalApp
}
// MARK: - Insertion
/// Copy to pasteboard and optionally simulate V in the target app.
/// Returns `true` only if the V event was actually synthesized. After a
/// successful paste the user's original clipboard is put back, so
/// dictation never permanently clobbers it.
@MainActor
static func insert(
_ text: String,
autoPaste: Bool
) throws -> Bool {
autoPaste: Bool,
targetApp: NSRunningApplication? = nil
) async throws -> Bool {
guard !text.isEmpty else { return false }
let pasteboard = NSPasteboard.general
let snapshot = snapshotItems(of: pasteboard)
pasteboard.clearContents()
pasteboard.setString(text, forType: .string)
guard autoPaste else { return false }
guard AXIsProcessTrusted() else { throw InsertionError.accessibilityNotGranted }
Thread.sleep(forTimeInterval: 0.08)
postCommandV()
// Make sure V lands in the app the user was dictating into, not in
// OSGKeyboard's own popover / window.
if let targetApp { await activate(targetApp) }
try? await Task.sleep(nanoseconds: 80_000_000)
guard postCommandV() else { return false }
// Give the target app time to read the transcript off the
// pasteboard, then restore whatever the user had on it.
try? await Task.sleep(nanoseconds: 300_000_000)
restoreItems(snapshot, to: pasteboard)
return true
}
private static func postCommandV() {
/// Brings `app` forward and waits (up to ~1 s) until it is frontmost so
/// the synthesized keystroke isn't swallowed mid-switch.
@MainActor
private static func activate(_ app: NSRunningApplication) async {
func isFront() -> Bool {
NSWorkspace.shared.frontmostApplication?.processIdentifier == app.processIdentifier
}
guard !isFront() else { return }
app.activate()
var attempts = 0
while !isFront(), attempts < 20 {
try? await Task.sleep(nanoseconds: 50_000_000)
attempts += 1
}
}
// MARK: - Pasteboard preservation
/// Every representation of every pasteboard item, so restore round-trips
/// rich content (images, files, multiple flavours) losslessly.
private static func snapshotItems(
of pasteboard: NSPasteboard
) -> [[NSPasteboard.PasteboardType: Data]] {
(pasteboard.pasteboardItems ?? []).map { item in
item.types.reduce(into: [NSPasteboard.PasteboardType: Data]()) { flavours, type in
flavours[type] = item.data(forType: type)
}
}
}
private static func restoreItems(
_ items: [[NSPasteboard.PasteboardType: Data]],
to pasteboard: NSPasteboard
) {
guard !items.isEmpty else { return }
pasteboard.clearContents()
pasteboard.writeObjects(items.map { flavours in
let item = NSPasteboardItem()
for (type, data) in flavours { item.setData(data, forType: type) }
return item
})
}
/// Returns `false` when the CGEvents could not be created in that case
/// nothing was pasted and callers must not report success.
private static func postCommandV() -> Bool {
let source = CGEventSource(stateID: .combinedSessionState)
let keyCode = CGKeyCode(kVK_ANSI_V)
let keyDown = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: true)
let keyUp = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: false)
keyDown?.flags = CGEventFlags.maskCommand
keyUp?.flags = CGEventFlags.maskCommand
keyDown?.post(tap: CGEventTapLocation.cghidEventTap)
keyUp?.post(tap: CGEventTapLocation.cghidEventTap)
guard
let keyDown = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: true),
let keyUp = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: false)
else { return false }
keyDown.flags = CGEventFlags.maskCommand
keyUp.flags = CGEventFlags.maskCommand
keyDown.post(tap: CGEventTapLocation.cghidEventTap)
keyUp.post(tap: CGEventTapLocation.cghidEventTap)
return true
}
}
// MARK: - Frontmost-app tracker
/// Remembers the most recent non-OSGKeyboard frontmost app. Needed because
/// the menu-bar popover activates OSGKeyboard, hiding the real paste target
/// from `NSWorkspace.frontmostApplication`.
@MainActor
private final class FrontmostAppTracker: NSObject {
static let shared = FrontmostAppTracker()
private(set) var lastExternalApp: NSRunningApplication?
private override init() {
super.init()
// Seed with whatever is frontmost now (usually not us at launch).
if let front = NSWorkspace.shared.frontmostApplication,
front.processIdentifier != NSRunningApplication.current.processIdentifier {
lastExternalApp = front
}
NSWorkspace.shared.notificationCenter.addObserver(
self,
selector: #selector(appDidActivate(_:)),
name: NSWorkspace.didActivateApplicationNotification,
object: nil
)
}
deinit {
NSWorkspace.shared.notificationCenter.removeObserver(self)
}
@objc private func appDidActivate(_ notification: Notification) {
guard
let app = notification.userInfo?[NSWorkspace.applicationUserInfoKey] as? NSRunningApplication,
app.processIdentifier != NSRunningApplication.current.processIdentifier
else { return }
lastExternalApp = app
}
}
+10 -5
View File
@@ -6,7 +6,8 @@
// semantic colours, resolved to a concrete value for the *active* appearance.
// The brand green is kept only as the accent. Light mode uses a warm,
// iOS-matched surface set (the default `windowBackgroundColor` reads cold
// grey on macOS); Dark mode keeps the native AppKit semantic colours.
// grey on macOS); Dark mode uses stepped elevated greys so cards stay
// readable against the page background.
import AppKit
import SwiftUI
@@ -33,8 +34,8 @@ enum MacSystemPalette {
surfaceMuted: resolved(dark ? darkMuted : warmMuted, dark: dark),
accent: Palette.accent,
accentMuted: Palette.accent.opacity(0.16),
accentGlow: Palette.accent.opacity(0.35),
accentMuted: Palette.accent.opacity(dark ? 0.22 : 0.14),
accentGlow: Palette.accent.opacity(dark ? 0.40 : 0.32),
danger: resolved(.systemRed, dark: dark),
success: Palette.accent,
@@ -45,8 +46,12 @@ enum MacSystemPalette {
textTertiary: resolved(.tertiaryLabelColor, dark: dark),
textOnAccent: Color.white,
divider: resolved(.separatorColor, dark: dark),
dividerStrong: resolved(.separatorColor, dark: dark),
divider: dark
? Color.white.opacity(0.08)
: Color.black.opacity(0.06),
dividerStrong: dark
? Color.white.opacity(0.12)
: Color.black.opacity(0.10),
recordRed: resolved(.systemRed, dark: dark)
)
+6 -1
View File
@@ -57,7 +57,9 @@ struct OSGKeyboardMacApp: App {
// very top, matching macOS System Settings.
.windowStyle(.hiddenTitleBar)
.windowResizability(.contentMinSize)
.defaultSize(width: 1_024, height: 720)
// Open at the minimum size same as `MacMetrics.windowMin*`, so the
// first launch already matches the smallest allowed window.
.defaultSize(width: MacMetrics.windowMinWidth, height: MacMetrics.windowMinHeight)
}
}
@@ -94,8 +96,11 @@ final class MacAppDelegate: NSObject, NSApplicationDelegate {
func applicationDidFinishLaunching(_ notification: Notification) {
MacAppearancePreference.applyToApp(.current)
MacTextInsertionService.beginTrackingFrontmostApp()
configurePopover()
configureStatusItem()
MacDictationOverlayController.shared.start(observing: MacDictationViewModel.shared)
CustomLanguageModelManager.shared.prepareInBackgroundIfNeeded()
// The menu bar always follows the *system* appearance, so the status
// item must ignore the app's forced light/dark override. Re-pin the
@@ -13,10 +13,18 @@ import Foundation
/// Keep this protocol narrow: only what the shared pipeline needs today.
/// Platform-specific settings UI and iCloud sync stay on concrete stores.
public protocol ConfigurationStore: Sendable {
/// Polish / LLM provider id.
var providerId: String { get }
var baseURL: String { get }
var apiKey: String { get }
var model: String { get }
/// Cloud ASR provider id independent from polish when `engineMode == "cloud"`.
var asrProviderId: String { get }
var asrBaseURL: String { get }
var asrApiKey: String { get }
var asrModel: String { get }
var engineMode: String { get }
var polishIntensity: PolishIntensity { get }
var personalDictionary: PersonalDictionary { get }
@@ -180,6 +180,8 @@ public enum TypeStyle {
public static let title3 = Font.system(size: 20, weight: .semibold)
public static let title2 = Font.system(size: 22, weight: .bold)
public static let title = Font.system(size: 28, weight: .bold)
/// Home brand line + History / Dictionary / Settings page titles.
public static let pageTitle = Font.system(size: 30, weight: .semibold)
public static let largeTitle = Font.system(size: 34, weight: .bold)
/// Subtle status line under the brand mark (home header).
public static let status = Font.system(size: 13, weight: .regular)
@@ -16,6 +16,10 @@ public struct AppGroupConfiguration: Sendable, Equatable {
/// Legacy plaintext slot migrated to Keychain on first read.
public static let apiKeyLegacy = "config.apiKey"
public static let model = "config.model"
/// Cloud ASR provider independent from polish `providerId`.
public static let asrProviderId = "config.asrProviderId"
public static let asrBaseURL = "config.asrBaseURL"
public static let asrModel = "config.asrModel"
public static let modeId = "config.modeId"
public static let localeId = "config.localeId"
public static let engineMode = "config.engineMode"
@@ -51,6 +55,10 @@ public struct AppGroupConfiguration: Sendable, Equatable {
public var providerId: String
public var baseURL: String
public var model: String
/// Cloud-engine speech-to-text provider (OpenLess-style split from polish).
public var asrProviderId: String
public var asrBaseURL: String
public var asrModel: String
public var modeId: String
public var localeId: String
public var engineMode: String
@@ -95,18 +103,32 @@ public struct AppGroupConfiguration: Sendable, Equatable {
: .polish
}
/// Local engine pins the LLM step to DeepSeek; cloud uses the user's provider.
public var polishProviderIdOverride: String? {
engineMode == "local" ? "deepseek" : nil
}
/// Polish LLM provider. Local engine no longer pins DeepSeek user picks in Settings.
public var polishProviderIdOverride: String? { nil }
public var isCloudAPIKeyMissingForVoiceInput: Bool {
public var isCloudLLMKeyMissing: Bool {
guard engineMode == "cloud" else { return false }
return apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
}
/// API key lives in the Keychain (cross-process, encrypted at rest).
/// When settings iCloud sync is on, reads synchronizable Keychain items first.
public var isCloudASRKeyMissing: Bool {
guard engineMode == "cloud" else { return false }
return asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
}
public var isPolishKeyMissing: Bool {
if !apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return false
}
return !PreconfiguredKeys.isDeepseekConfigured
}
public var isCloudAPIKeyMissingForVoiceInput: Bool {
guard engineMode == "cloud" else { return false }
return isCloudASRKeyMissing || isCloudLLMKeyMissing
}
/// Polish LLM uses `providerId` + Keychain `provider.<id>`.
public var apiKey: String {
Self.resolveAPIKey(
defaults: nil,
@@ -115,6 +137,15 @@ public struct AppGroupConfiguration: Sendable, Equatable {
)
}
/// Cloud ASR uses `asrProviderId` + Keychain `asr.<id>` (falls back to legacy `provider.<id>`).
public var asrApiKey: String {
Self.resolveASRAPIKey(
defaults: nil,
providerId: asrProviderId,
preferICloudSync: settingsICloudSyncEnabled
)
}
public func makeClient() -> LLMClient {
OpenAICompatibleClient(
baseURL: baseURL,
@@ -123,6 +154,20 @@ public struct AppGroupConfiguration: Sendable, Equatable {
)
}
/// Resolved cloud ASR model user override or catalog default.
public var resolvedASRModel: String {
let trimmed = asrModel.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty { return trimmed }
return CloudASRModelCatalog.defaultModel(for: asrProviderId)
}
/// Resolved cloud ASR base URL for prompt-style providers.
public var resolvedASRBaseURL: String {
let trimmed = asrBaseURL.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty { return trimmed }
return LLMProvider.provider(id: asrProviderId).defaultBaseURL
}
// MARK: - Detected app context
public func detectedAppContext(from defaults: UserDefaults) -> (context: AppContext, observedAt: Date)? {
@@ -152,9 +197,18 @@ public struct AppGroupConfiguration: Sendable, Equatable {
providerId: defaults.string(forKey: Keys.providerId) ?? "openai",
baseURL: "",
model: "",
asrProviderId: defaults.string(forKey: Keys.asrProviderId) ?? "",
asrBaseURL: "",
asrModel: "",
modeId: defaults.string(forKey: Keys.modeId) ?? "polish",
localeId: defaults.string(forKey: Keys.localeId) ?? "auto",
engineMode: defaults.string(forKey: Keys.engineMode) ?? "cloud",
// Privacy-critical default: `local` keeps raw audio on-device
// (SpeechAnalyzer). The `cloud` engine uploads recorded audio to
// the user's configured ASR provider and must stay an explicit,
// acknowledged opt-in (see `hasAcknowledgedCloudSharing`) a
// cloud default would contradict every privacy claim the app
// makes in its docs, App Store listing, and permission prompts.
engineMode: defaults.string(forKey: Keys.engineMode) ?? "local",
hasCompletedOnboarding: defaults.bool(forKey: Keys.hasCompletedOnboarding),
onboardingPage: {
let saved = defaults.integer(forKey: Keys.onboardingPage)
@@ -212,6 +266,19 @@ public struct AppGroupConfiguration: Sendable, Equatable {
config.model = defaults.string(forKey: Keys.model) ?? preset.defaultModel
}
if config.asrProviderId.isEmpty {
config.asrProviderId = config.providerId
defaults.set(config.asrProviderId, forKey: Keys.asrProviderId)
}
let asrPreset = LLMProvider.provider(id: config.asrProviderId)
if config.asrBaseURL.isEmpty {
config.asrBaseURL = defaults.string(forKey: Keys.asrBaseURL) ?? asrPreset.defaultBaseURL
}
if config.asrModel.isEmpty {
config.asrModel = defaults.string(forKey: Keys.asrModel)
?? CloudASRModelCatalog.defaultModel(for: config.asrProviderId)
}
// One-shot legacy migration: plaintext apiKey in UserDefaults Keychain.
_ = resolveAPIKey(
defaults: defaults,
@@ -219,6 +286,26 @@ public struct AppGroupConfiguration: Sendable, Equatable {
preferICloudSync: config.settingsICloudSyncEnabled
)
// One-shot default migration for installs that predate an explicit
// stored value. The privacy-safe defaults ("local", 30 min TTL) are
// for NEW installs only an existing user who ran on the old
// defaults must keep their behavior, both because silently changing
// engines under someone is wrong, and because iCloud settings sync
// would stamp the flip as a fresh "edit" and propagate it to every
// other device, overriding choices made there. Persisting the
// resolved value makes the decision stable and sync-invisible.
let isExistingInstall = defaults.bool(forKey: Keys.hasCompletedOnboarding)
if defaults.string(forKey: Keys.engineMode) == nil {
let resolved = isExistingInstall ? "cloud" : "local"
config.engineMode = resolved
defaults.set(resolved, forKey: Keys.engineMode)
}
if defaults.string(forKey: Keys.flowInactivityDuration) == nil {
let resolved: FlowInactivityDuration = isExistingInstall ? .twelveHours : .default
config.flowInactivityDuration = resolved
defaults.set(resolved.rawValue, forKey: Keys.flowInactivityDuration)
}
// Cloud no longer exposes off/transcribe; migrate legacy values.
if config.engineMode == "cloud", config.modeId != "polish" {
config.modeId = "polish"
@@ -234,6 +321,15 @@ public struct AppGroupConfiguration: Sendable, Equatable {
defaults.set(openAI.defaultBaseURL, forKey: Keys.baseURL)
defaults.set(openAI.defaultModel, forKey: Keys.model)
}
if config.engineMode == "cloud", config.asrProviderId == "deepseek" {
let openAI = LLMProvider.provider(id: "openai")
config.asrProviderId = openAI.id
config.asrBaseURL = openAI.defaultBaseURL
config.asrModel = CloudASRModelCatalog.defaultModel(for: openAI.id)
defaults.set(openAI.id, forKey: Keys.asrProviderId)
defaults.set(openAI.defaultBaseURL, forKey: Keys.asrBaseURL)
defaults.set(openAI.defaultModel, forKey: Keys.asrModel)
}
return config
}
@@ -242,6 +338,9 @@ public struct AppGroupConfiguration: Sendable, Equatable {
defaults.set(providerId, forKey: Keys.providerId)
defaults.set(baseURL, forKey: Keys.baseURL)
defaults.set(model, forKey: Keys.model)
defaults.set(asrProviderId, forKey: Keys.asrProviderId)
defaults.set(asrBaseURL, forKey: Keys.asrBaseURL)
defaults.set(asrModel, forKey: Keys.asrModel)
defaults.set(modeId, forKey: Keys.modeId)
defaults.set(localeId, forKey: Keys.localeId)
defaults.set(engineMode, forKey: Keys.engineMode)
@@ -328,4 +427,16 @@ public struct AppGroupConfiguration: Sendable, Equatable {
}
return ""
}
static func resolveASRAPIKey(
defaults: UserDefaults?,
providerId: String,
preferICloudSync: Bool = false
) -> String {
if let stored = Keychain.asrApiKey(for: providerId, preferICloudSync: preferICloudSync), !stored.isEmpty {
return stored
}
// Pre-split installs: one shared key under `provider.<id>`.
return resolveAPIKey(defaults: defaults, providerId: providerId, preferICloudSync: preferICloudSync)
}
}
@@ -0,0 +1,12 @@
// CloudProviderRole.swift
// OSGKeyboard · Shared
//
// Distinguishes cloud ASR credentials from polish LLM credentials
// (OpenLess-style split).
import Foundation
public enum CloudProviderRole: String, Sendable, Equatable {
case asr
case polish
}
@@ -15,7 +15,12 @@ public enum FlowInactivityDuration: String, CaseIterable, Identifiable, Sendable
public var id: String { rawValue }
public static let `default`: FlowInactivityDuration = .twelveHours
/// 30 minutes, not hours: competitors cap sessions at 560 min for a
/// reason a very long TTL keeps advertising "session active" long after
/// the host process is likely suspended or dead, amplifying every stale-
/// state bug into hours of confusing UI. Users can still opt into longer
/// windows explicitly.
public static let `default`: FlowInactivityDuration = .thirtyMinutes
public var timeInterval: TimeInterval {
switch self {
@@ -108,4 +108,11 @@ public struct LLMProvider: Identifiable, Codable, Hashable, Sendable {
public static var userSelectablePresets: [LLMProvider] {
presets.filter(\.isUserSelectable)
}
/// Cloud ASR presets (excludes providers without a cloud transcription API).
public static var asrSelectablePresets: [LLMProvider] {
userSelectablePresets.filter {
CloudASRModelCatalog.strategy(for: $0.id) != .localFallback
}
}
}
@@ -75,6 +75,9 @@ public struct LocalASRModelDefinition: Codable, Sendable, Equatable, Identifiabl
public let recommendedLocales: [String]
public let supportsHotwords: Bool
public let hotwordMode: LocalASRHotwordMode
/// Optional localization key for a short quality/speed badge
/// (e.g. `mac.localASR.badge.fastest`).
public let badgeKey: String?
public let installKind: LocalASRInstallKind
public let installRelativePath: String?
public let archiveBaseName: String?
@@ -9,7 +9,16 @@ import Foundation
extension PersonalDictionary {
public static let kvsKeyV2 = "personalDictionary.v2"
public static let legacyKVSKey = "personalDictionary.v1"
public static let tombstoneRetention: TimeInterval = 90 * 24 * 60 * 60
/// Tombstones guard against deleted entries "resurrecting" when a
/// long-offline device rejoins and re-merges them. A short wall-clock
/// retention re-opened that window after only 90 days; a year keeps the
/// window closed for any realistically dormant device while staying tiny
/// on the wire (a tombstone is ~60 bytes of JSON), and the count cap
/// bounds the worst case regardless of clock.
public static let tombstoneRetention: TimeInterval = 365 * 24 * 60 * 60
/// Hard cap independent of wall clock the oldest tombstones are
/// dropped first once exceeded.
public static let maxTombstones = 500
/// Merges two dictionary snapshots for cross-device sync.
///
@@ -100,11 +109,19 @@ extension PersonalDictionary {
clearedAt: Date?
) -> [UUID: Date] {
let cutoff = Date().addingTimeInterval(-tombstoneRetention)
return tombstones.filter { _, deletedAt in
var kept = tombstones.filter { _, deletedAt in
if deletedAt < cutoff { return false }
if let clearedAt, deletedAt <= clearedAt { return false }
return true
}
// Enforce the count cap that makes the 365-day retention safe on the
// KVS byte budget: keep the NEWEST tombstones (dropping an old one
// early only re-opens the resurrection window for that one entry).
if kept.count > maxTombstones {
let newest = kept.sorted { $0.value > $1.value }.prefix(maxTombstones)
kept = Dictionary(uniqueKeysWithValues: newest.map { ($0.key, $0.value) })
}
return kept
}
private static func later(of lhs: Date?, and rhs: Date?) -> Date? {
+104 -11
View File
@@ -53,6 +53,44 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
persistConfiguration()
}
}
@Published public var asrProviderId: String {
didSet {
guard !isApplyingConfiguration, asrProviderId != configuration.asrProviderId else { return }
configuration.asrProviderId = asrProviderId
isSyncingASRProviderAPIKey = true
asrApiKey = configuration.asrApiKey
isSyncingASRProviderAPIKey = false
persistConfiguration()
}
}
@Published public var asrBaseURL: String {
didSet {
guard !isApplyingConfiguration, asrBaseURL != configuration.asrBaseURL else { return }
configuration.asrBaseURL = asrBaseURL
persistConfiguration()
}
}
@Published public var asrApiKey: String {
didSet {
guard oldValue != asrApiKey, !isSyncingASRProviderAPIKey else { return }
do {
try Keychain.setASRAPIKey(
asrApiKey,
for: asrProviderId,
useICloudSync: configuration.settingsICloudSyncEnabled
)
} catch {
OSGLog.config.warning("ASR Keychain write failed: \(error.localizedDescription, privacy: .public)")
}
}
}
@Published public var asrModel: String {
didSet {
guard !isApplyingConfiguration, asrModel != configuration.asrModel else { return }
configuration.asrModel = asrModel
persistConfiguration()
}
}
@Published public var modeId: String {
didSet {
guard !isApplyingConfiguration, modeId != configuration.modeId else { return }
@@ -67,8 +105,8 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
persistConfiguration()
}
}
/// "local" on-device ASR + built-in DeepSeek polish.
/// "cloud" provider cloud ASR (with personal dictionary) + user's cloud LLM polish.
/// "local" on-device ASR + user's LLM polish (or built-in DeepSeek).
/// "cloud" user's cloud ASR + user's cloud LLM polish (independent picks).
@Published public var engineMode: String {
didSet {
guard !isApplyingConfiguration, engineMode != configuration.engineMode else { return }
@@ -213,25 +251,38 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
}
public var isConfigured: Bool {
// Local engine uses on-device ASR + built-in DeepSeek polish and
// does not need a user API key. Cloud needs base URL, key, and model.
if isLocalEngine { return true }
return !baseURL.isEmpty && !apiKey.isEmpty && !model.isEmpty
if isLocalEngine {
return isPolishConfigured
}
return isASRConfigured && isPolishConfigured
}
public var isPolishConfigured: Bool {
if !apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return !baseURL.isEmpty && !model.isEmpty
}
return PreconfiguredKeys.isDeepseekConfigured
}
public var isASRConfigured: Bool {
guard !isLocalEngine else { return true }
return !asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
&& (!asrBaseURL.isEmpty || CloudASRModelCatalog.strategy(for: asrProviderId) != .prompt)
}
/// On-device ASR only; no cloud API required.
public var isLocalEngine: Bool { configuration.isLocalEngine }
/// Local engine always polishes via the built-in DeepSeek path.
public var shouldPolishLocalTranscript: Bool { isLocalEngine }
/// Cloud engine uses `providerId`. Local engine pins DeepSeek.
public var localModeProviderId: String { "deepseek" }
/// Built-in DeepSeek path when the user has not supplied their own LLM key.
public var localModeProviderId: String {
apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty ? "deepseek" : providerId
}
private let defaults: UserDefaults
private var configuration: AppGroupConfiguration
private var isApplyingConfiguration = false
private var isSyncingProviderAPIKey = false
private var isSyncingASRProviderAPIKey = false
public init(defaults: UserDefaults? = nil) {
guard let resolvedDefaults = defaults ?? AppGroup.defaultsIfAvailable else {
@@ -270,6 +321,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
baseURL = configuration.baseURL
apiKey = configuration.apiKey
model = configuration.model
asrProviderId = configuration.asrProviderId
asrBaseURL = configuration.asrBaseURL
asrModel = configuration.asrModel
modeId = configuration.modeId
localeId = configuration.localeId
engineMode = configuration.engineMode
@@ -284,6 +338,12 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
flowSkipAppSwitch = configuration.flowSkipAppSwitch
flowInactivityDuration = configuration.flowInactivityDuration
localASRCustomLanguageModelEnabled = configuration.localASRCustomLanguageModelEnabled
isSyncingProviderAPIKey = true
apiKey = configuration.apiKey
isSyncingProviderAPIKey = false
isSyncingASRProviderAPIKey = true
asrApiKey = configuration.asrApiKey
isSyncingASRProviderAPIKey = false
isApplyingConfiguration = false
}
@@ -293,6 +353,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
if engineMode == "cloud", providerId == "deepseek" {
apply(preset: LLMProvider.provider(id: "openai"))
}
if engineMode == "cloud", asrProviderId == "deepseek" {
applyAsr(preset: LLMProvider.provider(id: "openai"))
}
}
private func persistConfiguration(postConfigChanged: Bool = false) {
@@ -324,6 +387,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
providerId = fresh.providerId
baseURL = fresh.baseURL
model = fresh.model
asrProviderId = fresh.asrProviderId
asrBaseURL = fresh.asrBaseURL
asrModel = fresh.asrModel
modeId = fresh.modeId
localeId = fresh.localeId
engineMode = fresh.engineMode
@@ -341,6 +407,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
isSyncingProviderAPIKey = true
apiKey = fresh.apiKey
isSyncingProviderAPIKey = false
isSyncingASRProviderAPIKey = true
asrApiKey = fresh.asrApiKey
isSyncingASRProviderAPIKey = false
isApplyingConfiguration = false
}
@@ -370,6 +439,23 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
persistConfiguration()
}
public func applyAsr(preset: LLMProvider) {
isApplyingConfiguration = true
asrProviderId = preset.id
if !preset.defaultBaseURL.isEmpty {
asrBaseURL = preset.defaultBaseURL
}
asrModel = CloudASRModelCatalog.defaultModel(for: preset.id)
configuration.asrProviderId = asrProviderId
configuration.asrBaseURL = asrBaseURL
configuration.asrModel = asrModel
isSyncingASRProviderAPIKey = true
asrApiKey = configuration.asrApiKey
isSyncingASRProviderAPIKey = false
isApplyingConfiguration = false
persistConfiguration()
}
public func reset() {
isApplyingConfiguration = true
let preset = LLMProvider.provider(id: "openai")
@@ -377,12 +463,19 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
baseURL = preset.defaultBaseURL
apiKey = ""
model = preset.defaultModel
asrProviderId = preset.id
asrBaseURL = preset.defaultBaseURL
asrModel = CloudASRModelCatalog.defaultModel(for: preset.id)
asrApiKey = ""
handednessPreference = .left
localASRCustomLanguageModelEnabled = true
hasAcknowledgedCloudSharing = false
configuration.providerId = preset.id
configuration.baseURL = preset.defaultBaseURL
configuration.model = preset.defaultModel
configuration.asrProviderId = preset.id
configuration.asrBaseURL = preset.defaultBaseURL
configuration.asrModel = CloudASRModelCatalog.defaultModel(for: preset.id)
configuration.handednessPreference = .left
configuration.localASRCustomLanguageModelEnabled = true
configuration.hasAcknowledgedCloudSharing = false
@@ -14,6 +14,10 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
public var providerId: SyncedField<String>
public var baseURL: SyncedField<String>
public var model: SyncedField<String>
/// Cloud ASR provider independent from polish `providerId`.
public var asrProviderId: SyncedField<String>
public var asrBaseURL: SyncedField<String>
public var asrModel: SyncedField<String>
public var modeId: SyncedField<String>
public var localeId: SyncedField<String>
public var engineMode: SyncedField<String>
@@ -31,6 +35,9 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
providerId: SyncedField<String>,
baseURL: SyncedField<String>,
model: SyncedField<String>,
asrProviderId: SyncedField<String>,
asrBaseURL: SyncedField<String>,
asrModel: SyncedField<String>,
modeId: SyncedField<String>,
localeId: SyncedField<String>,
engineMode: SyncedField<String>,
@@ -47,6 +54,9 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
self.providerId = providerId
self.baseURL = baseURL
self.model = model
self.asrProviderId = asrProviderId
self.asrBaseURL = asrBaseURL
self.asrModel = asrModel
self.modeId = modeId
self.localeId = localeId
self.engineMode = engineMode
@@ -60,12 +70,88 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
self.flowInactivityDuration = flowInactivityDuration
}
private enum CodingKeys: String, CodingKey {
case schemaVersion
case providerId
case baseURL
case model
case asrProviderId
case asrBaseURL
case asrModel
case modeId
case localeId
case engineMode
case hasAcknowledgedCloudSharing
case uiLanguage
case translationTargetLocaleId
case handednessPreference
case cursorDragNavigationEnabled
case polishIntensity
case flowSkipAppSwitch
case flowInactivityDuration
}
public init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
schemaVersion = try container.decode(Int.self, forKey: .schemaVersion)
providerId = try container.decode(SyncedField<String>.self, forKey: .providerId)
baseURL = try container.decode(SyncedField<String>.self, forKey: .baseURL)
model = try container.decode(SyncedField<String>.self, forKey: .model)
modeId = try container.decode(SyncedField<String>.self, forKey: .modeId)
localeId = try container.decode(SyncedField<String>.self, forKey: .localeId)
engineMode = try container.decode(SyncedField<String>.self, forKey: .engineMode)
hasAcknowledgedCloudSharing = try container.decode(SyncedField<Bool>.self, forKey: .hasAcknowledgedCloudSharing)
uiLanguage = try container.decode(SyncedField<AppUILanguage>.self, forKey: .uiLanguage)
translationTargetLocaleId = try container.decode(
SyncedField<String>.self,
forKey: .translationTargetLocaleId
)
handednessPreference = try container.decode(
SyncedField<HandednessPreference>.self,
forKey: .handednessPreference
)
cursorDragNavigationEnabled = try container.decode(
SyncedField<Bool>.self,
forKey: .cursorDragNavigationEnabled
)
polishIntensity = try container.decode(SyncedField<PolishIntensity>.self, forKey: .polishIntensity)
flowSkipAppSwitch = try container.decode(SyncedField<Bool>.self, forKey: .flowSkipAppSwitch)
flowInactivityDuration = try container.decode(
SyncedField<FlowInactivityDuration>.self,
forKey: .flowInactivityDuration
)
if let asrProvider = try container.decodeIfPresent(SyncedField<String>.self, forKey: .asrProviderId) {
asrProviderId = asrProvider
} else {
asrProviderId = providerId
}
if let asrURL = try container.decodeIfPresent(SyncedField<String>.self, forKey: .asrBaseURL) {
asrBaseURL = asrURL
} else {
asrBaseURL = baseURL
}
if let asrModelField = try container.decodeIfPresent(SyncedField<String>.self, forKey: .asrModel) {
asrModel = asrModelField
} else {
let fallbackModel = CloudASRModelCatalog.defaultModel(for: providerId.value)
asrModel = SyncedField(
value: fallbackModel,
updatedAt: model.updatedAt,
deviceID: model.deviceID
)
}
}
/// Monotonic stamp used for `settingsCloudUpdatedAt` bookkeeping.
public var latestUpdatedAt: Date {
[
providerId.updatedAt,
baseURL.updatedAt,
model.updatedAt,
asrProviderId.updatedAt,
asrBaseURL.updatedAt,
asrModel.updatedAt,
modeId.updatedAt,
localeId.updatedAt,
engineMode.updatedAt,
@@ -99,6 +185,9 @@ public extension SyncedAppSettingsV2 {
providerId: field(configuration.providerId),
baseURL: field(configuration.baseURL),
model: field(configuration.model),
asrProviderId: field(configuration.asrProviderId),
asrBaseURL: field(configuration.asrBaseURL),
asrModel: field(configuration.asrModel),
modeId: field(configuration.modeId),
localeId: field(configuration.localeId),
engineMode: field(configuration.engineMode),
@@ -119,10 +208,14 @@ public extension SyncedAppSettingsV2 {
func field<T>(_ value: T) -> SyncedField<T> {
SyncedField(value: value, updatedAt: stamp, deviceID: deviceID)
}
let provider = field(legacy.providerId)
return SyncedAppSettingsV2(
providerId: field(legacy.providerId),
providerId: provider,
baseURL: field(legacy.baseURL),
model: field(legacy.model),
asrProviderId: provider,
asrBaseURL: field(legacy.baseURL),
asrModel: field(CloudASRModelCatalog.defaultModel(for: legacy.providerId)),
modeId: field(legacy.modeId),
localeId: field(legacy.localeId),
engineMode: field(legacy.engineMode),
@@ -142,6 +235,9 @@ public extension SyncedAppSettingsV2 {
providerId: .merge(local: local.providerId, remote: remote.providerId),
baseURL: .merge(local: local.baseURL, remote: remote.baseURL),
model: .merge(local: local.model, remote: remote.model),
asrProviderId: .merge(local: local.asrProviderId, remote: remote.asrProviderId),
asrBaseURL: .merge(local: local.asrBaseURL, remote: remote.asrBaseURL),
asrModel: .merge(local: local.asrModel, remote: remote.asrModel),
modeId: .merge(local: local.modeId, remote: remote.modeId),
localeId: .merge(local: local.localeId, remote: remote.localeId),
engineMode: .merge(local: local.engineMode, remote: remote.engineMode),
@@ -172,6 +268,9 @@ public extension SyncedAppSettingsV2 {
configuration.providerId = providerId.value
configuration.baseURL = baseURL.value
configuration.model = model.value
configuration.asrProviderId = asrProviderId.value
configuration.asrBaseURL = asrBaseURL.value
configuration.asrModel = asrModel.value
configuration.modeId = modeId.value
configuration.localeId = localeId.value
configuration.engineMode = engineMode.value
@@ -195,6 +294,9 @@ public extension SyncedAppSettingsV2 {
patch(&copy.providerId, value: configuration.providerId)
patch(&copy.baseURL, value: configuration.baseURL)
patch(&copy.model, value: configuration.model)
patch(&copy.asrProviderId, value: configuration.asrProviderId)
patch(&copy.asrBaseURL, value: configuration.asrBaseURL)
patch(&copy.asrModel, value: configuration.asrModel)
patch(&copy.modeId, value: configuration.modeId)
patch(&copy.localeId, value: configuration.localeId)
patch(&copy.engineMode, value: configuration.engineMode)
@@ -221,6 +323,9 @@ public extension SyncedAppSettingsV2 {
touch(&copy.providerId, value: configuration.providerId)
touch(&copy.baseURL, value: configuration.baseURL)
touch(&copy.model, value: configuration.model)
touch(&copy.asrProviderId, value: configuration.asrProviderId)
touch(&copy.asrBaseURL, value: configuration.asrBaseURL)
touch(&copy.asrModel, value: configuration.asrModel)
touch(&copy.modeId, value: configuration.modeId)
touch(&copy.localeId, value: configuration.localeId)
touch(&copy.engineMode, value: configuration.engineMode)
+28 -3
View File
@@ -16,11 +16,36 @@ public struct SyncedField<T: Codable & Equatable & Sendable>: Codable, Equatable
self.deviceID = deviceID
}
/// A remote timestamp may be at most this far in OUR future before we
/// stop trusting it. Wall-clock LWW breaks down when one device's clock
/// runs fast: its edits would win every merge forever, silently
/// discarding later edits from correct-clock devices. Anything beyond
/// this skew is a broken clock, not a newer edit.
public static var maxTrustedFutureSkew: TimeInterval { 6 * 60 * 60 }
/// Pick the field with the newer `updatedAt`; ties break lexicographically on `deviceID`.
///
/// Broken-clock containment: comparing with clamped stamps alone is not
/// enough a far-future stamp stored in the winner would keep beating
/// every later genuine edit (whose stamps are merely "now") until that
/// wall-clock date actually arrived. So when the winner carries an
/// untrusted future stamp, the stamp itself is REWRITTEN to "now" in the
/// merged result: from then on any real edit, made later, outranks it.
public static func merge(local: SyncedField<T>, remote: SyncedField<T>) -> SyncedField<T> {
if remote.updatedAt > local.updatedAt { return remote }
if local.updatedAt > remote.updatedAt { return local }
return remote.deviceID >= local.deviceID ? remote : local
let now = Date()
let horizon = now.addingTimeInterval(maxTrustedFutureSkew)
let remoteAt = remote.updatedAt > horizon ? now : remote.updatedAt
let localAt = local.updatedAt > horizon ? now : local.updatedAt
let winner: SyncedField<T>
if remoteAt > localAt {
winner = remote
} else if localAt > remoteAt {
winner = local
} else {
winner = remote.deviceID >= local.deviceID ? remote : local
}
guard winner.updatedAt > horizon else { return winner }
return SyncedField(value: winner.value, updatedAt: now, deviceID: winner.deviceID)
}
public static func make(value: T, deviceID: String) -> SyncedField<T> {
@@ -12,7 +12,16 @@ public struct SyncedSpeechHistory: Codable, Equatable, Sendable {
public static let legacyKVSKey = "speechHistory.v1"
public static let maxEntries = 300
/// Tombstones older than this window may be pruned during merge.
public static let tombstoneRetention: TimeInterval = 90 * 24 * 60 * 60
/// Tombstones guard against deleted entries "resurrecting" when a
/// long-offline device rejoins and re-merges them. A short wall-clock
/// retention re-opened that window after only 90 days; a year keeps the
/// window closed for any realistically dormant device while staying tiny
/// on the wire (a tombstone is ~60 bytes of JSON), and the count cap
/// bounds the worst case regardless of clock.
public static let tombstoneRetention: TimeInterval = 365 * 24 * 60 * 60
/// Hard cap independent of wall clock the oldest tombstones are
/// dropped first once exceeded.
public static let maxTombstones = 500
public var schemaVersion: Int
public var updatedAt: Date
@@ -107,15 +116,19 @@ public struct SyncedSpeechHistory: Codable, Equatable, Sendable {
clearedAt: Date?
) -> [UUID: Date] {
let cutoff = Date().addingTimeInterval(-tombstoneRetention)
return tombstones.filter { _, deletedAt in
if deletedAt < cutoff {
return false
}
if let clearedAt, deletedAt <= clearedAt {
return false
}
var kept = tombstones.filter { _, deletedAt in
if deletedAt < cutoff { return false }
if let clearedAt, deletedAt <= clearedAt { return false }
return true
}
// Enforce the count cap that makes the 365-day retention safe on the
// KVS byte budget: keep the NEWEST tombstones (dropping an old one
// early only re-opens the resurrection window for that one entry).
if kept.count > maxTombstones {
let newest = kept.sorted { $0.value > $1.value }.prefix(maxTombstones)
kept = Dictionary(uniqueKeysWithValues: newest.map { ($0.key, $0.value) })
}
return kept
}
private static func later(of lhs: Date?, and rhs: Date?) -> Date? {
@@ -45,6 +45,7 @@
"recommendedLocales": ["zh-CN", "en-US"],
"supportsHotwords": true,
"hotwordMode": "recognizerScoped",
"badgeKey": "mac.localASR.badge.balanced",
"installKind": "archive",
"installRelativePath": "models/sherpa-qwen3-0.6b-int8",
"archiveBaseName": "sherpa-onnx-qwen3-asr-0.6B-int8-2026-03-25",
@@ -71,6 +72,7 @@
"recommendedLocales": ["zh-CN", "en-US"],
"supportsHotwords": true,
"hotwordMode": "recognizerScoped",
"badgeKey": "mac.localASR.badge.quality",
"installKind": "repository",
"installRelativePath": "models/sherpa-qwen3-1.7b-int8",
"archiveBaseName": "sherpa-onnx-qwen3-asr-1.7B-int8",
@@ -113,30 +115,6 @@
}
]
},
{
"id": "sherpa-paraformer-zh-int8",
"displayName": "Paraformer Large",
"backend": "sherpaParaformer",
"runtimePlatform": "macos",
"sizeBytes": 220000000,
"recommendedLocales": ["zh-CN", "en-US"],
"supportsHotwords": false,
"hotwordMode": "none",
"installKind": "archive",
"installRelativePath": "models/sherpa-paraformer-zh-int8",
"archiveBaseName": "sherpa-onnx-paraformer-zh-int8-2025-10-07",
"layout": {
"paraformerModel": "model.int8.onnx",
"tokens": "tokens.txt"
},
"sources": [
{
"type": "github",
"priority": 1,
"url": "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-paraformer-zh-int8-2025-10-07.tar.bz2"
}
]
},
{
"id": "sherpa-sensevoice-small-int8",
"displayName": "SenseVoice Small",
@@ -146,6 +124,7 @@
"recommendedLocales": ["zh-CN", "en-US", "ja-JP", "ko-KR"],
"supportsHotwords": false,
"hotwordMode": "none",
"badgeKey": "mac.localASR.badge.fastest",
"installKind": "archive",
"installRelativePath": "models/sherpa-sensevoice-small-int8",
"archiveBaseName": "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17",
@@ -0,0 +1,25 @@
// ASRChunkTranscribing.swift
// OSGKeyboard · Shared
//
// Minimal ASR surface for pipelined utterance chunking. Keeps
// `ChunkedUtterancePipeline` independent of iOS-only `SpeechAnalyzer`.
import Foundation
public enum ASRChunkResult: Sendable, Equatable {
case success(String)
case failure(String)
case cancelled
}
/// One-shot chunk transcription used by `ChunkedUtterancePipeline`.
public protocol ASRChunkTranscribing: Sendable {
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult
func cancel()
func resetForNewUtterance()
}
extension ASRChunkTranscribing {
public func cancel() {}
public func resetForNewUtterance() {}
}
+1 -7
View File
@@ -30,7 +30,7 @@ extension AVAudioPCMBuffer: @unchecked @retroactive Sendable {}
// MARK: - Protocol
public protocol ASRService: Sendable {
public protocol ASRService: ASRChunkTranscribing, Sendable {
/// Start a transcription session. The returned stream emits `.partial`
/// updates and exactly one `.final` (or `.error`) before finishing.
/// `SpeechAnalyzer` is always fully on-device, so there is no
@@ -54,12 +54,6 @@ public protocol ASRService: Sendable {
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult
}
public enum ASRChunkResult: Sendable, Equatable {
case success(String)
case failure(String)
case cancelled
}
extension ASRService {
public func resetForNewUtterance() {}
@@ -53,6 +53,10 @@ public struct AppGroupStore: @unchecked Sendable {
public var baseURL: String { configuration.baseURL }
public var apiKey: String { configuration.apiKey }
public var model: String { configuration.model }
public var asrProviderId: String { configuration.asrProviderId }
public var asrBaseURL: String { configuration.resolvedASRBaseURL }
public var asrApiKey: String { configuration.asrApiKey }
public var asrModel: String { configuration.resolvedASRModel }
public var modeId: String { configuration.modeId }
public var localeId: String { configuration.localeId }
public var engineMode: String { configuration.engineMode }
@@ -91,6 +95,12 @@ public struct AppGroupStore: @unchecked Sendable {
config.baseURL = openAI.defaultBaseURL
config.model = openAI.defaultModel
}
if mode == "cloud", config.asrProviderId == "deepseek" {
let openAI = LLMProvider.provider(id: "openai")
config.asrProviderId = openAI.id
config.asrBaseURL = openAI.defaultBaseURL
config.asrModel = CloudASRModelCatalog.defaultModel(for: openAI.id)
}
}
AppGroupConfigDarwin.postConfigChanged()
}
@@ -67,13 +67,13 @@ private actor ChunkWorkQueue {
}
public actor ChunkedUtterancePipeline {
private let asr: ASRService
private let asr: any ASRChunkTranscribing
private let locale: Locale
private let config: FlowUtteranceChunkConfig
private var cancelled = false
public init(
asr: ASRService,
asr: any ASRChunkTranscribing,
locale: Locale,
config: FlowUtteranceChunkConfig = .flowDefault
) {
@@ -17,31 +17,35 @@ public protocol CloudASRTranscribing: Sendable {
public enum CloudASRClientFactory {
public static func make(store: any ConfigurationStore, session: URLSession = .shared) -> CloudASRTranscribing {
let strategy = CloudASRModelCatalog.strategy(for: store.providerId)
let providerId = store.asrProviderId
let strategy = CloudASRModelCatalog.strategy(for: providerId)
let asrModel = store.asrModel.isEmpty
? CloudASRModelCatalog.defaultModel(for: providerId)
: store.asrModel
switch strategy {
case .zhipuHotwords:
return ZhipuCloudASRClient(
apiKey: store.apiKey,
model: CloudASRModelCatalog.defaultModel(for: store.providerId),
apiKey: store.asrApiKey,
model: asrModel,
session: session
)
case .alibabaVocabulary:
return AlibabaFunASRClient(
apiKey: store.apiKey,
model: CloudASRModelCatalog.defaultModel(for: store.providerId),
apiKey: store.asrApiKey,
model: asrModel,
persistence: store.cloudASRPersistence,
session: session
)
case .prompt:
return PromptCloudASRClient(
providerId: store.providerId,
baseURL: store.baseURL,
apiKey: store.apiKey,
model: CloudASRModelCatalog.defaultModel(for: store.providerId),
providerId: providerId,
baseURL: store.asrBaseURL,
apiKey: store.asrApiKey,
model: asrModel,
session: session
)
case .localFallback:
return UnsupportedCloudASRClient(providerId: store.providerId)
return UnsupportedCloudASRClient(providerId: providerId)
}
}
}
@@ -133,7 +133,7 @@ public final class CloudASRService: ASRService, @unchecked Sendable {
}
private func bindClientIfNeeded() {
let providerId = store.providerId
let providerId = store.asrProviderId
let strategy = CloudASRModelCatalog.strategy(for: providerId)
lock.withLock {
guard boundProviderId != providerId else { return }
@@ -70,7 +70,9 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
/// Fire-and-forget preparation for the host app. Safe to call repeatedly.
/// Retries after exponential backoff when a prior attempt failed.
public func prepareInBackgroundIfNeeded() {
#if os(iOS)
guard AppGroup.isAvailable else { return }
#endif
let shouldStart = lock.withLock { () -> Bool in
if case .preparing = state { return false }
@@ -167,8 +169,8 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
throw PrepareError.missingPreparedArtifacts
}
AppGroup.defaultsIfAvailable?.set(fingerprint, forKey: Storage.fingerprintKey)
AppGroup.defaultsIfAvailable?.set(Date().timeIntervalSince1970, forKey: Storage.preparedAtKey)
Self.persistenceDefaults.set(fingerprint, forKey: Storage.fingerprintKey)
Self.persistenceDefaults.set(Date().timeIntervalSince1970, forKey: Storage.preparedAtKey)
Self.clearRetryState()
lock.withLock {
@@ -180,8 +182,9 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
return configuration
}
// MARK: - DictationTranscriber factory
// MARK: - DictationTranscriber factory (iOS host app)
#if os(iOS)
public static func makeDictationTranscriber(
locale: Locale,
lmConfiguration: SFSpeechLanguageModel.Configuration?
@@ -202,6 +205,34 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
attributeOptions: preset.attributeOptions
)
}
#endif
// MARK: - Legacy Speech request (macOS Apple Speech fallback)
/// Up to 100 short phrases for `SFSpeechRecognitionRequest.contextualStrings`.
public static func contextualStringsForRecognition(
bias: LocalASRBiasPayload?,
maxCount: Int = 100
) -> [String] {
guard let bias, !bias.hardHotwords.isEmpty else { return [] }
return Array(bias.hardHotwords.prefix(max(1, maxCount)))
}
/// Applies bundled CLM + optional contextual strings to a legacy on-device request.
public static func applyCustomLanguageModel(
to request: SFSpeechURLRecognitionRequest,
locale: Locale,
bias: LocalASRBiasPayload?
) {
request.requiresOnDeviceRecognition = true
if let configuration = shared.configurationForTranscription(locale: locale) {
request.customizedLanguageModel = configuration
}
let phrases = contextualStringsForRecognition(bias: bias)
if !phrases.isEmpty {
request.contextualStrings = phrases
}
}
// MARK: - Bundle / disk helpers
@@ -238,14 +269,28 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
}
static func preparedDirectoryURL() -> URL? {
guard let container = FileManager.default.containerURL(
if let container = FileManager.default.containerURL(
forSecurityApplicationGroupIdentifier: AppGroup.identifier
) else {
) {
let directory = container.appendingPathComponent(Storage.subdirectory, isDirectory: true)
try? FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
return directory
}
#if os(macOS)
guard let appSupport = FileManager.default.urls(
for: .applicationSupportDirectory,
in: .userDomainMask
).first else {
return nil
}
let directory = container.appendingPathComponent(Storage.subdirectory, isDirectory: true)
let directory = appSupport
.appendingPathComponent("OSGKeyboard", isDirectory: true)
.appendingPathComponent(Storage.subdirectory, isDirectory: true)
try? FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
return directory
#else
return nil
#endif
}
static func loadCachedConfigurationFromDisk() -> SFSpeechLanguageModel.Configuration? {
@@ -279,7 +324,7 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
}
private static func storedFingerprint() -> String? {
AppGroup.defaultsIfAvailable?.string(forKey: Storage.fingerprintKey)
persistenceDefaults.string(forKey: Storage.fingerprintKey)
}
private static func removeItemIfExists(at url: URL) throws {
@@ -310,26 +355,28 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
// MARK: - Retry / backoff
private static var persistenceDefaults: UserDefaults {
AppGroup.defaultsIfAvailable ?? .standard
}
private static func storedAttemptCount() -> Int {
AppGroup.defaultsIfAvailable?.integer(forKey: Storage.attemptCountKey) ?? 0
persistenceDefaults.integer(forKey: Storage.attemptCountKey)
}
private static func storedLastFailureAt() -> TimeInterval? {
let value = AppGroup.defaultsIfAvailable?.double(forKey: Storage.lastFailureAtKey) ?? 0
let value = persistenceDefaults.double(forKey: Storage.lastFailureAtKey)
return value > 0 ? value : nil
}
private static func recordFailure() {
guard let defaults = AppGroup.defaultsIfAvailable else { return }
let nextAttempt = storedAttemptCount() + 1
defaults.set(nextAttempt, forKey: Storage.attemptCountKey)
defaults.set(Date().timeIntervalSince1970, forKey: Storage.lastFailureAtKey)
persistenceDefaults.set(nextAttempt, forKey: Storage.attemptCountKey)
persistenceDefaults.set(Date().timeIntervalSince1970, forKey: Storage.lastFailureAtKey)
}
private static func clearRetryState() {
guard let defaults = AppGroup.defaultsIfAvailable else { return }
defaults.removeObject(forKey: Storage.attemptCountKey)
defaults.removeObject(forKey: Storage.lastFailureAtKey)
persistenceDefaults.removeObject(forKey: Storage.attemptCountKey)
persistenceDefaults.removeObject(forKey: Storage.lastFailureAtKey)
}
/// Returns false when retry budget is exhausted or backoff has not elapsed.
@@ -159,28 +159,78 @@ private final class FlowAudioProofStore: @unchecked Sendable {
/// incoming buffer's format actually changes, so downsampling to the ASR target
/// rate is always valid regardless of route churn.
private final class AdaptiveDownsampler: @unchecked Sendable {
// `AVAudioConverter` / `AVAudioFormat` are not `Sendable`, so the state and
// the returned converter are guarded manually via the unchecked lock APIs.
private let lock = OSAllocatedUnfairLock<(converter: AVAudioConverter, source: AVAudioFormat)?>(uncheckedState: nil)
// `AVAudioConverter` / `AVAudioFormat` / `AVAudioPCMBuffer` are not
// `Sendable`, so the state is guarded manually via the unchecked lock
// APIs. The scratch output buffer is REUSED across tap callbacks
// allocating on the realtime audio thread risks priority inversion, and
// taps on one bus are serialized, so a single scratch is safe as long as
// callers copy its contents out before returning (AudioBufferSnapshot
// does exactly that).
private struct State {
var converter: AVAudioConverter
var source: AVAudioFormat
var scratch: AVAudioPCMBuffer
}
private let lock = OSAllocatedUnfairLock<State?>(uncheckedState: nil)
let targetFormat: AVAudioFormat
/// Frame headroom for the reusable output buffer. Taps deliver 4096
/// input frames; output frames = input × (16k / hardwareRate), which
/// exceeds input only for sub-16 kHz hardware (rare telephony routes),
/// so 2× the tap size covers every realistic ratio.
private static let scratchCapacity: AVAudioFrameCount = 8_192
init(targetFormat: AVAudioFormat) {
self.targetFormat = targetFormat
}
/// Returns a converter valid for `sourceFormat`, rebuilding it lazily when
/// the hardware route (and thus the buffer format) changes.
func converter(for sourceFormat: AVAudioFormat) -> AVAudioConverter? {
lock.withLockUnchecked { state in
if let state, state.source == sourceFormat {
return state.converter
/// Downsamples `buffer` into the reusable scratch buffer and returns it,
/// rebuilding the converter lazily when the hardware route (and thus the
/// source format) changes. The returned buffer is only valid until the
/// next call copy its samples out synchronously.
func convertReusingScratch(_ buffer: AVAudioPCMBuffer) -> AVAudioPCMBuffer? {
let sourceFormat = buffer.format
guard sourceFormat.sampleRate > 0 else { return nil }
return lock.withLockUnchecked { state -> AVAudioPCMBuffer? in
if state == nil || state!.source != sourceFormat {
guard let converter = AVAudioConverter(from: sourceFormat, to: targetFormat),
let scratch = AVAudioPCMBuffer(
pcmFormat: targetFormat,
frameCapacity: Self.scratchCapacity
) else {
state = nil
return nil
}
state = State(converter: converter, source: sourceFormat, scratch: scratch)
}
guard let converter = AVAudioConverter(from: sourceFormat, to: targetFormat) else {
state = nil
return nil
guard let current = state else { return nil }
let wanted = AVAudioFrameCount(
Double(buffer.frameLength) * targetFormat.sampleRate / sourceFormat.sampleRate
)
guard wanted > 0, wanted <= current.scratch.frameCapacity else { return nil }
current.scratch.frameLength = 0
// ONE-SHOT input: the converter keeps pulling until the output
// buffer's frameCapacity is full, and the scratch is deliberately
// oversized feeding the same tap buffer on every pull would
// duplicate the audio ~6× (stuttering ASR input). After the
// single feed we report "ran dry", so the expected status is
// `.inputRanDry` (output not full), not `.haveData`.
var provided = false
var error: NSError?
let status = current.converter.convert(to: current.scratch, error: &error) { _, outStatus in
if provided {
outStatus.pointee = .noDataNow
return nil
}
provided = true
outStatus.pointee = .haveData
return buffer
}
state = (converter, sourceFormat)
return converter
guard status != .error, error == nil, current.scratch.frameLength > 0 else { return nil }
return current.scratch
}
}
}
@@ -240,6 +290,10 @@ public final class FlowContinuousCapture {
private var didInstallTap = false
private var isRunning = false
private var isRebuilding = false
private var interrupted = false
/// When the engine last (re)activated a freshly started engine has
/// produced no frames yet and must not be misclassified as a zombie.
private var lastActivationAt = Date.distantPast
private var routeObserver: NSObjectProtocol?
private var interruptionObserver: NSObjectProtocol?
@@ -250,6 +304,11 @@ public final class FlowContinuousCapture {
public var running: Bool { isRunning }
/// True between interruption `.began` and `.ended` (phone call, Siri).
/// While set, `setActive(true)` is guaranteed to fail owners should
/// wait for `.ended` (which rebuilds the engine) instead of retrying.
public var isInterrupted: Bool { interrupted }
/// True when the capture session flag, tap, and audio engine are all live.
public var engineIsLive: Bool {
isRunning && didInstallTap && audioEngine.isRunning
@@ -264,9 +323,33 @@ public final class FlowContinuousCapture {
/// Called on the main actor when `engineIsLive` may have changed.
public var onEngineLiveChanged: ((Bool) -> Void)?
/// Called on the main actor when the system interrupted capture (phone
/// call, Siri). The session owner should fail any mic-open utterance
/// audio frames stop arriving, so continuing to "record" only captures
/// a silence gap the user cannot see.
public var onInterruptionBegan: (() -> Void)?
/// Configure `.playAndRecord`, install a permanent input tap, start the engine.
///
/// Idempotent: "already running and healthy" is a warm-start fast path,
/// while "already running but producing no audio" is a zombie state
/// (force-quit relaunch, failed cold start, mediaserverd reset) that is
/// torn down and rebuilt in place. It must never be a silent no-op
/// a `guard !isRunning` early-return here turned every cold-start retry
/// into a guaranteed audio-proof timeout.
public func start() throws {
guard !isRunning else { return }
if isRunning {
let startedMomentsAgo = Date().timeIntervalSince(lastActivationAt) < 2
if engineIsLive && (engineHasRecentAudio(maxAge: 2) || startedMomentsAgo) {
// Healthy warm engine or one so fresh it simply hasn't
// produced its first frame yet (interleaved start attempts
// land here; rebuilding a 100 ms-old engine only multiplies
// audio-session churn in the fragile post-relaunch window).
return
}
log.info("start(): zombie engine detected (running but no live audio) — forcing rebuild")
stop()
}
audioProofStore.reset()
try activateEngine()
isRunning = true
@@ -353,6 +436,7 @@ public final class FlowContinuousCapture {
} catch {
throw StartError.engineStartFailed(error.localizedDescription)
}
lastActivationAt = Date()
}
/// Tear down the engine and release the audio session.
@@ -371,6 +455,7 @@ public final class FlowContinuousCapture {
audioEngine.stop()
}
isRunning = false
interrupted = false
audioProofStore.reset()
downsampler = nil
targetFormat = nil
@@ -384,6 +469,13 @@ public final class FlowContinuousCapture {
/// Re-activate capture after returning from background without
/// reinstalling the tap (iOS may deactivate the audio session).
///
/// Doubles as the interruption-recovery probe: `setActive(true)` FAILS
/// while a call/Siri interruption is live and succeeds once it ends, so a
/// successful reassert proves the interruption is over. iOS does not
/// guarantee delivery of `.ended` (commonly dropped when the app was
/// suspended during the call), so this is the only reliable way to clear
/// the `interrupted` latch in that case.
@discardableResult
public func reassertIfRunning() -> Bool {
guard isRunning else { return false }
@@ -395,6 +487,7 @@ public final class FlowContinuousCapture {
options: [.defaultToSpeaker, .allowBluetoothHFP, .mixWithOthers]
)
try session.setActive(true, options: .notifyOthersOnDeactivation)
interrupted = false
if !audioEngine.isRunning {
try audioEngine.start()
}
@@ -415,7 +508,14 @@ public final class FlowContinuousCapture {
if engineHasRecentAudio(maxAge: recentFrameMaxAge) {
return true
}
try? await Task.sleep(nanoseconds: 50_000_000)
do {
try await Task.sleep(nanoseconds: 50_000_000)
} catch {
// Cancelled bail out instead of busy-spinning the main
// actor for the rest of the window (a cancelled Task.sleep
// returns immediately, starving concurrent start attempts).
return false
}
}
return engineHasRecentAudio(maxAge: recentFrameMaxAge)
}
@@ -497,8 +597,11 @@ public final class FlowContinuousCapture {
switch type {
case .began:
log.info("Audio interruption began")
interrupted = true
notifyEngineLiveChanged()
onInterruptionBegan?()
case .ended:
interrupted = false
guard isRunning else { return }
let shouldResume: Bool
if let optionsRaw {
@@ -627,26 +730,12 @@ public final class FlowContinuousCapture {
audioProofStore.markFrameReceived()
levelStore.update(from: buffer, barCount: FlowCaptureConstants.levelBarCount)
// Derive the converter from the *live* buffer format so a mid-session
// route change (e.g. 48 kHz 24 kHz) is handled transparently.
let sourceFormat = buffer.format
let targetFormat = downsampler.targetFormat
guard sourceFormat.sampleRate > 0,
let converter = downsampler.converter(for: sourceFormat) else { return }
let outFrames = AVAudioFrameCount(
Double(buffer.frameLength) * targetFormat.sampleRate / sourceFormat.sampleRate
)
guard outFrames > 0,
let outBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: outFrames)
else { return }
var error: NSError?
let status = converter.convert(to: outBuffer, error: &error) { _, outStatus in
outStatus.pointee = .haveData
return buffer
}
guard status == .haveData, error == nil, outBuffer.frameLength > 0 else { return }
// The downsampler derives its converter from the *live* buffer
// format (mid-session route changes handled transparently) and
// returns a REUSED scratch buffer no per-callback allocation
// on the realtime thread. The snapshot below copies the samples
// out before the next tap callback can overwrite the scratch.
guard let outBuffer = downsampler.convertReusingScratch(buffer) else { return }
let snapshot = AudioBufferSnapshot(buffer: outBuffer)
guard !snapshot.samples.isEmpty else { return }
@@ -130,6 +130,12 @@ public struct FlowReadySnapshot: Codable, Equatable, Sendable {
public let localeId: String
public let busyUtteranceId: UUID?
public let sessionExpiresAt: TimeInterval?
/// Host process generation that wrote this snapshot. A snapshot whose
/// generation no longer matches `FlowSessionKeys.hostGeneration` was
/// written by a dead process and is void immediately no need to wait
/// out the heartbeat-zombie window. Optional for wire compatibility with
/// snapshots written before this field existed.
public let hostGeneration: String?
public init(
protocolVersion: Int = 1,
@@ -142,7 +148,8 @@ public struct FlowReadySnapshot: Codable, Equatable, Sendable {
engineMode: String,
localeId: String,
busyUtteranceId: UUID? = nil,
sessionExpiresAt: TimeInterval? = nil
sessionExpiresAt: TimeInterval? = nil,
hostGeneration: String? = nil
) {
self.protocolVersion = protocolVersion
self.sessionId = sessionId
@@ -155,6 +162,7 @@ public struct FlowReadySnapshot: Codable, Equatable, Sendable {
self.localeId = localeId
self.busyUtteranceId = busyUtteranceId
self.sessionExpiresAt = sessionExpiresAt
self.hostGeneration = hostGeneration
}
}
@@ -266,13 +274,27 @@ public enum FlowSessionBridge {
store.set(readyAt, forKey: FlowSessionKeys.flowHostReadyAt)
}
} else {
// Keep the not-ready payload. The keyboard needs `reason`
// (recording / processing / waitingForAudioProof / ) to tell
// "host is busy" apart from "host is still starting". Deleting
// the payload here forced every mid-utterance ready=false into
// a permanent orange `preparingSession` state.
clearHostReady(defaults: store, notify: false)
store.removeObject(forKey: FlowSessionKeys.flowReadyPayload)
}
if let expires = snapshot.sessionExpiresAt {
store.set(expires, forKey: FlowSessionKeys.flowSessionExpires)
}
store.set(snapshot.heartbeatAt, forKey: FlowSessionKeys.flowHeartbeat)
// Only a genuinely live host ready, or actively serving an
// utterance may refresh the heartbeat here. A host stuck in a
// failed cold start would otherwise keep "reviving" itself on every
// engine-state flap, flickering the keyboard between reachable and
// dead and postponing zombie-state cleanup indefinitely.
let provesHostAlive = snapshot.ready
|| snapshot.reason == .recording
|| snapshot.reason == .processing
if provesHostAlive {
store.set(snapshot.heartbeatAt, forKey: FlowSessionKeys.flowHeartbeat)
}
flush(store)
FlowSessionDarwin.postHostReadyChanged()
}
@@ -309,7 +331,8 @@ public enum FlowSessionBridge {
heartbeatAt: now,
engineMode: AppGroupConfiguration.load(fromAvailable: store).engineMode,
localeId: AppGroupConfiguration.load(fromAvailable: store).localeId,
sessionExpiresAt: expires
sessionExpiresAt: expires,
hostGeneration: store.string(forKey: FlowSessionKeys.hostGeneration)
)
if let data = encode(snapshot) {
store.set(data, forKey: FlowSessionKeys.flowReadyPayload)
@@ -418,6 +441,54 @@ public enum FlowSessionBridge {
return staleness <= FlowSessionKeys.heartbeatStaleInterval
}
// MARK: - Host process generation
/// Host app: rotate the per-process generation token. Call exactly once,
/// as early as possible in the host launch path. Returns the previous
/// generation (nil on first-ever launch) so the caller can log it.
///
/// Rationale: `applicationWillTerminate` is best-effort it never runs
/// when a *suspended* app is force-quit (the common case after a failed
/// cold start). Instead of anchoring cleanup on a termination callback
/// that may not fire, each launch proves the previous process is dead and
/// voids whatever session state it left behind.
@discardableResult
public static func rotateHostGeneration(defaults: UserDefaults? = nil) -> String? {
let store = resolvedDefaults(defaults)
let previous = store.string(forKey: FlowSessionKeys.hostGeneration)
store.set(UUID().uuidString, forKey: FlowSessionKeys.hostGeneration)
flush(store)
return previous
}
public static func currentHostGeneration(defaults: UserDefaults? = nil) -> String? {
let store = resolvedDefaults(defaults)
return store.string(forKey: FlowSessionKeys.hostGeneration)
}
/// Host launch reconciliation: clear every piece of persisted session
/// state a previous (dead) generation left behind. Unlike
/// `clearFlowState()` this keeps `pendingHostBundleId` on a keyboard
/// `startflow` cold launch the scene delegate stores the host bundle id
/// *before* the SwiftUI hierarchy (and thus the session manager) exists,
/// and wiping it here would break the return-to-host affordance.
public static func clearFlowStateOnHostLaunch(defaults: UserDefaults? = nil) {
let store = resolvedDefaults(defaults)
store.set(false, forKey: FlowSessionKeys.flowSessionActive)
store.removeObject(forKey: FlowSessionKeys.flowSessionExpires)
store.removeObject(forKey: FlowSessionKeys.flowHeartbeat)
store.removeObject(forKey: FlowSessionKeys.keyboardRecordingState)
store.removeObject(forKey: FlowSessionKeys.flowCommandPayload)
store.removeObject(forKey: FlowSessionKeys.flowResultPayload)
store.removeObject(forKey: FlowSessionKeys.flowAckPayload)
store.removeObject(forKey: FlowSessionKeys.flowReadyPayload)
clearTranscription(defaults: store)
store.removeObject(forKey: FlowSessionKeys.audioLevels)
store.removeObject(forKey: FlowSessionKeys.lastActivityAt)
clearHostReady(defaults: store, notify: false)
flush(store)
}
// MARK: - Host ready contract (host app keyboard)
/// Host app: publish whether Flow can accept a new utterance right now.
@@ -446,6 +517,13 @@ public enum FlowSessionBridge {
let store = resolvedDefaults(defaults)
if let snapshot = readySnapshot(defaults: store) {
guard snapshot.ready else { return false }
// Snapshot written by a dead host generation void immediately,
// without waiting out the heartbeat-zombie window.
if let snapshotGeneration = snapshot.hostGeneration,
let currentGeneration = store.string(forKey: FlowSessionKeys.hostGeneration),
snapshotGeneration != currentGeneration {
return false
}
guard isHostReachable(defaults: store) else { return false }
if let readyAt = snapshot.readyAt {
let skew = abs(snapshot.heartbeatAt - readyAt)
@@ -33,6 +33,11 @@ public enum FlowSessionKeys {
public static let pendingHostBundleId = "flow.pendingHostBundleId"
/// Wall-clock timestamp of the last utterance completion or session start.
public static let lastActivityAt = "flow.lastActivityAt"
/// One-shot token rotated by every host-process launch. State written by
/// a previous generation is void by definition a fresh launch proves the
/// previous process is dead, whether or not its `applicationWillTerminate`
/// cleanup ever ran (it does NOT run when a suspended app is force-quit).
public static let hostGeneration = "flow.hostGeneration.v1"
/// Heartbeat older than this host is not actively reachable for recording.
public static let heartbeatStaleInterval: TimeInterval = 3
@@ -59,14 +64,23 @@ public enum FlowSessionKeys {
public static let localASRWaitTimeout: TimeInterval = 120
public static let cloudASRWaitTimeout: TimeInterval = 120
/// Keyboard watchdog after the user stops recording (not utterance max length).
/// Must cover worst-case post-stop backlog: remaining SpeechAnalyzer chunks
/// plus cloud LLM polish (see `PolishingService.effectiveTimeout` cap).
/// Hard cap on a single LLM polish request. `PolishingService`'s scaled
/// per-request timeout clamps to this value, so it participates in the
/// keyboard-watchdog budget below.
public static let maxPolishTimeout: TimeInterval = 120
/// Extra slack for result serialization, cross-process propagation, and
/// the host's own polling cadence.
public static let resultDeliveryMargin: TimeInterval = 20
/// Keyboard watchdog after the user stops recording (not utterance max
/// length). Derived from the host-side budget so it always outlasts the
/// host's worst case (ASR drain wait + polish cap + margin) hand-tuned
/// constants drifted below the real host maximum, making the keyboard
/// report a timeout for transcriptions that were still going to succeed.
public static func keyboardResultTimeout(engineMode: String) -> TimeInterval {
if engineMode == "local" {
return 180
}
return 240
let asrWait = engineMode == "local" ? localASRWaitTimeout : cloudASRWaitTimeout
return asrWait + maxPolishTimeout + resultDeliveryMargin
}
public enum RecordingState: String, Sendable, Equatable {
@@ -39,20 +39,48 @@ public final class AppCloudSync {
?? SpeechHistoryCloudSync(kvs: kvs, makeStore: makeStore, historyDefaults: historyDefaults)
}
/// Serializes external-change pulls: KVS posts change notifications in
/// bursts (one per key at times), and overlapping pull-merge-apply runs
/// can interleave their read/write phases. `wantsAnotherPull` coalesces
/// every burst into at most one trailing re-pull.
private var isPulling = false
private var wantsAnotherPull = false
public func startObservingExternalChanges() {
guard externalChangeObserver == nil else { return }
externalChangeObserver = NotificationCenter.default.addObserver(
forName: NSUbiquitousKeyValueStore.didChangeExternallyNotification,
object: nil,
queue: .main
) { [weak self] _ in
) { [weak self] note in
guard let self else { return }
// Distinguish WHY the store changed. `.accountChange` means the
// user switched iCloud accounts the incoming values belong to a
// DIFFERENT account and must not be merged into this one's data
// (deleted-entry resurrection, foreign history, wrong settings).
let reason = note.userInfo?[NSUbiquitousKeyValueStoreChangeReasonKey] as? Int
if reason == NSUbiquitousKeyValueStoreAccountChange {
return
}
Task { @MainActor in
await self.pullAllIfEnabled()
await self.pullAllCoalesced()
}
}
}
private func pullAllCoalesced() async {
guard !isPulling else {
wantsAnotherPull = true
return
}
isPulling = true
defer { isPulling = false }
repeat {
wantsAnotherPull = false
await pullAllIfEnabled()
} while wantsAnotherPull
}
public func stopObservingExternalChanges() {
if let externalChangeObserver {
NotificationCenter.default.removeObserver(externalChangeObserver)
@@ -79,18 +107,27 @@ public final class AppCloudSync {
}
/// Low-risk manual sync: pull remote changes, merge, then push local state.
/// Each push runs independently one payload failing must not abort the
/// others (a too-large history would otherwise also kill the dictionary
/// push). The first error is rethrown after every push has been tried.
public func syncNow() async throws {
let store = makeStore()
await pullAllIfEnabled()
var firstError: Error?
func attempt(_ body: () async throws -> Void) async {
do { try await body() } catch { if firstError == nil { firstError = error } }
}
if store.settingsICloudSyncEnabled {
try await settingsSync.pushLocalIfEnabled()
try await usageStatisticsSync.pushLocalIfEnabled()
try await speechHistorySync.pushLocalIfEnabled()
await attempt { try await settingsSync.pushLocalIfEnabled() }
await attempt { try await usageStatisticsSync.pushLocalIfEnabled() }
await attempt { try await speechHistorySync.pushLocalIfEnabled() }
}
if store.personalDictionaryICloudSyncEnabled {
try await dictionarySync.pushLocalIfEnabled(store.personalDictionary)
await attempt { try await dictionarySync.pushLocalIfEnabled(store.personalDictionary) }
}
if let firstError { throw firstError }
}
public var settingsSyncService: SettingsCloudSync { settingsSync }
@@ -24,8 +24,12 @@ public final class SpeechHistoryCloudSync {
public static let kvsKey = SyncedSpeechHistory.kvsKey
public static let legacyKVSKey = SyncedSpeechHistory.legacyKVSKey
/// Stay below the ~1 MB per-key KVS limit.
public static let maxPayloadBytes = 900_000
/// The 1 MB iCloud KVS quota is for the WHOLE store, not per key.
/// History and the personal dictionary must fit together (plus settings
/// and usage stats) once the store exceeds 1 MB, KVS rejects writes
/// for ALL keys with `QuotaViolation` and every sync silently stops.
/// Budget: ~400 KB history + ~400 KB dictionary + headroom for the rest.
public static let maxPayloadBytes = 400_000
private let kvs: UbiquitousKeyValueStoreing
private let makeStore: () -> AppGroupStore
@@ -50,8 +54,16 @@ public final class SpeechHistoryCloudSync {
public func pushLocalIfEnabled() async throws {
let store = makeStore()
guard store.settingsICloudSyncEnabled else { return }
let local = SpeechHistoryStorage.load(from: historyDefaults())
try push(local)
// Read-merge-write: pushing the local view verbatim would overwrite
// entries another device added since our last pull (KVS is
// last-writer-wins with no server-side merge).
let defaults = historyDefaults()
let local = SpeechHistoryStorage.load(from: defaults)
let merged = loadRemote().map { SyncedSpeechHistory.merge(local: local, remote: $0) } ?? local
if merged != local {
apply(merged, to: defaults, postNotification: true)
}
try push(merged)
}
/// Called when settings sync is first enabled to union local + remote history.
@@ -81,11 +93,34 @@ public final class SpeechHistoryCloudSync {
}
public func push(_ history: SyncedSpeechHistory) throws {
let data = try encode(history)
let data = try encodeFittingBudget(history)
kvs.set(data, forKey: Self.kvsKey)
_ = kvs.synchronize()
}
/// Encode, dropping the oldest entries until the payload fits the KVS
/// budget. Without this, a history that once fit under the old 900 KB
/// cap (300 long dictations easily exceed 400 KB) would make EVERY push
/// throw forever automatic pushes are fire-and-forget, so sync would
/// just silently die with no way back short of clearing all history.
/// Only the *uploaded* copy is trimmed; local history keeps its full
/// 300 entries.
func encodeFittingBudget(_ history: SyncedSpeechHistory) throws -> Data {
var payload = history
while true {
do {
return try encode(payload)
} catch SpeechHistoryCloudSyncError.payloadTooLarge {
guard payload.entries.count > 1 else { throw SpeechHistoryCloudSyncError.payloadTooLarge(byteCount: 0) }
// Drop the oldest ~10% per pass; entries are kept
// newest-first by the store, so trim from the tail.
let sorted = payload.entries.sorted { $0.createdAt > $1.createdAt }
let keep = max(1, sorted.count - max(1, sorted.count / 10))
payload.entries = Array(sorted.prefix(keep))
}
}
}
public func loadRemote() -> SyncedSpeechHistory? {
if let data = kvs.data(forKey: Self.kvsKey) {
return try? decode(data)
@@ -45,8 +45,16 @@ public final class UsageStatisticsCloudSync {
public func pushLocalIfEnabled() async throws {
let store = makeStore()
guard store.settingsICloudSyncEnabled else { return }
// Read-merge-write: this fires after every utterance, so pushing the
// local view verbatim would clobber counter slices another device
// advanced since our last pull (KVS is last-writer-wins). The
// G-Counter merge makes the push commutative instead.
let local = SyncedUsageStatisticsStorage.load(from: store.defaults)
try push(local)
let merged = loadRemote().map { SyncedUsageStatisticsV2.merge(local: local, remote: $0) } ?? local
if merged != local {
apply(merged, to: store.defaults, postNotification: true)
}
try push(merged)
}
/// Called when settings sync is first enabled to union local + remote totals.
+45 -2
View File
@@ -91,8 +91,11 @@ public final class KeyboardState: ObservableObject {
@Published public var micDisabled: Bool = false
/// One-line helper shown above the mic while `micDisabled == true`.
@Published public var micDisabledHint: String = ""
/// "local" on-device ASR only. "cloud" ASR + LLM polish.
@Published public var engineMode: String = "cloud"
/// "local" on-device ASR only. "cloud" cloud ASR + LLM polish.
/// Boot value must match the privacy-safe app default (`local`) so the
/// keyboard never assumes the audio-uploading engine before the App
/// Group config has been read.
@Published public var engineMode: String = "local"
/// v0.2.1 follow-up: derived translation is on iff a target
/// locale has been selected (mirrors `ProviderConfig.translationEnabled`
/// so the chip / pipeline read the same source of truth).
@@ -148,6 +151,46 @@ public final class KeyboardState: ObservableObject {
case openSettings
}
// MARK: - Temporary Flow debug (remove after orange-mic investigation)
/// Mirrored from `KeyboardFlowCoordinator` for the on-screen debug panel.
@Published public var debugPendingFlowStart: Bool = false
@Published public var debugFlowRecording: Bool = false
@Published public var debugAwaitingFlowResult: Bool = false
@Published public var debugHasFullAccess: Bool = false
/// Snapshot for the keyboard debug panel.
public func makeFlowDebugRows(hasFullAccess: Bool) -> [FlowDebugRow] {
debugHasFullAccess = hasFullAccess
let micLabel: String = {
switch micVoiceAvailability {
case .ready: return "ready"
case .recording: return "recording"
case .processing: return "processing"
case .unavailable(let reason):
switch reason {
case .hostNotReady: return "unavailable(hostNotReady)"
case .preparingSession: return "unavailable(preparingSession)"
case .noFullAccess: return "unavailable(noFullAccess)"
case .appGroupUnavailable: return "unavailable(appGroupUnavailable)"
case .missingAPIKey: return "unavailable(missingAPIKey)"
}
}
}()
let localRows: [FlowDebugRow] = [
FlowDebugRow("mic", micLabel),
FlowDebugRow("phase", String(describing: phase)),
FlowDebugRow("pendingStart", debugPendingFlowStart ? "1" : "0"),
FlowDebugRow("kb.recording", debugFlowRecording ? "1" : "0"),
FlowDebugRow("kb.awaiting", debugAwaitingFlowResult ? "1" : "0"),
FlowDebugRow("fullAccess", hasFullAccess ? "1" : "0"),
FlowDebugRow("micDisabled", micDisabled ? "1" : "0"),
FlowDebugRow("flowSessionPub", flowSessionActive ? "1" : "0"),
FlowDebugRow("engine", engineMode)
]
return localRows + FlowDebugAppGroupSnapshot.rows()
}
// Action hooks injected by the view controller at install time.
public var beginRecording: () -> Void = {}
public var endRecording: () -> Void = {}
+192 -8
View File
@@ -22,12 +22,29 @@ public enum Keychain: @unchecked Sendable {
private static let legacyAccount = "current"
private static let defaultProviderId = "openai"
private static func account(for providerId: String) -> String {
private static func normalizedProviderId(_ providerId: String) -> String {
let trimmed = providerId.trimmingCharacters(in: .whitespacesAndNewlines)
let normalized = trimmed.isEmpty ? defaultProviderId : trimmed.lowercased()
return "provider.\(normalized)"
return trimmed.isEmpty ? defaultProviderId : trimmed.lowercased()
}
/// LLM polish credentials (`provider.<id>`).
private static func account(for providerId: String) -> String {
"provider.\(normalizedProviderId(providerId))"
}
/// Cloud ASR credentials (`asr.<id>`), independent from polish keys.
private static func asrAccount(for providerId: String) -> String {
"asr.\(normalizedProviderId(providerId))"
}
// NOTE on kSecAttrAccessGroup: we deliberately rely on the DEFAULT
// access group (the first entry in each target's keychain-access-groups,
// which project.yml pins to `$(AppIdentifierPrefix)com.osgkeyboard.shared`
// for every target). Setting the attribute explicitly would require the
// team-prefixed string at runtime, which is not portably available
// without injecting TeamID through the build system. If a SECOND access
// group is ever added to any target, revisit this reordered groups
// would silently change which store these queries hit.
private static func baseQuery(providerId: String, synchronizable: Bool) -> [String: Any] {
var query: [String: Any] = [
kSecClass as String: kSecClassGenericPassword,
@@ -43,6 +60,130 @@ public enum Keychain: @unchecked Sendable {
// MARK: - Read
// MARK: - ASR keys
public static func asrApiKey(for providerId: String, preferICloudSync: Bool = false) -> String? {
if preferICloudSync, let synced = readASRKey(providerId: providerId, synchronizable: true) {
return synced
}
if let local = readASRKey(providerId: providerId, synchronizable: false) {
return local
}
if preferICloudSync {
return readASRKey(providerId: providerId, synchronizable: true)
}
return nil
}
public static func asrApiKeyOutcome(
for providerId: String,
preferICloudSync: Bool = false
) -> ReadOutcome {
let first = readASRKeyOutcome(providerId: providerId, synchronizable: preferICloudSync)
if case .found = first { return first }
let second = readASRKeyOutcome(providerId: providerId, synchronizable: !preferICloudSync)
if case .found = second { return second }
if case .unavailable = first { return first }
if case .unavailable = second { return second }
return .notFound
}
public static func setASRAPIKey(_ key: String, for providerId: String, useICloudSync: Bool = false) throws {
if key.isEmpty {
try deleteASRAPIKey(for: providerId, useICloudSync: useICloudSync)
return
}
if useICloudSync {
try writeASRKey(key, providerId: providerId, synchronizable: true)
try? deleteASRKey(providerId: providerId, synchronizable: false)
} else {
try writeASRKey(key, providerId: providerId, synchronizable: false)
}
}
public static func deleteASRAPIKey(for providerId: String, useICloudSync: Bool = false) throws {
try deleteASRKey(providerId: providerId, synchronizable: false)
if useICloudSync {
try deleteASRKey(providerId: providerId, synchronizable: true)
}
}
private static func readASRKey(providerId: String, synchronizable: Bool) -> String? {
if case .found(let value) = readASRKeyOutcome(providerId: providerId, synchronizable: synchronizable) {
return value
}
return nil
}
private static func readASRKeyOutcome(providerId: String, synchronizable: Bool) -> ReadOutcome {
var query = baseASRQuery(providerId: providerId, synchronizable: synchronizable)
query[kSecReturnData as String] = true
query[kSecMatchLimit as String] = kSecMatchLimitOne
var result: CFTypeRef?
let status = SecItemCopyMatching(query as CFDictionary, &result)
switch status {
case errSecSuccess:
guard let data = result as? Data,
let str = String(data: data, encoding: .utf8) else {
return .notFound
}
return .found(str)
case errSecItemNotFound:
// Pre-split installs stored one key under `provider.<id>` for both stages.
return readKeyOutcome(providerId: providerId, synchronizable: synchronizable)
default:
#if DEBUG
print("⚠️ [OSGKeyboard] ASR Keychain read returned OSStatus \(status); reporting unavailable.")
#endif
return .unavailable(status)
}
}
private static func baseASRQuery(providerId: String, synchronizable: Bool) -> [String: Any] {
var query: [String: Any] = [
kSecClass as String: kSecClassGenericPassword,
kSecAttrService as String: service,
kSecAttrAccount as String: asrAccount(for: providerId),
kSecAttrSynchronizable as String: synchronizable ? kCFBooleanTrue! : kCFBooleanFalse!,
]
#if os(macOS)
query[kSecUseDataProtectionKeychain as String] = true
#endif
return query
}
private static func writeASRKey(_ key: String, providerId: String, synchronizable: Bool) throws {
let data = Data(key.utf8)
var baseQuery = baseASRQuery(providerId: providerId, synchronizable: synchronizable)
let updateAttrs: [String: Any] = [kSecValueData as String: data]
let updateStatus = SecItemUpdate(baseQuery as CFDictionary, updateAttrs as CFDictionary)
switch updateStatus {
case errSecSuccess:
return
case errSecItemNotFound:
baseQuery[kSecValueData as String] = data
baseQuery[kSecAttrAccessible as String] = synchronizable
? kSecAttrAccessibleAfterFirstUnlock
: kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly
let addStatus = SecItemAdd(baseQuery as CFDictionary, nil)
if addStatus != errSecSuccess {
throw KeychainError.unexpectedStatus(addStatus)
}
default:
throw KeychainError.unexpectedStatus(updateStatus)
}
}
private static func deleteASRKey(providerId: String, synchronizable: Bool) throws {
let query = baseASRQuery(providerId: providerId, synchronizable: synchronizable)
let status = SecItemDelete(query as CFDictionary)
if status != errSecSuccess, status != errSecItemNotFound {
throw KeychainError.unexpectedStatus(status)
}
}
// MARK: - LLM keys
public static func apiKey(for providerId: String, preferICloudSync: Bool = false) -> String? {
if preferICloudSync, let synced = readKey(providerId: providerId, synchronizable: true) {
return synced
@@ -60,7 +201,43 @@ public enum Keychain: @unchecked Sendable {
apiKey(for: defaultProviderId)
}
/// Distinguishes "no key stored" from "keychain temporarily unreadable".
public enum ReadOutcome: Equatable {
case found(String)
case notFound
/// The keychain could not be read (e.g. `errSecInteractionNotAllowed`
/// while the device is locked before first unlock). NOT the same as
/// "no key configured" telling the user to re-enter their key in
/// this state would be wrong; the read succeeds once unlocked.
case unavailable(OSStatus)
}
/// Like `apiKey(for:)`, but reports WHY a key was not returned so
/// callers can distinguish a missing key (user action needed) from a
/// transiently locked keychain (retry later).
public static func apiKeyOutcome(
for providerId: String,
preferICloudSync: Bool = false
) -> ReadOutcome {
let first = readKeyOutcome(providerId: providerId, synchronizable: preferICloudSync)
if case .found = first { return first }
let second = readKeyOutcome(providerId: providerId, synchronizable: !preferICloudSync)
if case .found = second { return second }
// Neither store had it: surface "unavailable" when either read was
// blocked, since the key may well exist behind the lock.
if case .unavailable = first { return first }
if case .unavailable = second { return second }
return .notFound
}
private static func readKey(providerId: String, synchronizable: Bool) -> String? {
if case .found(let value) = readKeyOutcome(providerId: providerId, synchronizable: synchronizable) {
return value
}
return nil
}
private static func readKeyOutcome(providerId: String, synchronizable: Bool) -> ReadOutcome {
var query = baseQuery(providerId: providerId, synchronizable: synchronizable)
query[kSecReturnData as String] = true
query[kSecMatchLimit as String] = kSecMatchLimitOne
@@ -70,16 +247,16 @@ public enum Keychain: @unchecked Sendable {
case errSecSuccess:
guard let data = result as? Data,
let str = String(data: data, encoding: .utf8) else {
return nil
return .notFound
}
return str
return .found(str)
case errSecItemNotFound:
return nil
return .notFound
default:
#if DEBUG
print("⚠️ [OSGKeyboard] Keychain read returned OSStatus \(status); treating as no key.")
print("⚠️ [OSGKeyboard] Keychain read returned OSStatus \(status); reporting unavailable.")
#endif
return nil
return .unavailable(status)
}
}
@@ -193,6 +370,13 @@ public enum Keychain: @unchecked Sendable {
try? writeKey(local, providerId: provider.id, synchronizable: true)
try? deleteKey(providerId: provider.id, synchronizable: false)
}
for provider in LLMProvider.asrSelectablePresets {
guard let local = readASRKey(providerId: provider.id, synchronizable: false), !local.isEmpty else {
continue
}
try? writeASRKey(local, providerId: provider.id, synchronizable: true)
try? deleteASRKey(providerId: provider.id, synchronizable: false)
}
}
// MARK: - Onboarding completion (reboot-durable flag)
@@ -26,8 +26,12 @@ public final class PersonalDictionaryCloudSync {
public static let kvsKey = PersonalDictionary.kvsKeyV2
public static let legacyKVSKey = PersonalDictionary.legacyKVSKey
/// Stay below the ~1 MB per-key KVS limit.
public static let maxPayloadBytes = 900_000
/// The 1 MB iCloud KVS quota covers the WHOLE store, not one key
/// this payload shares it with speech history, settings, and usage
/// stats. Exceeding the total quota makes KVS reject writes for ALL
/// keys (`QuotaViolation`), silently stopping every sync.
/// Budget: ~400 KB dictionary + ~400 KB history + headroom.
public static let maxPayloadBytes = 400_000
private let kvs: UbiquitousKeyValueStoreing
private let makeStore: () -> AppGroupStore
@@ -75,7 +79,14 @@ public final class PersonalDictionaryCloudSync {
public func pushLocalIfEnabled(_ dictionary: PersonalDictionary) async throws {
let store = makeStore()
guard store.personalDictionaryICloudSyncEnabled else { return }
try push(dictionary)
// Read-merge-write: KVS is last-writer-wins; uploading the local
// view verbatim would drop entries another device added since our
// last pull. Tombstones in `merge` keep deletions intact.
let merged = loadRemote().map { PersonalDictionary.merge(local: dictionary, remote: $0) } ?? dictionary
if merged != dictionary {
store.setPersonalDictionary(merged)
}
try push(merged)
}
/// Enable sync: merge local + remote, persist locally, then upload.
@@ -10,8 +10,8 @@
// English dictation while halving the network round-trip.
//
// Engine matrix:
// - `engineMode == "cloud"` provider cloud ASR + user's cloud LLM
// - `engineMode == "local"` on-device ASR + built-in DeepSeek
// - `engineMode == "cloud"` user's cloud ASR + user's cloud LLM (independent)
// - `engineMode == "local"` on-device ASR + user's LLM (or built-in DeepSeek)
// - Ultra-short, structure-free utterances skip the LLM entirely
// - Cloud without API key raw + `.missingAPIKey` warning
// - Local without build key raw + `.missingAPIKey` warning
@@ -37,6 +37,10 @@ public actor PolishingService {
/// Local engine DeepSeek step: `PreconfiguredKeys.deepseek` is
/// still the repo placeholder, or cloud engine Keychain is empty.
case missingAPIKey
/// The keychain was unreadable (device locked before first unlock)
/// the key likely EXISTS; treat as transient, never as "please
/// re-enter your API key".
case keychainLocked
}
/// v0.2.1: what the LLM should do with the raw transcript. The
@@ -95,8 +99,13 @@ public actor PolishingService {
return TranscriptPostProcessor.localClean(trimmed)
}
if store.engineMode == "cloud", injectedClient == nil {
guard !store.apiKey.isEmpty else {
if injectedClient == nil {
let providerId = Self.resolvedProviderId(store: store, providerIdOverride: providerIdOverride)
let hasPolishKey = Self.hasPolishAPIKey(store: store, providerId: providerId)
guard hasPolishKey else {
if case .unavailable = Keychain.apiKeyOutcome(for: providerId, preferICloudSync: true) {
throw PolishError.keychainLocked
}
throw PolishError.missingAPIKey
}
}
@@ -150,10 +159,14 @@ public actor PolishingService {
)
let apiKey: String
if effectiveProviderId == "deepseek" {
guard PreconfiguredKeys.isDeepseekConfigured else {
let userKey = store.apiKey.trimmingCharacters(in: .whitespacesAndNewlines)
if !userKey.isEmpty {
apiKey = userKey
} else if PreconfiguredKeys.isDeepseekConfigured {
apiKey = PreconfiguredKeys.deepseek
} else {
throw PolishError.missingAPIKey
}
apiKey = PreconfiguredKeys.deepseek
} else {
apiKey = store.apiKey
}
@@ -356,7 +369,11 @@ public actor PolishingService {
/// (unpolished, unsegmented) ASR text.
internal func effectiveTimeout(for text: String) -> TimeInterval {
let scaled = timeout + (Double(text.count) / 100.0) * 10.0
return min(max(scaled, timeout), 120)
// The cap participates in the keyboard-watchdog budget see
// `FlowSessionKeys.keyboardResultTimeout`. Raising it here without
// going through that constant would silently break the invariant
// "keyboard timeout > host worst case".
return min(max(scaled, timeout), FlowSessionKeys.maxPolishTimeout)
}
internal static func resolvedProviderId(
@@ -366,11 +383,25 @@ public actor PolishingService {
if let providerIdOverride {
return providerIdOverride
}
if store.engineMode == "local" {
let id = store.providerId
// Local installs without a user LLM key keep using the built-in DeepSeek path.
if store.engineMode == "local",
id != "deepseek",
store.apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty,
PreconfiguredKeys.isDeepseekConfigured {
return "deepseek"
}
let id = store.providerId
return id == "deepseek" ? "openai" : id
return id == "deepseek" && store.engineMode == "cloud" ? "openai" : id
}
internal static func hasPolishAPIKey(store: any ConfigurationStore, providerId: String) -> Bool {
if !store.apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return true
}
if providerId == "deepseek", PreconfiguredKeys.isDeepseekConfigured {
return true
}
return false
}
internal static func resolveLLMEndpoint(
@@ -396,6 +427,8 @@ extension PolishingService.PolishError: LocalizedError {
return "LLM polish timed out."
case .missingAPIKey:
return "Missing API key (cloud: Settings API key; local: build configuration)."
case .keychainLocked:
return "API key unavailable while the device is locked — will work after unlock."
}
}
}
@@ -34,6 +34,7 @@ public final class SpeechHistoryStore: ObservableObject {
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return }
rebaseOnPersistedStateBeforeMutation()
let entry = SpeechHistoryEntry(text: trimmed, engineMode: engineMode)
payload.entries.insert(entry, at: 0)
payload.trimEntries()
@@ -42,6 +43,7 @@ public final class SpeechHistoryStore: ObservableObject {
}
public func delete(id: UUID) {
rebaseOnPersistedStateBeforeMutation()
guard payload.entries.contains(where: { $0.id == id }) else { return }
payload.deletedEntryIDs[id] = Date()
payload.entries.removeAll { $0.id == id }
@@ -51,12 +53,24 @@ public final class SpeechHistoryStore: ObservableObject {
}
public func clearAll() {
rebaseOnPersistedStateBeforeMutation()
payload.recordClearAll()
payload.updatedAt = Date()
payload.pruneTombstonesIfNeeded()
applyPayload(postCloudPush: true)
}
/// Cloud pulls write the merged history to disk but only *schedule* the
/// in-memory reload (the notification observer hops through a Task).
/// Mutating a stale snapshot and saving it wholesale would erase whatever
/// that merge just brought in always rebase on the persisted state
/// before mutating.
private func rebaseOnPersistedStateBeforeMutation() {
let disk = SpeechHistoryStorage.load(from: defaults)
guard disk != payload else { return }
payload = SyncedSpeechHistory.merge(local: payload, remote: disk)
}
public func snapshot() -> SyncedSpeechHistory {
payload
}
@@ -0,0 +1,47 @@
// TranscriptionPolishFallback.swift
// OSGKeyboard · Shared
//
// Shared polish-failure handling: conservative raw ASR cleanup plus
// bilingual user-visible warnings (iOS Flow + macOS dictation).
import Foundation
public enum TranscriptionPolishFallback: Sendable {
public static func makeDelivery(
rawText: String,
error: Error,
engineMode: String,
chunkWarning: String?
) -> TranscriptionDelivery {
let fallbackText = TranscriptPostProcessor.cleanRawASRFallback(rawText)
let warning = warning(for: error, engineMode: engineMode)
?? degradedWarning()
?? chunkWarning
return TranscriptionDelivery(text: fallbackText, polishWarning: warning)
}
public static func warning(for error: Error, engineMode: String) -> String? {
if let polishError = error as? PolishingService.PolishError {
switch polishError {
case .missingAPIKey:
if engineMode == "local" {
return SharedL10n.string("flow.warning.localPolishUnavailable")
}
return SharedL10n.string("flow.warning.cloudPolishMissingKey")
case .timeout, .keychainLocked:
return degradedWarning()
case .noTranscript:
return nil
}
}
if error is LLMError {
return degradedWarning()
}
return nil
}
public static func degradedWarning() -> String? {
SharedL10n.string("flow.warning.polishDegraded")
}
}
@@ -60,6 +60,27 @@ public enum DictationTextComposer {
return isCJK(last) && isCJK(first)
}
/// Separator to place between existing document text and an inserted
/// transcript. Inserting at a cursor that sits right after "Hello" must
/// produce "Hello world", not "Helloworld" but CJK, whitespace, and
/// opening-punctuation boundaries take no space.
public static func insertionSeparator(previousContext: String?, insertion: String) -> String {
guard let previousContext,
let last = previousContext.unicodeScalars.last,
let first = insertion.unicodeScalars.first else {
return ""
}
if CharacterSet.whitespacesAndNewlines.contains(last) { return "" }
if isCJK(last) || isCJK(first) { return "" }
// No space after opening brackets/quotes ("(", "[", "", """).
if CharacterSet(charactersIn: "([{\u{201C}\u{2018}\u{300C}\u{300E}\u{3010}\u{FF08}").contains(last) {
return ""
}
// No space before closing/clause punctuation (".", ",", ")", "!").
if CharacterSet.punctuationCharacters.contains(first) { return "" }
return " "
}
static func normalizeForOverlap(_ text: String) -> String {
text.unicodeScalars.filter {
!CharacterSet.whitespacesAndNewlines.contains($0)
@@ -0,0 +1,174 @@
// FlowDebugPanel.swift
// OSGKeyboard · Shared
//
// TEMPORARY debug overlay for cross-process Flow state. Remove after the
// orange-mic investigation. Shows the same App Group contract fields on both
// the host app and the keyboard extension so we can see where they diverge.
import SwiftUI
/// One labeled row in the temporary Flow debug panel.
public struct FlowDebugRow: Equatable, Sendable {
public let label: String
public let value: String
public init(_ label: String, _ value: String) {
self.label = label
self.value = value
}
}
/// Builds the App Group half of the debug snapshot (readable from both processes).
public enum FlowDebugAppGroupSnapshot {
public static func rows(defaults: UserDefaults? = nil) -> [FlowDebugRow] {
FlowSessionBridge.reloadFromDisk(defaults: defaults)
let snapshot = FlowSessionBridge.readySnapshot(defaults: defaults)
let staleness = FlowSessionBridge.heartbeatStaleness(defaults: defaults)
let generation = FlowSessionBridge.currentHostGeneration(defaults: defaults)
let shortGen: String = {
guard let generation, generation.count >= 8 else { return generation ?? "nil" }
return String(generation.prefix(8))
}()
let snapGen: String = {
guard let g = snapshot?.hostGeneration, g.count >= 8 else {
return snapshot?.hostGeneration ?? "nil"
}
return String(g.prefix(8))
}()
let expires: String = {
guard let ts = FlowSessionBridge.sessionExpiresAt(defaults: defaults) else { return "nil" }
let remaining = ts - Date().timeIntervalSince1970
return String(format: "%.0fs", remaining)
}()
return [
FlowDebugRow("sessionActive", FlowSessionBridge.isSessionActive(defaults: defaults) ? "1" : "0"),
FlowDebugRow("expiresIn", expires),
FlowDebugRow("hostReachable", FlowSessionBridge.isHostReachable(defaults: defaults) ? "1" : "0"),
FlowDebugRow("hostReady", FlowSessionBridge.isHostReady(defaults: defaults) ? "1" : "0"),
FlowDebugRow("hostStale", FlowSessionBridge.isHostStale(defaults: defaults) ? "1" : "0"),
FlowDebugRow("hbStale", staleness.map { String(format: "%.1fs", $0) } ?? "nil"),
FlowDebugRow("snap.ready", snapshot.map { $0.ready ? "1" : "0" } ?? "nil"),
FlowDebugRow("snap.reason", snapshot?.reason.rawValue ?? "nil"),
FlowDebugRow("snap.session", shortUUID(snapshot?.sessionId)),
FlowDebugRow("gen.now", shortGen),
FlowDebugRow("gen.snap", snapGen),
FlowDebugRow("gen.match", {
guard let a = snapshot?.hostGeneration,
let b = generation else { return "n/a" }
return a == b ? "1" : "0"
}()),
FlowDebugRow("pendingHost", FlowSessionBridge.pendingHostBundleId(defaults: defaults) ?? "nil"),
FlowDebugRow("recState", FlowSessionBridge.recordingState(defaults: defaults).rawValue),
FlowDebugRow("appGroup", AppGroup.isAvailable ? "1" : "0")
]
}
private static func shortUUID(_ id: UUID?) -> String {
guard let id else { return "nil" }
return String(id.uuidString.prefix(8))
}
}
/// Collapsible monospaced status panel. Temporary for investigation only.
public struct FlowDebugPanel: View {
public let title: String
public let rows: [FlowDebugRow]
@Binding public var isExpanded: Bool
public var maxContentHeight: CGFloat
public init(
title: String,
rows: [FlowDebugRow],
isExpanded: Binding<Bool>,
maxContentHeight: CGFloat = 180
) {
self.title = title
self.rows = rows
self._isExpanded = isExpanded
self.maxContentHeight = maxContentHeight
}
public var body: some View {
VStack(alignment: .leading, spacing: 4) {
Button {
isExpanded.toggle()
} label: {
HStack(spacing: 6) {
Text(isExpanded ? "" : "")
.font(.system(size: 10, weight: .bold, design: .monospaced))
Text(title)
.font(.system(size: 11, weight: .semibold, design: .monospaced))
Spacer(minLength: 0)
Text(summaryChip)
.font(.system(size: 10, weight: .bold, design: .monospaced))
.foregroundStyle(summaryColor)
}
.foregroundStyle(Color.primary)
.contentShape(Rectangle())
}
.buttonStyle(.plain)
if isExpanded {
ScrollView {
LazyVStack(alignment: .leading, spacing: 2) {
ForEach(Array(rows.enumerated()), id: \.offset) { _, row in
HStack(alignment: .top, spacing: 6) {
Text(row.label)
.font(.system(size: 10, weight: .medium, design: .monospaced))
.foregroundStyle(Color.secondary)
.frame(width: 92, alignment: .leading)
Text(row.value)
.font(.system(size: 10, weight: .regular, design: .monospaced))
.foregroundStyle(Color.primary)
.textSelection(.enabled)
.frame(maxWidth: .infinity, alignment: .leading)
}
}
}
}
.frame(maxHeight: maxContentHeight)
}
}
.padding(8)
.background(
RoundedRectangle(cornerRadius: 8, style: .continuous)
.fill(.ultraThinMaterial)
)
.overlay(
RoundedRectangle(cornerRadius: 8, style: .continuous)
.stroke(Color.orange.opacity(0.7), lineWidth: 1)
)
}
private var summaryChip: String {
let hostReady = rows.first(where: { $0.label == "hostReady" })?.value
?? rows.first(where: { $0.label == "bridgeReady" })?.value
?? "?"
let mic = rows.first(where: { $0.label == "mic" })?.value
if let mic {
return "mic=\(shortMic(mic)) hr=\(hostReady)"
}
let active = rows.first(where: { $0.label == "isActive" })?.value ?? "?"
return "active=\(active) hr=\(hostReady)"
}
private var summaryColor: Color {
let hostReady = rows.first(where: { $0.label == "hostReady" })?.value
?? rows.first(where: { $0.label == "bridgeReady" })?.value
if hostReady == "1" { return .green }
return .orange
}
private func shortMic(_ value: String) -> String {
if value.hasPrefix("ready") { return "ready" }
if value.contains("preparing") { return "prep" }
if value.contains("hostNotReady") { return "notReady" }
if value.contains("recording") { return "rec" }
if value.contains("processing") { return "proc" }
if value.contains("noFullAccess") { return "noFA" }
if value.contains("appGroup") { return "noAG" }
if value.contains("missingAPIKey") { return "noKey" }
return String(value.prefix(12))
}
}
+36 -5
View File
@@ -101,18 +101,25 @@
"keyboard.translation.a11yHint" = "Toggle translation or change the target language.";
/* macOS app */
"mac.section.dashboard" = "Dashboard";
"mac.section.dashboard" = "Home";
"mac.section.history" = "History";
"mac.section.dictionary" = "Personal Dictionary";
"mac.section.dictionary" = "Dictionary";
"mac.section.settings" = "Settings";
"mac.brand.subtitle" = "AI DICTATION";
"mac.brand.tagline" = "Speak it. Its typed.";
"mac.brand.tagline.subtitle" = "Local-first · Cross-device · One-tap dictation";
"mac.devices" = "Devices";
"mac.status.ready" = "Ready to dictate…";
"mac.page.history.subtitle" = "Every dictation, kept in order.";
"mac.page.dictionary.subtitle" = "Words that teach recognition your voice.";
"mac.page.settings.subtitle" = "Engine, shortcuts, and appearance.";
"mac.status.ready" = "Ready when you are…";
"mac.status.listening" = "Listening…";
"mac.status.transcribing" = "Transcribing…";
"mac.status.polishing" = "Polishing…";
"mac.status.copied" = "Copied to clipboard";
"mac.status.pasted" = "Inserted into front app";
"mac.status.copiedAndPasted" = "Copied and inserted";
"mac.status.deliveryWithNote" = "%@ — %@";
"mac.stat.dictationTime" = "Dictation Time";
"mac.stat.words" = "Dictation Chars";
"mac.stat.translation" = "Translation Chars";
@@ -123,6 +130,12 @@
"mac.stat.customTerms" = "Custom terms";
"mac.status.chipReady" = "Ready";
"mac.status.chipProcessing" = "Processing";
"mac.overlay.listening" = "Listening";
"mac.overlay.preparing" = "Starting…";
"mac.overlay.transcribing" = "Transcribing";
"mac.overlay.polishing" = "Polishing";
"mac.overlay.live" = "Live";
"mac.overlay.done" = "Done";
"mac.record.start" = "Record";
"mac.record.stop" = "Stop";
"mac.record.pressStop" = "Press Stop";
@@ -133,8 +146,8 @@
"mac.mode.local" = "Local Mode";
"mac.connected" = "Connected";
"mac.offline" = "Offline";
"mac.history.recent" = "Recent";
"mac.history.empty" = "No voice transcripts yet.";
"mac.history.empty" = "No dictations yet";
"mac.history.emptyBody" = "Hold Option anywhere to speak — transcripts land here.";
"mac.history.select" = "Select a dictation";
"mac.history.clearTitle" = "Clear all history?";
"mac.history.clearMessage" = "This cannot be undone.";
@@ -151,6 +164,12 @@
"mac.dict.deleteMessage" = "This cannot be undone.";
"mac.hint.holdOption" = "Hold Option to dictate";
"mac.settings.cloudProvider" = "CLOUD PROVIDER";
"mac.settings.polishProvider" = "TEXT POLISH (LLM)";
"mac.settings.asrProvider" = "SPEECH RECOGNITION (ASR)";
"mac.settings.asrService" = "ASR service";
"mac.settings.asrApiKey" = "ASR API key";
"mac.settings.asrModel" = "ASR model";
"mac.settings.baseURL" = "Base URL";
"mac.settings.service" = "Service";
"mac.settings.apiKey" = "API Key";
"mac.settings.model" = "Model";
@@ -174,6 +193,15 @@
"mac.settings.autoPasteDesc" = "Simulate ⌘V in the front app after transcription (requires Accessibility).";
"mac.settings.hotkey" = "Global shortcut";
"mac.settings.hotkeyDesc" = "Hold Option (⌥) to dictate from any app.";
"mac.settings.hotkeyTrigger" = "Shortcut key";
"mac.settings.hotkeyTriggerDesc" = "Which Option (⌥) key starts dictation when held.";
"mac.hotkeyTrigger.rightOption" = "Right Option (⌥)";
"mac.hotkeyTrigger.leftOption" = "Left Option (⌥)";
"mac.hotkeyTrigger.eitherOption" = "Either Option key";
"mac.hint.hold.rightOption" = "Hold right Option (⌥) to dictate";
"mac.hint.hold.leftOption" = "Hold left Option (⌥) to dictate";
"mac.hint.hold.eitherOption" = "Hold either Option (⌥) to dictate";
"mac.hint.holdOption" = "Hold Option to dictate";
"mac.settings.qwen3Model" = "Qwen3 model folder";
"mac.settings.qwen3ModelDesc" = "Folder with config.json, model.safetensors, vocab.json, and merges.txt.";
"mac.settings.qwen3Browse" = "Choose folder…";
@@ -241,6 +269,9 @@
"mac.localASR.installed" = "Installed";
"mac.localASR.notInstalled" = "Not installed";
"mac.localASR.personalDictionaryTag" = "Personal dictionary";
"mac.localASR.badge.fastest" = "Fastest";
"mac.localASR.badge.balanced" = "Most balanced";
"mac.localASR.badge.quality" = "Best quality";
"mac.localASR.hotwordsYes" = "Hotwords";
"mac.localASR.hotwordsNo" = "No hotwords";
"mac.localASR.catalogMissing" = "Local ASR catalog is missing from the app bundle.";
+37 -6
View File
@@ -101,18 +101,25 @@
"keyboard.translation.a11yHint" = "切换翻译或更改目标语言。";
/* macOS 应用 */
"mac.section.dashboard" = "仪表盘";
"mac.section.dashboard" = "首页";
"mac.section.history" = "历史";
"mac.section.dictionary" = "个性词库";
"mac.section.dictionary" = "词库";
"mac.section.settings" = "设置";
"mac.brand.subtitle" = "AI 听写";
"mac.brand.tagline" = "开口即文字。";
"mac.brand.tagline.subtitle" = "本地优先 · 跨端同步 · 一键听写";
"mac.devices" = "设备";
"mac.status.ready" = "准备听写…";
"mac.status.listening" = "录音中…";
"mac.page.history.subtitle" = "每一次听写,按时间妥善保存。";
"mac.page.dictionary.subtitle" = "让识别更懂你的用词。";
"mac.page.settings.subtitle" = "引擎、快捷键与外观。";
"mac.status.ready" = "准备好了,随时开口…";
"mac.status.listening" = "正在聆听…";
"mac.status.transcribing" = "识别中…";
"mac.status.polishing" = "润色中…";
"mac.status.copied" = "已复制到剪贴板";
"mac.status.pasted" = "已插入前台应用";
"mac.status.copiedAndPasted" = "已复制并插入";
"mac.status.deliveryWithNote" = "%@ — %@";
"mac.stat.dictationTime" = "听写时长";
"mac.stat.words" = "听写字数";
"mac.stat.translation" = "翻译字数";
@@ -123,6 +130,12 @@
"mac.stat.customTerms" = "自定义词条";
"mac.status.chipReady" = "就绪";
"mac.status.chipProcessing" = "处理中";
"mac.overlay.listening" = "聆听中";
"mac.overlay.preparing" = "启动中…";
"mac.overlay.transcribing" = "识别中";
"mac.overlay.polishing" = "润色中";
"mac.overlay.live" = "实时";
"mac.overlay.done" = "已完成";
"mac.record.start" = "开始录音";
"mac.record.stop" = "停止";
"mac.record.pressStop" = "点击停止";
@@ -133,8 +146,8 @@
"mac.mode.local" = "本地模式";
"mac.connected" = "已连接";
"mac.offline" = "离线";
"mac.history.recent" = "最近";
"mac.history.empty" = "还没有语音识别记录。";
"mac.history.empty" = "还没有听写记录";
"mac.history.emptyBody" = "在任意应用按住 Option 开口说话,记录会出现在这里。";
"mac.history.select" = "选择一条记录";
"mac.history.clearTitle" = "清空全部历史?";
"mac.history.clearMessage" = "此操作无法撤销。";
@@ -151,6 +164,12 @@
"mac.dict.deleteMessage" = "此操作无法撤销。";
"mac.hint.holdOption" = "长按 Option 开始听写";
"mac.settings.cloudProvider" = "云端服务商";
"mac.settings.polishProvider" = "文本润色(LLM";
"mac.settings.asrProvider" = "语音转写(ASR";
"mac.settings.asrService" = "转写服务";
"mac.settings.asrApiKey" = "转写 API 密钥";
"mac.settings.asrModel" = "转写模型";
"mac.settings.baseURL" = "接口地址";
"mac.settings.service" = "服务商";
"mac.settings.apiKey" = "API 密钥";
"mac.settings.model" = "模型";
@@ -174,6 +193,15 @@
"mac.settings.autoPasteDesc" = "转写完成后向前台应用模拟 ⌘V(需辅助功能权限)。";
"mac.settings.hotkey" = "全局快捷键";
"mac.settings.hotkeyDesc" = "按住 Option (⌥) 键即可从任意应用开始听写。";
"mac.settings.hotkeyTrigger" = "快捷键按键";
"mac.settings.hotkeyTriggerDesc" = "按住哪个 Option (⌥) 键开始听写。";
"mac.hotkeyTrigger.rightOption" = "右 Option (⌥)";
"mac.hotkeyTrigger.leftOption" = "左 Option (⌥)";
"mac.hotkeyTrigger.eitherOption" = "任一 Option 键";
"mac.hint.hold.rightOption" = "长按右 Option(⌥)开始听写";
"mac.hint.hold.leftOption" = "长按左 Option(⌥)开始听写";
"mac.hint.hold.eitherOption" = "长按任一 Option(⌥)开始听写";
"mac.hint.holdOption" = "长按 Option 开始听写";
"mac.settings.qwen3Model" = "Qwen3 模型目录";
"mac.settings.qwen3ModelDesc" = "需包含 config.json、model.safetensors、vocab.json 与 merges.txt。";
"mac.settings.qwen3Browse" = "选择文件夹…";
@@ -241,6 +269,9 @@
"mac.localASR.installed" = "已安装";
"mac.localASR.notInstalled" = "未安装";
"mac.localASR.personalDictionaryTag" = "个性词库";
"mac.localASR.badge.fastest" = "速度最快";
"mac.localASR.badge.balanced" = "最平衡";
"mac.localASR.badge.quality" = "质量最好";
"mac.localASR.hotwordsYes" = "支持热词";
"mac.localASR.hotwordsNo" = "无热词";
"mac.localASR.catalogMissing" = "应用包内缺少本地 ASR 模型目录。";
@@ -20,7 +20,8 @@ final class AppGroupConfigurationTests: XCTestCase {
XCTAssertEqual(config.providerId, "openai")
XCTAssertEqual(config.modeId, "polish")
XCTAssertEqual(config.localeId, "auto")
XCTAssertEqual(config.engineMode, "cloud")
// Privacy-critical: the default engine must keep audio on-device.
XCTAssertEqual(config.engineMode, "local")
XCTAssertFalse(config.hasCompletedOnboarding)
XCTAssertEqual(config.onboardingPage, 0)
XCTAssertFalse(config.hasAcknowledgedCloudSharing)
@@ -31,7 +32,7 @@ final class AppGroupConfigurationTests: XCTestCase {
XCTAssertEqual(config.polishIntensity, .default)
XCTAssertTrue(config.personalDictionary.entries.isEmpty)
XCTAssertTrue(config.flowSkipAppSwitch)
XCTAssertEqual(config.flowInactivityDuration, .twelveHours)
XCTAssertEqual(config.flowInactivityDuration, .thirtyMinutes)
}
func testSaveAndLoadRoundTrip() {
@@ -40,9 +41,13 @@ final class AppGroupConfigurationTests: XCTestCase {
config.providerId = "anthropic"
config.baseURL = "https://example.com/v1"
config.model = "claude-test"
config.asrProviderId = "zhipu"
config.asrBaseURL = "https://open.bigmodel.cn/api/paas/v4"
config.asrModel = "glm-asr-2512"
config.modeId = "polish"
config.localeId = "zh-Hans"
config.engineMode = "local"
// Non-default value so the round-trip proves persistence.
config.engineMode = "cloud"
config.hasCompletedOnboarding = true
config.onboardingPage = 2
config.hasAcknowledgedCloudSharing = true
@@ -52,15 +57,19 @@ final class AppGroupConfigurationTests: XCTestCase {
config.cursorDragNavigationEnabled = false
config.polishIntensity = .light
config.flowSkipAppSwitch = false
config.flowInactivityDuration = .thirtyMinutes
// Use a non-default value so the round-trip actually proves persistence.
config.flowInactivityDuration = .threeHours
config.save(to: defaults)
let loaded = AppGroupConfiguration.load(fromAvailable: defaults)
XCTAssertEqual(loaded.providerId, "anthropic")
XCTAssertEqual(loaded.baseURL, "https://example.com/v1")
XCTAssertEqual(loaded.model, "claude-test")
XCTAssertEqual(loaded.asrProviderId, "zhipu")
XCTAssertEqual(loaded.asrBaseURL, "https://open.bigmodel.cn/api/paas/v4")
XCTAssertEqual(loaded.asrModel, "glm-asr-2512")
XCTAssertEqual(loaded.localeId, "zh-Hans")
XCTAssertEqual(loaded.engineMode, "local")
XCTAssertEqual(loaded.engineMode, "cloud")
XCTAssertTrue(loaded.hasCompletedOnboarding)
XCTAssertEqual(loaded.onboardingPage, 2)
XCTAssertTrue(loaded.hasAcknowledgedCloudSharing)
@@ -71,7 +80,37 @@ final class AppGroupConfigurationTests: XCTestCase {
XCTAssertFalse(loaded.cursorDragNavigationEnabled)
XCTAssertEqual(loaded.polishIntensity, .light)
XCTAssertFalse(loaded.flowSkipAppSwitch)
XCTAssertEqual(loaded.flowInactivityDuration, .thirtyMinutes)
XCTAssertEqual(loaded.flowInactivityDuration, .threeHours)
}
/// Existing installs (onboarding completed, no explicit engineMode key)
/// ran on the old "cloud"/12h defaults a silent flip to the new
/// privacy defaults would change their engine under them AND propagate
/// through settings sync as a fake fresh edit to their other devices.
func testDefaultMigrationPreservesExistingInstallBehavior() {
let defaults = makeDefaults()
defaults.set(true, forKey: AppGroupConfiguration.Keys.hasCompletedOnboarding)
let config = AppGroupConfiguration.load(fromAvailable: defaults)
XCTAssertEqual(config.engineMode, "cloud", "pre-picker installs stay on their old default")
XCTAssertEqual(config.flowInactivityDuration, .twelveHours)
// The resolution is persisted so it is stable and sync-invisible.
XCTAssertEqual(defaults.string(forKey: AppGroupConfiguration.Keys.engineMode), "cloud")
XCTAssertEqual(
defaults.string(forKey: AppGroupConfiguration.Keys.flowInactivityDuration),
FlowInactivityDuration.twelveHours.rawValue
)
}
func testDefaultMigrationGivesFreshInstallPrivacyDefaults() {
let defaults = makeDefaults()
let config = AppGroupConfiguration.load(fromAvailable: defaults)
XCTAssertEqual(config.engineMode, "local")
XCTAssertEqual(config.flowInactivityDuration, .thirtyMinutes)
XCTAssertEqual(defaults.string(forKey: AppGroupConfiguration.Keys.engineMode), "local")
}
func testTranslationEnabledDerivedFromTargetLocale() {
@@ -41,4 +41,30 @@ final class ConfigurationStoreTests: XCTestCase {
let service = ASRServiceFactory.make(store: store as any ConfigurationStore)
XCTAssertTrue(service is SpeechAnalyzerASR)
}
func testASRAndPolishProvidersAreIndependent() throws {
try Keychain.setAPIKey("sk-llm", for: "openai", useICloudSync: false)
try Keychain.setASRAPIKey("sk-asr", for: "zhipu", useICloudSync: false)
var config = AppGroupConfiguration.load(fromAvailable: defaults)
config.engineMode = "cloud"
config.providerId = "openai"
config.asrProviderId = "zhipu"
config.save(to: defaults)
let loaded = AppGroupStore(defaults: defaults)
XCTAssertEqual(loaded.providerId, "openai")
XCTAssertEqual(loaded.asrProviderId, "zhipu")
XCTAssertEqual(loaded.apiKey, "sk-llm")
XCTAssertEqual(loaded.asrApiKey, "sk-asr")
let asrClient = CloudASRClientFactory.make(store: loaded)
XCTAssertTrue(asrClient is ZhipuCloudASRClient)
}
func testLegacyInstallCopiesProviderIdToAsrProviderId() {
defaults.set("qwen", forKey: AppGroupConfiguration.Keys.providerId)
let config = AppGroupConfiguration.load(fromAvailable: defaults)
XCTAssertEqual(config.asrProviderId, "qwen")
}
}
@@ -0,0 +1,212 @@
// FlowBudgetAndMergeTests.swift
// OSGKeyboardTests
//
// Guards the cross-cutting invariants introduced by the reliability
// overhaul: timeout budgets derived from a single source, LWW clock
// clamping, and mutation-rebase for the speech history store.
import XCTest
@testable import OSGKeyboardShared
final class FlowBudgetAndMergeTests: XCTestCase {
// MARK: - Timeout budget invariant
/// The keyboard's post-stop watchdog must outlast the host's worst case
/// (ASR drain wait + LLM polish cap) with real margin otherwise the
/// keyboard reports a timeout for transcriptions that are still going
/// to succeed, and hand-tuned constants have drifted below the host
/// maximum before.
func testKeyboardResultTimeoutOutlastsHostWorstCase() {
for engineMode in ["local", "cloud"] {
let hostWorstCase = (engineMode == "local"
? FlowSessionKeys.localASRWaitTimeout
: FlowSessionKeys.cloudASRWaitTimeout)
+ FlowSessionKeys.maxPolishTimeout
let keyboardTimeout = FlowSessionKeys.keyboardResultTimeout(engineMode: engineMode)
XCTAssertGreaterThanOrEqual(
keyboardTimeout,
hostWorstCase + 10,
"keyboard watchdog (\(engineMode)) must exceed host worst case with margin"
)
}
}
// MARK: - SyncedField future-clock clamping
func testMergePrefersGenuinelyNewerRemote() {
let older = SyncedField(value: "a", updatedAt: Date(timeIntervalSinceNow: -100), deviceID: "A")
let newer = SyncedField(value: "b", updatedAt: Date(timeIntervalSinceNow: -10), deviceID: "B")
XCTAssertEqual(SyncedField.merge(local: older, remote: newer).value, "b")
XCTAssertEqual(SyncedField.merge(local: newer, remote: older).value, "b")
}
/// A device with a clock years in the future must not win every merge
/// forever: its timestamp is clamped to "now" for comparison, so an
/// edit carrying a trusted (within-skew) later stamp still beats it
/// with unclamped LWW the year-ahead stamp would win against everything
/// until that wall-clock date actually arrived.
func testMergeClampsAbsurdFutureRemoteTimestamp() {
let farFuture = Date().addingTimeInterval(365 * 24 * 3600)
let brokenClock = SyncedField(value: "broken", updatedAt: farFuture, deviceID: "B")
// Sane edit one minute ahead of now: inside the trusted skew window,
// so it is NOT clamped while the broken stamp collapses to ~now.
let local = SyncedField(value: "sane", updatedAt: Date().addingTimeInterval(60), deviceID: "A")
XCTAssertEqual(
SyncedField.merge(local: local, remote: brokenClock).value,
"sane",
"a year-ahead stamp must lose to a trusted, genuinely newer edit"
)
XCTAssertEqual(
SyncedField.merge(local: brokenClock, remote: local).value,
"sane",
"clamping must be symmetric regardless of which side is remote"
)
}
/// The winner's untrusted future stamp must be REWRITTEN to now in the
/// merged result otherwise the stored far-future stamp keeps beating
/// every later genuine edit until that wall-clock date arrives.
func testMergeFlattensUntrustedWinnerStamp() {
let farFuture = Date().addingTimeInterval(365 * 24 * 3600)
let broken = SyncedField(value: "broken", updatedAt: farFuture, deviceID: "B")
let old = SyncedField(value: "old", updatedAt: Date(timeIntervalSinceNow: -9999), deviceID: "A")
let merged = SyncedField.merge(local: old, remote: broken)
XCTAssertEqual(merged.value, "broken", "newer (clamped) edit still wins this merge")
XCTAssertLessThan(
merged.updatedAt.timeIntervalSinceNow, 60,
"the far-future stamp must be flattened so later real edits can outrank it"
)
}
func testMergeTrustsModestFutureSkew() {
// Small forward skew (minutes) is normal clock drift and stays trusted.
let slightlyAhead = SyncedField(value: "ahead", updatedAt: Date().addingTimeInterval(120), deviceID: "A")
let past = SyncedField(value: "past", updatedAt: Date(timeIntervalSinceNow: -3600), deviceID: "B")
XCTAssertEqual(SyncedField.merge(local: past, remote: slightlyAhead).value, "ahead")
}
// MARK: - History push byte budget
/// A history that outgrew the KVS byte budget must be trimmed (oldest
/// entries first) for upload, not fail forever automatic pushes are
/// fire-and-forget, so a throwing encode would silently kill sync with
/// no recovery path short of clearing all history.
@MainActor
func testOversizedHistoryPushTrimsOldestEntriesToFitBudget() throws {
let sync = SpeechHistoryCloudSync(
kvs: FakeUbiquitousKeyValueStore(),
makeStore: { AppGroupStore(defaults: self.makeDefaults()) },
historyDefaults: { self.makeDefaults() }
)
// ~300 entries × ~2.4 KB 720 KB encoded over the 400 KB budget.
let filler = String(repeating: "很长的听写内容 long dictation text ", count: 80)
let now = Date()
var history = SyncedSpeechHistory.empty
history.entries = (0..<300).map { index in
SpeechHistoryEntry(
text: "\(filler)#\(index)",
createdAt: now.addingTimeInterval(TimeInterval(-index)),
engineMode: "local"
)
}
let data = try sync.encodeFittingBudget(history)
XCTAssertLessThanOrEqual(data.count, SpeechHistoryCloudSync.maxPayloadBytes)
let decoded = try sync.decode(data)
XCTAssertFalse(decoded.entries.isEmpty)
// Newest entries must survive the trim.
XCTAssertTrue(decoded.entries.contains { $0.text.hasSuffix("#0") })
XCTAssertFalse(decoded.entries.contains { $0.text.hasSuffix("#299") })
}
// MARK: - Insertion word-boundary hygiene
func testInsertionSeparatorAddsSpaceBetweenLatinWords() {
XCTAssertEqual(
DictationTextComposer.insertionSeparator(previousContext: "Hello", insertion: "world"),
" "
)
XCTAssertEqual(
DictationTextComposer.insertionSeparator(previousContext: "version 2", insertion: "is out"),
" "
)
}
func testInsertionSeparatorSkipsWhitespaceCJKAndPunctuationBoundaries() {
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "Hello ", insertion: "world"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "line\n", insertion: "next"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "你好", insertion: "世界"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "说英文", insertion: "now"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "see (", insertion: "note"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "wait", insertion: ", then go"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: nil, insertion: "fresh"), "")
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "", insertion: "fresh"), "")
}
// MARK: - SpeechHistoryStore rebase-before-mutation
private func makeDefaults() -> UserDefaults {
let suite = "group.com.osgkeyboard.shared.tests.history.\(UUID().uuidString)"
let defaults = UserDefaults(suiteName: suite)!
defaults.removePersistentDomain(forName: suite)
return defaults
}
/// Cloud pulls write merged history to disk and only *schedule* the
/// in-memory reload. A mutation racing that reload must not wipe what
/// the merge brought in.
@MainActor
func testAppendDoesNotEraseEntriesMergedToDiskBehindItsBack() {
let defaults = makeDefaults()
let store = SpeechHistoryStore(defaults: defaults)
store.append(text: "本地第一条", engineMode: "local")
XCTAssertEqual(store.entries.count, 1)
// Simulate a cloud merge landing on disk without the store's
// in-memory payload being reloaded yet.
var onDisk = SpeechHistoryStorage.load(from: defaults)
let remoteEntry = SpeechHistoryEntry(text: "远端合并进来的一条", engineMode: "cloud")
onDisk.entries.append(remoteEntry)
onDisk.updatedAt = Date()
SpeechHistoryStorage.save(onDisk, to: defaults)
// Mutate through the store pre-fix this overwrote the disk state
// with the stale in-memory payload, deleting the remote entry.
store.append(text: "本地第二条", engineMode: "local")
let persisted = SpeechHistoryStorage.load(from: defaults)
XCTAssertTrue(
persisted.entries.contains { $0.id == remoteEntry.id },
"append must rebase on the persisted state instead of clobbering the cloud merge"
)
XCTAssertTrue(persisted.entries.contains { $0.text == "本地第二条" })
XCTAssertTrue(persisted.entries.contains { $0.text == "本地第一条" })
}
@MainActor
func testDeleteAfterExternalDiskMergeStillTombstones() {
let defaults = makeDefaults()
let store = SpeechHistoryStore(defaults: defaults)
store.append(text: "要删除的一条", engineMode: "local")
guard let target = store.entries.first else {
return XCTFail("expected an entry")
}
// External merge adds an unrelated entry on disk.
var onDisk = SpeechHistoryStorage.load(from: defaults)
onDisk.entries.append(SpeechHistoryEntry(text: "外部条目", engineMode: "cloud"))
onDisk.updatedAt = Date()
SpeechHistoryStorage.save(onDisk, to: defaults)
store.delete(id: target.id)
let persisted = SpeechHistoryStorage.load(from: defaults)
XCTAssertFalse(persisted.entries.contains { $0.id == target.id })
XCTAssertNotNil(persisted.deletedEntryIDs[target.id], "delete must record a tombstone")
XCTAssertTrue(persisted.entries.contains { $0.text == "外部条目" }, "external entry must survive")
}
}
@@ -238,6 +238,142 @@ final class FlowSessionBridgeTests: XCTestCase {
XCTAssertEqual(FlowSessionBridge.latestAck(defaults: defaults), ack)
}
func testNotReadySnapshotDoesNotRefreshHeartbeat() {
let defaults = makeDefaults()
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
let zombieHeartbeat = Date().timeIntervalSince1970 - 120
defaults.set(zombieHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
// A host stuck in a failed cold start writes not-ready snapshots on
// every engine flap; those must NOT revive the heartbeat, or zombie
// detection is postponed forever.
FlowSessionBridge.writeReadySnapshot(
FlowReadySnapshot(
sessionId: UUID(),
ready: false,
reason: .waitingForAudioProof,
engineMode: "local",
localeId: "zh-Hans"
),
defaults: defaults
)
XCTAssertTrue(FlowSessionBridge.isHostStale(defaults: defaults))
}
func testBusySnapshotStillRefreshesHeartbeat() {
let defaults = makeDefaults()
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
let staleHeartbeat = Date().timeIntervalSince1970 - 10
defaults.set(staleHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
// Recording/processing proves the host is alive even though the
// snapshot is not "ready" the heartbeat must keep flowing so the
// keyboard does not declare a mid-utterance host dead.
let sessionId = UUID()
FlowSessionBridge.writeReadySnapshot(
FlowReadySnapshot(
sessionId: sessionId,
ready: false,
reason: .recording,
engineMode: "local",
localeId: "zh-Hans",
busyUtteranceId: UUID()
),
defaults: defaults
)
XCTAssertTrue(FlowSessionBridge.isHostReachable(defaults: defaults))
// Not-ready busy snapshots must remain readable so the keyboard can
// distinguish "host is recording" from "host is still starting".
let snap = FlowSessionBridge.readySnapshot(defaults: defaults)
XCTAssertEqual(snap?.reason, .recording)
XCTAssertEqual(snap?.ready, false)
XCTAssertEqual(snap?.sessionId, sessionId)
}
func testNotReadyStartingSnapshotIsRetainedWithoutRevivingHeartbeat() {
let defaults = makeDefaults()
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
let zombieHeartbeat = Date().timeIntervalSince1970 - 120
defaults.set(zombieHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
FlowSessionBridge.writeReadySnapshot(
FlowReadySnapshot(
sessionId: UUID(),
ready: false,
reason: .waitingForAudioProof,
engineMode: "local",
localeId: "zh-Hans"
),
defaults: defaults
)
XCTAssertTrue(FlowSessionBridge.isHostStale(defaults: defaults))
XCTAssertEqual(
FlowSessionBridge.readySnapshot(defaults: defaults)?.reason,
.waitingForAudioProof
)
}
func testStaleGenerationSnapshotIsNotReady() {
let defaults = makeDefaults()
let sessionId = UUID()
let now = Date().timeIntervalSince1970
FlowSessionBridge.rotateHostGeneration(defaults: defaults)
let liveGeneration = FlowSessionBridge.currentHostGeneration(defaults: defaults)
FlowSessionBridge.markSessionActive(duration: 60, sessionId: sessionId, defaults: defaults)
FlowSessionBridge.writeReadySnapshot(
FlowReadySnapshot(
sessionId: sessionId,
ready: true,
reason: .ready,
heartbeatAt: now,
readyAt: now,
engineMode: "local",
localeId: "zh-Hans",
hostGeneration: liveGeneration
),
defaults: defaults
)
XCTAssertTrue(FlowSessionBridge.isHostReady(defaults: defaults))
// Host relaunches (force-quit path) new generation. The old ready
// snapshot must be void instantly, without waiting out the 60 s
// heartbeat-zombie window.
FlowSessionBridge.rotateHostGeneration(defaults: defaults)
XCTAssertFalse(FlowSessionBridge.isHostReady(defaults: defaults))
}
func testClearFlowStateOnHostLaunchPreservesPendingHost() {
let defaults = makeDefaults()
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
FlowSessionBridge.setHostReady(true, defaults: defaults)
FlowSessionBridge.setPendingHostBundleId("com.example.host", defaults: defaults)
FlowSessionBridge.clearFlowStateOnHostLaunch(defaults: defaults)
XCTAssertFalse(FlowSessionBridge.isSessionActive(defaults: defaults))
XCTAssertFalse(FlowSessionBridge.isHostReady(defaults: defaults))
// The startflow scene-delegate write happens before the session
// manager exists launch reconciliation must not eat it.
XCTAssertEqual(
FlowSessionBridge.pendingHostBundleId(defaults: defaults),
"com.example.host"
)
}
func testRotateHostGenerationReturnsPreviousToken() {
let defaults = makeDefaults()
XCTAssertNil(FlowSessionBridge.rotateHostGeneration(defaults: defaults))
let first = FlowSessionBridge.currentHostGeneration(defaults: defaults)
XCTAssertNotNil(first)
let previous = FlowSessionBridge.rotateHostGeneration(defaults: defaults)
XCTAssertEqual(previous, first)
XCTAssertNotEqual(FlowSessionBridge.currentHostGeneration(defaults: defaults), first)
}
func testReadySnapshotDrivesHostReady() {
let defaults = makeDefaults()
let sessionId = UUID()
@@ -17,10 +17,10 @@ final class FlowSessionPolicyTests: XCTestCase {
XCTAssertTrue(FlowSessionPolicy.skipAppSwitch(defaults: defaults))
}
func testInactivityDurationDefaultsToTwelveHours() {
func testInactivityDurationDefaultsToThirtyMinutes() {
let defaults = makeDefaults()
XCTAssertEqual(FlowSessionPolicy.inactivityDuration(defaults: defaults), .twelveHours)
XCTAssertEqual(FlowSessionPolicy.sessionDuration(defaults: defaults), 12 * 60 * 60)
XCTAssertEqual(FlowSessionPolicy.inactivityDuration(defaults: defaults), .thirtyMinutes)
XCTAssertEqual(FlowSessionPolicy.sessionDuration(defaults: defaults), 30 * 60)
}
func testTouchLastActivityExtendsExpiry() {
+15 -1
View File
@@ -340,7 +340,7 @@ final class IntelligentPolishTests: XCTestCase {
@MainActor
func testFlowFallbackDeliveryCleansTextAndCarriesWeakNetworkWarning() {
let delivery = FlowSessionManager.makeFallbackDelivery(
let delivery = TranscriptionPolishFallback.makeDelivery(
rawText: " 你 是不是 已经 解决了 这个 问题 ? ",
error: LLMError.transport("offline"),
engineMode: "cloud",
@@ -352,6 +352,20 @@ final class IntelligentPolishTests: XCTestCase {
XCTAssertEqual(delivery.polishWarning, SharedL10n.string("flow.warning.polishDegraded"))
}
func testTranscriptionPolishFallbackLocalMissingKeyWarning() {
let delivery = TranscriptionPolishFallback.makeDelivery(
rawText: "测试文本",
error: PolishingService.PolishError.missingAPIKey,
engineMode: "local",
chunkWarning: nil
)
XCTAssertEqual(delivery.text, "测试文本")
XCTAssertEqual(
delivery.polishWarning,
SharedL10n.string("flow.warning.localPolishUnavailable")
)
}
func testHasStructureSignalDetectsChineseEnumeration() {
XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "首先测试其次上线"))
XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "第一点修复"))
@@ -13,7 +13,20 @@ final class LocalASRModelCatalogTests: XCTestCase {
XCTAssertFalse(catalog.models.contains { $0.id == "qwen3-mlx-1.7b" })
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-qwen3-0.6b-int8" })
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-qwen3-1.7b-int8" })
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-paraformer-zh-int8" })
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-sensevoice-small-int8" })
XCTAssertFalse(catalog.models.contains { $0.id == "sherpa-paraformer-zh-int8" })
XCTAssertEqual(
LocalASRModelCatalog.model("sherpa-sensevoice-small-int8", in: catalog)?.badgeKey,
"mac.localASR.badge.fastest"
)
XCTAssertEqual(
LocalASRModelCatalog.model("sherpa-qwen3-0.6b-int8", in: catalog)?.badgeKey,
"mac.localASR.badge.balanced"
)
XCTAssertEqual(
LocalASRModelCatalog.model("sherpa-qwen3-1.7b-int8", in: catalog)?.badgeKey,
"mac.localASR.badge.quality"
)
}
func testSherpaQwen317BUsesRepositoryInstall() throws {
@@ -32,9 +45,9 @@ final class LocalASRModelCatalogTests: XCTestCase {
XCTAssertTrue(model.supportsHotwords)
}
func testCapabilitiesForParaformer() throws {
func testCapabilitiesForSenseVoice() throws {
let catalog = try LocalASRModelCatalog.loadBundled()
let model = try XCTUnwrap(LocalASRModelCatalog.model("sherpa-paraformer-zh-int8", in: catalog))
let model = try XCTUnwrap(LocalASRModelCatalog.model("sherpa-sensevoice-small-int8", in: catalog))
let caps = LocalASRModelCatalog.capabilities(for: model)
XCTAssertEqual(caps.hotwordMode, .none)
XCTAssertFalse(model.supportsHotwords)
@@ -44,6 +44,9 @@ final class SettingsCloudSyncTests: XCTestCase {
providerId: SyncedField(value: "openai", updatedAt: stampA, deviceID: deviceA),
baseURL: SyncedField(value: "https://local.example", updatedAt: stampA, deviceID: deviceA),
model: SyncedField(value: "gpt-local", updatedAt: stampA, deviceID: deviceA),
asrProviderId: SyncedField(value: "zhipu", updatedAt: stampA, deviceID: deviceA),
asrBaseURL: SyncedField(value: "https://asr-local.example", updatedAt: stampA, deviceID: deviceA),
asrModel: SyncedField(value: "glm-asr-local", updatedAt: stampA, deviceID: deviceA),
modeId: SyncedField(value: "polish", updatedAt: stampA, deviceID: deviceA),
localeId: SyncedField(value: "auto", updatedAt: stampA, deviceID: deviceA),
engineMode: SyncedField(value: "cloud", updatedAt: stampA, deviceID: deviceA),
@@ -64,6 +67,9 @@ final class SettingsCloudSyncTests: XCTestCase {
providerId: SyncedField(value: "openai", updatedAt: stampA, deviceID: deviceB),
baseURL: SyncedField(value: "https://remote.example", updatedAt: stampB, deviceID: deviceB),
model: SyncedField(value: "gpt-remote", updatedAt: stampB, deviceID: deviceB),
asrProviderId: SyncedField(value: "qwen", updatedAt: stampB, deviceID: deviceB),
asrBaseURL: SyncedField(value: "https://asr-remote.example", updatedAt: stampB, deviceID: deviceB),
asrModel: SyncedField(value: "fun-asr-remote", updatedAt: stampB, deviceID: deviceB),
modeId: SyncedField(value: "polish", updatedAt: stampA, deviceID: deviceB),
localeId: SyncedField(value: "ja", updatedAt: stampB, deviceID: deviceB),
engineMode: SyncedField(value: "local", updatedAt: stampB, deviceID: deviceB),
@@ -80,6 +86,7 @@ final class SettingsCloudSyncTests: XCTestCase {
let merged = SyncedAppSettingsV2.merge(local: local, remote: remote)
XCTAssertEqual(merged.baseURL.value, "https://remote.example")
XCTAssertEqual(merged.asrProviderId.value, "qwen")
XCTAssertEqual(merged.localeId.value, "ja")
XCTAssertEqual(merged.engineMode.value, "local")
}
+17 -6
View File
@@ -21,7 +21,7 @@ OSGKeyboard is a free, source-available alternative to commercial voice-input to
2. Speak naturally (up to 3.5 minutes / 210 seconds per take)
3. Tap again to stop — the AI polishes your words into clean text and inserts at the cursor
Audio is transcribed **on-device** by Apple's `SpeechAnalyzer` + `DictationTranscriber` (iOS 26+). Only the **polished transcript** is sent to your chosen cloud LLM. **No audio ever leaves your phone.**
By default, audio is transcribed **on-device** by Apple's `SpeechAnalyzer` + `DictationTranscriber` (iOS 26+)**no audio leaves your phone** unless you say so. If polish is enabled, only the transcript text goes to your chosen LLM. Optionally, you can switch to a **cloud ASR engine** (explicit opt-in with a confirmation): in that mode your recordings are uploaded to the ASR provider you configure.
Under the hood, OSGKeyboard uses a **Flow session model**: a long-lived audio session runs in the host app, the keyboard extension writes tiny "start / stop" signals to the App Group, and the polished text is delivered back to the keyboard for insertion. You do not need to jump back to the host app between recordings.
@@ -34,7 +34,7 @@ Under the hood, OSGKeyboard uses a **Flow session model**: a long-lived audio se
- ✍️ **AI polishing** — adds structure, punctuation, fixes grammar, optionally produces lists
- 🧩 **Local + cloud polish toggle** — local engine is ASR-only by default; opt into a post-ASR cloud polish step (DeepSeek by default) when the iOS speech recognition isn't strong enough for your environment (noisy far-field audio, strong accents, etc.)
- 🔌 **Bring-your-own API** — works with any OpenAI-compatible endpoint (OpenAI, DeepSeek, Qwen DashScope, Moonshot, Zhipu, your own self-hosted server, …)
- 🔒 **Privacy first** — audio never leaves your device; only the final transcript is sent to the LLM you choose
- 🔒 **Privacy first** on-device ASR by default, so audio never leaves your device unless you explicitly opt into the cloud engine; polish sends only the transcript to the LLM you choose
- 🎨 **Native SwiftUI** — dark theme, frosted glass, pure Swift 6, ~3,600 lines of code
- 🪶 **Zero dependencies** — no SwiftPM packages, no CocoaPods, no Carthage
- 🔁 **Flow session** — keep recording across multiple takes without bouncing back to the host app
@@ -46,7 +46,7 @@ Under the hood, OSGKeyboard uses a **Flow session model**: a long-lived audio se
### Requirements
- macOS with **Xcode 26** (matches `project.yml` deployment target iOS 26)
- iPhone running **iOS 26.0+**
- iPhone or iPad running **iOS 26.0+** (iPad fully supported: Split View / Stage Manager, adaptive layout)
- [XcodeGen](https://github.com/yonaskolb/XcodeGen): `brew install xcodegen`
- An OpenAI-compatible API key (e.g. from [OpenAI](https://platform.openai.com/api-keys), [DeepSeek](https://platform.deepseek.com/api_keys), or [Qwen DashScope](https://dashscope.console.aliyun.com/apiKey)). Not needed if you stay on the "local ASR only" engine.
@@ -65,6 +65,17 @@ xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
> `project.yml` by `Scripts/generate-xcodeproj.sh`. Always re-run the script
> after `git pull` if `project.yml` has changed.
### macOS distribution (decision)
The macOS menu-bar app ships via **Developer ID direct distribution** (notarized,
non-sandboxed), NOT the Mac App Store. This is deliberate: its core features —
global hold-to-talk hotkey, Accessibility-based text insertion into other apps,
and synthesized ⌘V — are incompatible with the Mac App Store sandbox, and a
sandboxed Accessibility grant also tends to reset after every app update.
`OSGKeyboardMac.entitlements` therefore keeps `com.apple.security.app-sandbox`
set to `false`; do not flip it back on without redesigning the insertion path.
(The iOS app targets the iOS App Store as usual — see `AUDIT_APPSTORE.md`.)
### Enable the keyboard in iOS
The host app walks you through a **5-step onboarding**:
@@ -130,9 +141,9 @@ OSGKeyboard/
**Engine modes:**
- `cloud` (default) — on-device ASR via `SpeechAnalyzer`, transcript is sent to your configured LLM for polish.
- `local` — on-device ASR via `SpeechAnalyzer` only; transcript is inserted as-is. No network round-trip.
- `local` + "Cloud polish after ASR" toggle (Settings → Engine) — same on-device ASR, but the transcript is routed through your configured LLM before insertion. Useful when iOS speech recognition isn't accurate enough in your environment.
- `local` (default) — on-device ASR via `SpeechAnalyzer`; transcript is inserted as-is. No network round-trip.
- `local` + "Cloud polish after ASR" toggle (Settings → Engine) — same on-device ASR, but the transcript (text only) is routed through your configured LLM before insertion. Useful when iOS speech recognition isn't accurate enough in your environment.
- `cloud` (opt-in, requires an explicit confirmation) — **your voice recordings are uploaded** to the ASR provider you configure (e.g. OpenAI `/audio/transcriptions`, DashScope, Zhipu), and the resulting transcript is sent to your LLM for polish. Choose this only when you accept your provider's privacy terms.
**Cross-process plumbing (host app ↔ keyboard extension):**
+5 -5
View File
@@ -20,7 +20,7 @@ OSGKeyboard 是商业语音输入工具的免费、源码可见替代方案。
2. 自由说话(单次上限 3.5 分钟 / 210 秒)
3. 再按一下结束 —— AI 自动整理成干净的文字并插入光标
**音频始终在设备本地转写**(iOS 26+ 的 `SpeechAnalyzer` + `DictationTranscriber`**只有润色后的文本** 会发到你选择的云端 LLM。**音频永不离开你的手机。**
默认情况下,**音频在设备本地转写**(iOS 26+ 的 `SpeechAnalyzer` + `DictationTranscriber`——**除非你主动选择,音频不会离开你的手机**;开启润色时也只有文本会发到你选择的 LLM。你也可以显式切换到**云端识别引擎**(需二次确认的 opt-in):该模式下你的录音会上传到你配置的识别服务商。
项目内部采用 **Flow 会话模型**:主 App 维护一个长生命周期的音频会话,键盘扩展只通过 App Group 写入"开始 / 停止"等轻量信号,润色后的文本再由主 App 回传给键盘插入。**多次录音之间无需反复跳回主 App**。
@@ -33,7 +33,7 @@ OSGKeyboard 是商业语音输入工具的免费、源码可见替代方案。
- ✍️ **AI 润色** —— 自动加结构、补标点、修正语法、可生成列表
- 🧩 **本地 + 云端润色开关** —— 本地模式默认仅在设备上识别;若 iOS 语音识别效果不理想(远场、噪声、方言),可开启「识别后云端润色」,默认走 DeepSeek
- 🔌 **自带 API 接入** —— 兼容任何 OpenAI 兼容协议端点(OpenAI / DeepSeek / Qwen DashScope / Moonshot / 智谱 / 自建服务器 ……)
- 🔒 **隐私优先** —— 音频不离开设备;只有润色文本会发给你选择的 LLM
- 🔒 **隐私优先** —— 默认端侧识别,音频不离开设备(除非显式开启云端识别引擎);润色只发送文本给你选择的 LLM
- 🎨 **原生 SwiftUI** —— 暗色主题、毛玻璃、纯 Swift 6 实现,约 3,600 行代码
- 🪶 **零依赖** —— 无 SwiftPM 包、无 CocoaPods、无 Carthage
- 🔁 **Flow 会话** —— 多次录音无需跳回主 App,会话自动维持心跳与续期
@@ -127,9 +127,9 @@ OSGKeyboard/
**引擎模式:**
- `cloud`(默认)—— 端侧 `SpeechAnalyzer` 识别,文本发到云端 LLM 润色
- `local` —— 仅端侧 `SpeechAnalyzer` 识别,原始文本直接插入,不联网
- `local` + 「识别后云端润色」开关(设置 → 引擎)—— 同样走端侧 ASR,但识别完成后送 LLM 润色再插入。适用于 iOS 识别效果不理想的场景
- `local`(默认)—— 端侧 `SpeechAnalyzer` 识别,原始文本直接插入,不联网
- `local` + 「识别后云端润色」开关(设置 → 引擎)—— 同样走端侧 ASR,但识别完成后送 LLM 润色(仅文本)再插入。适用于 iOS 识别效果不理想的场景
- `cloud`(opt-in,需显式确认)—— **你的语音录音会上传**到你配置的识别服务商(如 OpenAI `/audio/transcriptions`、DashScope、智谱),识别文本再送 LLM 润色。请在接受服务商隐私条款的前提下选用
**跨进程管道(主 App ↔ 键盘扩展):**
+13 -9
View File
@@ -50,10 +50,12 @@
OSGKeyboard is a free, open-source custom keyboard for iOS 26 that turns
your voice into clean, AI-polished text — in any app.
Hold the mic key, speak naturally, release. The keyboard transcribes
your voice entirely on-device (Apple's iOS 26 SpeechAnalyzer +
DictationTranscriber), and only the final text is sent to the AI you
choose to polish it. Your audio never leaves your iPhone.
Hold the mic key, speak naturally, release. By default the keyboard
transcribes your voice entirely on-device (Apple's iOS 26
SpeechAnalyzer + DictationTranscriber), and only the final text is
sent to the AI you choose to polish it — your audio never leaves your
device unless you explicitly opt into the cloud ASR engine, which
uploads recordings to the provider you configure.
WHY OSGKEYBOARD
@@ -62,8 +64,9 @@ WHY OSGKEYBOARD
types for you.
• Push-to-talk, the way voice should work. No more "Hey Siri" mode that
listens to the whole room.
• On-device speech recognition. Powered by Apple's iOS 26 speech
pipeline — no cloud ASR, no audio upload.
• On-device speech recognition by default. Powered by Apple's iOS 26
speech pipeline — no audio upload unless you explicitly enable the
optional cloud ASR engine (confirmation required).
• Bring-your-own AI. Connect any OpenAI-compatible endpoint (OpenAI,
DeepSeek, Qwen DashScope, Moonshot, Zhipu, your own self-hosted
server). Your API key stays in the iOS Keychain.
@@ -76,12 +79,13 @@ WHY OSGKEYBOARD
• Zero dependencies. No trackers, no analytics, no crash reporters.
The whole project is ~8,700 lines of Swift you can audit in an
afternoon.
• Privacy first. PrivacyInfo.xcprivacy declares zero collected data;
we don't run a server.
• Privacy first. PrivacyInfo.xcprivacy declares exactly what the app
touches (voice audio + transcripts, on-device by default, never
linked or tracked); we don't run a server.
BUILT FOR
• iOS 26 and later, iPhone only.
• iOS 26 and later, iPhone and iPad.
• Anyone who types more than 100 words a day on their phone.
• Developers, writers, students, and translators who want voice input
that respects their privacy.

Some files were not shown because too many files have changed in this diff Show More