feat: harden Flow cold-start/force-quit and polish macOS dictation UX
Fix cold-start overlay recursion that overflowed the main-thread stack when recording began while the ready overlay was still up; also remove temporary on-screen Flow DEBUG panels after the orange-mic investigation, and land the macOS overlay/catalog/layout polish plus related Flow recovery hardening.
This commit is contained in:
+1
-1
@@ -116,7 +116,7 @@
|
||||
- **6.7"** (1290×2796) — 必需
|
||||
- **6.1"** (1179×2556) — 必需(iPhone 17 Pro)
|
||||
- 5.5" 已被苹果官方文档降级为"可选"(iPhone 8 Plus 等已停产机型)
|
||||
- iPad 截图 — 项目声明 iPhone only,无需提供
|
||||
- iPad 截图 — **现已必需**:项目自 TARGETED_DEVICE_FAMILY "1,2" 起支持 iPad,App Store Connect 要求提供 13″ iPad Pro 截图套组(本条为后续更新覆盖原「iPhone only 无需提供」的结论)
|
||||
- 每套至少 3 张、最多 10 张
|
||||
|
||||
**本审计交付**:
|
||||
|
||||
@@ -7,7 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **macOS dictation overlay**: a bottom-centered floating pill appears for any recording path (global hotkey, menu bar, or main window) — shows listening / transcribing state, a stronger live waveform, a one-line live transcript preview (partials when chunked ASR runs), front-app name, and a stop control, then briefly confirms success before fading out without stealing focus. / **macOS 听写浮层**:任意录音路径(全局热键、菜单栏或主窗口)都会在屏幕底部居中出现胶囊浮层——显示聆听 / 识别状态、更强的实时波形、单行转写预览(分块识别时显示 partial)、前台应用名与停止按钮,成功后短暂确认再淡出,且不抢前台焦点。
|
||||
- **macOS Option-key picker**: Settings → Input lets you choose Left / Right / Either Option as the hold-to-talk key, so the shortcut can avoid conflicts with other apps. / **macOS Option 键选择**:设置 → 输入与快捷键可选择左 / 右 / 任一 Option 作为按住听写键,避免与其他应用冲突。
|
||||
|
||||
### Changed
|
||||
- **Removed temporary Flow DEBUG panels**: the on-screen App Group / session debug text boxes on Home and the keyboard extension are gone now that the orange-mic investigation is closed. / **移除临时 Flow DEBUG 面板**:橙色麦克风排查结束后,首页与键盘扩展上的 App Group / 会话调试文本框已去掉。
|
||||
- **macOS local ASR catalog**: removed offline Paraformer; SenseVoice / Qwen3 0.6B / Qwen3 1.7B now show Fastest / Most balanced / Best quality badges (users still on Paraformer migrate to Qwen3 0.6B). / **macOS 本地 ASR 目录**:移除 offline Paraformer;SenseVoice / Qwen3 0.6B / Qwen3 1.7B 分别标注速度最快 / 最平衡 / 质量最好(仍选 Paraformer 的用户迁移到 Qwen3 0.6B)。
|
||||
- **macOS visual system**: full-app redesign around the brand line “Speak it. It’s typed.” / 「开口即文字。」— grouped sidebar, restrained accent selection, asymmetric Home stats (chars as hero), unified page headers, quieter status footer, and clearer dark-mode card elevation. / **macOS 视觉体系**:围绕品牌句「开口即文字。」/ “Speak it. It’s typed.” 做全 App 设计升级——侧栏分组、克制的选中态、首页不对称统计(字数主卡)、统一页头、降权状态栏,以及更清晰的暗色卡片层次。
|
||||
- **macOS Home stat cards**: the hero word-count card is now a full-width horizontal bar sized to its content (icon + label + big number) instead of a tall card stretched to match its neighbors, removing the large dead space below the number; the "Recent" list was removed from Home since it duplicated the History page. / **macOS 首页统计卡**:字数主卡改为按内容自适应高度的满宽横条(图标+标题+大数字),不再被拉伸到与相邻卡片等高、留出大片空白;首页底部与历史页重复的「最近」列表已移除。
|
||||
- **macOS page margins**: Home / History / Dictionary / Settings share `pageHorizontalInset` on titles and scroll *content*; ScrollViews / Forms stay full-bleed so the scrollbar sits on the window edge, while cards stay aligned with the page title. Settings keeps native grouped Form for control layout. / **macOS 页边距**:首页 / 历史 / 词库 / 设置在标题与滚动*内容*上共用 `pageHorizontalInset`;ScrollView / Form 通栏使滚动条贴窗口右缘,卡片仍与页标题对齐。设置保留原生分组 Form 以保证控件排版。
|
||||
|
||||
### Fixed
|
||||
- **Cold-start overlay recursion crash**: dismissing the ready overlay while an utterance is already recording no longer recurses `refreshHostReady` → `reconcile` → `dismiss` on the main thread until stack overflow (`EXC_BAD_ACCESS`). Handoff flags are cleared before any refresh. / **冷启动浮层递归崩溃**:就绪浮层仍在时若已开始录音,不再在主线程上递归 `refreshHostReady` → `reconcile` → `dismiss` 直至栈溢出(`EXC_BAD_ACCESS`);交接标志会在任何 refresh 之前先清除。
|
||||
- **macOS Qwen3 “language” garbage transcript**: Sherpa Qwen3 results that still include the model scaffold (`language Chinese<asr_text>…`) are now stripped to the spoken text; incomplete outputs that stop at the bare word `language` are treated as empty instead of being inserted. / **macOS Qwen3「language」乱码转写**:Sherpa Qwen3 结果若仍带模型脚手架(`language Chinese<asr_text>…`)会剥到真实口语文案;不完整输出停在单词 `language` 时按空结果处理,不再插入。
|
||||
- **macOS local ASR silence garbage output**: dictating with no speech (silence) on a Sherpa-backed local model (SenseVoice/Qwen3/Paraformer) no longer inserts the raw JSON result line (`{"lang": "", "emotion": "", ...}`) as the transcript — it now correctly reports "no speech recognized". / **macOS 本地识别静音乱码**:使用 Sherpa 本地模型(SenseVoice/Qwen3/Paraformer)听写时若未检测到语音,不再把原始 JSON 结果行(`{"lang": "", "emotion": "", ...}`)当作转写文本插入,现在会正确提示「没有识别到语音」。
|
||||
- **Force-quit mic release**: on termination the host app now synchronously stops `AVAudioEngine`, deactivates `AVAudioSession`, and ends Live Activities (Dynamic Island + Lock Screen) in `applicationWillTerminate`, reducing “microphone in use” errors after reopening. / **强杀麦克风释放**:进程终止时在 `applicationWillTerminate` 内同步停止 `AVAudioEngine`、释放 `AVAudioSession` 并结束 Live Activity(灵动岛 + 锁屏),降低强杀后重开提示麦克风被占用的概率。
|
||||
|
||||
## [0.5.2] - 2026-07-09
|
||||
|
||||
@@ -117,5 +117,12 @@
|
||||
<array>
|
||||
<string>UIInterfaceOrientationPortrait</string>
|
||||
</array>
|
||||
<key>UISupportedInterfaceOrientations~ipad</key>
|
||||
<array>
|
||||
<string>UIInterfaceOrientationPortrait</string>
|
||||
<string>UIInterfaceOrientationPortraitUpsideDown</string>
|
||||
<string>UIInterfaceOrientationLandscapeLeft</string>
|
||||
<string>UIInterfaceOrientationLandscapeRight</string>
|
||||
</array>
|
||||
</dict>
|
||||
</plist>
|
||||
|
||||
@@ -8,6 +8,21 @@
|
||||
<array/>
|
||||
<key>NSPrivacyCollectedDataTypes</key>
|
||||
<array>
|
||||
<dict>
|
||||
<!-- Voice recordings: processed on-device by default; uploaded to
|
||||
the user's configured ASR provider only when the cloud engine
|
||||
is explicitly enabled (opt-in with acknowledgment). -->
|
||||
<key>NSPrivacyCollectedDataType</key>
|
||||
<string>NSPrivacyCollectedDataTypeAudioData</string>
|
||||
<key>NSPrivacyCollectedDataTypeLinked</key>
|
||||
<false/>
|
||||
<key>NSPrivacyCollectedDataTypeTracking</key>
|
||||
<false/>
|
||||
<key>NSPrivacyCollectedDataTypePurposes</key>
|
||||
<array>
|
||||
<string>NSPrivacyCollectedDataTypePurposeAppFunctionality</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>NSPrivacyCollectedDataType</key>
|
||||
<string>NSPrivacyCollectedDataTypeOtherUserContent</string>
|
||||
|
||||
@@ -97,8 +97,11 @@ final class AppSceneDelegate: NSObject, UIWindowSceneDelegate {
|
||||
for item in items {
|
||||
// `sourceApplication` is only non-nil when the caller belongs to
|
||||
// the same Apple Developer Team (our own keyboard extension) —
|
||||
// exactly what the host-return whitelist relies on.
|
||||
if let source = item.source {
|
||||
// exactly what the host-return whitelist relies on. Record it
|
||||
// ONLY for the `startflow` handoff: overwriting it for every
|
||||
// deep link (e.g. `osgkeyboard://settings`) could point a later
|
||||
// cold-start "return to host" at the wrong app.
|
||||
if let source = item.source, item.url.host == "startflow" {
|
||||
FlowSessionBridge.setPendingHostBundleId(source)
|
||||
}
|
||||
AppOpenURLRouter.shared.route(item.url)
|
||||
|
||||
@@ -15,11 +15,17 @@ enum FlowLiveActivityController {
|
||||
nonisolated(unsafe) private static var currentPhase: FlowActivityAttributes.ContentState.Phase = .idle
|
||||
|
||||
/// If the host app is force-quit its `endSession()` never runs, orphaning
|
||||
/// the Live Activity. A short `staleDate` lets the system grey it out and
|
||||
/// reclaim it on its own within ~45s of the process dying. While the host is
|
||||
/// alive the heartbeat calls `keepAlive()` well inside this window, so a
|
||||
/// genuinely active session never looks stale.
|
||||
private static let staleWindow: TimeInterval = 45
|
||||
/// the Live Activity. `staleDate` semantics (verified against ActivityKit
|
||||
/// behaviour, not folklore): the *Dynamic Island* presentation is reliably
|
||||
/// removed shortly after the stale date passes, but the *lock-screen*
|
||||
/// banner may linger greyed-out depending on the iOS version — it is NOT
|
||||
/// guaranteed to be dismissed. Treating staleDate as "auto-cleanup" is
|
||||
/// therefore wrong on its own; the full zombie defence is this short
|
||||
/// window + launch-time reconciliation (`clearOrphanedActivities`) + the
|
||||
/// widget rendering an explicit "disconnected" state via
|
||||
/// `context.isStale`. While the host is alive the heartbeat calls
|
||||
/// `keepAlive()` every ~10 s, well inside this window.
|
||||
private static let staleWindow: TimeInterval = 30
|
||||
|
||||
private static func freshContent(
|
||||
phase: FlowActivityAttributes.ContentState.Phase
|
||||
@@ -118,6 +124,8 @@ enum FlowLiveActivityController {
|
||||
}
|
||||
|
||||
/// `applicationWillTerminate` 专用:阻塞到所有 `end` 完成,避免进程先退出而锁屏卡片残留。
|
||||
/// 等待必须带超时:ActivityKit 的 `end` 走异步 XPC,若在 watchdog 杀进程前
|
||||
/// 没有返回,无限期 `wait()` 会吞掉整个 ~5 秒终止窗口,反而让后续清理全部没跑。
|
||||
nonisolated static func endAllSynchronouslyOnTerminate() {
|
||||
let semaphore = DispatchSemaphore(value: 0)
|
||||
Task.detached(priority: .userInitiated) {
|
||||
@@ -129,7 +137,7 @@ enum FlowLiveActivityController {
|
||||
FlowDiagnostics.log("Live Activity ended synchronously on terminate (count=\(count))")
|
||||
semaphore.signal()
|
||||
}
|
||||
semaphore.wait()
|
||||
_ = semaphore.wait(timeout: .now() + 2)
|
||||
currentPhase = .idle
|
||||
currentActivity = nil
|
||||
}
|
||||
|
||||
@@ -54,9 +54,10 @@ final class FlowSessionManager: ObservableObject {
|
||||
private var currentUtteranceId: UUID?
|
||||
private var currentCommandSeq: Int64 = 0
|
||||
private var lastHandledCommandSeq: Int64 = 0
|
||||
private var isUtteranceRecording = false
|
||||
/// Published so Home / debug UI can show "recording" instead of a false "ready".
|
||||
@Published private(set) var isUtteranceRecording = false
|
||||
/// True from `stopped` until the result/error is written back to App Group.
|
||||
private var isUtteranceProcessing = false
|
||||
@Published private(set) var isUtteranceProcessing = false
|
||||
private var finalizeTask: Task<Void, Never>?
|
||||
private var asrTask: Task<Void, Never>?
|
||||
private var utteranceSafetyTask: Task<Void, Never>?
|
||||
@@ -77,17 +78,50 @@ final class FlowSessionManager: ObservableObject {
|
||||
private var coldStartRecoveryTask: Task<Void, Never>?
|
||||
/// Initial proof window — cold mic sessions often need >2.5s after app switch.
|
||||
private static let coldStartAudioProofTimeout: TimeInterval = 6
|
||||
/// Extra window after the first timeout while the overlay shows a failure hint.
|
||||
private static let coldStartRecoveryProofTimeout: TimeInterval = 12
|
||||
/// Guards the once-per-process launch reconciliation (scene reconnects
|
||||
/// recreate the `@StateObject`-owned manager within the same process).
|
||||
private static var didRunLaunchReconciliation = false
|
||||
|
||||
init() {
|
||||
// Sessions are (re)started explicitly on app foreground via
|
||||
// `activateOnForeground()`. We deliberately do NOT silently reattach a
|
||||
// stored session here — after a force-quit that would resurrect capture
|
||||
// (and keep a stale Live Activity alive) without the user re-opening.
|
||||
//
|
||||
// Launch reconciliation: a brand-new process can never own an
|
||||
// in-flight session, so whatever the previous generation persisted
|
||||
// (force-quit skips `applicationWillTerminate` entirely when the app
|
||||
// was suspended) is void. Rotating the generation token also lets the
|
||||
// keyboard invalidate stale ready snapshots instantly instead of
|
||||
// waiting out the 60 s heartbeat-zombie window.
|
||||
//
|
||||
// Once per PROCESS, not per manager: iOS can disconnect and later
|
||||
// reconnect the sole scene without killing the process, which
|
||||
// recreates the `@StateObject` (and thus this init). Re-rotating then
|
||||
// would wipe live state that belongs to this very process.
|
||||
if AppGroup.isAvailable, !Self.didRunLaunchReconciliation {
|
||||
Self.didRunLaunchReconciliation = true
|
||||
let previous = FlowSessionBridge.rotateHostGeneration()
|
||||
if previous != nil || FlowSessionBridge.isSessionActive() {
|
||||
FlowSessionBridge.clearFlowStateOnHostLaunch()
|
||||
FlowLiveActivityController.clearOrphanedActivities()
|
||||
FlowSessionDarwin.postSessionChanged()
|
||||
debug("launch reconciliation: voided previous-generation Flow state")
|
||||
}
|
||||
}
|
||||
|
||||
capture.onEngineLiveChanged = { [weak self] _ in
|
||||
self?.refreshHostReady()
|
||||
}
|
||||
// A system interruption (call / Siri) stops audio frames mid-utterance;
|
||||
// fail fast so the user is not silently recording into a gap.
|
||||
capture.onInterruptionBegan = { [weak self] in
|
||||
guard let self, self.isUtteranceRecording else { return }
|
||||
self.failUtterance(
|
||||
message: AppL10n.string("flow.error.recognitionInterrupted"),
|
||||
kind: .recognitionInterrupted
|
||||
)
|
||||
}
|
||||
FlowTerminationCoordinator.register(self)
|
||||
}
|
||||
|
||||
@@ -126,6 +160,10 @@ final class FlowSessionManager: ObservableObject {
|
||||
traceState("startSession.ignored", extra: "reason=alreadyStarting")
|
||||
return
|
||||
}
|
||||
// Claim the flag synchronously: on a cold start the URL router and
|
||||
// `activateOnForeground()` both fire in the same runloop turn, and
|
||||
// setting it inside the async body let two start bodies interleave.
|
||||
isStarting = true
|
||||
|
||||
startTask?.cancel()
|
||||
startTask = Task { @MainActor [weak self] in
|
||||
@@ -190,11 +228,15 @@ final class FlowSessionManager: ObservableObject {
|
||||
func dismissColdStartOverlay() {
|
||||
coldStartRecoveryTask?.cancel()
|
||||
coldStartRecoveryTask = nil
|
||||
// Clear handoff flags BEFORE any refreshHostReady call. Otherwise
|
||||
// refresh → reconcileColdStartOverlayIfRecovered → dismiss → refresh
|
||||
// recurses until the main-thread stack overflows (EXC_BAD_ACCESS,
|
||||
// "Thread stack size exceeded due to excessive recursion").
|
||||
coldStartContext = nil
|
||||
isColdStartHandoff = false
|
||||
if isActive {
|
||||
refreshHostReady()
|
||||
}
|
||||
coldStartContext = nil
|
||||
isColdStartHandoff = false
|
||||
}
|
||||
|
||||
func returnToPendingHostFromColdStart() {
|
||||
@@ -204,6 +246,21 @@ final class FlowSessionManager: ObservableObject {
|
||||
|
||||
func retryColdStartReadiness() {
|
||||
guard AppGroup.isAvailable else { return }
|
||||
// A failed cold start leaves capture in a running-but-dead state on
|
||||
// purpose (the recovery loop keeps probing it). A user-initiated
|
||||
// retry must instead begin from a clean pipeline: tear down capture
|
||||
// and the cached ASR instance so `startSession` rebuilds both —
|
||||
// otherwise the retry reuses the zombie engine and is guaranteed to
|
||||
// hit the same audio-proof timeout.
|
||||
coldStartRecoveryTask?.cancel()
|
||||
coldStartRecoveryTask = nil
|
||||
if capture.running {
|
||||
capture.stop()
|
||||
}
|
||||
sessionASR?.cancel()
|
||||
sessionASR = nil
|
||||
sessionASREngineMode = nil
|
||||
sessionASRWarmedLocaleID = nil
|
||||
startSession(coldStart: true)
|
||||
}
|
||||
|
||||
@@ -241,7 +298,10 @@ final class FlowSessionManager: ObservableObject {
|
||||
|
||||
if isUtteranceRecording || isUtteranceProcessing {
|
||||
capture.cancelUtterance()
|
||||
asr.cancel()
|
||||
// `asr` is a computed property that ALLOCATES a fresh ASRService
|
||||
// when `sessionASR` is nil — never do that inside the ~5 s
|
||||
// termination window; only cancel an instance that exists.
|
||||
sessionASR?.cancel()
|
||||
}
|
||||
|
||||
capture.cancelUtterance()
|
||||
@@ -407,17 +467,43 @@ final class FlowSessionManager: ObservableObject {
|
||||
|
||||
private func reactivateCaptureIfNeeded() async {
|
||||
guard isActive else { return }
|
||||
// A system interruption (call / Siri) may be in progress. Probe it:
|
||||
// `setActive(true)` inside `reassertIfRunning` fails while the
|
||||
// interruption is live and succeeds once it ends — which also covers
|
||||
// the documented case where iOS never delivers `.ended` (the latch
|
||||
// must not depend on that notification, or the session is dead until
|
||||
// its TTL). While the probe fails we deliberately do NOT stop or
|
||||
// rebuild: tearing the engine down would remove the observers the
|
||||
// `.ended` rebuild relies on and churn the shared session mid-call.
|
||||
if capture.isInterrupted {
|
||||
guard capture.reassertIfRunning(), !capture.isInterrupted else { return }
|
||||
}
|
||||
|
||||
if capture.running {
|
||||
let reasserted = capture.reassertIfRunning()
|
||||
if reasserted, capture.engineHasRecentAudio() {
|
||||
if reasserted, await capture.awaitAudioFlowing(timeout: 2) {
|
||||
sessionWarning = nil
|
||||
} else if !reasserted {
|
||||
sessionWarning = AppL10n.string("flow.error.audioUnavailable")
|
||||
}
|
||||
refreshHostReady()
|
||||
return
|
||||
}
|
||||
// The await above is a suspension point: the session may have
|
||||
// ended (expiry, user, teardown) while we waited. Never restart
|
||||
// the microphone for a session that no longer exists.
|
||||
guard isActive, !Task.isCancelled else { return }
|
||||
// Never tear capture down underneath a live utterance either — a
|
||||
// stalled route transition mid-recording must surface through the
|
||||
// utterance pipeline (safety timer / empty-transcript error), not
|
||||
// as a silent stop that truncates the take with no error at all.
|
||||
guard !isUtteranceRecording, !isUtteranceProcessing else {
|
||||
refreshHostReady()
|
||||
return
|
||||
}
|
||||
// Reassert failed, or the engine reports running yet produces no
|
||||
// frames (zombie after suspend / mediaserverd reset) — fall
|
||||
// through to a full rebuild instead of leaving it half-dead.
|
||||
capture.stop()
|
||||
debug("capture zombie after foreground — rebuilding")
|
||||
}
|
||||
|
||||
do {
|
||||
try capture.start()
|
||||
@@ -442,7 +528,8 @@ final class FlowSessionManager: ObservableObject {
|
||||
reason: .noSession,
|
||||
engineMode: store.engineMode,
|
||||
localeId: store.localeId,
|
||||
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt()
|
||||
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt(),
|
||||
hostGeneration: FlowSessionBridge.currentHostGeneration()
|
||||
)
|
||||
)
|
||||
return
|
||||
@@ -486,7 +573,8 @@ final class FlowSessionManager: ObservableObject {
|
||||
engineMode: store.engineMode,
|
||||
localeId: store.localeId,
|
||||
busyUtteranceId: isUtteranceRecording || isUtteranceProcessing ? currentUtteranceId : nil,
|
||||
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt()
|
||||
sessionExpiresAt: FlowSessionBridge.sessionExpiresAt(),
|
||||
hostGeneration: FlowSessionBridge.currentHostGeneration()
|
||||
)
|
||||
)
|
||||
let signature = [
|
||||
@@ -509,9 +597,19 @@ final class FlowSessionManager: ObservableObject {
|
||||
/// shows a stale preparing/failed snapshot, heal automatically.
|
||||
private func reconcileColdStartOverlayIfRecovered() {
|
||||
guard isColdStartHandoff, isActive else { return }
|
||||
guard FlowSessionBridge.isHostReady() else { return }
|
||||
guard let context = coldStartContext else { return }
|
||||
|
||||
// Mid-utterance is not "ready" — dismiss the ready overlay so Home
|
||||
// does not keep advertising "语音已就绪" while utt.rec=1.
|
||||
if isUtteranceRecording || isUtteranceProcessing {
|
||||
if case .ready = context.state {
|
||||
dismissColdStartOverlay()
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
guard FlowSessionBridge.isHostReady() else { return }
|
||||
|
||||
switch context.state {
|
||||
case .preparing:
|
||||
presentColdStartReadyOverlay()
|
||||
@@ -556,10 +654,11 @@ final class FlowSessionManager: ObservableObject {
|
||||
// MARK: - Session start
|
||||
|
||||
private func startSessionAsync(duration: TimeInterval?) async {
|
||||
traceState("startSessionAsync.begin")
|
||||
isStarting = true
|
||||
sessionWarning = nil
|
||||
// `isStarting` was claimed synchronously in `startSession()`.
|
||||
defer { isStarting = false }
|
||||
guard !Task.isCancelled else { return }
|
||||
traceState("startSessionAsync.begin")
|
||||
sessionWarning = nil
|
||||
|
||||
guard AppPermissions.flowRequirementsMet else {
|
||||
sessionWarning = permissionWarningMessage()
|
||||
@@ -586,7 +685,9 @@ final class FlowSessionManager: ObservableObject {
|
||||
return
|
||||
}
|
||||
|
||||
guard await waitForAudioProof() else {
|
||||
let audioProved = await waitForAudioProof()
|
||||
guard !Task.isCancelled else { return }
|
||||
guard audioProved else {
|
||||
let message = AppL10n.string("flow.coldStart.error.audioTimeout")
|
||||
sessionWarning = message
|
||||
traceState("startSessionAsync.failed", extra: "reason=audioProofTimeout")
|
||||
@@ -658,6 +759,16 @@ final class FlowSessionManager: ObservableObject {
|
||||
|
||||
refreshHostReady()
|
||||
guard FlowSessionBridge.isHostReady() else {
|
||||
// Busy ≠ broken: a startflow arriving mid-utterance (e.g. tapping
|
||||
// the Live Activity while dictating) finds a healthy session that
|
||||
// is simply recording/processing. Showing the audio-failure
|
||||
// overlay here would be a lie — and its recovery loop could even
|
||||
// stop capture and kill the live utterance.
|
||||
if isUtteranceRecording || isUtteranceProcessing {
|
||||
dismissColdStartOverlay()
|
||||
debug("cold-start handoff ignored: session busy with an utterance")
|
||||
return
|
||||
}
|
||||
let message = AppL10n.string("flow.coldStart.error.audioTimeout")
|
||||
sessionWarning = message
|
||||
showColdStartAudioFailure(message: message)
|
||||
@@ -687,18 +798,54 @@ final class FlowSessionManager: ObservableObject {
|
||||
}
|
||||
}
|
||||
|
||||
/// Keeps proving mic readiness after the first timeout instead of tearing
|
||||
/// capture down — many handoffs become ready a few seconds later.
|
||||
/// Actively rebuilds the audio pipeline after a failed cold start instead
|
||||
/// of passively waiting for frames that a dead engine will never produce.
|
||||
/// Escalates per attempt: reassert the session → full engine rebuild →
|
||||
/// bounce the audio session and rebuild. Force-quit relaunches routinely
|
||||
/// inherit stale mediaserverd state that only a rebuild clears.
|
||||
private func scheduleColdStartRecovery(duration: TimeInterval?) {
|
||||
coldStartRecoveryTask?.cancel()
|
||||
coldStartRecoveryTask = Task { @MainActor [weak self] in
|
||||
guard let self else { return }
|
||||
let recovered = await self.capture.awaitAudioFlowing(
|
||||
timeout: Self.coldStartRecoveryProofTimeout
|
||||
var recovered = false
|
||||
for attempt in 1...3 {
|
||||
guard !Task.isCancelled, self.isColdStartHandoff else { return }
|
||||
switch attempt {
|
||||
case 1:
|
||||
_ = self.capture.reassertIfRunning()
|
||||
case 2:
|
||||
self.capture.stop()
|
||||
try? self.capture.start()
|
||||
default:
|
||||
self.capture.stop()
|
||||
try? await Task.sleep(nanoseconds: 300_000_000)
|
||||
guard !Task.isCancelled else { return }
|
||||
try? self.capture.start()
|
||||
}
|
||||
recovered = await self.capture.awaitAudioFlowing(
|
||||
timeout: TimeInterval(attempt + 1)
|
||||
)
|
||||
self.traceState(
|
||||
"coldStartRecovery.attempt",
|
||||
extra: "attempt=\(attempt) recovered=\(recovered)"
|
||||
)
|
||||
if recovered { break }
|
||||
}
|
||||
guard !Task.isCancelled else { return }
|
||||
guard self.isColdStartHandoff else { return }
|
||||
guard recovered else { return }
|
||||
guard recovered else {
|
||||
// Out of attempts — leave the failure overlay up; its retry
|
||||
// button now performs a full teardown so the user always has
|
||||
// a working escape hatch (no more force-quit loops). Only
|
||||
// tear capture down when no session owns it: for an active
|
||||
// session the 1 Hz heartbeat keeps self-healing, and a stop
|
||||
// here would just fight it.
|
||||
if !self.isActive {
|
||||
self.capture.stop()
|
||||
}
|
||||
self.traceState("coldStartRecovery.exhausted")
|
||||
return
|
||||
}
|
||||
|
||||
self.sessionWarning = nil
|
||||
self.traceState("coldStartRecovery.recovered")
|
||||
@@ -1027,12 +1174,22 @@ final class FlowSessionManager: ObservableObject {
|
||||
|
||||
// Do NOT cancel `asrTask` or `asr` — drain trailing PCM, then finalize.
|
||||
|
||||
// Capture ids now: a cancelled finalize must still clear *this*
|
||||
// utterance's processing gate even if currentUtteranceId was cleared
|
||||
// by a racing fail/abort path.
|
||||
let drainingSessionId = activeSessionId
|
||||
let drainingUtteranceId = currentUtteranceId
|
||||
let drainingCommandSeq = currentCommandSeq
|
||||
finalizeTask?.cancel()
|
||||
finalizeTask = Task { @MainActor [weak self] in
|
||||
guard let self else { return }
|
||||
let drainReport = await self.capture.endUtteranceAndDrain()
|
||||
FlowDiagnostics.logDrain(drainReport)
|
||||
await self.finalizeUtterance()
|
||||
await self.finalizeUtterance(
|
||||
sessionId: drainingSessionId,
|
||||
utteranceId: drainingUtteranceId,
|
||||
commandSeq: drainingCommandSeq
|
||||
)
|
||||
}
|
||||
debug("utterance stopped, draining tail")
|
||||
}
|
||||
@@ -1109,20 +1266,22 @@ final class FlowSessionManager: ObservableObject {
|
||||
debug("utterance processing failed: \(message)")
|
||||
}
|
||||
|
||||
private func finalizeUtterance() async {
|
||||
let finalizeSessionId = activeSessionId
|
||||
let finalizeUtteranceId = currentUtteranceId
|
||||
private func finalizeUtterance(
|
||||
sessionId finalizeSessionId: UUID?,
|
||||
utteranceId finalizeUtteranceId: UUID?,
|
||||
commandSeq finalizeCommandSeq: Int64
|
||||
) async {
|
||||
let pipelineStarted = Date()
|
||||
// ALWAYS clear the processing gate for this utterance. The previous
|
||||
// guard required currentUtteranceId to still match; a racing
|
||||
// fail/abort/cancel path could nil the id (or leave processing stuck)
|
||||
// and then skip refreshHostReady — keyboard stayed white forever
|
||||
// while host logs still said "utterance finalized".
|
||||
defer {
|
||||
if activeSessionId == finalizeSessionId,
|
||||
currentUtteranceId == finalizeUtteranceId {
|
||||
isUtteranceProcessing = false
|
||||
FlowLiveActivityController.update(phase: .idle)
|
||||
touchSessionActivity()
|
||||
currentUtteranceId = nil
|
||||
currentCommandSeq = 0
|
||||
refreshHostReady()
|
||||
}
|
||||
completeFinalizeCleanup(
|
||||
sessionId: finalizeSessionId,
|
||||
utteranceId: finalizeUtteranceId
|
||||
)
|
||||
}
|
||||
|
||||
let asrWait = asrWaitTimeout()
|
||||
@@ -1134,6 +1293,10 @@ final class FlowSessionManager: ObservableObject {
|
||||
while Date() < asrDeadline {
|
||||
if !lastFinal.isEmpty { break }
|
||||
if asrTask?.isCancelled == true { break }
|
||||
// Honour cooperative cancel so a replaced finalize exits promptly,
|
||||
// but still run defer cleanup (unlike an early `return` mid-polish
|
||||
// that used to leave processing=true when ids no longer matched).
|
||||
if Task.isCancelled { break }
|
||||
try? await Task.sleep(nanoseconds: 100_000_000)
|
||||
}
|
||||
|
||||
@@ -1151,14 +1314,21 @@ final class FlowSessionManager: ObservableObject {
|
||||
text = currentPartial.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
guard !text.isEmpty else {
|
||||
let key = (asrTask?.isCancelled == true)
|
||||
let key = (asrTask?.isCancelled == true || Task.isCancelled)
|
||||
? "flow.error.recognitionInterrupted"
|
||||
: "flow.error.noSpeech"
|
||||
let kind: FlowSessionKeys.TranscriptionErrorKind =
|
||||
(asrTask?.isCancelled == true) ? .recognitionInterrupted : .noSpeech
|
||||
(asrTask?.isCancelled == true || Task.isCancelled)
|
||||
? .recognitionInterrupted : .noSpeech
|
||||
FlowDiagnostics.log("finalize failed: empty transcript after \(String(format: "%.1f", asrElapsed))s")
|
||||
utteranceRecordingStartedAt = nil
|
||||
storeCurrentError(AppL10n.string(key), kind: kind)
|
||||
storeFinalizedError(
|
||||
AppL10n.string(key),
|
||||
kind: kind,
|
||||
sessionId: finalizeSessionId,
|
||||
utteranceId: finalizeUtteranceId,
|
||||
commandSeq: finalizeCommandSeq
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -1178,23 +1348,33 @@ final class FlowSessionManager: ObservableObject {
|
||||
"translationTarget=\(pipelineStore.translationTargetLocaleId)"
|
||||
)
|
||||
do {
|
||||
// If the finalize task was cancelled (cold-start churn / abort),
|
||||
// skip the LLM round-trip and deliver the raw transcript so the
|
||||
// keyboard is not left waiting on a result that never arrives.
|
||||
if Task.isCancelled {
|
||||
throw CancellationError()
|
||||
}
|
||||
let polished = try await polisher.polish(
|
||||
text,
|
||||
mode: polishMode,
|
||||
providerIdOverride: pipelineStore.polishProviderIdOverride
|
||||
)
|
||||
delivered = polished
|
||||
storeCurrentFinal(polished, warning: chunkNote)
|
||||
storeFinalizedResult(
|
||||
polished,
|
||||
warning: chunkNote,
|
||||
sessionId: finalizeSessionId,
|
||||
utteranceId: finalizeUtteranceId,
|
||||
commandSeq: finalizeCommandSeq
|
||||
)
|
||||
FlowDiagnostics.log(
|
||||
"polish done in \(String(format: "%.1f", Date().timeIntervalSince(polishStarted)))s " +
|
||||
"total=\(String(format: "%.1f", Date().timeIntervalSince(pipelineStarted)))s"
|
||||
)
|
||||
} catch {
|
||||
// v0.2.0: local + cloud-polish-on + no API key surfaces
|
||||
// `.missingAPIKey`. We translate it into a polishWarning
|
||||
// so the keyboard can show the "fill in your key" hint
|
||||
// inline rather than a generic failure message. The raw
|
||||
// transcript is still delivered — no data loss.
|
||||
// CancellationError is common when the user jumps back via
|
||||
// startflow mid-polish; still deliver raw text. Other errors
|
||||
// keep the existing polish-warning fallback.
|
||||
let fallback = Self.makeFallbackDelivery(
|
||||
rawText: text,
|
||||
error: error,
|
||||
@@ -1206,7 +1386,13 @@ final class FlowSessionManager: ObservableObject {
|
||||
"\(error.localizedDescription)"
|
||||
)
|
||||
delivered = fallback.text
|
||||
storeCurrentFinal(fallback.text, warning: fallback.polishWarning)
|
||||
storeFinalizedResult(
|
||||
fallback.text,
|
||||
warning: fallback.polishWarning,
|
||||
sessionId: finalizeSessionId,
|
||||
utteranceId: finalizeUtteranceId,
|
||||
commandSeq: finalizeCommandSeq
|
||||
)
|
||||
}
|
||||
|
||||
SpeechHistoryStore.shared.recordUtterance(
|
||||
@@ -1223,6 +1409,92 @@ final class FlowSessionManager: ObservableObject {
|
||||
debug("utterance finalized length=\(text.count)")
|
||||
}
|
||||
|
||||
/// Drop the processing gate and republish hostReady after finalize.
|
||||
/// Must not depend on a perfect id match — a racing fail/abort/cancel
|
||||
/// used to skip this block and leave the keyboard stuck on white「识别中」
|
||||
/// even after "utterance finalized" was logged.
|
||||
private func completeFinalizeCleanup(sessionId: UUID?, utteranceId: UUID?) {
|
||||
// A newer utterance may have started; never clobber its gate.
|
||||
if let current = currentUtteranceId,
|
||||
let finished = utteranceId,
|
||||
current != finished {
|
||||
debug(
|
||||
"finalize cleanup skipped — newer utterance live " +
|
||||
"finished=\(finished.uuidString.prefix(8)) current=\(current.uuidString.prefix(8))"
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
let wasProcessing = isUtteranceProcessing
|
||||
isUtteranceProcessing = false
|
||||
FlowLiveActivityController.update(phase: .idle)
|
||||
if isActive {
|
||||
touchSessionActivity()
|
||||
}
|
||||
if currentUtteranceId == utteranceId || currentUtteranceId == nil {
|
||||
currentUtteranceId = nil
|
||||
currentCommandSeq = 0
|
||||
}
|
||||
refreshHostReady()
|
||||
debug(
|
||||
"finalize cleanup done wasProcessing=\(wasProcessing ? 1 : 0) " +
|
||||
"utterance=\(utteranceId?.uuidString.prefix(8) ?? "nil") " +
|
||||
"session=\(sessionId?.uuidString.prefix(8) ?? "nil")"
|
||||
)
|
||||
}
|
||||
|
||||
private func storeFinalizedResult(
|
||||
_ text: String,
|
||||
warning: String?,
|
||||
sessionId: UUID?,
|
||||
utteranceId: UUID?,
|
||||
commandSeq: Int64
|
||||
) {
|
||||
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty else {
|
||||
storeFinalizedError(
|
||||
AppL10n.string("flow.error.noSpeech"),
|
||||
kind: .noSpeech,
|
||||
sessionId: sessionId,
|
||||
utteranceId: utteranceId,
|
||||
commandSeq: commandSeq
|
||||
)
|
||||
return
|
||||
}
|
||||
guard let sessionId, let utteranceId else { return }
|
||||
FlowSessionBridge.writeResult(
|
||||
FlowResult(
|
||||
sessionId: sessionId,
|
||||
utteranceId: utteranceId,
|
||||
commandSeq: commandSeq,
|
||||
status: .final,
|
||||
text: trimmed,
|
||||
warning: warning
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
private func storeFinalizedError(
|
||||
_ message: String,
|
||||
kind: FlowSessionKeys.TranscriptionErrorKind,
|
||||
sessionId: UUID?,
|
||||
utteranceId: UUID?,
|
||||
commandSeq: Int64,
|
||||
status: FlowResult.Status = .error
|
||||
) {
|
||||
guard let sessionId, let utteranceId else { return }
|
||||
FlowSessionBridge.writeResult(
|
||||
FlowResult(
|
||||
sessionId: sessionId,
|
||||
utteranceId: utteranceId,
|
||||
commandSeq: commandSeq,
|
||||
status: status,
|
||||
text: message,
|
||||
errorKind: kind
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
private static func polishModeLogLabel(_ mode: PolishingService.PolishMode) -> String {
|
||||
switch mode {
|
||||
case .polish:
|
||||
@@ -1252,35 +1524,12 @@ final class FlowSessionManager: ObservableObject {
|
||||
engineMode: String,
|
||||
chunkWarning: String?
|
||||
) -> TranscriptionDelivery {
|
||||
let fallbackText = TranscriptPostProcessor.cleanRawASRFallback(rawText)
|
||||
let warning = warningFromPolishError(error, engineMode: engineMode)
|
||||
?? polishDegradedWarning()
|
||||
?? chunkWarning
|
||||
return TranscriptionDelivery(text: fallbackText, polishWarning: warning)
|
||||
}
|
||||
|
||||
private static func warningFromPolishError(_ error: Error, engineMode: String) -> String? {
|
||||
if let polishError = error as? PolishingService.PolishError {
|
||||
switch polishError {
|
||||
case .missingAPIKey:
|
||||
if engineMode == "local" {
|
||||
return SharedL10n.string("flow.warning.localPolishUnavailable")
|
||||
}
|
||||
return SharedL10n.string("flow.warning.cloudPolishMissingKey")
|
||||
case .timeout:
|
||||
return polishDegradedWarning()
|
||||
case .noTranscript:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if error is LLMError {
|
||||
return polishDegradedWarning()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
private static func polishDegradedWarning() -> String? {
|
||||
SharedL10n.string("flow.warning.polishDegraded")
|
||||
TranscriptionPolishFallback.makeDelivery(
|
||||
rawText: rawText,
|
||||
error: error,
|
||||
engineMode: engineMode,
|
||||
chunkWarning: chunkWarning
|
||||
)
|
||||
}
|
||||
|
||||
private func asrWaitTimeout() -> TimeInterval {
|
||||
@@ -1318,8 +1567,8 @@ final class FlowSessionManager: ObservableObject {
|
||||
// Refresh the Live Activity `staleDate` every N heartbeat ticks
|
||||
// (1 Hz) — well inside `FlowLiveActivityController.staleWindow` so a
|
||||
// live session never looks stale, while a force-quit stops these
|
||||
// refreshes and lets the system reclaim the orphaned island.
|
||||
let liveActivityKeepAliveEveryTicks = 15
|
||||
// refreshes and lets the island go stale within ~30 s.
|
||||
let liveActivityKeepAliveEveryTicks = 10
|
||||
var tick = 0
|
||||
while !Task.isCancelled {
|
||||
guard let self else { break }
|
||||
@@ -1351,6 +1600,30 @@ final class FlowSessionManager: ObservableObject {
|
||||
FlowDiagnostics.log(message)
|
||||
}
|
||||
|
||||
// MARK: - Temporary Flow debug panel (remove after orange-mic investigation)
|
||||
|
||||
/// Snapshot for the on-screen debug panel. Safe to call from the main actor.
|
||||
func makeDebugRows() -> [FlowDebugRow] {
|
||||
let snapshot = FlowSessionBridge.readySnapshot()
|
||||
let hostRows = FlowDebugAppGroupSnapshot.rows()
|
||||
let memRows: [FlowDebugRow] = [
|
||||
FlowDebugRow("isActive", isActive ? "1" : "0"),
|
||||
FlowDebugRow("isStarting", isStarting ? "1" : "0"),
|
||||
FlowDebugRow("coldStart", isColdStartHandoff ? "1" : "0"),
|
||||
FlowDebugRow("engineLive", capture.engineIsLive ? "1" : "0"),
|
||||
FlowDebugRow("audioFresh", capture.engineHasRecentAudio(maxAge: 2) ? "1" : "0"),
|
||||
FlowDebugRow("mem.reason", snapshot?.reason.rawValue ?? "nil"),
|
||||
FlowDebugRow("utt.rec", isUtteranceRecording ? "1" : "0"),
|
||||
FlowDebugRow("utt.proc", isUtteranceProcessing ? "1" : "0"),
|
||||
FlowDebugRow("sessionId", activeSessionId.map { String($0.uuidString.prefix(8)) } ?? "nil"),
|
||||
FlowDebugRow("warning", sessionWarning == nil ? "0" : "1"),
|
||||
FlowDebugRow("overlay", coldStartContext.map { String(describing: $0.state) } ?? "nil"),
|
||||
FlowDebugRow("bridgeReady", FlowSessionBridge.isHostReady() ? "1" : "0")
|
||||
]
|
||||
// Prefer App Group snap.reason near the top of the shared block.
|
||||
return memRows + hostRows
|
||||
}
|
||||
|
||||
private func traceIgnoredCommand(reason: String, command: FlowCommand, detail: String) {
|
||||
let signature = "\(reason)|\(command.action.rawValue)|\(command.commandSeq)|\(command.sessionId.uuidString)|\(command.utteranceId.uuidString)|\(detail)"
|
||||
guard signature != lastIgnoredCommandSignature else { return }
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
// ASRSettingsCard.swift
|
||||
// OSGKeyboard · Main App
|
||||
//
|
||||
// Cloud ASR credentials — independent from the polish LLM card.
|
||||
|
||||
import SwiftUI
|
||||
import OSGKeyboardShared
|
||||
|
||||
struct ASRSettingsCard: View {
|
||||
@Environment(\.themePalette) private var palette: ThemePalette
|
||||
|
||||
@ObservedObject var config: ProviderConfig
|
||||
@State private var showKey: Bool = false
|
||||
@State private var testStatus: TestStatus = .idle
|
||||
|
||||
private enum TestStatus: Equatable {
|
||||
case idle
|
||||
case running
|
||||
case success
|
||||
case failure(String)
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 0) {
|
||||
if CloudASRModelCatalog.strategy(for: config.asrProviderId) == .prompt {
|
||||
field(
|
||||
title: AppL10n.string("api.baseUrl"),
|
||||
placeholder: "https://api.openai.com/v1",
|
||||
text: $config.asrBaseURL,
|
||||
autocap: false
|
||||
)
|
||||
Divider().background(palette.divider)
|
||||
}
|
||||
keyField
|
||||
Divider().background(palette.divider)
|
||||
field(
|
||||
title: AppL10n.string("settings.asr.model"),
|
||||
placeholder: CloudASRModelCatalog.defaultModel(for: config.asrProviderId),
|
||||
text: $config.asrModel,
|
||||
autocap: false
|
||||
)
|
||||
if let url = LLMProvider.provider(id: config.asrProviderId).apiKeyURL {
|
||||
Divider().background(palette.divider)
|
||||
Button {
|
||||
UIApplication.shared.open(url)
|
||||
} label: {
|
||||
HStack {
|
||||
Text("api.getKey")
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
Spacer()
|
||||
Image(systemName: "arrow.up.right.square")
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.frame(minHeight: SettingsListMetrics.singleLineMinHeight)
|
||||
.contentShape(Rectangle())
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
}
|
||||
Divider().background(palette.divider)
|
||||
testConnectionRow
|
||||
}
|
||||
.background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: Radius.large, style: .continuous)
|
||||
.stroke(palette.divider, lineWidth: 0.5)
|
||||
)
|
||||
}
|
||||
|
||||
private var keyField: some View {
|
||||
VStack(alignment: .leading, spacing: 6) {
|
||||
HStack {
|
||||
Text("api.key")
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
Spacer()
|
||||
Button(action: { showKey.toggle() }) {
|
||||
Image(systemName: showKey ? "eye.slash.fill" : "eye.fill")
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
}
|
||||
Group {
|
||||
if showKey {
|
||||
TextField("sk-…", text: $config.asrApiKey)
|
||||
} else {
|
||||
SecureField("sk-…", text: $config.asrApiKey)
|
||||
}
|
||||
}
|
||||
.keyboardType(.asciiCapable)
|
||||
.textInputAutocapitalization(.never)
|
||||
.autocorrectionDisabled(true)
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
}
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.frame(minHeight: SettingsListMetrics.doubleLineMinHeight, alignment: .center)
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private func field(
|
||||
title: String,
|
||||
placeholder: String,
|
||||
text: Binding<String>,
|
||||
autocap: Bool
|
||||
) -> some View {
|
||||
VStack(alignment: .leading, spacing: 6) {
|
||||
Text(title)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
TextField(placeholder, text: text)
|
||||
.keyboardType(.asciiCapable)
|
||||
.autocorrectionDisabled(true)
|
||||
.textInputAutocapitalization(autocap ? .sentences : .never)
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
}
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.frame(minHeight: SettingsListMetrics.doubleLineMinHeight, alignment: .center)
|
||||
}
|
||||
|
||||
private var testConnectionRow: some View {
|
||||
VStack(alignment: .leading, spacing: 6) {
|
||||
HStack {
|
||||
Text("settings.asr.testConnection")
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
Spacer()
|
||||
Button(action: runTest) {
|
||||
Group {
|
||||
if testStatus == .running {
|
||||
ProgressView().controlSize(.mini)
|
||||
} else {
|
||||
Text(testButtonLabel)
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(testTint)
|
||||
}
|
||||
}
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
.disabled(testStatus == .running)
|
||||
}
|
||||
if let detail = testDetail {
|
||||
Text(detail)
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(testTint)
|
||||
.lineLimit(3)
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.padding(.vertical, Spacing.xs)
|
||||
.frame(minHeight: SettingsListMetrics.singleLineMinHeight, alignment: .center)
|
||||
}
|
||||
|
||||
private var testButtonLabel: String {
|
||||
switch testStatus {
|
||||
case .idle: return AppL10n.string("api.test.idle")
|
||||
case .running: return AppL10n.string("api.test.running")
|
||||
case .success: return AppL10n.string("api.test.success")
|
||||
case .failure: return AppL10n.string("api.test.failure")
|
||||
}
|
||||
}
|
||||
|
||||
private var testTint: Color {
|
||||
switch testStatus {
|
||||
case .idle, .running: return palette.accent
|
||||
case .success: return palette.accent
|
||||
case .failure: return palette.danger
|
||||
}
|
||||
}
|
||||
|
||||
private var testDetail: String? {
|
||||
switch testStatus {
|
||||
case .idle, .running, .success: return nil
|
||||
case .failure(let message): return message
|
||||
}
|
||||
}
|
||||
|
||||
private func runTest() {
|
||||
testStatus = .running
|
||||
let store = AppGroupStore()
|
||||
let client = CloudASRClientFactory.make(store: store)
|
||||
Task {
|
||||
do {
|
||||
try await client.prepare(dictionary: store.personalDictionary)
|
||||
let samples = [Float](repeating: 0.01, count: 16_000)
|
||||
_ = try await client.transcribe(
|
||||
samples: samples,
|
||||
sampleRate: 16_000,
|
||||
locale: Locale(identifier: store.localeId == "auto" ? "zh-CN" : store.localeId),
|
||||
dictionary: store.personalDictionary
|
||||
)
|
||||
testStatus = .success
|
||||
} catch CloudASRError.noAPIKey {
|
||||
testStatus = .failure(AppL10n.string("api.test.missing"))
|
||||
} catch let error as CloudASRError {
|
||||
testStatus = .failure(error.localizedDescription ?? "\(error)")
|
||||
} catch {
|
||||
testStatus = .failure((error as? LocalizedError)?.errorDescription ?? "\(error)")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -39,6 +39,18 @@ enum AppTab: Int, CaseIterable {
|
||||
case .settings: return "tab.settings"
|
||||
}
|
||||
}
|
||||
|
||||
/// Sidebar label for iPad `NavigationSplitView` (SF Symbol + title).
|
||||
var sidebarTitle: LocalizedStringKey { accessibilityKey }
|
||||
|
||||
var sidebarSystemImage: String {
|
||||
switch self {
|
||||
case .keyboard: return "house"
|
||||
case .history: return "clock.arrow.circlepath"
|
||||
case .dictionary: return "character.book.closed"
|
||||
case .settings: return "gearshape"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct MinimalTabBar: View {
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
// WideLayoutComponents.swift
|
||||
// OSGKeyboard · Main App
|
||||
//
|
||||
// Reusable layout pieces for iPad / regular-width surfaces. Styled with the
|
||||
// shared design tokens so the wide Home dashboard can mirror the macOS shell
|
||||
// without pulling in AppKit-only types from OSGKeyboardMac.
|
||||
|
||||
import SwiftUI
|
||||
import OSGKeyboardShared
|
||||
|
||||
// MARK: - Layout metrics
|
||||
|
||||
/// Fixed metrics that keep wide surfaces on the same grid as the macOS app.
|
||||
enum WideLayoutMetrics {
|
||||
static let sidebarWidth: CGFloat = 240
|
||||
static let sidebarInset: CGFloat = Spacing.md
|
||||
static let sidebarContentInset: CGFloat = sidebarInset + Spacing.sm
|
||||
static let pageHorizontalInset: CGFloat = 40
|
||||
static let dictationCanvasMinHeight: CGFloat = 120
|
||||
}
|
||||
|
||||
// MARK: - Card container
|
||||
|
||||
/// Elevated surface used for stat tiles and the dictation canvas.
|
||||
struct WideCard<Content: View>: View {
|
||||
@Environment(\.themePalette) private var palette
|
||||
var padding: CGFloat = Spacing.md
|
||||
var cornerRadius: CGFloat = Radius.medium
|
||||
@ViewBuilder var content: () -> Content
|
||||
|
||||
var body: some View {
|
||||
let shape = RoundedRectangle(cornerRadius: cornerRadius, style: .continuous)
|
||||
|
||||
content()
|
||||
.padding(padding)
|
||||
.background(palette.surface, in: shape)
|
||||
.overlay(
|
||||
shape.stroke(palette.divider, lineWidth: 0.5)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Stat tile
|
||||
|
||||
struct WideStatCard: View {
|
||||
@Environment(\.themePalette) private var palette
|
||||
let title: String
|
||||
let value: String
|
||||
let caption: String
|
||||
var systemImage: String?
|
||||
var accent: Bool = false
|
||||
/// Hero metric: wide horizontal layout for the primary word count.
|
||||
var prominent: Bool = false
|
||||
|
||||
var body: some View {
|
||||
WideCard(padding: Spacing.md) {
|
||||
if prominent {
|
||||
prominentBody
|
||||
} else {
|
||||
compactBody
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var compactBody: some View {
|
||||
VStack(alignment: .leading, spacing: Spacing.xs) {
|
||||
HStack {
|
||||
Text(title.uppercased())
|
||||
.font(TypeStyle.caption2)
|
||||
.tracking(0.6)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
Spacer()
|
||||
if let systemImage {
|
||||
Image(systemName: systemImage)
|
||||
.font(.system(size: 13, weight: .semibold))
|
||||
.foregroundStyle(accent ? palette.accent : palette.textTertiary)
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
}
|
||||
}
|
||||
Text(value)
|
||||
.font(TypeStyle.title2)
|
||||
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
.minimumScaleFactor(0.7)
|
||||
.contentTransition(.numericText())
|
||||
.animation(Motion.soft, value: value)
|
||||
Text(caption)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
|
||||
private var prominentBody: some View {
|
||||
HStack(spacing: Spacing.md) {
|
||||
if let systemImage {
|
||||
ZStack {
|
||||
Circle()
|
||||
.fill(palette.accentMuted)
|
||||
.frame(width: 44, height: 44)
|
||||
Image(systemName: systemImage)
|
||||
.font(.system(size: 17, weight: .semibold))
|
||||
.foregroundStyle(palette.accent)
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
}
|
||||
}
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Text(title.uppercased())
|
||||
.font(TypeStyle.caption2)
|
||||
.tracking(0.6)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
Text(caption)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
Spacer(minLength: Spacing.md)
|
||||
Text(value)
|
||||
.font(.system(size: 34, weight: .bold))
|
||||
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
.minimumScaleFactor(0.6)
|
||||
.contentTransition(.numericText())
|
||||
.animation(Motion.soft, value: value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Home stats cluster
|
||||
|
||||
/// Dashboard-style stat cluster for the wide Home layout.
|
||||
struct WideHomeStatsCluster: View {
|
||||
@ObservedObject private var stats = UsageStatisticsStore.shared
|
||||
@ObservedObject private var config = ProviderConfig.shared
|
||||
|
||||
@State private var dictionaryCount = 0
|
||||
|
||||
private var language: AppUILanguage { config.uiLanguage }
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: Spacing.md) {
|
||||
WideStatCard(
|
||||
title: AppL10n.string("home.stats.dictationCharacters", language: language),
|
||||
value: UsageStatisticsStore.formatCount(
|
||||
stats.dictationCharacterCount,
|
||||
language: language
|
||||
),
|
||||
caption: AppL10n.string("home.wide.stat.transcribed", language: language),
|
||||
systemImage: "text.alignleft",
|
||||
accent: true,
|
||||
prominent: true
|
||||
)
|
||||
|
||||
HStack(spacing: Spacing.md) {
|
||||
WideStatCard(
|
||||
title: AppL10n.string("home.stats.dictationDuration", language: language),
|
||||
value: UsageStatisticsStore.formatDuration(
|
||||
stats.dictationDurationSeconds,
|
||||
language: language
|
||||
),
|
||||
caption: AppL10n.string("home.wide.stat.cumulativeDuration", language: language),
|
||||
systemImage: "waveform"
|
||||
)
|
||||
WideStatCard(
|
||||
title: AppL10n.string("home.stats.translationCharacters", language: language),
|
||||
value: UsageStatisticsStore.formatCount(
|
||||
stats.translationCharacterCount,
|
||||
language: language
|
||||
),
|
||||
caption: AppL10n.string("home.wide.stat.cumulativeTranslation", language: language),
|
||||
systemImage: "character.bubble"
|
||||
)
|
||||
WideStatCard(
|
||||
title: AppL10n.string("home.stats.dictionaryEntries", language: language),
|
||||
value: UsageStatisticsStore.formatCount(
|
||||
dictionaryCount,
|
||||
language: language
|
||||
),
|
||||
caption: AppL10n.string("home.wide.stat.customTerms", language: language),
|
||||
systemImage: "character.book.closed"
|
||||
)
|
||||
}
|
||||
}
|
||||
.onAppear(perform: refreshDictionaryCount)
|
||||
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
|
||||
refreshDictionaryCount()
|
||||
}
|
||||
.onReceive(NotificationCenter.default.publisher(for: .personalDictionaryDidSyncFromCloud)) { _ in
|
||||
refreshDictionaryCount()
|
||||
}
|
||||
.onReceive(NotificationCenter.default.publisher(for: .usageStatisticsDidSyncFromCloud)) { _ in
|
||||
stats.reloadFromDisk()
|
||||
}
|
||||
}
|
||||
|
||||
private func refreshDictionaryCount() {
|
||||
dictionaryCount = AppGroupStore().personalDictionary.entries.count
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,7 @@ import UIKit
|
||||
struct HomeView: View {
|
||||
@Environment(\.themePalette) private var palette: ThemePalette
|
||||
@Environment(\.scenePhase) private var scenePhase
|
||||
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
|
||||
|
||||
@ObservedObject private var config = ProviderConfig.shared
|
||||
@EnvironmentObject private var flowManager: FlowSessionManager
|
||||
@@ -24,6 +25,10 @@ struct HomeView: View {
|
||||
@State private var micStatus = AppPermissions.micStatus
|
||||
@State private var speechStatus = AppPermissions.speechStatus
|
||||
|
||||
private var usesWideLayout: Bool {
|
||||
horizontalSizeClass == .regular
|
||||
}
|
||||
|
||||
private var sessionIsLive: Bool {
|
||||
flowManager.isActive || flowManager.isStarting
|
||||
}
|
||||
@@ -52,6 +57,32 @@ struct HomeView: View {
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
Group {
|
||||
if usesWideLayout {
|
||||
wideBody
|
||||
} else {
|
||||
phoneBody
|
||||
}
|
||||
}
|
||||
.onAppear {
|
||||
refreshPermissionStatuses()
|
||||
}
|
||||
.onChange(of: scenePhase) { _, phase in
|
||||
guard phase == .active else { return }
|
||||
refreshPermissionStatuses()
|
||||
}
|
||||
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
|
||||
refreshPermissionStatuses()
|
||||
}
|
||||
.onChange(of: previewFocused) { _, focused in
|
||||
guard focused else { return }
|
||||
Task { await flowManager.refreshForInlineKeyboardFocus() }
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Phone layout
|
||||
|
||||
private var phoneBody: some View {
|
||||
GeometryReader { geo in
|
||||
let gradientHeight = geo.size.height * 0.30 + geo.safeAreaInsets.top
|
||||
|
||||
@@ -98,20 +129,63 @@ struct HomeView: View {
|
||||
}
|
||||
}
|
||||
}
|
||||
.onAppear {
|
||||
refreshPermissionStatuses()
|
||||
}
|
||||
.onChange(of: scenePhase) { _, phase in
|
||||
guard phase == .active else { return }
|
||||
refreshPermissionStatuses()
|
||||
|
||||
// MARK: - Wide layout (iPad / regular width)
|
||||
|
||||
private var wideBody: some View {
|
||||
VStack(spacing: 0) {
|
||||
VStack(alignment: .leading, spacing: Spacing.lg) {
|
||||
wideHeroHeader
|
||||
|
||||
WideHomeStatsCluster()
|
||||
|
||||
if showsFlowSessionExtras {
|
||||
flowSessionExtras
|
||||
}
|
||||
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
|
||||
refreshPermissionStatuses()
|
||||
|
||||
widePreviewStage
|
||||
}
|
||||
.onChange(of: previewFocused) { _, focused in
|
||||
guard focused else { return }
|
||||
Task { await flowManager.refreshForInlineKeyboardFocus() }
|
||||
.padding(.horizontal, WideLayoutMetrics.pageHorizontalInset)
|
||||
.padding(.top, Spacing.sm)
|
||||
.padding(.bottom, Spacing.md)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
|
||||
}
|
||||
.background(palette.background)
|
||||
.contentShape(Rectangle())
|
||||
.onTapGesture {
|
||||
if previewFocused {
|
||||
previewFocused = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var wideHeroHeader: some View {
|
||||
VStack(alignment: .leading, spacing: Spacing.xs) {
|
||||
Text("home.wide.tagline")
|
||||
.font(.system(size: 30, weight: .semibold))
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineLimit(2)
|
||||
.minimumScaleFactor(0.85)
|
||||
|
||||
Text("home.wide.tagline.subtitle")
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
|
||||
private var widePreviewStage: some View {
|
||||
WideCard(padding: Spacing.md, cornerRadius: Radius.large) {
|
||||
previewFieldContent
|
||||
.frame(
|
||||
maxWidth: .infinity,
|
||||
minHeight: WideLayoutMetrics.dictationCanvasMinHeight,
|
||||
maxHeight: .infinity,
|
||||
alignment: .topLeading
|
||||
)
|
||||
}
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top)
|
||||
}
|
||||
|
||||
private func refreshPermissionStatuses() {
|
||||
@@ -187,7 +261,18 @@ struct HomeView: View {
|
||||
.foregroundStyle(palette.warning)
|
||||
.lineLimit(1)
|
||||
.minimumScaleFactor(0.85)
|
||||
} else if flowManager.isUtteranceRecording {
|
||||
Text("home.flow.recording")
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
} else if flowManager.isUtteranceProcessing {
|
||||
Text("home.flow.processing")
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
} else if flowManager.isActive,
|
||||
FlowSessionBridge.isHostReady(),
|
||||
let expires = flowManager.sessionExpiresAt {
|
||||
Text("home.flow.label")
|
||||
.font(TypeStyle.caption2)
|
||||
@@ -322,10 +407,14 @@ struct HomeView: View {
|
||||
|
||||
private var flowStatusColor: Color {
|
||||
if needsCloudSetup { return palette.warning }
|
||||
if flowManager.isActive { return palette.accent }
|
||||
if flowManager.isUtteranceRecording { return palette.accent }
|
||||
if flowManager.isUtteranceProcessing { return palette.accent }
|
||||
if flowManager.isActive, FlowSessionBridge.isHostReady() { return palette.accent }
|
||||
if flowManager.isStarting { return palette.accent }
|
||||
if needsPermissionSetup { return palette.warning }
|
||||
if flowManager.sessionWarning != nil { return palette.warning }
|
||||
// Active but not host-ready (e.g. mid-utterance / audio proof) — amber.
|
||||
if flowManager.isActive { return palette.warning }
|
||||
return palette.textTertiary
|
||||
}
|
||||
|
||||
@@ -337,21 +426,26 @@ struct HomeView: View {
|
||||
if flowManager.isStarting {
|
||||
return AppL10n.string("home.flow.starting")
|
||||
}
|
||||
if flowManager.isActive {
|
||||
if flowManager.isUtteranceRecording {
|
||||
return AppL10n.string("home.flow.recording")
|
||||
}
|
||||
if flowManager.isUtteranceProcessing {
|
||||
return AppL10n.string("home.flow.processing")
|
||||
}
|
||||
if flowManager.isActive, FlowSessionBridge.isHostReady() {
|
||||
return AppL10n.string("home.flow.label")
|
||||
}
|
||||
if flowManager.isActive {
|
||||
// Session flag is up but the ready contract is not — do not lie.
|
||||
return AppL10n.string("home.flow.notReady")
|
||||
}
|
||||
return AppL10n.string("home.flow.inactive")
|
||||
}
|
||||
|
||||
// MARK: - Preview field
|
||||
|
||||
private var previewField: some View {
|
||||
TextField("home.preview.placeholder", text: $previewText, axis: .vertical)
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.tint(palette.accent)
|
||||
.focused($previewFocused)
|
||||
.lineLimit(1...100)
|
||||
previewFieldContent
|
||||
.frame(maxWidth: .infinity, minHeight: 180, maxHeight: .infinity, alignment: .topLeading)
|
||||
.padding(Spacing.md)
|
||||
.background(palette.surfaceElevated, in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
|
||||
@@ -359,11 +453,19 @@ struct HomeView: View {
|
||||
RoundedRectangle(cornerRadius: Radius.large, style: .continuous)
|
||||
.stroke(previewFocused ? palette.dividerStrong : palette.dividerStrong.opacity(0.75), lineWidth: 1)
|
||||
)
|
||||
// TextField only hit-tests the text line(s); expand taps to the full card.
|
||||
.contentShape(RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
|
||||
.onTapGesture { previewFocused = true }
|
||||
}
|
||||
|
||||
private var previewFieldContent: some View {
|
||||
TextField("home.preview.placeholder", text: $previewText, axis: .vertical)
|
||||
.font(TypeStyle.body)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.tint(palette.accent)
|
||||
.focused($previewFocused)
|
||||
.lineLimit(1...100)
|
||||
}
|
||||
|
||||
private var engineStatusLine: some View {
|
||||
Text(
|
||||
EngineServiceLabel.summary(
|
||||
|
||||
@@ -39,14 +39,18 @@ struct MainAppRoot: View {
|
||||
.animation(.easeInOut(duration: 0.2), value: flowManager.coldStartContext != nil)
|
||||
.onAppear {
|
||||
flowManager.setAppForeground(scenePhase == .active)
|
||||
flowManager.activateOnForeground()
|
||||
AppCloudSync.shared.startObservingExternalChanges()
|
||||
// Registering here also flushes any URL buffered during a cold
|
||||
// launch (the keyboard → app `startflow` handoff arrives via the
|
||||
// scene delegate before this view is on screen).
|
||||
// Register the URL handler BEFORE the foreground auto-start.
|
||||
// Registering flushes any URL buffered during a cold launch (the
|
||||
// keyboard → app `startflow` handoff arrives via the scene
|
||||
// delegate before this view is on screen), so a cold start takes
|
||||
// the cold-start path first and `activateOnForeground()`'s plain
|
||||
// start then no-ops on the isStarting guard — instead of two
|
||||
// start bodies racing each other on the main actor.
|
||||
AppOpenURLRouter.shared.register { url in
|
||||
handleIncomingURL(url)
|
||||
}
|
||||
flowManager.activateOnForeground()
|
||||
AppCloudSync.shared.startObservingExternalChanges()
|
||||
Task {
|
||||
await AppCloudSync.shared.pullAllIfEnabled()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
// MainSplitView.swift
|
||||
// OSGKeyboard · Main App
|
||||
//
|
||||
// iPad / regular-width shell: sidebar navigation + detail workspace.
|
||||
// Mirrors the macOS `NavigationSplitView` structure while keeping iOS tabs
|
||||
// and Flow session behaviour unchanged underneath.
|
||||
|
||||
import SwiftUI
|
||||
import OSGKeyboardShared
|
||||
import UIKit
|
||||
|
||||
struct MainSplitView: View {
|
||||
@Binding var selection: AppTab
|
||||
|
||||
@Environment(\.themePalette) private var palette
|
||||
|
||||
@State private var columnVisibility: NavigationSplitViewVisibility = .all
|
||||
|
||||
var body: some View {
|
||||
NavigationSplitView(columnVisibility: $columnVisibility) {
|
||||
sidebar
|
||||
.navigationSplitViewColumnWidth(WideLayoutMetrics.sidebarWidth)
|
||||
} detail: {
|
||||
detail
|
||||
}
|
||||
.navigationSplitViewStyle(.balanced)
|
||||
.background(palette.background)
|
||||
// No floating dock in split mode — child scroll views should not
|
||||
// reserve bottom clearance for the phone tab bar.
|
||||
.environment(\.isTabBarVisible, false)
|
||||
}
|
||||
|
||||
// MARK: - Sidebar
|
||||
|
||||
private var sidebar: some View {
|
||||
VStack(spacing: 0) {
|
||||
brandHeader
|
||||
VStack(spacing: 4) {
|
||||
ForEach(AppTab.allCases, id: \.rawValue) { tab in
|
||||
WideSidebarRow(
|
||||
tab: tab,
|
||||
isSelected: selection == tab
|
||||
) {
|
||||
withAnimation(Motion.soft) { selection = tab }
|
||||
}
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, WideLayoutMetrics.sidebarInset)
|
||||
Spacer()
|
||||
devicesFooter
|
||||
}
|
||||
.background(palette.background)
|
||||
}
|
||||
|
||||
private var brandHeader: some View {
|
||||
HStack {
|
||||
Image("osglogo")
|
||||
.resizable()
|
||||
.scaledToFit()
|
||||
.frame(height: 28)
|
||||
.accessibilityLabel("OSGKeyboard")
|
||||
Spacer()
|
||||
}
|
||||
.padding(.leading, WideLayoutMetrics.sidebarContentInset)
|
||||
.padding(.trailing, WideLayoutMetrics.sidebarInset)
|
||||
.padding(.top, Spacing.lg)
|
||||
.padding(.bottom, Spacing.md)
|
||||
}
|
||||
|
||||
private var devicesFooter: some View {
|
||||
Label("home.wide.devices", systemImage: "ipad.and.iphone")
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(.horizontal, WideLayoutMetrics.sidebarInset + Spacing.sm)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
}
|
||||
|
||||
// MARK: - Detail
|
||||
|
||||
private var detail: some View {
|
||||
VStack(spacing: 0) {
|
||||
MainTabContent(tab: selection)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
.id(selection)
|
||||
.transition(.opacity)
|
||||
WideStatusFooter()
|
||||
}
|
||||
.background(palette.background)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Sidebar row
|
||||
|
||||
private struct WideSidebarRow: View {
|
||||
let tab: AppTab
|
||||
let isSelected: Bool
|
||||
let action: () -> Void
|
||||
|
||||
@Environment(\.themePalette) private var palette
|
||||
|
||||
var body: some View {
|
||||
Button(action: action) {
|
||||
Label(tab.sidebarTitle, systemImage: tab.sidebarSystemImage)
|
||||
.font(.system(size: 13, weight: isSelected ? .semibold : .regular))
|
||||
.foregroundStyle(isSelected ? palette.accent : palette.textPrimary)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(.horizontal, Spacing.sm)
|
||||
.padding(.vertical, 7)
|
||||
.background(
|
||||
rowBackground,
|
||||
in: RoundedRectangle(cornerRadius: 7, style: .continuous)
|
||||
)
|
||||
.contentShape(Rectangle())
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
.animation(Motion.quick, value: isSelected)
|
||||
.accessibilityAddTraits(isSelected ? .isSelected : [])
|
||||
}
|
||||
|
||||
private var rowBackground: Color {
|
||||
isSelected ? palette.accentMuted : .clear
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Status footer
|
||||
|
||||
/// Quiet bottom strip: engine mode + translation target + Flow readiness.
|
||||
private struct WideStatusFooter: View {
|
||||
@Environment(\.themePalette) private var palette
|
||||
@ObservedObject private var config = ProviderConfig.shared
|
||||
@EnvironmentObject private var flowManager: FlowSessionManager
|
||||
|
||||
@State private var micStatus = AppPermissions.micStatus
|
||||
@State private var speechStatus = AppPermissions.speechStatus
|
||||
|
||||
private var needsCloudSetup: Bool {
|
||||
!config.isLocalEngine && !config.isConfigured
|
||||
}
|
||||
|
||||
private var needsPermissionSetup: Bool {
|
||||
micStatus != .granted || speechStatus != .granted
|
||||
}
|
||||
|
||||
private var canManuallyStartSession: Bool {
|
||||
!flowManager.isActive && !flowManager.isStarting && !needsPermissionSetup
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: Spacing.sm) {
|
||||
Spacer()
|
||||
Label(
|
||||
config.engineMode == "cloud" ? "home.wide.mode.cloud" : "home.wide.mode.local",
|
||||
systemImage: config.engineMode == "cloud" ? "cloud" : "cpu"
|
||||
)
|
||||
.contentTransition(.opacity)
|
||||
|
||||
Text("·")
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.5))
|
||||
|
||||
Label(
|
||||
translationLabel,
|
||||
systemImage: "translate"
|
||||
)
|
||||
.contentTransition(.opacity)
|
||||
|
||||
Text("·")
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.5))
|
||||
|
||||
flowStatusControl
|
||||
}
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.labelStyle(.titleAndIcon)
|
||||
.padding(.horizontal, WideLayoutMetrics.pageHorizontalInset)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
.animation(Motion.quick, value: config.engineMode)
|
||||
.animation(Motion.quick, value: config.translationTargetLocaleId)
|
||||
.animation(Motion.soft, value: flowManager.isActive)
|
||||
.onAppear { refreshPermissionStatuses() }
|
||||
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)) { _ in
|
||||
refreshPermissionStatuses()
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var flowStatusControl: some View {
|
||||
HStack(spacing: Spacing.xs) {
|
||||
Circle()
|
||||
.fill(flowStatusColor)
|
||||
.frame(width: 6, height: 6)
|
||||
|
||||
if needsCloudSetup {
|
||||
Text("home.flow.notReady")
|
||||
.foregroundStyle(palette.warning)
|
||||
} else if flowManager.isUtteranceRecording {
|
||||
Text("home.flow.recording")
|
||||
} else if flowManager.isUtteranceProcessing {
|
||||
Text("home.flow.processing")
|
||||
} else if flowManager.isActive,
|
||||
FlowSessionBridge.isHostReady(),
|
||||
let expires = flowManager.sessionExpiresAt {
|
||||
Text("home.flow.label")
|
||||
Text(":")
|
||||
Text(expires, style: .timer)
|
||||
.monospacedDigit()
|
||||
} else {
|
||||
Text(flowStatusLabel)
|
||||
}
|
||||
|
||||
if flowManager.isActive {
|
||||
Button {
|
||||
flowManager.endSession()
|
||||
} label: {
|
||||
Text("home.flow.endShort")
|
||||
.foregroundStyle(palette.accent)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
} else if canManuallyStartSession && !needsCloudSetup {
|
||||
Button {
|
||||
flowManager.activateOnForeground()
|
||||
} label: {
|
||||
Text("home.flow.startShort")
|
||||
.foregroundStyle(palette.accent)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func refreshPermissionStatuses() {
|
||||
micStatus = AppPermissions.micStatus
|
||||
speechStatus = AppPermissions.speechStatus
|
||||
}
|
||||
|
||||
private var translationLabel: String {
|
||||
let resolved = TranslationLanguageCatalog.resolve(config.translationTargetLocaleId)
|
||||
if TranslationLanguageCatalog.isOff(resolved.id) {
|
||||
return AppL10n.string("keyboard.translation.offMenu", language: config.uiLanguage)
|
||||
}
|
||||
return resolved.nativeName
|
||||
}
|
||||
|
||||
private var flowStatusColor: Color {
|
||||
if !config.isLocalEngine && !config.isConfigured { return palette.warning }
|
||||
if flowManager.isUtteranceRecording || flowManager.isUtteranceProcessing {
|
||||
return palette.accent
|
||||
}
|
||||
if flowManager.isActive, FlowSessionBridge.isHostReady() { return palette.accent }
|
||||
if flowManager.isStarting { return palette.accent }
|
||||
if flowManager.isActive { return palette.warning }
|
||||
return palette.textTertiary
|
||||
}
|
||||
|
||||
private var flowStatusLabel: LocalizedStringKey {
|
||||
if !config.isLocalEngine && !config.isConfigured {
|
||||
return "home.flow.notReady"
|
||||
}
|
||||
if flowManager.isStarting {
|
||||
return "home.flow.starting"
|
||||
}
|
||||
if flowManager.isUtteranceRecording {
|
||||
return "home.flow.recording"
|
||||
}
|
||||
if flowManager.isUtteranceProcessing {
|
||||
return "home.flow.processing"
|
||||
}
|
||||
if flowManager.isActive, FlowSessionBridge.isHostReady() {
|
||||
return "home.flow.label"
|
||||
}
|
||||
if flowManager.isActive {
|
||||
return "home.flow.notReady"
|
||||
}
|
||||
return "home.flow.inactive"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
// MainTabContent.swift
|
||||
// OSGKeyboard · Main App
|
||||
//
|
||||
// Shared tab destination switcher used by both the phone dock and the iPad
|
||||
// split-view detail column.
|
||||
|
||||
import SwiftUI
|
||||
import OSGKeyboardShared
|
||||
|
||||
struct MainTabContent: View {
|
||||
let tab: AppTab
|
||||
|
||||
var body: some View {
|
||||
switch tab {
|
||||
case .keyboard:
|
||||
HomeView()
|
||||
case .history:
|
||||
HistoryView()
|
||||
case .dictionary:
|
||||
PersonalDictionaryView()
|
||||
case .settings:
|
||||
SettingsView(presentation: .tab)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,27 +6,35 @@ import OSGKeyboardShared
|
||||
|
||||
struct MainTabView: View {
|
||||
@Environment(\.themePalette) private var palette: ThemePalette
|
||||
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
|
||||
@EnvironmentObject private var flowManager: FlowSessionManager
|
||||
|
||||
@State private var tab: AppTab = .keyboard
|
||||
@State private var isTabBarHidden = false
|
||||
|
||||
private var usesSplitLayout: Bool {
|
||||
horizontalSizeClass == .regular
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
Group {
|
||||
if usesSplitLayout {
|
||||
MainSplitView(selection: $tab)
|
||||
} else {
|
||||
phoneTabLayout
|
||||
}
|
||||
}
|
||||
.background(palette.background)
|
||||
.ignoresSafeArea(.keyboard, edges: .bottom)
|
||||
}
|
||||
|
||||
// MARK: - Phone layout
|
||||
|
||||
private var phoneTabLayout: some View {
|
||||
ZStack(alignment: .bottom) {
|
||||
palette.background.ignoresSafeArea()
|
||||
|
||||
Group {
|
||||
switch tab {
|
||||
case .keyboard:
|
||||
HomeView()
|
||||
case .history:
|
||||
HistoryView()
|
||||
case .dictionary:
|
||||
PersonalDictionaryView()
|
||||
case .settings:
|
||||
SettingsView(presentation: .tab)
|
||||
}
|
||||
}
|
||||
MainTabContent(tab: tab)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
.environment(\.isTabBarVisible, !isTabBarHidden)
|
||||
.safeAreaInset(edge: .bottom, spacing: 0) {
|
||||
@@ -45,8 +53,5 @@ struct MainTabView: View {
|
||||
.transition(.move(edge: .bottom).combined(with: .opacity))
|
||||
}
|
||||
}
|
||||
// Keep home card/input/tab layout fixed when system keyboard appears.
|
||||
// Let the keyboard overlay the content instead of pushing it.
|
||||
.ignoresSafeArea(.keyboard, edges: .bottom)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,17 +8,22 @@ struct ProviderPickerSection: View {
|
||||
@Environment(\.themePalette) private var palette: ThemePalette
|
||||
|
||||
@ObservedObject var config: ProviderConfig
|
||||
var role: CloudProviderRole = .polish
|
||||
|
||||
private var selectedProviderId: String {
|
||||
role == .asr ? config.asrProviderId : config.providerId
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
// v0.2.1 follow-up: filter out presets marked as
|
||||
// `isUserSelectable == false` (DeepSeek is local-engine only).
|
||||
let visiblePresets = LLMProvider.userSelectablePresets
|
||||
let visiblePresets = role == .asr
|
||||
? LLMProvider.asrSelectablePresets
|
||||
: LLMProvider.userSelectablePresets
|
||||
VStack(spacing: 0) {
|
||||
ForEach(Array(visiblePresets.enumerated()), id: \.element.id) { index, provider in
|
||||
Button {
|
||||
select(provider)
|
||||
} label: {
|
||||
row(provider, selected: provider.id == config.providerId)
|
||||
row(provider, selected: provider.id == selectedProviderId)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
if index < visiblePresets.count - 1 {
|
||||
@@ -35,7 +40,12 @@ struct ProviderPickerSection: View {
|
||||
|
||||
private func select(_ provider: LLMProvider) {
|
||||
withAnimation(Motion.quick) {
|
||||
switch role {
|
||||
case .polish:
|
||||
config.apply(preset: provider)
|
||||
case .asr:
|
||||
config.applyAsr(preset: provider)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -43,16 +43,11 @@ struct SettingsView: View {
|
||||
dictionaryAndPolishSection
|
||||
flowSessionSection
|
||||
engineSection
|
||||
// v0.2.1: hide provider/api card when the
|
||||
// local engine is active regardless of the
|
||||
// cloud-polish toggle. Local mode is
|
||||
// contractually ASR-only, so provider/model/
|
||||
// base URL/API key controls have no use —
|
||||
// and exposing them invites the user to fill
|
||||
// out a DeepSeek key they can't use.
|
||||
polishProviderSection
|
||||
polishApiSection
|
||||
if config.engineMode == "cloud" {
|
||||
providerSection
|
||||
apiSection
|
||||
asrProviderSection
|
||||
asrApiSection
|
||||
}
|
||||
if config.engineMode == "local" {
|
||||
localEngineSettingsSection
|
||||
@@ -253,22 +248,42 @@ struct SettingsView: View {
|
||||
}
|
||||
}
|
||||
|
||||
private var providerSection: some View {
|
||||
private var polishProviderSection: some View {
|
||||
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
|
||||
sectionHeader("settings.provider.title")
|
||||
ProviderPickerSection(config: config)
|
||||
sectionHeader("settings.polishProvider.title")
|
||||
Text("settings.polishProvider.subtitle")
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
ProviderPickerSection(config: config, role: .polish)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - API
|
||||
|
||||
private var apiSection: some View {
|
||||
private var asrProviderSection: some View {
|
||||
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
|
||||
sectionHeader("settings.api.title")
|
||||
sectionHeader("settings.asrProvider.title")
|
||||
Text("settings.asrProvider.subtitle")
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
ProviderPickerSection(config: config, role: .asr)
|
||||
}
|
||||
}
|
||||
|
||||
private var polishApiSection: some View {
|
||||
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
|
||||
sectionHeader("settings.polishApi.title")
|
||||
APISettingsCard(config: config)
|
||||
}
|
||||
}
|
||||
|
||||
private var asrApiSection: some View {
|
||||
VStack(alignment: .leading, spacing: SettingsListMetrics.sectionLabelSpacing) {
|
||||
sectionHeader("settings.asrApi.title")
|
||||
ASRSettingsCard(config: config)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Language helpers
|
||||
|
||||
/// Falls back to a static list while dynamic locales are loading.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/* Permission prompts — English */
|
||||
|
||||
"NSMicrophoneUsageDescription" = "OSGKeyboard uses the microphone for voice dictation and keeps a background audio session active while a voice session is running.";
|
||||
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard uses on-device speech recognition to transcribe your voice. Audio is processed on your device and is not uploaded for transcription.";
|
||||
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard transcribes your voice with on-device speech recognition by default. If you explicitly switch to a cloud engine in Settings, recordings are sent to the ASR provider you configure.";
|
||||
|
||||
@@ -98,11 +98,19 @@
|
||||
"settings.engine.local.ios26" = "Always on-device, no network.";
|
||||
"settings.engine.local.legacy" = "On-device ASR. Transcription only, no polish.";
|
||||
"settings.engine.cloud.title" = "Cloud recognition & polish";
|
||||
"settings.engine.cloud.subtitle" = "Cloud ASR (with your dictionary) + API polish. Audio is sent to your provider.";
|
||||
"settings.engine.cloud.subtitle" = "Cloud ASR and polish LLM are configured separately. Audio goes to your ASR provider.";
|
||||
"settings.engine.cloud.badge" = "Cloud engine";
|
||||
"settings.provider.title" = "Provider";
|
||||
"settings.provider.personalDictionaryBadge" = "Personal dictionary";
|
||||
"settings.provider.subtitle" = "Pick the LLM that polishes your dictation.";
|
||||
"settings.polishProvider.title" = "Text polish (LLM)";
|
||||
"settings.polishProvider.subtitle" = "Cleans up the transcript after recognition. Independent from the ASR provider.";
|
||||
"settings.polishApi.title" = "Polish API";
|
||||
"settings.asrProvider.title" = "Speech recognition (ASR)";
|
||||
"settings.asrProvider.subtitle" = "Transcribes your audio in cloud mode. Can differ from the polish provider.";
|
||||
"settings.asrApi.title" = "ASR API";
|
||||
"settings.asr.model" = "ASR model";
|
||||
"settings.asr.testConnection" = "Test ASR";
|
||||
"provider.openai" = "OpenAI";
|
||||
"provider.deepseek" = "DeepSeek";
|
||||
"provider.qwen" = "Qwen (DashScope)";
|
||||
@@ -161,8 +169,8 @@
|
||||
"settings.privacy.fullAccess.title" = "About Full Access";
|
||||
"settings.privacy.fullAccess.body" = "Full Access is required for the microphone and to read your API key. OSGKeyboard does not record or upload what you type with the keyboard.";
|
||||
"settings.privacy.cloud.body" = "In Cloud polish mode, transcribed text is sent to the API endpoint you configure (e.g. OpenAI or your own server). OSGKeyboard does not operate servers and does not store transcripts in the cloud.";
|
||||
"settings.privacy.cloud.alert.title" = "Third-party API";
|
||||
"settings.privacy.cloud.alert.message" = "Cloud polish sends transcribed text to the third-party API you configure. OSGKeyboard never stores data on our servers. Continue?";
|
||||
"settings.privacy.cloud.alert.title" = "Audio leaves your device";
|
||||
"settings.privacy.cloud.alert.message" = "The cloud engine uploads your voice recordings to the third-party ASR provider you configure, and sends the transcript to its API for polish. That provider's privacy policy applies. OSGKeyboard never stores data on our servers. Continue?";
|
||||
"settings.link.support" = "Help & Feedback";
|
||||
"settings.link.github" = "GitHub";
|
||||
"settings.support.footer" = "OSGKeyboard is open source. Report bugs and share ideas on GitHub.";
|
||||
@@ -287,6 +295,8 @@
|
||||
"home.flow.label" = "Ready";
|
||||
"home.flow.inactive" = "Voice session inactive";
|
||||
"home.flow.notReady" = "Not ready";
|
||||
"home.flow.recording" = "Recording…";
|
||||
"home.flow.processing" = "Processing…";
|
||||
"home.flow.hint" = "Switch to any app and tap the keyboard mic to dictate.";
|
||||
"home.setup.permission.mic" = "Microphone access is off — voice input won't work.";
|
||||
"home.setup.permission.speech" = "Speech recognition is off — voice input won't work.";
|
||||
@@ -307,6 +317,15 @@
|
||||
"home.stats.dictationCharacters" = "Dictation chars";
|
||||
"home.stats.translationCharacters" = "Translation chars";
|
||||
"home.stats.dictionaryEntries" = "Dictionary";
|
||||
"home.wide.tagline" = "Voice dictation, anywhere.";
|
||||
"home.wide.tagline.subtitle" = "Switch to any app and tap the keyboard mic to dictate.";
|
||||
"home.wide.stat.transcribed" = "Total dictated";
|
||||
"home.wide.stat.cumulativeDuration" = "Cumulative duration";
|
||||
"home.wide.stat.cumulativeTranslation" = "Cumulative translation";
|
||||
"home.wide.stat.customTerms" = "Custom terms";
|
||||
"home.wide.mode.cloud" = "Cloud";
|
||||
"home.wide.mode.local" = "On-device";
|
||||
"home.wide.devices" = "iPhone & iPad";
|
||||
"home.engine.unsupportedOS" = "Qwen3-ASR CoreML requires iOS 18+";
|
||||
"home.engine.warming" = "Loading ASR model into memory…";
|
||||
"home.engine.downloading" = "Downloading ASR model…";
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
/* 权限说明 — 简体中文 */
|
||||
|
||||
"NSMicrophoneUsageDescription" = "OSGKeyboard 使用麦克风进行语音听写,并在语音会话运行期间保持后台音频会话。";
|
||||
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard 使用设备端语音识别将你的语音转为文字。音频仅在设备上处理,不会上传用于转写。";
|
||||
"NSSpeechRecognitionUsageDescription" = "OSGKeyboard 默认使用设备端语音识别将你的语音转为文字。若你在设置中主动切换到云端引擎,录音会发送到你配置的识别服务商。";
|
||||
|
||||
@@ -98,11 +98,19 @@
|
||||
"settings.engine.local.ios26" = "全程在手机本地,不用联网";
|
||||
"settings.engine.local.legacy" = "端侧 ASR,仅转录,无润色。";
|
||||
"settings.engine.cloud.title" = "云端识别与润色";
|
||||
"settings.engine.cloud.subtitle" = "云端 ASR(含个性词库)+ API 润色,音频将发往第三方服务";
|
||||
"settings.engine.cloud.subtitle" = "云端 ASR 与润色 LLM 分开配置;音频发送至转写服务商。";
|
||||
"settings.engine.cloud.badge" = "云端引擎";
|
||||
"settings.provider.title" = "云端引擎";
|
||||
"settings.provider.personalDictionaryBadge" = "个性词库";
|
||||
"settings.provider.subtitle" = "选择 LLM 提供商。";
|
||||
"settings.polishProvider.title" = "文本润色(LLM)";
|
||||
"settings.polishProvider.subtitle" = "识别完成后整理文字,可与 ASR 服务商不同。";
|
||||
"settings.polishApi.title" = "润色接口";
|
||||
"settings.asrProvider.title" = "语音转写(ASR)";
|
||||
"settings.asrProvider.subtitle" = "云端模式下负责听写转文字,可与润色模型分开配置。";
|
||||
"settings.asrApi.title" = "转写接口";
|
||||
"settings.asr.model" = "ASR 模型";
|
||||
"settings.asr.testConnection" = "测试转写";
|
||||
"provider.openai" = "OpenAI";
|
||||
"provider.deepseek" = "DeepSeek";
|
||||
"provider.qwen" = "通义千问";
|
||||
@@ -161,8 +169,8 @@
|
||||
"settings.privacy.fullAccess.title" = "关于完全访问";
|
||||
"settings.privacy.fullAccess.body" = "用来调用麦克风和读取 API Key;不会读取或上传你的输入内容。";
|
||||
"settings.privacy.cloud.body" = "云端润色模式下,转写文字会发送到你配置的 API(如 OpenAI 或自建服务)。OSGKeyboard 不运营服务器,也不会把转写内容存到云端。";
|
||||
"settings.privacy.cloud.alert.title" = "第三方 API";
|
||||
"settings.privacy.cloud.alert.message" = "云端润色会把转写文字发到你配置的第三方 API。OSGKeyboard 不会在自有服务器上存储数据。是否继续?";
|
||||
"settings.privacy.cloud.alert.title" = "音频将离开你的设备";
|
||||
"settings.privacy.cloud.alert.message" = "云端引擎会把你的语音录音上传到你配置的第三方识别服务,并把转写文字发送到其 API 进行润色,适用该服务商的隐私政策。OSGKeyboard 不会在自有服务器上存储数据。是否继续?";
|
||||
"settings.link.support" = "帮助与反馈";
|
||||
"settings.link.github" = "GitHub";
|
||||
"settings.support.footer" = "OSGKeyboard 为开源项目,欢迎在 GitHub 提交问题与建议。";
|
||||
@@ -286,6 +294,8 @@
|
||||
"home.flow.label" = "就绪";
|
||||
"home.flow.inactive" = "语音会话未启动";
|
||||
"home.flow.notReady" = "未就绪";
|
||||
"home.flow.recording" = "录音中…";
|
||||
"home.flow.processing" = "处理中…";
|
||||
"home.flow.hint" = "切到别的 App,点键盘麦克风就能说。";
|
||||
"home.setup.permission.mic" = "麦克风还没授权,语音输入用不了。";
|
||||
"home.setup.permission.speech" = "语音识别还没授权,语音输入用不了。";
|
||||
@@ -306,6 +316,15 @@
|
||||
"home.stats.dictationCharacters" = "听写字数";
|
||||
"home.stats.translationCharacters" = "翻译字数";
|
||||
"home.stats.dictionaryEntries" = "个性词库";
|
||||
"home.wide.tagline" = "随处语音听写";
|
||||
"home.wide.tagline.subtitle" = "切换到任意 App,点键盘麦克风即可听写。";
|
||||
"home.wide.stat.transcribed" = "累计听写字数";
|
||||
"home.wide.stat.cumulativeDuration" = "累计听写时长";
|
||||
"home.wide.stat.cumulativeTranslation" = "累计翻译字数";
|
||||
"home.wide.stat.customTerms" = "自定义词条";
|
||||
"home.wide.mode.cloud" = "云端";
|
||||
"home.wide.mode.local" = "本机";
|
||||
"home.wide.devices" = "iPhone 与 iPad";
|
||||
"home.engine.unsupportedOS" = "Qwen3-ASR CoreML 需要 iOS 18 或更高版本";
|
||||
"home.engine.warming" = "正在加载语音识别模型…";
|
||||
"home.engine.downloading" = "正在下载语音识别模型…";
|
||||
|
||||
@@ -57,7 +57,11 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
// Voice-first keyboard — hide the misleading "English" subtitle in Settings.
|
||||
primaryLanguage = "mis"
|
||||
OSGLog.keyboardExt.info("viewDidLoad — extension booted")
|
||||
CustomLanguageModelManager.shared.prepareInBackgroundIfNeeded()
|
||||
// Deliberately NO CustomLanguageModelManager prewarm here: the
|
||||
// extension never runs ASR (the host app owns the microphone and
|
||||
// the SpeechAnalyzer pipeline), and compiling/caching an LM inside
|
||||
// the keyboard's ~60 MB jetsam budget risks the system killing the
|
||||
// keyboard outright. The host app prewarms it on session start.
|
||||
setNeedsUpdateOfScreenEdgesDeferringSystemGestures()
|
||||
installKeyboardHeight()
|
||||
configureDictationBehavior()
|
||||
@@ -84,6 +88,7 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
setNeedsUpdateOfScreenEdgesDeferringSystemGestures()
|
||||
configureDictationBehavior()
|
||||
KeyboardSetupBridge.markExtensionAppearance(hasFullAccess: hasFullAccess)
|
||||
state.debugHasFullAccess = hasFullAccess
|
||||
flowCoordinator.refreshSessionState()
|
||||
flowCoordinator.startSessionMonitor()
|
||||
configSync.syncOnboardingStateFromAppGroup()
|
||||
@@ -126,6 +131,7 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
textInserter = KeyboardTextInserter(
|
||||
state: state,
|
||||
insertText: { [weak self] text in self?.textDocumentProxy.insertText(text) },
|
||||
contextBeforeInput: { [weak self] in self?.textDocumentProxy.documentContextBeforeInput },
|
||||
scheduleAutoClearError: { [weak self] in self?.scheduleAutoClearError() }
|
||||
)
|
||||
|
||||
|
||||
@@ -4,15 +4,17 @@
|
||||
// Opens the host app from the keyboard extension.
|
||||
//
|
||||
// Reality check (verified against iOS 18–26 behaviour):
|
||||
// • `extensionContext.open` is documented for Today widgets only; for a
|
||||
// keyboard extension it resolves `false`, so we do not use it.
|
||||
// • The deprecated `openURL:` selector hack was disabled in iOS 18
|
||||
// ("BUG IN CLIENT OF UIKIT … migrate to open(_:options:completionHandler:)").
|
||||
// • The still-working path is: walk the responder chain to `UIApplication`
|
||||
// and call the non-deprecated `open(_:options:completionHandler:)`. This
|
||||
// requires Full Access and grows less reliable on newer iOS, so we report
|
||||
// the *real* success from the completion handler instead of assuming it
|
||||
// worked — callers degrade to on-keyboard guidance when it returns false.
|
||||
// • Primary path: walk the responder chain to `UIApplication` and call the
|
||||
// non-deprecated `open(_:options:completionHandler:)`. This requires Full
|
||||
// Access and grows less reliable on newer iOS, so we report the *real*
|
||||
// success from the completion handler instead of assuming it worked.
|
||||
// • Fallback: `extensionContext.open`. Historically documented for Today
|
||||
// widgets only (and it used to resolve `false` for keyboards), but it is
|
||||
// the Apple-documented API for extensions to open URLs and ships in
|
||||
// production keyboards on current iOS — worth trying before giving up.
|
||||
// When both paths fail, callers degrade to on-keyboard guidance.
|
||||
|
||||
import UIKit
|
||||
|
||||
@@ -27,13 +29,52 @@ enum HostAppLauncher {
|
||||
while let current = responder {
|
||||
if let application = current as? UIApplication {
|
||||
application.open(url, options: [:]) { success in
|
||||
Task { @MainActor in completion(success) }
|
||||
Task { @MainActor in
|
||||
if success {
|
||||
completion(true)
|
||||
} else {
|
||||
openViaExtensionContext(url: url, from: controller, completion: completion)
|
||||
}
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
responder = current.next
|
||||
}
|
||||
// No `UIApplication` in the responder chain — cannot open the host app.
|
||||
// No `UIApplication` in the responder chain — try the extension context.
|
||||
openViaExtensionContext(url: url, from: controller, completion: completion)
|
||||
}
|
||||
|
||||
@MainActor
|
||||
private static func openViaExtensionContext(
|
||||
url: URL,
|
||||
from controller: KeyboardViewController,
|
||||
completion: @escaping @MainActor (Bool) -> Void
|
||||
) {
|
||||
guard let context = controller.extensionContext else {
|
||||
completion(false)
|
||||
return
|
||||
}
|
||||
// `NSExtensionContext.open` from keyboards has historically been
|
||||
// flaky about ever invoking its completion on some iOS versions.
|
||||
// Callers rely on a real answer to fail fast (instead of spinning
|
||||
// the 30 s start watchdog), so race the callback against a timeout
|
||||
// and report the first result only.
|
||||
var didComplete = false
|
||||
let finish: @MainActor (Bool) -> Void = { success in
|
||||
guard !didComplete else { return }
|
||||
didComplete = true
|
||||
completion(success)
|
||||
}
|
||||
Task { @MainActor in
|
||||
// 1.5 s: long enough for a real open to call back, short enough
|
||||
// that a dead completion degrades to on-keyboard guidance before
|
||||
// the user gives up staring at nothing.
|
||||
try? await Task.sleep(nanoseconds: 1_500_000_000)
|
||||
finish(false)
|
||||
}
|
||||
context.open(url) { success in
|
||||
Task { @MainActor in finish(success) }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,6 +44,10 @@ final class KeyboardFlowCoordinator {
|
||||
private var isAwaitingFlowResult = false
|
||||
private var activeSessionId: UUID?
|
||||
private var currentUtteranceId: UUID?
|
||||
/// Utterance whose final result we already inserted (or failed). Prevents
|
||||
/// `adoptHostBusyStateIfNeeded` from re-entering `.processing` after a
|
||||
/// stale App Group snapshot still says `reason=processing`.
|
||||
private var lastConsumedUtteranceId: UUID?
|
||||
private var currentCommandSeq: Int64 = 0
|
||||
private var lastAvailabilityTraceSignature = ""
|
||||
|
||||
@@ -139,6 +143,12 @@ final class KeyboardFlowCoordinator {
|
||||
FlowSessionBridge.reloadFromDisk()
|
||||
let readySnapshot = FlowSessionBridge.readySnapshot()
|
||||
activeSessionId = readySnapshot?.sessionId ?? activeSessionId
|
||||
|
||||
// If the host is mid-utterance but this extension process lost local
|
||||
// ownership (jetsam / recreate after app switch), re-adopt it so we
|
||||
// show red/white instead of a fake orange "starting" state.
|
||||
adoptHostBusyStateIfNeeded(snapshot: readySnapshot)
|
||||
|
||||
let hostReady = readySnapshot?.ready == true && FlowSessionBridge.isHostReady()
|
||||
let now = Date().timeIntervalSince1970
|
||||
if hostReady { lastHostReadyAt = now }
|
||||
@@ -148,10 +158,20 @@ final class KeyboardFlowCoordinator {
|
||||
// across cross-process read jitter and anchors this smoothing.
|
||||
let withinReadyGrace = lastHostReadyAt > 0
|
||||
&& (now - lastHostReadyAt) <= Self.hostReadyGrace
|
||||
// Host busy (recording/processing) is NOT "still starting". Treating
|
||||
// it as preparingSession was the orange-stuck bug after cold start:
|
||||
// host utt.rec=1 → ready=false → keyboard forever "正在启动…".
|
||||
let hostBusy = readySnapshot?.reason == .recording
|
||||
|| readySnapshot?.reason == .processing
|
||||
let hostWarming = !hostReady
|
||||
&& !hostBusy
|
||||
&& FlowSessionBridge.isSessionActive()
|
||||
&& (FlowSessionBridge.isHostReachable() || isPendingFlowStart || withinReadyGrace)
|
||||
state.flowSessionActive = FlowSessionBridge.isSessionActive()
|
||||
state.debugPendingFlowStart = isPendingFlowStart
|
||||
state.debugFlowRecording = isFlowRecording
|
||||
state.debugAwaitingFlowResult = isAwaitingFlowResult
|
||||
state.debugHasFullAccess = hasFullAccess()
|
||||
state.micVoiceAvailability = MicVoiceAvailabilityResolver.resolve(
|
||||
phase: state.phase,
|
||||
micDisabled: state.micDisabled,
|
||||
@@ -176,6 +196,84 @@ final class KeyboardFlowCoordinator {
|
||||
}
|
||||
}
|
||||
|
||||
/// Re-attach to a host utterance this keyboard process no longer owns.
|
||||
private func adoptHostBusyStateIfNeeded(snapshot: FlowReadySnapshot?) {
|
||||
guard let snapshot, let sessionId = snapshot.sessionId else { return }
|
||||
// Ignore snapshots from a dead host generation.
|
||||
if let snapGen = snapshot.hostGeneration,
|
||||
let liveGen = FlowSessionBridge.currentHostGeneration(),
|
||||
snapGen != liveGen {
|
||||
return
|
||||
}
|
||||
|
||||
// Host already finished — never re-adopt a consumed utterance, and
|
||||
// clear sticky local processing left behind by a stale busy snapshot.
|
||||
if snapshot.reason != .recording, snapshot.reason != .processing {
|
||||
clearStickyProcessingIfNeeded(hostReady: snapshot.ready)
|
||||
return
|
||||
}
|
||||
|
||||
switch snapshot.reason {
|
||||
case .recording:
|
||||
guard !isFlowRecording else { return }
|
||||
guard !isAwaitingFlowResult else { return }
|
||||
// Require the host's utterance id — inventing one makes matchingResult
|
||||
// forever miss the real delivery and leaves the mic white forever.
|
||||
guard let busyId = snapshot.busyUtteranceId else { return }
|
||||
guard busyId != lastConsumedUtteranceId else { return }
|
||||
activeSessionId = sessionId
|
||||
currentUtteranceId = busyId
|
||||
isPendingFlowStart = false
|
||||
flowStartDeadline = 0
|
||||
stopHostReadyWait()
|
||||
isFlowRecording = true
|
||||
state.phase = .recording
|
||||
if state.lastTranscript.isEmpty {
|
||||
state.lastTranscript = ""
|
||||
}
|
||||
if let view = wakeLockView() {
|
||||
ExtensionScreenWakeLock.acquire(from: view)
|
||||
}
|
||||
startUtteranceCountdown()
|
||||
startFlowLevelWatchdog()
|
||||
traceState("adoptHostBusy.recording", extra: "session=\(sessionId)")
|
||||
case .processing:
|
||||
guard !isAwaitingFlowResult else { return }
|
||||
guard let busyId = snapshot.busyUtteranceId else { return }
|
||||
guard busyId != lastConsumedUtteranceId else { return }
|
||||
activeSessionId = sessionId
|
||||
currentUtteranceId = busyId
|
||||
isPendingFlowStart = false
|
||||
flowStartDeadline = 0
|
||||
isFlowRecording = false
|
||||
stopUtteranceCountdown()
|
||||
ExtensionScreenWakeLock.release()
|
||||
state.phase = .processing
|
||||
if state.lastTranscript.isEmpty {
|
||||
state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing")
|
||||
}
|
||||
startFlowResultWatchdog()
|
||||
traceState("adoptHostBusy.processing", extra: "session=\(sessionId)")
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
/// After insert, a stale `reason=processing` snapshot can bounce the mic
|
||||
/// back to white loading. When the host is no longer busy, force idle.
|
||||
private func clearStickyProcessingIfNeeded(hostReady: Bool) {
|
||||
guard !isAwaitingFlowResult, !isFlowRecording else { return }
|
||||
guard case .processing = state.phase else { return }
|
||||
state.phase = .idle
|
||||
state.lastTranscript = ""
|
||||
stopFlowWatchdog()
|
||||
currentUtteranceId = nil
|
||||
traceState(
|
||||
"stickyProcessing.cleared",
|
||||
extra: hostReady ? "hostReady=1" : "hostReady=0"
|
||||
)
|
||||
}
|
||||
|
||||
/// Session is live but the ready contract has not landed yet — poll
|
||||
/// quickly instead of sticking on "session inactive" orange.
|
||||
private func startHostReadyWaitIfNeeded() {
|
||||
@@ -184,6 +282,12 @@ final class KeyboardFlowCoordinator {
|
||||
stopHostReadyWait()
|
||||
return
|
||||
}
|
||||
// Host busy ≠ waiting for ready. Do not spin the ready-wait poll.
|
||||
if let reason = FlowSessionBridge.readySnapshot()?.reason,
|
||||
reason == .recording || reason == .processing {
|
||||
stopHostReadyWait()
|
||||
return
|
||||
}
|
||||
guard !FlowSessionBridge.isHostReady() else {
|
||||
stopHostReadyWait()
|
||||
return
|
||||
@@ -196,7 +300,9 @@ final class KeyboardFlowCoordinator {
|
||||
guard let self, !Task.isCancelled else { return }
|
||||
FlowSessionBridge.reloadFromDisk()
|
||||
self.recomputeMicVoiceAvailability()
|
||||
if self.state.micVoiceAvailability.isReady {
|
||||
if self.state.micVoiceAvailability.isReady
|
||||
|| self.state.micVoiceAvailability == .recording
|
||||
|| self.state.micVoiceAvailability == .processing {
|
||||
return
|
||||
}
|
||||
try? await Task.sleep(nanoseconds: 150_000_000)
|
||||
@@ -365,6 +471,7 @@ final class KeyboardFlowCoordinator {
|
||||
)
|
||||
)
|
||||
FlowSessionBridge.clearResult()
|
||||
lastConsumedUtteranceId = result.utteranceId
|
||||
currentUtteranceId = nil
|
||||
textInserter.handleFlowTranscript(
|
||||
TranscriptionDelivery(text: text, polishWarning: result.warning)
|
||||
@@ -375,6 +482,7 @@ final class KeyboardFlowCoordinator {
|
||||
isAwaitingFlowResult = false
|
||||
stopFlowWatchdog()
|
||||
FlowSessionBridge.clearResult()
|
||||
lastConsumedUtteranceId = result.utteranceId
|
||||
currentUtteranceId = nil
|
||||
let error = FlowTranscriptionError(
|
||||
message: result.text ?? ExtL10n.string("keyboard.flow.resultTimeout"),
|
||||
@@ -626,6 +734,7 @@ final class KeyboardFlowCoordinator {
|
||||
)
|
||||
)
|
||||
FlowSessionBridge.clearResult()
|
||||
self.lastConsumedUtteranceId = result.utteranceId
|
||||
self.currentUtteranceId = nil
|
||||
self.debug("resultWatchdog consumed delivery len=\(text.count)")
|
||||
self.textInserter.handleFlowTranscript(
|
||||
@@ -637,6 +746,7 @@ final class KeyboardFlowCoordinator {
|
||||
self.isAwaitingFlowResult = false
|
||||
self.stopFlowWatchdog()
|
||||
FlowSessionBridge.clearResult()
|
||||
self.lastConsumedUtteranceId = result.utteranceId
|
||||
self.currentUtteranceId = nil
|
||||
let error = FlowTranscriptionError(
|
||||
message: result.text ?? ExtL10n.string("keyboard.flow.resultTimeout"),
|
||||
|
||||
@@ -10,15 +10,18 @@ import OSGKeyboardShared
|
||||
final class KeyboardTextInserter {
|
||||
private let state: KeyboardState
|
||||
private let insertText: (String) -> Void
|
||||
private let contextBeforeInput: () -> String?
|
||||
private let scheduleAutoClearError: () -> Void
|
||||
|
||||
init(
|
||||
state: KeyboardState,
|
||||
insertText: @escaping (String) -> Void,
|
||||
contextBeforeInput: @escaping () -> String?,
|
||||
scheduleAutoClearError: @escaping () -> Void
|
||||
) {
|
||||
self.state = state
|
||||
self.insertText = insertText
|
||||
self.contextBeforeInput = contextBeforeInput
|
||||
self.scheduleAutoClearError = scheduleAutoClearError
|
||||
}
|
||||
|
||||
@@ -30,7 +33,13 @@ final class KeyboardTextInserter {
|
||||
return
|
||||
}
|
||||
// Host app already polished when configured; keyboard only inserts.
|
||||
insertText(trimmed)
|
||||
// Word-boundary hygiene: dictating "world" with the cursor right
|
||||
// after "Hello" must yield "Hello world", not "Helloworld".
|
||||
let separator = DictationTextComposer.insertionSeparator(
|
||||
previousContext: contextBeforeInput(),
|
||||
insertion: trimmed
|
||||
)
|
||||
insertText(separator + trimmed)
|
||||
state.lastTranscript = ""
|
||||
state.level = 0
|
||||
if let warning = delivery.polishWarning {
|
||||
|
||||
@@ -33,6 +33,11 @@ private enum KeyboardLayoutMetrics {
|
||||
static let topBarToTranscriptSpacing: CGFloat = Spacing.xs / 2
|
||||
/// Outer inset for the bottom action row from screen edges (8 pt → 24 pt, +200%).
|
||||
static let sideActionHorizontalInset: CGFloat = Spacing.xs * 3
|
||||
/// iPad: cap the content column. A full-width (~1180 pt) keyboard would
|
||||
/// park delete/return at the far screen edges and turn each cursor-drag
|
||||
/// pad into a ~450 pt runway — capping keeps the reach ergonomics of the
|
||||
/// phone layout. iPhone widths are all below this, so it is a no-op there.
|
||||
static let contentMaxWidth: CGFloat = 700
|
||||
|
||||
// MARK: - Content-driven keyboard height (single source of truth)
|
||||
static let outerPaddingTop: CGFloat = 2
|
||||
@@ -109,6 +114,8 @@ public struct KeyboardRootView: View {
|
||||
.padding(.bottom, KeyboardLayoutMetrics.outerPaddingBottom)
|
||||
// 透明背景:让系统键盘 chrome 透出,不自行铺色(深浅模式一致)。
|
||||
.background(Color.clear)
|
||||
.frame(maxWidth: KeyboardLayoutMetrics.contentMaxWidth)
|
||||
.frame(maxWidth: .infinity)
|
||||
.frame(height: Self.totalHeight)
|
||||
// Feed the resolved palette to all nested chips/buttons.
|
||||
.environment(\.themePalette, palette)
|
||||
|
||||
@@ -9,37 +9,50 @@ import SwiftUI
|
||||
import WidgetKit
|
||||
|
||||
struct FlowLiveActivityWidget: Widget {
|
||||
/// Deep link that restarts the Flow session. When the host process is
|
||||
/// dead the activity goes stale; tapping it must take the *cold-start*
|
||||
/// path (same as the keyboard's mic button), not just open the app.
|
||||
private static let reconnectURL = URL(string: "osgkeyboard://startflow")
|
||||
|
||||
var body: some WidgetConfiguration {
|
||||
ActivityConfiguration(for: FlowActivityAttributes.self) { context in
|
||||
FlowLiveActivityLockScreenView(phase: context.state.phase)
|
||||
FlowLiveActivityLockScreenView(
|
||||
phase: context.state.phase,
|
||||
isStale: context.isStale
|
||||
)
|
||||
.activityBackgroundTint(Color.black.opacity(0.82))
|
||||
.activitySystemActionForegroundColor(.white)
|
||||
// Deep-link to a session restart only when the host is dead —
|
||||
// tapping a HEALTHY activity should just open the app, not
|
||||
// force a cold-start handoff into a running session.
|
||||
.widgetURL(context.isStale ? Self.reconnectURL : nil)
|
||||
} dynamicIsland: { context in
|
||||
DynamicIsland {
|
||||
DynamicIslandExpandedRegion(.leading) {
|
||||
FlowLiveActivityBrandMark(height: 16)
|
||||
}
|
||||
DynamicIslandExpandedRegion(.trailing) {
|
||||
FlowLiveActivityPhaseLabel(phase: context.state.phase)
|
||||
FlowLiveActivityPhaseLabel(phase: context.state.phase, isStale: context.isStale)
|
||||
}
|
||||
DynamicIslandExpandedRegion(.center) {
|
||||
Text("OSGKeyboard")
|
||||
.font(.headline)
|
||||
}
|
||||
DynamicIslandExpandedRegion(.bottom) {
|
||||
FlowLiveActivityPhaseCaption(phase: context.state.phase)
|
||||
FlowLiveActivityPhaseCaption(phase: context.state.phase, isStale: context.isStale)
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
} compactLeading: {
|
||||
FlowLiveActivityBrandMark(height: 12)
|
||||
} compactTrailing: {
|
||||
FlowLiveActivityTrailingGlyph(phase: context.state.phase)
|
||||
FlowLiveActivityTrailingGlyph(phase: context.state.phase, isStale: context.isStale)
|
||||
} minimal: {
|
||||
// The minimal slot is a tiny circle; a short wordmark keeps
|
||||
// the natural ratio without overflowing its bounds.
|
||||
FlowLiveActivityBrandMark(height: 6)
|
||||
}
|
||||
.widgetURL(context.isStale ? Self.reconnectURL : nil)
|
||||
.keylineTint(Color(red: 0.35, green: 0.55, blue: 1.0))
|
||||
}
|
||||
}
|
||||
@@ -49,6 +62,7 @@ struct FlowLiveActivityWidget: Widget {
|
||||
|
||||
private struct FlowLiveActivityLockScreenView: View {
|
||||
let phase: FlowActivityAttributes.ContentState.Phase
|
||||
let isStale: Bool
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: 12) {
|
||||
@@ -56,13 +70,16 @@ private struct FlowLiveActivityLockScreenView: View {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Text("OSGKeyboard")
|
||||
.font(.headline)
|
||||
FlowLiveActivityPhaseCaption(phase: phase)
|
||||
FlowLiveActivityPhaseCaption(phase: phase, isStale: isStale)
|
||||
.font(.subheadline)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
Spacer(minLength: 0)
|
||||
FlowLiveActivityTrailingGlyph(phase: phase)
|
||||
FlowLiveActivityTrailingGlyph(phase: phase, isStale: isStale)
|
||||
}
|
||||
// A stale activity means the host process is gone — never advertise
|
||||
// "Voice session active" for a dead session; grey the card instead.
|
||||
.opacity(isStale ? 0.55 : 1)
|
||||
// iOS Live Activity lock-screen content needs margins so the leading
|
||||
// logo and trailing glyph don't touch the card edges.
|
||||
.padding(.horizontal, 16)
|
||||
@@ -91,8 +108,19 @@ private struct FlowLiveActivityBrandMark: View {
|
||||
|
||||
private struct FlowLiveActivityTrailingGlyph: View {
|
||||
let phase: FlowActivityAttributes.ContentState.Phase
|
||||
var isStale: Bool = false
|
||||
|
||||
var body: some View {
|
||||
if isStale {
|
||||
Image(systemName: "bolt.slash.circle")
|
||||
.foregroundStyle(.secondary)
|
||||
} else {
|
||||
phaseGlyph
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var phaseGlyph: some View {
|
||||
switch phase {
|
||||
case .recording:
|
||||
Image(systemName: "waveform")
|
||||
@@ -114,8 +142,19 @@ private struct FlowLiveActivityTrailingGlyph: View {
|
||||
|
||||
private struct FlowLiveActivityPhaseLabel: View {
|
||||
let phase: FlowActivityAttributes.ContentState.Phase
|
||||
var isStale: Bool = false
|
||||
|
||||
var body: some View {
|
||||
if isStale {
|
||||
Image(systemName: "bolt.slash.circle")
|
||||
.foregroundStyle(.secondary)
|
||||
} else {
|
||||
phaseLabel
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var phaseLabel: some View {
|
||||
switch phase {
|
||||
case .recording:
|
||||
Text("REC")
|
||||
@@ -133,8 +172,20 @@ private struct FlowLiveActivityPhaseLabel: View {
|
||||
|
||||
private struct FlowLiveActivityPhaseCaption: View {
|
||||
let phase: FlowActivityAttributes.ContentState.Phase
|
||||
var isStale: Bool = false
|
||||
|
||||
var body: some View {
|
||||
if isStale {
|
||||
// Host process is gone — be honest about it and turn the card
|
||||
// into a recovery entry point (tap deep-links to startflow).
|
||||
Text("Session disconnected · tap to reconnect")
|
||||
} else {
|
||||
phaseCaption
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var phaseCaption: some View {
|
||||
switch phase {
|
||||
case .idle:
|
||||
Text("Voice session active")
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
/* Live Activity captions. Keys are the English literals used in
|
||||
FlowLiveActivityWidget — SwiftUI Text(_:) resolves string literals as
|
||||
LocalizedStringKey against this table automatically. */
|
||||
"Voice session active" = "Voice session active";
|
||||
"Listening…" = "Listening…";
|
||||
"Transcribing…" = "Transcribing…";
|
||||
"Session disconnected · tap to reconnect" = "Session disconnected · tap to reconnect";
|
||||
@@ -0,0 +1,6 @@
|
||||
/* Live Activity 文案。键为 FlowLiveActivityWidget 中的英文字面量 —
|
||||
SwiftUI Text(_:) 会把字符串字面量按 LocalizedStringKey 在本表解析。 */
|
||||
"Voice session active" = "语音会话进行中";
|
||||
"Listening…" = "正在聆听…";
|
||||
"Transcribing…" = "正在转写…";
|
||||
"Session disconnected · tap to reconnect" = "会话已断开 · 点按重连";
|
||||
@@ -1,7 +1,8 @@
|
||||
// DashboardView.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Primary workspace: session stats, dictation canvas, floating record bar.
|
||||
// Primary workspace: brand voice, asymmetric stats, dictation stage, and
|
||||
// the record bar. History lives on its own page — no duplicate list here.
|
||||
|
||||
import SwiftUI
|
||||
|
||||
@@ -20,33 +21,24 @@ struct DashboardView: View {
|
||||
self._stats = ObservedObject(wrappedValue: viewModel.usageStatistics)
|
||||
}
|
||||
|
||||
// Four equal-width columns — same metrics as the iOS home stats card.
|
||||
private let columns = Array(
|
||||
repeating: GridItem(.flexible(minimum: 120), spacing: Spacing.md),
|
||||
count: 4
|
||||
)
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 0) {
|
||||
ScrollView {
|
||||
VStack(alignment: .leading, spacing: Spacing.lg) {
|
||||
if let appName = viewModel.foregroundAppName {
|
||||
Text(MacL10n.format("mac.foregroundApp", language: lang, appName))
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.transition(.opacity)
|
||||
heroHeader
|
||||
statCluster
|
||||
dictationStage
|
||||
}
|
||||
statGrid
|
||||
dictationCanvas
|
||||
}
|
||||
.animation(Motion.soft, value: viewModel.foregroundAppName)
|
||||
.padding(.horizontal, Spacing.lg)
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset)
|
||||
.padding(.top, Spacing.sm)
|
||||
.padding(.bottom, Spacing.lg)
|
||||
}
|
||||
|
||||
// Leftover window height splits evenly above / below the mic bar
|
||||
// so spacing stays balanced at any window size.
|
||||
Spacer(minLength: Spacing.xs)
|
||||
|
||||
BottomDictationBar(viewModel: viewModel)
|
||||
.padding(.horizontal, Spacing.lg)
|
||||
.padding(.bottom, Spacing.sm)
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset)
|
||||
|
||||
Spacer(minLength: Spacing.xs)
|
||||
}
|
||||
.onAppear { stats.reloadFromDisk() }
|
||||
.onReceive(NotificationCenter.default.publisher(for: .usageStatisticsDidSyncFromCloud)) { _ in
|
||||
@@ -54,18 +46,41 @@ struct DashboardView: View {
|
||||
}
|
||||
}
|
||||
|
||||
private var statGrid: some View {
|
||||
LazyVGrid(columns: columns, spacing: Spacing.md) {
|
||||
StatCard(
|
||||
title: MacL10n.string("mac.stat.dictationTime", language: lang),
|
||||
value: UsageStatisticsStore.formatDuration(
|
||||
stats.dictationDurationSeconds,
|
||||
language: lang
|
||||
),
|
||||
caption: MacL10n.string("mac.stat.cumulativeDuration", language: lang),
|
||||
systemImage: "waveform",
|
||||
accent: true
|
||||
)
|
||||
// MARK: - Hero
|
||||
|
||||
private var heroHeader: some View {
|
||||
VStack(alignment: .leading, spacing: Spacing.xs) {
|
||||
Text(MacL10n.string("mac.brand.tagline", language: lang))
|
||||
.font(TypeStyle.pageTitle)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineLimit(2)
|
||||
.minimumScaleFactor(0.85)
|
||||
|
||||
HStack(spacing: Spacing.sm) {
|
||||
Text(MacL10n.string("mac.brand.tagline.subtitle", language: lang))
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
|
||||
if let appName = viewModel.foregroundAppName {
|
||||
Text("·")
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.45))
|
||||
Text(MacL10n.format("mac.foregroundApp", language: lang, appName))
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.transition(.opacity)
|
||||
.lineLimit(1)
|
||||
}
|
||||
}
|
||||
.animation(Motion.soft, value: viewModel.foregroundAppName)
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
|
||||
// MARK: - Stats (hero word count, full width but content-height —
|
||||
// never stretched to match a taller sibling and left with dead air)
|
||||
|
||||
private var statCluster: some View {
|
||||
VStack(spacing: Spacing.md) {
|
||||
StatCard(
|
||||
title: MacL10n.string("mac.stat.words", language: lang),
|
||||
value: UsageStatisticsStore.formatCount(
|
||||
@@ -73,7 +88,20 @@ struct DashboardView: View {
|
||||
language: lang
|
||||
),
|
||||
caption: MacL10n.string("mac.stat.transcribed", language: lang),
|
||||
systemImage: "text.alignleft"
|
||||
systemImage: "text.alignleft",
|
||||
accent: true,
|
||||
prominent: true
|
||||
)
|
||||
|
||||
HStack(spacing: Spacing.md) {
|
||||
StatCard(
|
||||
title: MacL10n.string("mac.stat.dictationTime", language: lang),
|
||||
value: UsageStatisticsStore.formatDuration(
|
||||
stats.dictationDurationSeconds,
|
||||
language: lang
|
||||
),
|
||||
caption: MacL10n.string("mac.stat.cumulativeDuration", language: lang),
|
||||
systemImage: "waveform"
|
||||
)
|
||||
StatCard(
|
||||
title: MacL10n.string("mac.stat.translation", language: lang),
|
||||
@@ -92,9 +120,12 @@ struct DashboardView: View {
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var dictationCanvas: some View {
|
||||
MacCard(padding: Spacing.lg) {
|
||||
// MARK: - Dictation stage
|
||||
|
||||
private var dictationStage: some View {
|
||||
MacCard(padding: Spacing.md, cornerRadius: Radius.large) {
|
||||
ZStack(alignment: .topLeading) {
|
||||
if viewModel.transcript.isEmpty {
|
||||
Text(
|
||||
@@ -102,24 +133,37 @@ struct DashboardView: View {
|
||||
? MacL10n.string("mac.status.listening", language: lang)
|
||||
: MacL10n.string("mac.status.ready", language: lang)
|
||||
)
|
||||
.font(.system(size: 26, weight: .light))
|
||||
.font(.system(size: 22, weight: .light))
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.contentTransition(.opacity)
|
||||
.frame(maxWidth: .infinity, minHeight: 220, alignment: .topLeading)
|
||||
.frame(
|
||||
maxWidth: .infinity,
|
||||
minHeight: MacMetrics.dictationCanvasMinHeight,
|
||||
maxHeight: .infinity,
|
||||
alignment: .topLeading
|
||||
)
|
||||
.transition(.opacity)
|
||||
} else {
|
||||
Text(viewModel.transcript)
|
||||
.font(.system(size: 22, weight: .regular))
|
||||
.font(.system(size: 20, weight: .regular))
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineSpacing(4)
|
||||
.textSelection(.enabled)
|
||||
.frame(maxWidth: .infinity, minHeight: 220, alignment: .topLeading)
|
||||
.frame(
|
||||
maxWidth: .infinity,
|
||||
minHeight: MacMetrics.dictationCanvasMinHeight,
|
||||
maxHeight: .infinity,
|
||||
alignment: .topLeading
|
||||
)
|
||||
.transition(.opacity)
|
||||
}
|
||||
}
|
||||
.frame(minHeight: MacMetrics.dictationCanvasMinHeight, maxHeight: 160)
|
||||
}
|
||||
.animation(Motion.soft, value: viewModel.transcript.isEmpty)
|
||||
.animation(Motion.quick, value: viewModel.isRecording)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// MARK: - Floating record bar
|
||||
@@ -142,12 +186,10 @@ struct BottomDictationBar: View {
|
||||
recordControl
|
||||
}
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
.macGlassSurface(in: RoundedRectangle(cornerRadius: Radius.xl, style: .continuous), fillOpacity: 1)
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: Radius.xl, style: .continuous)
|
||||
.stroke(palette.dividerStrong, lineWidth: 0.5)
|
||||
)
|
||||
.padding(.vertical, Spacing.xs)
|
||||
// No surface fill — the mic bar sits on the page background so Home
|
||||
// stays flat and the canvas above can stay shorter without a second
|
||||
// floating card competing for height.
|
||||
}
|
||||
|
||||
private var readinessChip: some View {
|
||||
@@ -189,15 +231,14 @@ struct BottomDictationBar: View {
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
.padding(.horizontal, Spacing.sm)
|
||||
.padding(.vertical, 7)
|
||||
.macGlassSurface(in: Capsule(), fillOpacity: 0.66)
|
||||
.background(palette.surfaceElevated, in: Capsule())
|
||||
}
|
||||
.menuStyle(.borderlessButton)
|
||||
.fixedSize()
|
||||
}
|
||||
|
||||
// 麦克风按钮始终居中固定:录音时的波形放进按钮内部,
|
||||
// “按停止”提示作为浮层显示在按钮上方,二者均不参与布局,
|
||||
// 因此按下 Option 触发录音时按钮位置不会发生偏移。
|
||||
// Mic stays geometrically centred: waveform lives inside the button,
|
||||
// “press stop” floats above — neither participates in layout.
|
||||
private var recordControl: some View {
|
||||
recordButton
|
||||
.overlay(alignment: .top) {
|
||||
@@ -225,7 +266,6 @@ struct BottomDictationBar: View {
|
||||
)
|
||||
Group {
|
||||
if viewModel.isRecording {
|
||||
// 与 iOS 一致:录音时在红色按钮内部显示实时波形
|
||||
MiniWaveform(level: viewModel.audioLevel, barCount: 4, tint: palette.textOnAccent)
|
||||
} else {
|
||||
Image(systemName: "mic.fill")
|
||||
|
||||
@@ -11,11 +11,16 @@
|
||||
final class MacAudioRecorder: @unchecked Sendable {
|
||||
enum RecorderError: Error, LocalizedError {
|
||||
case converterUnavailable
|
||||
case microphoneAccessDenied
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .converterUnavailable:
|
||||
return "无法初始化音频转换器 / Failed to initialize audio converter"
|
||||
case .microphoneAccessDenied:
|
||||
return "麦克风权限被拒绝——请在「系统设置 → 隐私与安全性 → 麦克风」中启用"
|
||||
+ " / Microphone access denied — enable it in System Settings"
|
||||
+ " → Privacy & Security → Microphone"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -30,7 +35,20 @@ final class MacAudioRecorder: @unchecked Sendable {
|
||||
private var converter: AVAudioConverter?
|
||||
private let lock = NSLock()
|
||||
private var samples: [Float] = []
|
||||
private var snapshotContinuation: AsyncStream<AudioBufferSnapshot>.Continuation?
|
||||
private var isRunning = false
|
||||
/// Hard cap on accumulated audio: 10 minutes @16 kHz ≈ 38 MB of Float32.
|
||||
/// Recording is push-to-talk, but a stuck hotkey (or a latched Option
|
||||
/// key) would otherwise grow this buffer without bound; past the cap we
|
||||
/// keep the newest audio (drop from the front) so the take still ends
|
||||
/// with what the user last said.
|
||||
private static let maxSampleCount = 10 * 60 * 16_000
|
||||
/// Trim hysteresis: dropping from the front is an O(n) memmove of the
|
||||
/// whole ~38 MB buffer, done under the same lock the UI's level poll
|
||||
/// takes — doing it on EVERY tap callback once capped would stall the
|
||||
/// render thread ~12×/s. Let the buffer overshoot by 30 s and trim the
|
||||
/// whole excess in one move instead.
|
||||
private static let trimHysteresisSamples = 30 * 16_000
|
||||
private var smoothedLevel: Float = 0
|
||||
/// One-shot flag for the converter pull block. Taps are serialized per
|
||||
/// bus, so a plain instance property (not a captured local) is safe here.
|
||||
@@ -42,8 +60,51 @@ final class MacAudioRecorder: @unchecked Sendable {
|
||||
lock.withLock { smoothedLevel }
|
||||
}
|
||||
|
||||
func start() throws {
|
||||
lock.withLock { samples.removeAll(keepingCapacity: true) }
|
||||
/// Resolves microphone authorization before capture. Prompts on first
|
||||
/// use; throws `microphoneAccessDenied` once the user has declined so
|
||||
/// failures surface as a permission problem, not an empty transcription.
|
||||
private static func ensureMicrophoneAccess() async throws {
|
||||
switch AVCaptureDevice.authorizationStatus(for: .audio) {
|
||||
case .authorized:
|
||||
return
|
||||
case .notDetermined:
|
||||
guard await AVCaptureDevice.requestAccess(for: .audio) else {
|
||||
throw RecorderError.microphoneAccessDenied
|
||||
}
|
||||
case .denied, .restricted:
|
||||
throw RecorderError.microphoneAccessDenied
|
||||
@unknown default:
|
||||
throw RecorderError.microphoneAccessDenied
|
||||
}
|
||||
}
|
||||
|
||||
func start() async throws {
|
||||
try await Self.ensureMicrophoneAccess()
|
||||
try startEngine()
|
||||
}
|
||||
|
||||
/// Live 16 kHz mono snapshots for streaming ASR while the mic is open.
|
||||
/// The stream is finished automatically in `stop()`.
|
||||
func makeSnapshotStream() -> AsyncStream<AudioBufferSnapshot> {
|
||||
AsyncStream { continuation in
|
||||
lock.withLock {
|
||||
snapshotContinuation?.finish()
|
||||
snapshotContinuation = continuation
|
||||
}
|
||||
continuation.onTermination = { [weak self] _ in
|
||||
self?.lock.withLock {
|
||||
self?.snapshotContinuation = nil
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func startEngine() throws {
|
||||
lock.withLock {
|
||||
samples.removeAll(keepingCapacity: true)
|
||||
snapshotContinuation?.finish()
|
||||
snapshotContinuation = nil
|
||||
}
|
||||
|
||||
let input = engine.inputNode
|
||||
let inputFormat = input.outputFormat(forBus: 0)
|
||||
@@ -67,6 +128,8 @@ final class MacAudioRecorder: @unchecked Sendable {
|
||||
engine.stop()
|
||||
isRunning = false
|
||||
return lock.withLock {
|
||||
snapshotContinuation?.finish()
|
||||
snapshotContinuation = nil
|
||||
let out = samples
|
||||
samples.removeAll(keepingCapacity: false)
|
||||
return out
|
||||
@@ -105,8 +168,14 @@ final class MacAudioRecorder: @unchecked Sendable {
|
||||
|
||||
lock.withLock {
|
||||
samples.append(contentsOf: chunk)
|
||||
if samples.count > Self.maxSampleCount + Self.trimHysteresisSamples {
|
||||
samples.removeFirst(samples.count - Self.maxSampleCount)
|
||||
}
|
||||
let factor: Float = normalized > smoothedLevel ? 0.5 : 0.15
|
||||
smoothedLevel += (normalized - smoothedLevel) * factor
|
||||
snapshotContinuation?.yield(
|
||||
AudioBufferSnapshot(samples: chunk, sampleRate: 16_000)
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
// MacCloudASRChunkAdapter.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Adapts configured cloud ASR clients to the shared chunked utterance pipeline.
|
||||
|
||||
import Foundation
|
||||
import os
|
||||
|
||||
final class MacCloudASRChunkAdapter: ASRChunkTranscribing, @unchecked Sendable {
|
||||
private let store: AppGroupStore
|
||||
private let client: CloudASRTranscribing
|
||||
private let cancelled = OSAllocatedUnfairLock(initialState: false)
|
||||
|
||||
init(store: AppGroupStore) throws {
|
||||
let strategy = CloudASRModelCatalog.strategy(for: store.asrProviderId)
|
||||
guard strategy != .localFallback else {
|
||||
throw MacDictationError.providerHasNoCloudASR
|
||||
}
|
||||
self.store = store
|
||||
self.client = CloudASRClientFactory.make(store: store)
|
||||
}
|
||||
|
||||
func prepare() async throws {
|
||||
try await client.prepare(dictionary: store.personalDictionary)
|
||||
}
|
||||
|
||||
func resetForNewUtterance() {
|
||||
cancelled.withLock { $0 = false }
|
||||
}
|
||||
|
||||
func cancel() {
|
||||
cancelled.withLock { $0 = true }
|
||||
}
|
||||
|
||||
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult {
|
||||
let isCancelled = cancelled.withLock { $0 }
|
||||
if isCancelled || Task.isCancelled { return .cancelled }
|
||||
guard !samples.isEmpty else { return .success("") }
|
||||
|
||||
do {
|
||||
let text = try await client.transcribe(
|
||||
samples: samples,
|
||||
sampleRate: 16_000,
|
||||
locale: locale,
|
||||
dictionary: store.personalDictionary
|
||||
)
|
||||
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
return .success(trimmed)
|
||||
} catch is CancellationError {
|
||||
return .cancelled
|
||||
} catch {
|
||||
return .failure(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -25,6 +25,22 @@ enum MacMetrics {
|
||||
static let sidebarContentInset: CGFloat = sidebarInset + Spacing.sm
|
||||
/// Reading width for single-column content.
|
||||
static let contentMaxWidth: CGFloat = 720
|
||||
/// Horizontal inset for page titles and scroll *content* (cards).
|
||||
/// ScrollViews / Forms stay full-bleed so the scrollbar sits on the
|
||||
/// window edge; only the content inside is inset.
|
||||
/// Doubled from `Spacing.lg` so title + cards breathe from the edges.
|
||||
static let pageHorizontalInset: CGFloat = Spacing.lg * 2
|
||||
/// Built-in horizontal inset macOS grouped `Form` adds around its section
|
||||
/// cards, on top of any padding we apply. Subtracted from
|
||||
/// `pageHorizontalInset` on the Settings Form so its card outer edge lands
|
||||
/// on `pageHorizontalInset` — matching the History page and the page title.
|
||||
static let groupedFormSectionInset: CGFloat = Spacing.lg
|
||||
/// Default (= minimum) main-window size. Opening the app uses this size;
|
||||
/// the window cannot shrink below it.
|
||||
static let windowMinWidth: CGFloat = 860
|
||||
static let windowMinHeight: CGFloat = 600
|
||||
/// Compact dictation-canvas height so Home fits the min window without scrolling.
|
||||
static let dictationCanvasMinHeight: CGFloat = 120
|
||||
/// Top inset that clears the window traffic-light buttons now that the
|
||||
/// title bar is hidden.
|
||||
static let trafficLightInset: CGFloat = 28
|
||||
@@ -81,16 +97,67 @@ extension View {
|
||||
func macFieldStyle() -> some View { modifier(MacFieldStyleModifier()) }
|
||||
}
|
||||
|
||||
// MARK: - Page header
|
||||
|
||||
/// Page title for History / Dictionary / Settings. Applies the shared
|
||||
/// `pageHorizontalInset` so its left edge matches inset card content below.
|
||||
/// Type size matches Home's brand line (`TypeStyle.pageTitle`).
|
||||
struct MacPageHeader<Trailing: View>: View {
|
||||
@Environment(\.themePalette) private var palette
|
||||
let title: String
|
||||
var subtitle: String?
|
||||
@ViewBuilder var trailing: () -> Trailing
|
||||
|
||||
init(
|
||||
title: String,
|
||||
subtitle: String? = nil,
|
||||
@ViewBuilder trailing: @escaping () -> Trailing
|
||||
) {
|
||||
self.title = title
|
||||
self.subtitle = subtitle
|
||||
self.trailing = trailing
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
HStack(alignment: .firstTextBaseline, spacing: Spacing.md) {
|
||||
VStack(alignment: .leading, spacing: Spacing.xxs) {
|
||||
Text(title)
|
||||
.font(TypeStyle.pageTitle)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
.minimumScaleFactor(0.85)
|
||||
if let subtitle, !subtitle.isEmpty {
|
||||
Text(subtitle)
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
}
|
||||
}
|
||||
Spacer(minLength: 0)
|
||||
trailing()
|
||||
}
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset)
|
||||
.padding(.top, Spacing.sm)
|
||||
.padding(.bottom, Spacing.sm)
|
||||
}
|
||||
}
|
||||
|
||||
extension MacPageHeader where Trailing == EmptyView {
|
||||
init(title: String, subtitle: String? = nil) {
|
||||
self.init(title: title, subtitle: subtitle) { EmptyView() }
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Card container
|
||||
|
||||
/// Elevated surface used for stat tiles and the dictation canvas.
|
||||
struct MacCard<Content: View>: View {
|
||||
@Environment(\.themePalette) private var palette
|
||||
var padding: CGFloat = Spacing.md
|
||||
var cornerRadius: CGFloat = Radius.medium
|
||||
@ViewBuilder var content: () -> Content
|
||||
|
||||
var body: some View {
|
||||
let shape = RoundedRectangle(cornerRadius: Radius.medium, style: .continuous)
|
||||
let shape = RoundedRectangle(cornerRadius: cornerRadius, style: .continuous)
|
||||
|
||||
content()
|
||||
.padding(padding)
|
||||
@@ -111,9 +178,21 @@ struct StatCard: View {
|
||||
let caption: String
|
||||
var systemImage: String?
|
||||
var accent: Bool = false
|
||||
/// Hero metric: wide horizontal layout that uses full-card width without
|
||||
/// stretching to fill dead vertical space — used for the primary word count.
|
||||
var prominent: Bool = false
|
||||
|
||||
var body: some View {
|
||||
MacCard {
|
||||
MacCard(padding: prominent ? Spacing.md : Spacing.md) {
|
||||
if prominent {
|
||||
prominentBody
|
||||
} else {
|
||||
compactBody
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var compactBody: some View {
|
||||
VStack(alignment: .leading, spacing: Spacing.xs) {
|
||||
HStack {
|
||||
Text(title.uppercased())
|
||||
@@ -125,6 +204,7 @@ struct StatCard: View {
|
||||
Image(systemName: systemImage)
|
||||
.font(.system(size: 13, weight: .semibold))
|
||||
.foregroundStyle(accent ? palette.accent : palette.textTertiary)
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
}
|
||||
}
|
||||
Text(value)
|
||||
@@ -140,6 +220,41 @@ struct StatCard: View {
|
||||
}
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
|
||||
/// Wide "hero bar" layout: icon badge + title/caption on the left, the
|
||||
/// big number anchored right — fills the full card width edge-to-edge
|
||||
/// instead of a tall card with empty space below a small number.
|
||||
private var prominentBody: some View {
|
||||
HStack(spacing: Spacing.md) {
|
||||
if let systemImage {
|
||||
ZStack {
|
||||
Circle()
|
||||
.fill(palette.accentMuted)
|
||||
.frame(width: 44, height: 44)
|
||||
Image(systemName: systemImage)
|
||||
.font(.system(size: 17, weight: .semibold))
|
||||
.foregroundStyle(palette.accent)
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
}
|
||||
}
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Text(title.uppercased())
|
||||
.font(TypeStyle.caption2)
|
||||
.tracking(0.6)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
Text(caption)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
Spacer(minLength: Spacing.md)
|
||||
Text(value)
|
||||
.font(.system(size: 34, weight: .bold))
|
||||
.foregroundStyle(accent ? palette.accent : palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
.minimumScaleFactor(0.6)
|
||||
.contentTransition(.numericText())
|
||||
.animation(Motion.soft, value: value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -152,30 +267,39 @@ struct MiniWaveform: View {
|
||||
var barCount: Int = 5
|
||||
/// Pass nil to inherit the palette accent automatically.
|
||||
var tint: Color?
|
||||
/// Peak bar height; overlay HUD uses a taller meter than the mic button.
|
||||
var maxBarHeight: CGFloat = 22
|
||||
var barWidth: CGFloat = 3
|
||||
var barSpacing: CGFloat = 3
|
||||
|
||||
@State private var phase: CGFloat = 0
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: 3) {
|
||||
HStack(spacing: barSpacing) {
|
||||
ForEach(0..<barCount, id: \.self) { index in
|
||||
Capsule()
|
||||
.fill(tint ?? palette.accent)
|
||||
.frame(width: 3, height: barHeight(index))
|
||||
.frame(width: barWidth, height: barHeight(index))
|
||||
}
|
||||
}
|
||||
.frame(height: 22)
|
||||
.frame(height: maxBarHeight)
|
||||
.animation(Motion.instant, value: level)
|
||||
.onAppear {
|
||||
withAnimation(.linear(duration: 0.9).repeatForever(autoreverses: true)) {
|
||||
withAnimation(.linear(duration: 0.75).repeatForever(autoreverses: true)) {
|
||||
phase = 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func barHeight(_ index: Int) -> CGFloat {
|
||||
let base = CGFloat(level) * 22
|
||||
let wobble = sin((phase * .pi * 2) + CGFloat(index)) * 4 + 4
|
||||
return max(4, min(22, base * (0.6 + CGFloat(index % 2) * 0.4) + wobble))
|
||||
// Stronger level coupling + staggered phase so the meter reads as
|
||||
// "alive" even at modest mic levels.
|
||||
let boosted = min(1, CGFloat(level) * 1.35 + 0.08)
|
||||
let base = boosted * maxBarHeight
|
||||
let wobble = sin((phase * .pi * 2) + CGFloat(index) * 0.85) * (maxBarHeight * 0.22)
|
||||
+ (maxBarHeight * 0.12)
|
||||
let parity = 0.55 + CGFloat(index % 3) * 0.2
|
||||
return max(maxBarHeight * 0.18, min(maxBarHeight, base * parity + wobble))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -195,8 +319,8 @@ enum MacTranslationDisplay {
|
||||
|
||||
// MARK: - Status footer
|
||||
|
||||
/// Bottom status strip: engine mode (cloud/local), translation target, and
|
||||
/// the connection state — icons and wording mirror the dashboard record bar.
|
||||
/// Bottom status strip: engine mode, translation target, connection —
|
||||
/// kept visually quiet so it never competes with the record bar.
|
||||
struct MacStatusFooter: View {
|
||||
@ObservedObject var viewModel: MacDictationViewModel
|
||||
@Environment(\.themePalette) private var palette
|
||||
@@ -204,7 +328,7 @@ struct MacStatusFooter: View {
|
||||
private var lang: AppUILanguage { viewModel.config.uiLanguage }
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: Spacing.md) {
|
||||
HStack(spacing: Spacing.sm) {
|
||||
Spacer()
|
||||
Label(
|
||||
viewModel.isCloudMode
|
||||
@@ -212,24 +336,47 @@ struct MacStatusFooter: View {
|
||||
: MacL10n.string("mac.mode.local", language: lang),
|
||||
systemImage: viewModel.isCloudMode ? "cloud" : "cpu"
|
||||
)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
.contentTransition(.opacity)
|
||||
|
||||
Text("·")
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.5))
|
||||
|
||||
Label(
|
||||
MacTranslationDisplay.label(for: viewModel.config.translationTargetLocaleId, language: lang),
|
||||
systemImage: "translate"
|
||||
)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
.contentTransition(.opacity)
|
||||
|
||||
Text("·")
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.5))
|
||||
|
||||
Label(MacL10n.string("mac.connected", language: lang), systemImage: "link")
|
||||
.foregroundStyle(palette.accent)
|
||||
.foregroundStyle(palette.accent.opacity(0.85))
|
||||
}
|
||||
.font(TypeStyle.caption)
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.labelStyle(.titleAndIcon)
|
||||
.padding(.horizontal, Spacing.lg)
|
||||
.padding(.vertical, Spacing.xs)
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
.animation(Motion.quick, value: viewModel.isCloudMode)
|
||||
.animation(Motion.quick, value: viewModel.config.translationTargetLocaleId)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Form alignment
|
||||
|
||||
private struct MacFormPageAlignModifier: ViewModifier {
|
||||
func body(content: Content) -> some View {
|
||||
// Form stays full-bleed (scrollbar on the window edge). Section
|
||||
// cards are inset to match `MacPageHeader`.
|
||||
content
|
||||
.contentMargins(.horizontal, MacMetrics.pageHorizontalInset, for: .scrollContent)
|
||||
}
|
||||
}
|
||||
|
||||
extension View {
|
||||
/// Insets grouped-`Form` section cards to `pageHorizontalInset`.
|
||||
func macFormPageAligned() -> some View {
|
||||
modifier(MacFormPageAlignModifier())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ struct MacContentView: View {
|
||||
VStack(alignment: .leading, spacing: Spacing.sm) {
|
||||
header
|
||||
recordButton
|
||||
Text(MacL10n.string("mac.hint.holdOption", language: lang))
|
||||
Text(MacL10n.string(viewModel.hotkeyTrigger.hintKey, language: lang))
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .center)
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
// MacDictationOverlayController.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Owns a borderless, non-activating floating NSPanel that hosts the
|
||||
// dictation HUD. Shown for any recording path (hotkey, menu bar, main
|
||||
// window) and dismissed after a short success beat when processing ends.
|
||||
|
||||
import AppKit
|
||||
import Combine
|
||||
import SwiftUI
|
||||
|
||||
@MainActor
|
||||
final class MacDictationOverlayController {
|
||||
static let shared = MacDictationOverlayController()
|
||||
|
||||
private var panel: NSPanel?
|
||||
private var hosting: NSHostingView<AnyView>?
|
||||
private var cancellables = Set<AnyCancellable>()
|
||||
private var hideWorkItem: DispatchWorkItem?
|
||||
/// Keeps the pill visible briefly after a successful delivery.
|
||||
private var showingCompletion = false
|
||||
private var wasBusy = false
|
||||
|
||||
private let bottomMargin: CGFloat = 36
|
||||
private let fallbackSize = NSSize(width: 400, height: 52)
|
||||
|
||||
private init() {}
|
||||
|
||||
func start(observing viewModel: MacDictationViewModel) {
|
||||
guard cancellables.isEmpty else { return }
|
||||
|
||||
Publishers.CombineLatest3(
|
||||
viewModel.$isRecording,
|
||||
viewModel.$isPreparingToRecord,
|
||||
viewModel.$isProcessing
|
||||
)
|
||||
.receive(on: RunLoop.main)
|
||||
.sink { [weak self] recording, preparing, processing in
|
||||
self?.handleBusyChange(
|
||||
recording: recording,
|
||||
preparing: preparing,
|
||||
processing: processing,
|
||||
viewModel: viewModel
|
||||
)
|
||||
}
|
||||
.store(in: &cancellables)
|
||||
|
||||
// Keep waveform / app name / copy fresh while visible.
|
||||
viewModel.objectWillChange
|
||||
.receive(on: RunLoop.main)
|
||||
.sink { [weak self] _ in
|
||||
guard let self, self.panel?.isVisible == true else { return }
|
||||
self.refreshContent(viewModel: viewModel)
|
||||
self.resizeToFit()
|
||||
}
|
||||
.store(in: &cancellables)
|
||||
|
||||
NotificationCenter.default.publisher(for: NSApplication.didChangeScreenParametersNotification)
|
||||
.receive(on: RunLoop.main)
|
||||
.sink { [weak self] _ in self?.reposition() }
|
||||
.store(in: &cancellables)
|
||||
}
|
||||
|
||||
private func handleBusyChange(
|
||||
recording: Bool,
|
||||
preparing: Bool,
|
||||
processing: Bool,
|
||||
viewModel: MacDictationViewModel
|
||||
) {
|
||||
let busy = recording || preparing || processing
|
||||
|
||||
if busy {
|
||||
hideWorkItem?.cancel()
|
||||
hideWorkItem = nil
|
||||
showingCompletion = false
|
||||
wasBusy = true
|
||||
present(viewModel: viewModel)
|
||||
return
|
||||
}
|
||||
|
||||
// Transition: busy → idle. Flash a short "done" state, then hide.
|
||||
if wasBusy {
|
||||
wasBusy = false
|
||||
showingCompletion = true
|
||||
present(viewModel: viewModel)
|
||||
scheduleHide()
|
||||
return
|
||||
}
|
||||
|
||||
if !showingCompletion {
|
||||
hideImmediately()
|
||||
}
|
||||
}
|
||||
|
||||
private func present(viewModel: MacDictationViewModel) {
|
||||
ensurePanel(viewModel: viewModel)
|
||||
refreshContent(viewModel: viewModel)
|
||||
resizeToFit()
|
||||
reposition()
|
||||
|
||||
guard let panel else { return }
|
||||
if panel.isVisible {
|
||||
// Already up — still bump to front in case another space stole it.
|
||||
panel.orderFrontRegardless()
|
||||
return
|
||||
}
|
||||
|
||||
panel.alphaValue = 0
|
||||
panel.orderFrontRegardless()
|
||||
NSAnimationContext.runAnimationGroup { ctx in
|
||||
ctx.duration = 0.22
|
||||
panel.animator().alphaValue = 1
|
||||
}
|
||||
}
|
||||
|
||||
private func ensurePanel(viewModel: MacDictationViewModel) {
|
||||
if panel != nil { return }
|
||||
|
||||
let host = NSHostingView(rootView: makeRoot(viewModel: viewModel))
|
||||
host.frame = NSRect(origin: .zero, size: fallbackSize)
|
||||
hosting = host
|
||||
|
||||
let panel = NSPanel(
|
||||
contentRect: NSRect(origin: .zero, size: fallbackSize),
|
||||
styleMask: [.borderless, .nonactivatingPanel],
|
||||
backing: .buffered,
|
||||
defer: false
|
||||
)
|
||||
panel.contentView = host
|
||||
panel.isOpaque = false
|
||||
panel.backgroundColor = .clear
|
||||
panel.hasShadow = false
|
||||
// Above normal floating windows so the HUD stays visible over browsers /
|
||||
// full-screen apps, without going as high as the screen saver.
|
||||
panel.level = NSWindow.Level(rawValue: Int(CGWindowLevelForKey(.floatingWindow)) + 1)
|
||||
panel.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary, .stationary]
|
||||
panel.isMovableByWindowBackground = false
|
||||
panel.hidesOnDeactivate = false
|
||||
panel.ignoresMouseEvents = false
|
||||
panel.becomesKeyOnlyIfNeeded = true
|
||||
self.panel = panel
|
||||
}
|
||||
|
||||
private func refreshContent(viewModel: MacDictationViewModel) {
|
||||
hosting?.rootView = makeRoot(viewModel: viewModel)
|
||||
}
|
||||
|
||||
private func makeRoot(viewModel: MacDictationViewModel) -> AnyView {
|
||||
AnyView(
|
||||
MacDictationOverlayView(viewModel: viewModel)
|
||||
.macSystemPalette()
|
||||
.environment(\.locale, viewModel.config.uiLanguage.swiftUILocale)
|
||||
.preferredColorScheme(MacAppearancePreference.current.colorScheme)
|
||||
)
|
||||
}
|
||||
|
||||
private func resizeToFit() {
|
||||
guard let panel, let hosting else { return }
|
||||
hosting.layoutSubtreeIfNeeded()
|
||||
let fitting = hosting.fittingSize
|
||||
let width = fitting.width.isFinite && fitting.width > 1
|
||||
? min(max(fitting.width, 300), 520)
|
||||
: fallbackSize.width
|
||||
let height = fitting.height.isFinite && fitting.height > 1
|
||||
? max(fitting.height, fallbackSize.height)
|
||||
: fallbackSize.height
|
||||
var frame = panel.frame
|
||||
let midX = frame.midX
|
||||
frame.size = NSSize(width: width, height: height)
|
||||
if midX.isFinite {
|
||||
frame.origin.x = midX - width / 2
|
||||
}
|
||||
panel.setFrame(frame, display: true)
|
||||
hosting.frame = NSRect(origin: .zero, size: frame.size)
|
||||
}
|
||||
|
||||
private func reposition() {
|
||||
guard let panel else { return }
|
||||
let screen = NSScreen.main ?? NSScreen.screens.first
|
||||
guard let visible = screen?.visibleFrame else { return }
|
||||
let size = panel.frame.size
|
||||
let origin = NSPoint(
|
||||
x: visible.midX - size.width / 2,
|
||||
y: visible.minY + bottomMargin
|
||||
)
|
||||
panel.setFrameOrigin(origin)
|
||||
}
|
||||
|
||||
private func scheduleHide() {
|
||||
hideWorkItem?.cancel()
|
||||
let work = DispatchWorkItem { [weak self] in
|
||||
self?.showingCompletion = false
|
||||
self?.hideAnimated()
|
||||
}
|
||||
hideWorkItem = work
|
||||
DispatchQueue.main.asyncAfter(deadline: .now() + 1.15, execute: work)
|
||||
}
|
||||
|
||||
private func hideImmediately() {
|
||||
hideWorkItem?.cancel()
|
||||
hideWorkItem = nil
|
||||
showingCompletion = false
|
||||
panel?.orderOut(nil)
|
||||
panel?.alphaValue = 1
|
||||
}
|
||||
|
||||
private func hideAnimated() {
|
||||
guard let panel, panel.isVisible else {
|
||||
hideImmediately()
|
||||
return
|
||||
}
|
||||
NSAnimationContext.runAnimationGroup({ ctx in
|
||||
ctx.duration = 0.2
|
||||
panel.animator().alphaValue = 0
|
||||
}, completionHandler: { [weak self] in
|
||||
Task { @MainActor in
|
||||
self?.panel?.orderOut(nil)
|
||||
self?.panel?.alphaValue = 1
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
// MacDictationOverlayView.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Compact bottom-of-screen HUD shown while dictating. Lives inside a
|
||||
// non-activating NSPanel so it never steals focus from the front app.
|
||||
// One-line layout: status / live transcript preview + waveform + stop.
|
||||
|
||||
import SwiftUI
|
||||
|
||||
struct MacDictationOverlayView: View {
|
||||
@ObservedObject var viewModel: MacDictationViewModel
|
||||
@Environment(\.themePalette) private var palette
|
||||
|
||||
private var lang: AppUILanguage { viewModel.config.uiLanguage }
|
||||
|
||||
private var isBusy: Bool {
|
||||
viewModel.isRecording || viewModel.isPreparingToRecord || viewModel.isProcessing
|
||||
}
|
||||
|
||||
/// Trimmed live / final transcript for the single-line preview.
|
||||
private var previewText: String {
|
||||
viewModel.transcript.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
|
||||
private var hasPreview: Bool { !previewText.isEmpty }
|
||||
|
||||
private var showsLiveBadge: Bool {
|
||||
viewModel.isRecording && viewModel.isStreamingPartial
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: Spacing.sm) {
|
||||
statusDot
|
||||
primaryLine
|
||||
Spacer(minLength: Spacing.xs)
|
||||
trailingControl
|
||||
}
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.padding(.vertical, 11)
|
||||
.frame(minWidth: 300, idealWidth: 400, maxWidth: 520)
|
||||
.fixedSize(horizontal: true, vertical: true)
|
||||
.background(palette.surface, in: Capsule(style: .continuous))
|
||||
.overlay(
|
||||
Capsule(style: .continuous)
|
||||
.stroke(palette.dividerStrong, lineWidth: 0.5)
|
||||
)
|
||||
.shadow(color: Color.black.opacity(0.22), radius: 14, y: 5)
|
||||
.padding(2)
|
||||
.animation(Motion.soft, value: hasPreview)
|
||||
.animation(Motion.quick, value: viewModel.isRecording)
|
||||
.animation(Motion.quick, value: viewModel.isStreamingPartial)
|
||||
}
|
||||
|
||||
// MARK: - Primary line (status or one-line transcript)
|
||||
|
||||
@ViewBuilder
|
||||
private var primaryLine: some View {
|
||||
if hasPreview {
|
||||
HStack(spacing: 6) {
|
||||
if showsLiveBadge {
|
||||
liveBadge
|
||||
}
|
||||
Text(previewText)
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.lineLimit(1)
|
||||
.truncationMode(.head)
|
||||
.frame(maxWidth: showsLiveBadge ? 280 : 320, alignment: .leading)
|
||||
.contentTransition(.opacity)
|
||||
.animation(Motion.quick, value: previewText)
|
||||
.accessibilityLabel(previewText)
|
||||
}
|
||||
} else {
|
||||
HStack(spacing: 6) {
|
||||
Text(statusText)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.contentTransition(.opacity)
|
||||
if let appName = viewModel.foregroundAppName, isBusy {
|
||||
Text("·")
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.45))
|
||||
Text(appName)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.lineLimit(1)
|
||||
}
|
||||
}
|
||||
.animation(Motion.quick, value: statusText)
|
||||
}
|
||||
}
|
||||
|
||||
private var liveBadge: some View {
|
||||
Text(MacL10n.string("mac.overlay.live", language: lang))
|
||||
.font(.system(size: 9, weight: .semibold))
|
||||
.foregroundStyle(palette.recordRed)
|
||||
.padding(.horizontal, 5)
|
||||
.padding(.vertical, 2)
|
||||
.background(palette.recordRed.opacity(0.12), in: Capsule(style: .continuous))
|
||||
.accessibilityHidden(true)
|
||||
}
|
||||
|
||||
private var statusDot: some View {
|
||||
Circle()
|
||||
.fill(dotColor)
|
||||
.frame(width: 8, height: 8)
|
||||
.animation(Motion.quick, value: viewModel.isRecording)
|
||||
.animation(Motion.quick, value: viewModel.isProcessing)
|
||||
.animation(Motion.quick, value: viewModel.isPreparingToRecord)
|
||||
.animation(Motion.quick, value: viewModel.isStreamingPartial)
|
||||
}
|
||||
|
||||
private var dotColor: Color {
|
||||
if viewModel.isRecording {
|
||||
return viewModel.isStreamingPartial ? palette.accent : palette.recordRed
|
||||
}
|
||||
if viewModel.isPreparingToRecord || viewModel.isProcessing { return palette.warning }
|
||||
return palette.accent
|
||||
}
|
||||
|
||||
private var statusText: String {
|
||||
if viewModel.isRecording {
|
||||
return MacL10n.string("mac.overlay.listening", language: lang)
|
||||
}
|
||||
if viewModel.isPreparingToRecord {
|
||||
return MacL10n.string("mac.overlay.preparing", language: lang)
|
||||
}
|
||||
if viewModel.isProcessing {
|
||||
if viewModel.isStreamingPartial {
|
||||
return MacL10n.string("mac.overlay.polishing", language: lang)
|
||||
}
|
||||
return MacL10n.string("mac.overlay.transcribing", language: lang)
|
||||
}
|
||||
return MacL10n.string("mac.overlay.done", language: lang)
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var trailingControl: some View {
|
||||
if viewModel.isRecording {
|
||||
MiniWaveform(
|
||||
level: viewModel.audioLevel,
|
||||
barCount: 7,
|
||||
tint: (viewModel.isStreamingPartial ? palette.accent : palette.recordRed)
|
||||
.opacity(0.9),
|
||||
maxBarHeight: 28,
|
||||
barWidth: 3.5,
|
||||
barSpacing: 2.5
|
||||
)
|
||||
stopButton
|
||||
} else if viewModel.isPreparingToRecord || viewModel.isProcessing {
|
||||
ProgressView()
|
||||
.controlSize(.small)
|
||||
} else {
|
||||
Image(systemName: "checkmark.circle.fill")
|
||||
.foregroundStyle(palette.accent)
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
}
|
||||
}
|
||||
|
||||
private var stopButton: some View {
|
||||
Button(action: viewModel.toggleRecording) {
|
||||
Image(systemName: "stop.fill")
|
||||
.font(.system(size: 10, weight: .bold))
|
||||
.foregroundStyle(palette.textOnAccent)
|
||||
.frame(width: 28, height: 28)
|
||||
.background(palette.recordRed, in: Circle())
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
.accessibilityLabel(MacL10n.string("mac.record.stop", language: lang))
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,7 @@
|
||||
// MacDictationPipeline.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Dictation pipeline: samples → ASR (cloud or local) → polish.
|
||||
// Cloud path reuses `CloudASRClientFactory`; local path uses Qwen3-ASR (MLX)
|
||||
// with Apple Speech fallback when weights are missing.
|
||||
// Dictation pipeline: samples → ASR (cloud or local, chunked when long) → polish.
|
||||
|
||||
import Foundation
|
||||
|
||||
@@ -24,39 +22,60 @@ enum MacDictationError: Error, LocalizedError {
|
||||
}
|
||||
}
|
||||
|
||||
/// Outcome of ASR that ran while the microphone was still open.
|
||||
struct MacLiveASRCaptureResult: Sendable {
|
||||
let raw: String
|
||||
let chunkWarning: String?
|
||||
let localBias: LocalASRBiasPayload?
|
||||
/// When true, callers should fall back to batch ASR on the recorded samples.
|
||||
let shouldFallbackToBatch: Bool
|
||||
}
|
||||
|
||||
enum MacDictationPipeline {
|
||||
/// Runs ASR then best-effort polish. Polish failures fall back to raw text.
|
||||
static func run(samples: [Float], store: AppGroupStore) async throws -> String {
|
||||
/// First-chunk threshold: longer local utterances use pipelined chunk ASR.
|
||||
private static let chunkedLocalThresholdSamples = Int(
|
||||
FlowUtteranceChunkConfig.flowDefault.maxChunkDurationSeconds(forChunkIndex: 0) * 16_000
|
||||
)
|
||||
|
||||
/// Whether the active engine can surface `onPartial` text while recording.
|
||||
static func supportsLivePartials(store: AppGroupStore) -> Bool {
|
||||
if store.engineMode == "local" { return true }
|
||||
return CloudASRModelCatalog.strategy(for: store.asrProviderId) != .localFallback
|
||||
}
|
||||
|
||||
/// Runs ASR then polish. Polish failures return cleaned raw ASR plus a warning.
|
||||
static func run(
|
||||
samples: [Float],
|
||||
store: AppGroupStore,
|
||||
onPartial: (@Sendable (String) -> Void)? = nil
|
||||
) async throws -> MacDictationResult {
|
||||
guard !samples.isEmpty else { throw MacDictationError.noAudio }
|
||||
|
||||
let locale = Locale(identifier: store.localeId.isEmpty ? "zh-CN" : store.localeId)
|
||||
let locale = resolvedLocale(store: store)
|
||||
var chunkWarning: String?
|
||||
let raw: String
|
||||
var localBias: LocalASRBiasPayload?
|
||||
|
||||
if store.engineMode == "local" {
|
||||
MacAppContextService.captureAndPersist(to: store)
|
||||
let capabilities = MacLocalASRService.currentCapabilities()
|
||||
let bias = LocalASRBiasAdapter.adapt(
|
||||
LocalASRBiasRequest(
|
||||
dictionary: store.personalDictionary,
|
||||
localBias = resolveLocalBias(store: store, locale: locale)
|
||||
if samples.count > chunkedLocalThresholdSamples {
|
||||
let chunked = try await transcribeLocalChunked(
|
||||
samples: samples,
|
||||
locale: locale,
|
||||
frontAppBundleId: MacAppContextService.frontmostBundleIdentifier(),
|
||||
capabilities: capabilities
|
||||
)
|
||||
)
|
||||
localBias = bias
|
||||
LocalASRBiasDiagnosticsStore.save(
|
||||
payload: bias,
|
||||
modelId: MacLocalASRService.selectedModelDefinition()?.id,
|
||||
backendLabel: MacLocalASRService.currentBackendLabel()
|
||||
bias: localBias,
|
||||
onPartial: onPartial
|
||||
)
|
||||
raw = chunked.text
|
||||
chunkWarning = chunked.chunkWarning
|
||||
} else {
|
||||
raw = try await MacLocalASRService.transcribe(
|
||||
samples: samples,
|
||||
locale: locale,
|
||||
bias: bias
|
||||
bias: localBias
|
||||
)
|
||||
}
|
||||
} else {
|
||||
let strategy = CloudASRModelCatalog.strategy(for: store.providerId)
|
||||
let strategy = CloudASRModelCatalog.strategy(for: store.asrProviderId)
|
||||
guard strategy != .localFallback else { throw MacDictationError.providerHasNoCloudASR }
|
||||
|
||||
let client = CloudASRClientFactory.make(store: store)
|
||||
@@ -69,6 +88,85 @@ enum MacDictationPipeline {
|
||||
)
|
||||
}
|
||||
|
||||
return try await polishCapturedASR(
|
||||
raw: raw,
|
||||
store: store,
|
||||
localBias: localBias,
|
||||
chunkWarning: chunkWarning
|
||||
)
|
||||
}
|
||||
|
||||
/// Consumes a live mic snapshot stream until finished; yields stitched partials.
|
||||
static func captureLive(
|
||||
stream: AsyncStream<AudioBufferSnapshot>,
|
||||
store: AppGroupStore,
|
||||
onPartial: @escaping @Sendable (String) -> Void
|
||||
) async -> MacLiveASRCaptureResult {
|
||||
let locale = resolvedLocale(store: store)
|
||||
let localBias: LocalASRBiasPayload?
|
||||
if store.engineMode == "local" {
|
||||
localBias = resolveLocalBias(store: store, locale: locale)
|
||||
} else {
|
||||
localBias = nil
|
||||
}
|
||||
|
||||
do {
|
||||
let adapter = try makeChunkASRAdapter(
|
||||
store: store,
|
||||
locale: locale,
|
||||
bias: localBias
|
||||
)
|
||||
if let cloudAdapter = adapter as? MacCloudASRChunkAdapter {
|
||||
try? await cloudAdapter.prepare()
|
||||
}
|
||||
|
||||
let pipeline = ChunkedUtterancePipeline(
|
||||
asr: adapter,
|
||||
locale: locale,
|
||||
config: .flowDefault
|
||||
)
|
||||
let outcome = await pipeline.transcribe(stream: stream, onPartial: onPartial)
|
||||
|
||||
switch outcome {
|
||||
case .success(let success):
|
||||
return MacLiveASRCaptureResult(
|
||||
raw: success.text,
|
||||
chunkWarning: success.chunkWarnings.first,
|
||||
localBias: localBias,
|
||||
shouldFallbackToBatch: false
|
||||
)
|
||||
case .failure:
|
||||
return MacLiveASRCaptureResult(
|
||||
raw: "",
|
||||
chunkWarning: nil,
|
||||
localBias: localBias,
|
||||
shouldFallbackToBatch: true
|
||||
)
|
||||
case .cancelled:
|
||||
return MacLiveASRCaptureResult(
|
||||
raw: "",
|
||||
chunkWarning: nil,
|
||||
localBias: localBias,
|
||||
shouldFallbackToBatch: true
|
||||
)
|
||||
}
|
||||
} catch {
|
||||
return MacLiveASRCaptureResult(
|
||||
raw: "",
|
||||
chunkWarning: nil,
|
||||
localBias: localBias,
|
||||
shouldFallbackToBatch: true
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Polish-only step after live or batch ASR has produced raw text.
|
||||
static func polishCapturedASR(
|
||||
raw: String,
|
||||
store: AppGroupStore,
|
||||
localBias: LocalASRBiasPayload?,
|
||||
chunkWarning: String?
|
||||
) async throws -> MacDictationResult {
|
||||
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty else { throw MacDictationError.emptyTranscript }
|
||||
|
||||
@@ -91,14 +189,127 @@ enum MacDictationPipeline {
|
||||
polishContext = nil
|
||||
}
|
||||
|
||||
if let polished = try? await PolishingService(store: store).polish(
|
||||
do {
|
||||
let polished = try await PolishingService(store: store).polish(
|
||||
postASR,
|
||||
mode: store.polishModeForPipeline,
|
||||
context: polishContext
|
||||
),
|
||||
!polished.isEmpty {
|
||||
return polished
|
||||
)
|
||||
guard !polished.isEmpty else {
|
||||
throw PolishingService.PolishError.noTranscript
|
||||
}
|
||||
return MacDictationResult(
|
||||
text: polished,
|
||||
polishWarning: nil,
|
||||
chunkWarning: chunkWarning
|
||||
)
|
||||
} catch {
|
||||
let delivery = TranscriptionPolishFallback.makeDelivery(
|
||||
rawText: postASR,
|
||||
error: error,
|
||||
engineMode: store.engineMode,
|
||||
chunkWarning: chunkWarning
|
||||
)
|
||||
return MacDictationResult(
|
||||
text: delivery.text,
|
||||
polishWarning: delivery.polishWarning,
|
||||
chunkWarning: nil
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Chunked local ASR
|
||||
|
||||
private struct ChunkedLocalResult {
|
||||
let text: String
|
||||
let chunkWarning: String?
|
||||
}
|
||||
|
||||
private static func resolvedLocale(store: AppGroupStore) -> Locale {
|
||||
Locale(identifier: store.localeId.isEmpty ? "zh-CN" : store.localeId)
|
||||
}
|
||||
|
||||
private static func resolveLocalBias(
|
||||
store: AppGroupStore,
|
||||
locale: Locale
|
||||
) -> LocalASRBiasPayload? {
|
||||
MacAppContextService.captureAndPersist(to: store)
|
||||
let capabilities = MacLocalASRService.currentCapabilities()
|
||||
let bias = LocalASRBiasAdapter.adapt(
|
||||
LocalASRBiasRequest(
|
||||
dictionary: store.personalDictionary,
|
||||
locale: locale,
|
||||
frontAppBundleId: MacAppContextService.frontmostBundleIdentifier(),
|
||||
capabilities: capabilities
|
||||
)
|
||||
)
|
||||
LocalASRBiasDiagnosticsStore.save(
|
||||
payload: bias,
|
||||
modelId: MacLocalASRService.selectedModelDefinition()?.id,
|
||||
backendLabel: MacLocalASRService.currentBackendLabel()
|
||||
)
|
||||
return bias
|
||||
}
|
||||
|
||||
private static func makeChunkASRAdapter(
|
||||
store: AppGroupStore,
|
||||
locale: Locale,
|
||||
bias: LocalASRBiasPayload?
|
||||
) throws -> any ASRChunkTranscribing {
|
||||
if store.engineMode == "local" {
|
||||
return MacLocalASRChunkAdapter(locale: locale, bias: bias)
|
||||
}
|
||||
return try MacCloudASRChunkAdapter(store: store)
|
||||
}
|
||||
|
||||
private static func transcribeLocalChunked(
|
||||
samples: [Float],
|
||||
locale: Locale,
|
||||
bias: LocalASRBiasPayload?,
|
||||
onPartial: (@Sendable (String) -> Void)?
|
||||
) async throws -> ChunkedLocalResult {
|
||||
let adapter = MacLocalASRChunkAdapter(locale: locale, bias: bias)
|
||||
let pipeline = ChunkedUtterancePipeline(
|
||||
asr: adapter,
|
||||
locale: locale,
|
||||
config: .flowDefault
|
||||
)
|
||||
let outcome = await pipeline.transcribe(
|
||||
stream: audioStream(from: samples),
|
||||
onPartial: { partial in
|
||||
onPartial?(partial)
|
||||
}
|
||||
)
|
||||
|
||||
switch outcome {
|
||||
case .success(let success):
|
||||
let warning = success.chunkWarnings.first
|
||||
return ChunkedLocalResult(text: success.text, chunkWarning: warning)
|
||||
case .failure(let message):
|
||||
throw MacLocalASRError.qwen3InferenceFailed(message)
|
||||
case .cancelled:
|
||||
throw MacLocalASRError.qwen3InferenceFailed("Cancelled")
|
||||
}
|
||||
}
|
||||
|
||||
/// Feeds recorded PCM into the chunker as if it arrived incrementally.
|
||||
private static func audioStream(
|
||||
from samples: [Float],
|
||||
sliceSamples: Int = 8_000
|
||||
) -> AsyncStream<AudioBufferSnapshot> {
|
||||
AsyncStream { continuation in
|
||||
var offset = 0
|
||||
while offset < samples.count {
|
||||
let end = min(offset + sliceSamples, samples.count)
|
||||
continuation.yield(
|
||||
AudioBufferSnapshot(
|
||||
samples: Array(samples[offset..<end]),
|
||||
sampleRate: 16_000
|
||||
)
|
||||
)
|
||||
offset = end
|
||||
}
|
||||
continuation.finish()
|
||||
}
|
||||
return postASR
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
// MacDictationResult.swift
|
||||
// OSGKeyboard · Mac
|
||||
|
||||
import Foundation
|
||||
|
||||
struct MacDictationResult: Sendable, Equatable {
|
||||
let text: String
|
||||
/// Shown when DeepSeek / cloud polish failed but raw ASR was delivered.
|
||||
let polishWarning: String?
|
||||
/// Non-fatal per-chunk ASR issues from long utterance chunking.
|
||||
let chunkWarning: String?
|
||||
}
|
||||
@@ -28,7 +28,7 @@ enum MacSection: String, CaseIterable, Identifiable {
|
||||
|
||||
var systemImage: String {
|
||||
switch self {
|
||||
case .dashboard: return "square.grid.2x2"
|
||||
case .dashboard: return "house"
|
||||
case .history: return "clock.arrow.circlepath"
|
||||
case .dictionary: return "character.book.closed"
|
||||
case .settings: return "gearshape"
|
||||
@@ -45,7 +45,13 @@ final class MacDictationViewModel: ObservableObject {
|
||||
@Published var selectedSection: MacSection = .dashboard
|
||||
|
||||
@Published var isRecording = false
|
||||
/// True while microphone permission / engine start is in flight.
|
||||
/// Drives the overlay so the HUD appears on Option-down immediately,
|
||||
/// instead of waiting for the async `beginRecording` to finish.
|
||||
@Published private(set) var isPreparingToRecord = false
|
||||
@Published var isProcessing = false
|
||||
/// True once live ASR has surfaced at least one partial during this take.
|
||||
@Published private(set) var isStreamingPartial = false
|
||||
@Published var transcript = ""
|
||||
@Published var statusMessage = ""
|
||||
@Published var audioLevel: Float = 0
|
||||
@@ -55,6 +61,7 @@ final class MacDictationViewModel: ObservableObject {
|
||||
|
||||
@Published var autoPasteEnabled: Bool
|
||||
@Published var hotkeyEnabled: Bool
|
||||
@Published var hotkeyTrigger: MacHotkeyTrigger
|
||||
|
||||
@Published var config: ProviderConfig
|
||||
|
||||
@@ -64,6 +71,12 @@ final class MacDictationViewModel: ObservableObject {
|
||||
private var levelTimer: Timer?
|
||||
private var sessionTimer: Timer?
|
||||
private var cancellables = Set<AnyCancellable>()
|
||||
/// In-flight `beginRecording` started by the hotkey — cancelled if the
|
||||
/// key is released before the engine is ready (avoids a stuck session).
|
||||
private var hotkeyBeginTask: Task<Void, Never>?
|
||||
/// Live chunked ASR while recording (cloud / supported local paths).
|
||||
/// Finished in `finishRecording` so partials can become the final draft.
|
||||
private var liveCaptureTask: Task<MacLiveASRCaptureResult, Never>?
|
||||
|
||||
let usageStatistics: UsageStatisticsStore
|
||||
let speechHistory = SpeechHistoryStore.shared
|
||||
@@ -71,6 +84,7 @@ final class MacDictationViewModel: ObservableObject {
|
||||
private enum StoredKeys {
|
||||
static let autoPaste = "mac.autoPasteEnabled"
|
||||
static let hotkey = "mac.hotkeyEnabled"
|
||||
static let hotkeyTrigger = MacHotkeyTrigger.storageKey
|
||||
}
|
||||
|
||||
init(defaults: UserDefaults = .standard) {
|
||||
@@ -79,6 +93,9 @@ final class MacDictationViewModel: ObservableObject {
|
||||
self.usageStatistics = UsageStatisticsStore(defaults: defaults)
|
||||
self.autoPasteEnabled = defaults.object(forKey: StoredKeys.autoPaste) as? Bool ?? true
|
||||
self.hotkeyEnabled = defaults.object(forKey: StoredKeys.hotkey) as? Bool ?? true
|
||||
self.hotkeyTrigger = MacHotkeyTrigger(
|
||||
rawValue: defaults.string(forKey: StoredKeys.hotkeyTrigger) ?? ""
|
||||
) ?? .rightOption
|
||||
|
||||
MacICloudSyncBootstrap.configure(defaults: defaults)
|
||||
statusMessage = MacL10n.string("mac.status.ready", language: config.uiLanguage)
|
||||
@@ -114,10 +131,16 @@ final class MacDictationViewModel: ObservableObject {
|
||||
|
||||
// MARK: - Derived
|
||||
|
||||
var selectableProviders: [LLMProvider] {
|
||||
LLMProvider.presets.filter {
|
||||
$0.isUserSelectable && $0.cloudASRStrategy != .localFallback
|
||||
var polishSelectableProviders: [LLMProvider] {
|
||||
LLMProvider.userSelectablePresets
|
||||
}
|
||||
|
||||
var asrSelectableProviders: [LLMProvider] {
|
||||
LLMProvider.asrSelectablePresets
|
||||
}
|
||||
|
||||
var selectableProviders: [LLMProvider] {
|
||||
asrSelectableProviders
|
||||
}
|
||||
|
||||
var dictionaryTermCount: Int {
|
||||
@@ -185,6 +208,12 @@ final class MacDictationViewModel: ObservableObject {
|
||||
if enabled { hotkeyService.start() } else { hotkeyService.stop() }
|
||||
}
|
||||
|
||||
func setHotkeyTrigger(_ trigger: MacHotkeyTrigger) {
|
||||
hotkeyTrigger = trigger
|
||||
defaults.set(trigger.rawValue, forKey: StoredKeys.hotkeyTrigger)
|
||||
hotkeyService.trigger = trigger
|
||||
}
|
||||
|
||||
func setEngineMode(_ mode: String) {
|
||||
config.engineMode = mode
|
||||
}
|
||||
@@ -192,73 +221,212 @@ final class MacDictationViewModel: ObservableObject {
|
||||
// MARK: - Recording
|
||||
|
||||
func toggleRecording() {
|
||||
if isRecording { finishRecording() } else { beginRecording() }
|
||||
if isRecording || isPreparingToRecord {
|
||||
cancelOrFinishRecording()
|
||||
} else {
|
||||
Task { await beginRecording() }
|
||||
}
|
||||
}
|
||||
|
||||
func beginRecording() {
|
||||
guard !isProcessing else { return }
|
||||
func beginRecording() async {
|
||||
guard !isProcessing, !isRecording, !isPreparingToRecord else { return }
|
||||
isPreparingToRecord = true
|
||||
let store = AppGroupStore(defaults: defaults)
|
||||
MacAppContextService.captureAndPersist(to: store)
|
||||
refreshForegroundAppName()
|
||||
|
||||
do {
|
||||
try recorder.start()
|
||||
try await recorder.start()
|
||||
// Hotkey may have been released while we awaited mic permission /
|
||||
// engine start — abandon cleanly instead of latching a stuck session.
|
||||
isPreparingToRecord = false
|
||||
if Task.isCancelled {
|
||||
_ = recorder.stop()
|
||||
return
|
||||
}
|
||||
isRecording = true
|
||||
transcript = ""
|
||||
isStreamingPartial = false
|
||||
statusMessage = MacL10n.string("mac.status.listening", language: config.uiLanguage)
|
||||
startTimers()
|
||||
startLiveCaptureIfSupported(store: store)
|
||||
// Tiny race: Option released between the cancel check and
|
||||
// `isRecording = true`. Treat it as end-of-hold and finish.
|
||||
if Task.isCancelled {
|
||||
finishRecording()
|
||||
}
|
||||
} catch {
|
||||
isPreparingToRecord = false
|
||||
if !Task.isCancelled {
|
||||
statusMessage = error.localizedDescription
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func finishRecording() {
|
||||
guard isRecording else { return }
|
||||
isRecording = false
|
||||
isProcessing = true
|
||||
statusMessage = MacL10n.string("mac.status.transcribing", language: config.uiLanguage)
|
||||
let hadLivePartial = isStreamingPartial
|
||||
statusMessage = MacL10n.string(
|
||||
hadLivePartial ? "mac.status.polishing" : "mac.status.transcribing",
|
||||
language: config.uiLanguage
|
||||
)
|
||||
stopTimers()
|
||||
audioLevel = 0
|
||||
let samples = recorder.stop()
|
||||
let store = AppGroupStore(defaults: defaults)
|
||||
let liveTask = liveCaptureTask
|
||||
liveCaptureTask = nil
|
||||
|
||||
Task { [weak self] in
|
||||
guard let self else { return }
|
||||
do {
|
||||
let text = try await MacDictationPipeline.run(samples: samples, store: store)
|
||||
self.transcript = text
|
||||
let pasted = try self.deliver(text)
|
||||
self.recordUsage(for: text)
|
||||
self.speechHistory.append(text: text)
|
||||
self.statusMessage = self.statusAfterDelivery(pasted: pasted)
|
||||
let result: MacDictationResult
|
||||
if let liveTask {
|
||||
let capture = await liveTask.value
|
||||
let trimmedLive = capture.raw.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if !capture.shouldFallbackToBatch, !trimmedLive.isEmpty {
|
||||
if self.transcript.isEmpty {
|
||||
self.transcript = trimmedLive
|
||||
}
|
||||
result = try await MacDictationPipeline.polishCapturedASR(
|
||||
raw: capture.raw,
|
||||
store: store,
|
||||
localBias: capture.localBias,
|
||||
chunkWarning: capture.chunkWarning
|
||||
)
|
||||
} else {
|
||||
result = try await MacDictationPipeline.run(
|
||||
samples: samples,
|
||||
store: store,
|
||||
onPartial: { [weak self] partial in
|
||||
Task { @MainActor in
|
||||
self?.transcript = partial
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
} else {
|
||||
result = try await MacDictationPipeline.run(
|
||||
samples: samples,
|
||||
store: store,
|
||||
onPartial: { [weak self] partial in
|
||||
Task { @MainActor in
|
||||
self?.transcript = partial
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
self.transcript = result.text
|
||||
let pasted = try await self.deliver(result.text)
|
||||
self.recordUsage(for: result.text)
|
||||
self.speechHistory.append(text: result.text)
|
||||
self.statusMessage = self.statusAfterDelivery(
|
||||
pasted: pasted,
|
||||
polishWarning: result.polishWarning,
|
||||
chunkWarning: result.chunkWarning
|
||||
)
|
||||
} catch {
|
||||
self.statusMessage = error.localizedDescription
|
||||
}
|
||||
self.isStreamingPartial = false
|
||||
self.isProcessing = false
|
||||
}
|
||||
}
|
||||
|
||||
private func deliver(_ text: String) throws -> Bool {
|
||||
try MacTextInsertionService.insert(text, autoPaste: autoPasteEnabled)
|
||||
private func startLiveCaptureIfSupported(store: AppGroupStore) {
|
||||
guard MacDictationPipeline.supportsLivePartials(store: store) else { return }
|
||||
let stream = recorder.makeSnapshotStream()
|
||||
liveCaptureTask = Task { [weak self] in
|
||||
await MacDictationPipeline.captureLive(
|
||||
stream: stream,
|
||||
store: store,
|
||||
onPartial: { [weak self] partial in
|
||||
Task { @MainActor in
|
||||
guard let self else { return }
|
||||
guard self.isRecording || self.isProcessing else { return }
|
||||
let trimmed = partial.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty else { return }
|
||||
self.transcript = partial
|
||||
self.isStreamingPartial = true
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
private func statusAfterDelivery(pasted: Bool) -> String {
|
||||
private func cancelLiveCapture() {
|
||||
liveCaptureTask?.cancel()
|
||||
liveCaptureTask = nil
|
||||
isStreamingPartial = false
|
||||
}
|
||||
|
||||
/// Stops an in-flight prepare, or finishes an active recording.
|
||||
private func cancelOrFinishRecording() {
|
||||
if isRecording {
|
||||
finishRecording()
|
||||
return
|
||||
}
|
||||
if isPreparingToRecord {
|
||||
hotkeyBeginTask?.cancel()
|
||||
hotkeyBeginTask = nil
|
||||
// If the button-triggered prepare wasn't tracked by hotkeyBeginTask,
|
||||
// still clear the preparing flag and stop any engine that raced in.
|
||||
isPreparingToRecord = false
|
||||
cancelLiveCapture()
|
||||
_ = recorder.stop()
|
||||
}
|
||||
}
|
||||
|
||||
private func deliver(_ text: String) async throws -> Bool {
|
||||
try await MacTextInsertionService.insert(text, autoPaste: autoPasteEnabled)
|
||||
}
|
||||
|
||||
private func statusAfterDelivery(
|
||||
pasted: Bool,
|
||||
polishWarning: String? = nil,
|
||||
chunkWarning: String? = nil
|
||||
) -> String {
|
||||
let lang = config.uiLanguage
|
||||
let base: String
|
||||
if autoPasteEnabled, pasted {
|
||||
return MacL10n.string("mac.status.copiedAndPasted", language: lang)
|
||||
base = MacL10n.string("mac.status.copiedAndPasted", language: lang)
|
||||
} else if autoPasteEnabled, !pasted {
|
||||
base = MacL10n.string("mac.status.copied", language: lang)
|
||||
} else {
|
||||
base = MacL10n.string("mac.status.copied", language: lang)
|
||||
}
|
||||
if autoPasteEnabled, !pasted {
|
||||
return MacL10n.string("mac.status.copied", language: lang)
|
||||
|
||||
if let polishWarning, !polishWarning.isEmpty {
|
||||
return MacL10n.format("mac.status.deliveryWithNote", language: lang, base, polishWarning)
|
||||
}
|
||||
return MacL10n.string("mac.status.copied", language: lang)
|
||||
if let chunkWarning, !chunkWarning.isEmpty {
|
||||
return MacL10n.format("mac.status.deliveryWithNote", language: lang, base, chunkWarning)
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
||||
private func wireHotkeyService() {
|
||||
hotkeyService.trigger = hotkeyTrigger
|
||||
hotkeyService.onPressBegan = { [weak self] in
|
||||
self?.beginRecording()
|
||||
guard let self else { return }
|
||||
self.hotkeyBeginTask?.cancel()
|
||||
self.hotkeyBeginTask = Task { [weak self] in
|
||||
await self?.beginRecording()
|
||||
}
|
||||
}
|
||||
hotkeyService.onPressEnded = { [weak self] in
|
||||
self?.finishRecording()
|
||||
guard let self else { return }
|
||||
// Cancel a still-preparing start so a quick Option tap never
|
||||
// latches recording. If recording already began, finish it.
|
||||
if self.isRecording {
|
||||
self.hotkeyBeginTask = nil
|
||||
self.finishRecording()
|
||||
} else {
|
||||
self.hotkeyBeginTask?.cancel()
|
||||
self.hotkeyBeginTask = nil
|
||||
}
|
||||
}
|
||||
if hotkeyEnabled { hotkeyService.start() }
|
||||
}
|
||||
@@ -304,6 +472,10 @@ final class MacDictationViewModel: ObservableObject {
|
||||
config.apply(preset: provider)
|
||||
}
|
||||
|
||||
func selectAsrProvider(_ provider: LLMProvider) {
|
||||
config.applyAsr(preset: provider)
|
||||
}
|
||||
|
||||
func refreshForegroundAppName() {
|
||||
foregroundAppName = MacAppContextService.frontmostApplicationName()
|
||||
}
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
// MacDictionaryView.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Personal dictionary synced via iCloud KVS with the iOS app. Read-only on
|
||||
// the desktop (words are authored on iPhone / iPad): grouped cards (native
|
||||
// `Form`) with search, matching the Settings and History card style.
|
||||
// Personal dictionary synced via iCloud KVS. ScrollView is full-bleed
|
||||
// (scrollbar on the window edge); title + cards share `pageHorizontalInset`.
|
||||
|
||||
import SwiftUI
|
||||
|
||||
@@ -45,16 +44,27 @@ struct MacDictionaryView: View {
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 0) {
|
||||
MacPageHeader(
|
||||
title: MacL10n.string("mac.section.dictionary", language: lang),
|
||||
subtitle: MacL10n.string("mac.page.dictionary.subtitle", language: lang)
|
||||
) {
|
||||
if !entries.isEmpty {
|
||||
searchField
|
||||
}
|
||||
}
|
||||
|
||||
Group {
|
||||
if entries.isEmpty {
|
||||
emptyState
|
||||
.transition(.opacity)
|
||||
} else {
|
||||
form
|
||||
list
|
||||
.transition(.opacity)
|
||||
}
|
||||
}
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
}
|
||||
.background(palette.background)
|
||||
.animation(Motion.soft, value: entries.isEmpty)
|
||||
.task {
|
||||
@@ -64,33 +74,6 @@ struct MacDictionaryView: View {
|
||||
.onReceive(NotificationCenter.default.publisher(for: .personalDictionaryDidSyncFromCloud)) { _ in
|
||||
viewModel.refreshDictionaryFromCloud()
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Grouped cards
|
||||
|
||||
private var form: some View {
|
||||
Form {
|
||||
if sections.isEmpty {
|
||||
Section {
|
||||
Text(MacL10n.string("mac.dict.noMatch", language: lang))
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .center)
|
||||
}
|
||||
} else {
|
||||
ForEach(sections, id: \.category) { section in
|
||||
Section(MacL10n.string(section.category.labelKey, language: lang)) {
|
||||
ForEach(section.items) { entry in
|
||||
row(entry)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
.formStyle(.grouped)
|
||||
.scrollContentBackground(.hidden)
|
||||
.background(palette.background)
|
||||
.animation(Motion.soft, value: query)
|
||||
.safeAreaInset(edge: .top, spacing: 0) { centeredSearchField }
|
||||
.confirmationDialog(
|
||||
MacL10n.string("mac.dict.deleteTitle", language: lang),
|
||||
isPresented: deletionDialogBinding,
|
||||
@@ -110,6 +93,55 @@ struct MacDictionaryView: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - List
|
||||
|
||||
private var list: some View {
|
||||
// Full-bleed ScrollView → scrollbar on the detail pane's right edge.
|
||||
ScrollView {
|
||||
LazyVStack(alignment: .leading, spacing: Spacing.md) {
|
||||
if sections.isEmpty {
|
||||
MacCard {
|
||||
Text(MacL10n.string("mac.dict.noMatch", language: lang))
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .center)
|
||||
}
|
||||
} else {
|
||||
ForEach(sections, id: \.category) { section in
|
||||
categorySection(section)
|
||||
}
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset)
|
||||
.padding(.bottom, Spacing.md)
|
||||
.animation(Motion.soft, value: query)
|
||||
}
|
||||
}
|
||||
|
||||
private func categorySection(
|
||||
_ section: (category: PersonalDictionary.Entry.Category, items: [PersonalDictionary.Entry])
|
||||
) -> some View {
|
||||
VStack(alignment: .leading, spacing: Spacing.xs) {
|
||||
Text(MacL10n.string(section.category.labelKey, language: lang))
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
|
||||
MacCard(padding: 0) {
|
||||
VStack(spacing: 0) {
|
||||
ForEach(Array(section.items.enumerated()), id: \.element.id) { index, entry in
|
||||
row(entry)
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
if index < section.items.count - 1 {
|
||||
// Full-bleed like macOS list rows (not iOS inset separators).
|
||||
Divider()
|
||||
.overlay(palette.divider)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var deletionDialogBinding: Binding<Bool> {
|
||||
Binding(
|
||||
get: { entryPendingDeletion != nil },
|
||||
@@ -117,25 +149,19 @@ struct MacDictionaryView: View {
|
||||
)
|
||||
}
|
||||
|
||||
private var centeredSearchField: some View {
|
||||
HStack {
|
||||
Spacer()
|
||||
private var searchField: some View {
|
||||
HStack(spacing: Spacing.xs) {
|
||||
Image(systemName: "magnifyingglass")
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
TextField(MacL10n.string("mac.dict.search", language: lang), text: $query)
|
||||
.textFieldStyle(.plain)
|
||||
.font(TypeStyle.footnote)
|
||||
}
|
||||
.padding(.horizontal, Spacing.sm)
|
||||
.padding(.vertical, 7)
|
||||
.frame(width: 240)
|
||||
.macGlassSurface(in: Capsule(), fillOpacity: 0.72)
|
||||
.padding(.vertical, 6)
|
||||
.frame(width: 220)
|
||||
.background(palette.surface, in: Capsule())
|
||||
.overlay(Capsule().stroke(palette.divider, lineWidth: 0.5))
|
||||
Spacer()
|
||||
}
|
||||
.padding(.horizontal, Spacing.lg)
|
||||
.padding(.vertical, Spacing.xs)
|
||||
.background(palette.background)
|
||||
}
|
||||
|
||||
private func row(_ entry: PersonalDictionary.Entry) -> some View {
|
||||
@@ -165,19 +191,19 @@ struct MacDictionaryView: View {
|
||||
private var emptyState: some View {
|
||||
VStack(spacing: Spacing.sm) {
|
||||
Image(systemName: "character.book.closed")
|
||||
.font(.system(size: 34))
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.6))
|
||||
.font(.system(size: 34, weight: .light))
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.55))
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
Text(MacL10n.string("mac.dict.empty", language: lang))
|
||||
.font(TypeStyle.body)
|
||||
.font(TypeStyle.headline)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
Text(MacL10n.string("mac.dict.emptyBody", language: lang))
|
||||
.font(TypeStyle.caption)
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.multilineTextAlignment(.center)
|
||||
.frame(maxWidth: 360)
|
||||
.frame(maxWidth: 320)
|
||||
}
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
.padding(.horizontal, Spacing.xl)
|
||||
}
|
||||
|
||||
private func delete(_ entry: PersonalDictionary.Entry) {
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
// MacHistoryView.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Single-column, day-grouped transcript log rendered as grouped cards (the
|
||||
// same native `Form` container as Settings). Every entry shows its full text
|
||||
// inline — no master/detail split, so content never pushes the sidebar out.
|
||||
// Day-grouped transcript log. ScrollView is full-bleed (scrollbar on the
|
||||
// window edge); title + cards share `pageHorizontalInset` on their content.
|
||||
|
||||
import SwiftUI
|
||||
|
||||
@@ -31,36 +30,39 @@ struct MacHistoryView: View {
|
||||
}()
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 0) {
|
||||
MacPageHeader(
|
||||
title: MacL10n.string("mac.section.history", language: lang),
|
||||
subtitle: MacL10n.string("mac.page.history.subtitle", language: lang)
|
||||
) {
|
||||
if !historyStore.entries.isEmpty {
|
||||
Button {
|
||||
showClearConfirmation = true
|
||||
} label: {
|
||||
Label(
|
||||
MacL10n.string("mac.history.clearConfirm", language: lang),
|
||||
systemImage: "trash"
|
||||
)
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
}
|
||||
}
|
||||
|
||||
Group {
|
||||
if historyStore.entries.isEmpty {
|
||||
emptyState
|
||||
.transition(.opacity)
|
||||
} else {
|
||||
form
|
||||
list
|
||||
.transition(.opacity)
|
||||
}
|
||||
}
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
}
|
||||
.background(palette.background)
|
||||
.animation(Motion.soft, value: historyStore.entries.isEmpty)
|
||||
}
|
||||
|
||||
// MARK: - Grouped cards
|
||||
|
||||
private var form: some View {
|
||||
Form {
|
||||
ForEach(historyStore.groupedByDay, id: \.day) { group in
|
||||
Section(Self.dayFormatter.string(from: group.day)) {
|
||||
ForEach(group.items) { entry in
|
||||
row(entry)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
.formStyle(.grouped)
|
||||
.scrollContentBackground(.hidden)
|
||||
.background(palette.background)
|
||||
.safeAreaInset(edge: .top, spacing: 0) { toolbar }
|
||||
.confirmationDialog(
|
||||
MacL10n.string("mac.history.clearTitle", language: lang),
|
||||
isPresented: $showClearConfirmation,
|
||||
@@ -75,21 +77,43 @@ struct MacHistoryView: View {
|
||||
}
|
||||
}
|
||||
|
||||
private var toolbar: some View {
|
||||
HStack {
|
||||
Spacer()
|
||||
Button {
|
||||
showClearConfirmation = true
|
||||
} label: {
|
||||
Label(MacL10n.string("mac.history.clearConfirm", language: lang), systemImage: "trash")
|
||||
// MARK: - List
|
||||
|
||||
private var list: some View {
|
||||
// Full-bleed ScrollView → scrollbar on the detail pane's right edge.
|
||||
// Horizontal inset lives on the content so cards align with the title.
|
||||
ScrollView {
|
||||
LazyVStack(alignment: .leading, spacing: Spacing.md) {
|
||||
ForEach(historyStore.groupedByDay, id: \.day) { group in
|
||||
daySection(group)
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset)
|
||||
.padding(.bottom, Spacing.md)
|
||||
}
|
||||
}
|
||||
|
||||
private func daySection(_ group: (day: Date, items: [SpeechHistoryEntry])) -> some View {
|
||||
VStack(alignment: .leading, spacing: Spacing.xs) {
|
||||
Text(Self.dayFormatter.string(from: group.day))
|
||||
.font(TypeStyle.caption)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
|
||||
MacCard(padding: 0) {
|
||||
VStack(spacing: 0) {
|
||||
ForEach(Array(group.items.enumerated()), id: \.element.id) { index, entry in
|
||||
row(entry)
|
||||
.padding(.horizontal, Spacing.md)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
if index < group.items.count - 1 {
|
||||
// Full-bleed like macOS list rows (not iOS inset separators).
|
||||
Divider()
|
||||
.overlay(palette.divider)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
.buttonStyle(.borderless)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
}
|
||||
.padding(.horizontal, Spacing.lg)
|
||||
.padding(.vertical, Spacing.xs)
|
||||
.background(palette.background)
|
||||
}
|
||||
|
||||
private func row(_ entry: SpeechHistoryEntry) -> some View {
|
||||
@@ -107,11 +131,17 @@ struct MacHistoryView: View {
|
||||
private var emptyState: some View {
|
||||
VStack(spacing: Spacing.sm) {
|
||||
Image(systemName: "text.bubble")
|
||||
.font(.system(size: 34))
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.6))
|
||||
.font(.system(size: 34, weight: .light))
|
||||
.foregroundStyle(palette.textTertiary.opacity(0.55))
|
||||
.symbolRenderingMode(.hierarchical)
|
||||
Text(MacL10n.string("mac.history.empty", language: lang))
|
||||
.font(TypeStyle.body)
|
||||
.font(TypeStyle.headline)
|
||||
.foregroundStyle(palette.textSecondary)
|
||||
Text(MacL10n.string("mac.history.emptyBody", language: lang))
|
||||
.font(TypeStyle.footnote)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.multilineTextAlignment(.center)
|
||||
.frame(maxWidth: 320)
|
||||
}
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
}
|
||||
|
||||
@@ -1,33 +1,93 @@
|
||||
// MacHotkeyService.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Global hold-to-talk: while Option (⌥) is held, dictation runs. Mirrors
|
||||
// Typeless / SayIt push-to-talk from any foreground app.
|
||||
// Global hold-to-talk: while the configured Option (⌥) key is held, dictation
|
||||
// runs. Mirrors Typeless / SayIt push-to-talk from any foreground app.
|
||||
|
||||
import AppKit
|
||||
import Foundation
|
||||
|
||||
/// Which physical Option (⌥) key triggers global hold-to-talk.
|
||||
///
|
||||
/// Right Option is the default: the left key is a routine typing modifier
|
||||
/// (special characters, app shortcuts), so firing on any Option press
|
||||
/// constantly misfires during normal typing.
|
||||
enum MacHotkeyTrigger: String, CaseIterable, Identifiable {
|
||||
case rightOption
|
||||
case leftOption
|
||||
case eitherOption
|
||||
|
||||
var id: String { rawValue }
|
||||
|
||||
var labelKey: String {
|
||||
switch self {
|
||||
case .rightOption: return "mac.hotkeyTrigger.rightOption"
|
||||
case .leftOption: return "mac.hotkeyTrigger.leftOption"
|
||||
case .eitherOption: return "mac.hotkeyTrigger.eitherOption"
|
||||
}
|
||||
}
|
||||
|
||||
/// Main-window hint under the record button — must follow the picker,
|
||||
/// or the UI tells left-Option users to hold the right key.
|
||||
var hintKey: String {
|
||||
switch self {
|
||||
case .rightOption: return "mac.hint.hold.rightOption"
|
||||
case .leftOption: return "mac.hint.hold.leftOption"
|
||||
case .eitherOption: return "mac.hint.hold.eitherOption"
|
||||
}
|
||||
}
|
||||
|
||||
/// `@AppStorage`-compatible key; persisted via the view model's defaults.
|
||||
static let storageKey = "mac.hotkeyTrigger"
|
||||
|
||||
/// Device-dependent modifier bits (IOKit `NX_DEVICELALTKEYMASK` /
|
||||
/// `NX_DEVICERALTKEYMASK`) that `.flagsChanged` events carry alongside the
|
||||
/// device-independent `.option` flag, telling left and right apart.
|
||||
private static let leftOptionMask: UInt = 0x20
|
||||
private static let rightOptionMask: UInt = 0x40
|
||||
|
||||
/// Whether this trigger's key is currently down in a `.flagsChanged` event.
|
||||
func isPressed(in event: NSEvent) -> Bool {
|
||||
guard event.modifierFlags.contains(.option) else { return false }
|
||||
let raw = event.modifierFlags.rawValue
|
||||
switch self {
|
||||
case .rightOption: return raw & Self.rightOptionMask != 0
|
||||
case .leftOption: return raw & Self.leftOptionMask != 0
|
||||
case .eitherOption: return true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@MainActor
|
||||
final class MacHotkeyService {
|
||||
/// How long the trigger key must stay held before recording begins.
|
||||
/// Filters out quick ⌥-taps and ⌥+key combos (special characters, app
|
||||
/// shortcuts) that would otherwise start and immediately abort dictation.
|
||||
private static let holdDebounce: Duration = .milliseconds(150)
|
||||
|
||||
var onPressBegan: (() -> Void)?
|
||||
var onPressEnded: (() -> Void)?
|
||||
var trigger: MacHotkeyTrigger = .rightOption
|
||||
|
||||
private var globalFlagsMonitor: Any?
|
||||
private var localFlagsMonitor: Any?
|
||||
private var optionHeld = false
|
||||
/// The trigger key is physically down (debounce may still be pending).
|
||||
private var triggerKeyDown = false
|
||||
/// `onPressBegan` has fired and `onPressEnded` is owed.
|
||||
private var pressActive = false
|
||||
private var pendingBegin: Task<Void, Never>?
|
||||
private var isEnabled = true
|
||||
|
||||
func setEnabled(_ enabled: Bool) {
|
||||
isEnabled = enabled
|
||||
if !enabled, optionHeld {
|
||||
optionHeld = false
|
||||
onPressEnded?()
|
||||
}
|
||||
if !enabled { cancelPress() }
|
||||
}
|
||||
|
||||
func start() {
|
||||
guard globalFlagsMonitor == nil else { return }
|
||||
_ = MacTextInsertionService.requestAccessibilityIfNeeded()
|
||||
guard globalFlagsMonitor == nil, localFlagsMonitor == nil else { return }
|
||||
// Global monitors require Accessibility; without it the call returns
|
||||
// nil and Option-hold never fires outside our own windows.
|
||||
let trusted = MacTextInsertionService.requestAccessibilityIfNeeded()
|
||||
|
||||
globalFlagsMonitor = NSEvent.addGlobalMonitorForEvents(matching: .flagsChanged) { [weak self] event in
|
||||
Task { @MainActor in self?.handleFlagsChanged(event) }
|
||||
@@ -36,6 +96,12 @@ final class MacHotkeyService {
|
||||
Task { @MainActor in self?.handleFlagsChanged(event) }
|
||||
return event
|
||||
}
|
||||
|
||||
#if DEBUG
|
||||
if !trusted || globalFlagsMonitor == nil {
|
||||
NSLog("[OSGKeyboard] Hotkey global monitor unavailable — grant Accessibility in System Settings")
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
func stop() {
|
||||
@@ -47,20 +113,47 @@ final class MacHotkeyService {
|
||||
NSEvent.removeMonitor(localFlagsMonitor)
|
||||
self.localFlagsMonitor = nil
|
||||
}
|
||||
if optionHeld {
|
||||
optionHeld = false
|
||||
onPressEnded?()
|
||||
}
|
||||
cancelPress()
|
||||
}
|
||||
|
||||
private func handleFlagsChanged(_ event: NSEvent) {
|
||||
guard isEnabled else { return }
|
||||
let optionDown = event.modifierFlags.contains(.option)
|
||||
if optionDown, !optionHeld {
|
||||
optionHeld = true
|
||||
onPressBegan?()
|
||||
} else if !optionDown, optionHeld {
|
||||
optionHeld = false
|
||||
let triggerDown = trigger.isPressed(in: event)
|
||||
if triggerDown, !triggerKeyDown {
|
||||
triggerKeyDown = true
|
||||
scheduleBegin()
|
||||
} else if !triggerDown, triggerKeyDown {
|
||||
triggerKeyDown = false
|
||||
pendingBegin?.cancel()
|
||||
pendingBegin = nil
|
||||
if pressActive {
|
||||
pressActive = false
|
||||
onPressEnded?()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Debounce: begin only after the key has stayed held for `holdDebounce`.
|
||||
/// Releasing the key first cancels the pending start, so a quick
|
||||
/// Option+key combo never triggers recording.
|
||||
private func scheduleBegin() {
|
||||
pendingBegin?.cancel()
|
||||
pendingBegin = Task { [weak self] in
|
||||
try? await Task.sleep(for: Self.holdDebounce)
|
||||
guard let self, !Task.isCancelled else { return }
|
||||
self.pendingBegin = nil
|
||||
guard self.isEnabled, self.triggerKeyDown, !self.pressActive else { return }
|
||||
self.pressActive = true
|
||||
self.onPressBegan?()
|
||||
}
|
||||
}
|
||||
|
||||
private func cancelPress() {
|
||||
pendingBegin?.cancel()
|
||||
pendingBegin = nil
|
||||
triggerKeyDown = false
|
||||
if pressActive {
|
||||
pressActive = false
|
||||
onPressEnded?()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
// MacLocalASRChunkAdapter.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Adapts macOS local ASR to the shared chunked utterance pipeline.
|
||||
|
||||
import Foundation
|
||||
import os
|
||||
|
||||
final class MacLocalASRChunkAdapter: ASRChunkTranscribing, @unchecked Sendable {
|
||||
private let locale: Locale
|
||||
private let bias: LocalASRBiasPayload?
|
||||
private let cancelled = OSAllocatedUnfairLock(initialState: false)
|
||||
|
||||
init(locale: Locale, bias: LocalASRBiasPayload?) {
|
||||
self.locale = locale
|
||||
self.bias = bias
|
||||
}
|
||||
|
||||
func resetForNewUtterance() {
|
||||
cancelled.withLock { $0 = false }
|
||||
}
|
||||
|
||||
func cancel() {
|
||||
cancelled.withLock { $0 = true }
|
||||
}
|
||||
|
||||
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult {
|
||||
let isCancelled = cancelled.withLock { $0 }
|
||||
if isCancelled || Task.isCancelled { return .cancelled }
|
||||
guard !samples.isEmpty else { return .success("") }
|
||||
|
||||
do {
|
||||
let text = try await MacLocalASRService.transcribe(
|
||||
samples: samples,
|
||||
locale: locale,
|
||||
bias: bias
|
||||
)
|
||||
return .success(text)
|
||||
} catch {
|
||||
return .failure(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -276,14 +276,17 @@ struct MacLocalASRModelSettingsView: View {
|
||||
HStack(spacing: Spacing.xs) {
|
||||
Text(model.displayName)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
if let badgeKey = model.badgeKey {
|
||||
modelBadge(
|
||||
MacL10n.string(badgeKey, language: lang),
|
||||
emphasized: true
|
||||
)
|
||||
}
|
||||
if model.supportsHotwords {
|
||||
Text(MacL10n.string("mac.localASR.personalDictionaryTag", language: lang))
|
||||
.font(TypeStyle.caption2)
|
||||
.padding(.horizontal, 6)
|
||||
.padding(.vertical, 2)
|
||||
.background(palette.accent.opacity(0.15))
|
||||
.foregroundStyle(palette.accent)
|
||||
.clipShape(Capsule())
|
||||
modelBadge(
|
||||
MacL10n.string("mac.localASR.personalDictionaryTag", language: lang),
|
||||
emphasized: false
|
||||
)
|
||||
}
|
||||
}
|
||||
Text(modelSubtitle(model, installed: installed))
|
||||
@@ -305,6 +308,20 @@ struct MacLocalASRModelSettingsView: View {
|
||||
.padding(.vertical, 2)
|
||||
}
|
||||
|
||||
private func modelBadge(_ title: String, emphasized: Bool) -> some View {
|
||||
Text(title)
|
||||
.font(TypeStyle.caption2)
|
||||
.padding(.horizontal, 6)
|
||||
.padding(.vertical, 2)
|
||||
.background(
|
||||
emphasized
|
||||
? palette.accent.opacity(0.18)
|
||||
: palette.textTertiary.opacity(0.12)
|
||||
)
|
||||
.foregroundStyle(emphasized ? palette.accent : palette.textSecondary)
|
||||
.clipShape(Capsule())
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private func modelRowActions(
|
||||
model: LocalASRModelDefinition,
|
||||
|
||||
@@ -58,7 +58,7 @@ enum MacLocalASRPreferences {
|
||||
/// Maps removed catalog entries to the current default Sherpa model.
|
||||
static func migratedModelId(_ id: String) -> String {
|
||||
switch id {
|
||||
case "qwen3-mlx-1.7b":
|
||||
case "qwen3-mlx-1.7b", "sherpa-paraformer-zh-int8":
|
||||
return "sherpa-qwen3-0.6b-int8"
|
||||
default:
|
||||
return id
|
||||
@@ -120,7 +120,7 @@ enum MacLocalASRService {
|
||||
return try await transcribeWithModel(model, samples: samples, locale: locale, bias: bias)
|
||||
}
|
||||
|
||||
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale)
|
||||
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale, bias: bias)
|
||||
}
|
||||
|
||||
private static func transcribeWithModel(
|
||||
@@ -141,7 +141,7 @@ enum MacLocalASRService {
|
||||
bias: bias
|
||||
)
|
||||
case .appleSpeech:
|
||||
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale)
|
||||
return try await MacSpeechLocalASR.transcribe(samples: samples, locale: locale, bias: bias)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -198,7 +198,7 @@ struct MacOnboardingView: View {
|
||||
.frame(maxWidth: .infinity)
|
||||
}
|
||||
}
|
||||
.frame(minWidth: 860, minHeight: 600)
|
||||
.frame(minWidth: MacMetrics.windowMinWidth, minHeight: MacMetrics.windowMinHeight)
|
||||
.onAppear {
|
||||
applyDefaults()
|
||||
model.reload()
|
||||
|
||||
@@ -18,15 +18,6 @@ struct MacRootView: View {
|
||||
|
||||
private var uiLanguage: AppUILanguage { viewModel.config.uiLanguage }
|
||||
|
||||
/// `List` selection is optional; keep the view model's non-optional section
|
||||
/// in sync without letting a nil selection blank the detail pane.
|
||||
private var selection: Binding<MacSection?> {
|
||||
Binding(
|
||||
get: { viewModel.selectedSection },
|
||||
set: { if let new = $0 { viewModel.selectedSection = new } }
|
||||
)
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
NavigationSplitView(columnVisibility: $columnVisibility) {
|
||||
sidebar
|
||||
@@ -35,7 +26,7 @@ struct MacRootView: View {
|
||||
detail
|
||||
}
|
||||
.navigationSplitViewStyle(.balanced)
|
||||
.frame(minWidth: 860, minHeight: 600)
|
||||
.frame(minWidth: MacMetrics.windowMinWidth, minHeight: MacMetrics.windowMinHeight)
|
||||
.onAppear {
|
||||
// Let the AppKit status-bar popover reopen this window on demand.
|
||||
MacWindowBridge.shared.open = { openWindow(id: "main") }
|
||||
@@ -72,7 +63,7 @@ struct MacRootView: View {
|
||||
.renderingMode(.template)
|
||||
.resizable()
|
||||
.scaledToFit()
|
||||
.frame(height: 30)
|
||||
.frame(height: 28)
|
||||
.foregroundStyle(palette.accent)
|
||||
.accessibilityLabel("OSGKeyboard")
|
||||
Spacer()
|
||||
@@ -80,7 +71,7 @@ struct MacRootView: View {
|
||||
.padding(.leading, MacMetrics.sidebarContentInset)
|
||||
.padding(.trailing, MacMetrics.sidebarInset)
|
||||
.padding(.top, Spacing.lg)
|
||||
.padding(.bottom, Spacing.lg)
|
||||
.padding(.bottom, Spacing.md)
|
||||
}
|
||||
|
||||
private var devicesFooter: some View {
|
||||
@@ -88,10 +79,10 @@ struct MacRootView: View {
|
||||
MacL10n.string("mac.devices", language: uiLanguage),
|
||||
systemImage: "laptopcomputer.and.iphone"
|
||||
)
|
||||
.font(TypeStyle.caption)
|
||||
.font(TypeStyle.caption2)
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(.horizontal, MacMetrics.sidebarInset)
|
||||
.padding(.horizontal, MacMetrics.sidebarInset + Spacing.sm)
|
||||
.padding(.vertical, Spacing.sm)
|
||||
}
|
||||
|
||||
@@ -118,8 +109,8 @@ struct MacRootView: View {
|
||||
|
||||
// MARK: - Sidebar row
|
||||
|
||||
/// A navigation row with an animated hover highlight and selection state,
|
||||
/// matching the macOS System Settings feel.
|
||||
/// Navigation row with a restrained selected state: muted accent fill +
|
||||
/// accent label (not a solid green pill), matching System Settings polish.
|
||||
private struct MacSidebarRow: View {
|
||||
let section: MacSection
|
||||
let isSelected: Bool
|
||||
@@ -132,8 +123,8 @@ private struct MacSidebarRow: View {
|
||||
var body: some View {
|
||||
Button(action: action) {
|
||||
Label(section.title(language: language), systemImage: section.systemImage)
|
||||
.font(.system(size: 13))
|
||||
.foregroundStyle(isSelected ? palette.textOnAccent : palette.textPrimary)
|
||||
.font(.system(size: 13, weight: isSelected ? .semibold : .regular))
|
||||
.foregroundStyle(isSelected ? palette.accent : palette.textPrimary)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.padding(.horizontal, Spacing.sm)
|
||||
.padding(.vertical, 7)
|
||||
@@ -150,7 +141,7 @@ private struct MacSidebarRow: View {
|
||||
}
|
||||
|
||||
private var rowBackground: Color {
|
||||
if isSelected { return palette.accent }
|
||||
return isHovering ? palette.textPrimary.opacity(0.06) : .clear
|
||||
if isSelected { return palette.accentMuted }
|
||||
return isHovering ? palette.textPrimary.opacity(0.05) : .clear
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
// MacSettingsView.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Settings built on the native grouped `Form` — the same container macOS
|
||||
// System Settings uses. This gives system-accurate cards, dividers, insets
|
||||
// and right-aligned controls for free, on both light and dark.
|
||||
// Settings uses native grouped `Form` for correct control layout (Picker /
|
||||
// Toggle / LabeledContent). Title and Form share the same plain
|
||||
// `pageHorizontalInset` padding (Form scroll margins are zeroed first) so
|
||||
// card chrome lines up with the page title.
|
||||
|
||||
import SwiftUI
|
||||
#if os(macOS)
|
||||
@@ -20,6 +21,7 @@ struct MacSettingsView: View {
|
||||
private var hasCompletedMacOnboarding = true
|
||||
@State private var accessibilityTrusted = MacTextInsertionService.isAccessibilityTrusted
|
||||
@State private var showProviderPicker = false
|
||||
@State private var showAsrProviderPicker = false
|
||||
|
||||
private var lang: AppUILanguage { viewModel.config.uiLanguage }
|
||||
private let recognitionLocales: [(id: String, key: String, fallback: String)] = [
|
||||
@@ -33,11 +35,18 @@ struct MacSettingsView: View {
|
||||
|
||||
var body: some View {
|
||||
NavigationStack {
|
||||
VStack(spacing: 0) {
|
||||
MacPageHeader(
|
||||
title: MacL10n.string("mac.section.settings", language: lang),
|
||||
subtitle: MacL10n.string("mac.page.settings.subtitle", language: lang)
|
||||
)
|
||||
|
||||
Form {
|
||||
generalSection
|
||||
recognitionSection
|
||||
polishProviderSection
|
||||
if viewModel.config.engineMode == "cloud" {
|
||||
providerSection
|
||||
asrProviderSection
|
||||
.transition(.opacity)
|
||||
}
|
||||
if viewModel.config.engineMode == "local" {
|
||||
@@ -48,10 +57,23 @@ struct MacSettingsView: View {
|
||||
legalSection
|
||||
}
|
||||
.formStyle(.grouped)
|
||||
// Zero Form's own scroll margins, then inset via padding so the
|
||||
// section cards line up with MacPageHeader (contentMargins alone
|
||||
// does not match plain padding on macOS).
|
||||
//
|
||||
// grouped Form adds its own built-in section inset on top of our
|
||||
// padding, so cards sat ~`groupedFormSectionInset` wider than the
|
||||
// History page. Subtract that inset here so the card OUTER edge
|
||||
// lands on `pageHorizontalInset` (40pt), matching History and the
|
||||
// page title's left edge.
|
||||
.contentMargins(.horizontal, 0, for: .scrollContent)
|
||||
.padding(.horizontal, MacMetrics.pageHorizontalInset - MacMetrics.groupedFormSectionInset)
|
||||
.tint(palette.accent)
|
||||
.scrollContentBackground(.hidden)
|
||||
.background(palette.background)
|
||||
}
|
||||
.background(palette.background)
|
||||
}
|
||||
.onAppear { refreshAccessibilityState() }
|
||||
}
|
||||
|
||||
@@ -82,27 +104,21 @@ struct MacSettingsView: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Cloud provider
|
||||
// MARK: - Polish LLM
|
||||
|
||||
private var providerSection: some View {
|
||||
Section(MacL10n.string("mac.settings.cloudProvider", language: lang)) {
|
||||
LabeledContent(MacL10n.string("mac.settings.service", language: lang)) {
|
||||
Button {
|
||||
showProviderPicker = true
|
||||
} label: {
|
||||
HStack(spacing: 6) {
|
||||
providerLogo(currentProvider.id)
|
||||
Text(currentProvider.name)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
Image(systemName: "chevron.up.chevron.down")
|
||||
.font(.system(size: 9, weight: .semibold))
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
}
|
||||
.contentShape(Rectangle())
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
.popover(isPresented: $showProviderPicker, arrowEdge: .bottom) {
|
||||
providerPickerList
|
||||
private var polishProviderSection: some View {
|
||||
Section(MacL10n.string("mac.settings.polishProvider", language: lang)) {
|
||||
providerPickerRow(
|
||||
title: MacL10n.string("mac.settings.service", language: lang),
|
||||
provider: currentPolishProvider,
|
||||
isPresented: $showProviderPicker
|
||||
) {
|
||||
providerPickerList(
|
||||
providers: viewModel.polishSelectableProviders,
|
||||
selectedId: viewModel.config.providerId
|
||||
) { provider in
|
||||
viewModel.selectProvider(provider)
|
||||
showProviderPicker = false
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,6 +133,17 @@ struct MacSettingsView: View {
|
||||
Text(MacL10n.string("mac.settings.apiKey", language: lang))
|
||||
}
|
||||
|
||||
LabeledContent {
|
||||
TextField(text: $viewModel.config.baseURL, prompt: Text(verbatim: "")) {
|
||||
Text(MacL10n.string("mac.settings.baseURL", language: lang))
|
||||
}
|
||||
.labelsHidden()
|
||||
.macFieldStyle()
|
||||
.frame(maxWidth: MacMetrics.controlWidth)
|
||||
} label: {
|
||||
Text(MacL10n.string("mac.settings.baseURL", language: lang))
|
||||
}
|
||||
|
||||
LabeledContent {
|
||||
TextField(text: $viewModel.config.model, prompt: Text(verbatim: "")) {
|
||||
Text(MacL10n.string("mac.settings.model", language: lang))
|
||||
@@ -130,6 +157,86 @@ struct MacSettingsView: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Cloud ASR
|
||||
|
||||
private var asrProviderSection: some View {
|
||||
Section(MacL10n.string("mac.settings.asrProvider", language: lang)) {
|
||||
providerPickerRow(
|
||||
title: MacL10n.string("mac.settings.asrService", language: lang),
|
||||
provider: currentAsrProvider,
|
||||
isPresented: $showAsrProviderPicker
|
||||
) {
|
||||
providerPickerList(
|
||||
providers: viewModel.asrSelectableProviders,
|
||||
selectedId: viewModel.config.asrProviderId
|
||||
) { provider in
|
||||
viewModel.selectAsrProvider(provider)
|
||||
showAsrProviderPicker = false
|
||||
}
|
||||
}
|
||||
|
||||
LabeledContent {
|
||||
SecureField(text: $viewModel.config.asrApiKey, prompt: Text(verbatim: "sk-…")) {
|
||||
Text(MacL10n.string("mac.settings.apiKey", language: lang))
|
||||
}
|
||||
.labelsHidden()
|
||||
.macFieldStyle()
|
||||
.frame(maxWidth: MacMetrics.controlWidth)
|
||||
} label: {
|
||||
Text(MacL10n.string("mac.settings.asrApiKey", language: lang))
|
||||
}
|
||||
|
||||
if CloudASRModelCatalog.strategy(for: viewModel.config.asrProviderId) == .prompt {
|
||||
LabeledContent {
|
||||
TextField(text: $viewModel.config.asrBaseURL, prompt: Text(verbatim: "")) {
|
||||
Text(MacL10n.string("mac.settings.baseURL", language: lang))
|
||||
}
|
||||
.labelsHidden()
|
||||
.macFieldStyle()
|
||||
.frame(maxWidth: MacMetrics.controlWidth)
|
||||
} label: {
|
||||
Text(MacL10n.string("mac.settings.baseURL", language: lang))
|
||||
}
|
||||
}
|
||||
|
||||
LabeledContent {
|
||||
TextField(text: $viewModel.config.asrModel, prompt: Text(verbatim: "")) {
|
||||
Text(MacL10n.string("mac.settings.asrModel", language: lang))
|
||||
}
|
||||
.labelsHidden()
|
||||
.macFieldStyle()
|
||||
.frame(maxWidth: MacMetrics.controlWidth)
|
||||
} label: {
|
||||
Text(MacL10n.string("mac.settings.asrModel", language: lang))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func providerPickerRow<Content: View>(
|
||||
title: String,
|
||||
provider: LLMProvider,
|
||||
isPresented: Binding<Bool>,
|
||||
@ViewBuilder picker: @escaping () -> Content
|
||||
) -> some View {
|
||||
LabeledContent(title) {
|
||||
Button {
|
||||
isPresented.wrappedValue = true
|
||||
} label: {
|
||||
HStack(spacing: 6) {
|
||||
providerLogo(provider.id)
|
||||
Text(provider.name)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
Image(systemName: "chevron.up.chevron.down")
|
||||
.font(.system(size: 9, weight: .semibold))
|
||||
.foregroundStyle(palette.textTertiary)
|
||||
}
|
||||
.contentShape(Rectangle())
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
.popover(isPresented: isPresented, arrowEdge: .bottom, content: picker)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Recognition method
|
||||
|
||||
private var recognitionSection: some View {
|
||||
@@ -161,6 +268,19 @@ struct MacSettingsView: View {
|
||||
)
|
||||
}
|
||||
|
||||
Picker(selection: hotkeyTriggerBinding) {
|
||||
ForEach(MacHotkeyTrigger.allCases) { trigger in
|
||||
Text(MacL10n.string(trigger.labelKey, language: lang))
|
||||
.tag(trigger.rawValue)
|
||||
}
|
||||
} label: {
|
||||
rowLabel(
|
||||
MacL10n.string("mac.settings.hotkeyTrigger", language: lang),
|
||||
subtitle: MacL10n.string("mac.settings.hotkeyTriggerDesc", language: lang)
|
||||
)
|
||||
}
|
||||
.disabled(!viewModel.hotkeyEnabled)
|
||||
|
||||
Toggle(isOn: autoPasteBinding) {
|
||||
rowLabel(
|
||||
MacL10n.string("mac.settings.autoPaste", language: lang),
|
||||
@@ -258,28 +378,34 @@ struct MacSettingsView: View {
|
||||
.buttonStyle(.plain)
|
||||
}
|
||||
|
||||
private var currentProvider: LLMProvider {
|
||||
viewModel.selectableProviders.first { $0.id == viewModel.config.providerId }
|
||||
?? viewModel.selectableProviders.first
|
||||
private var currentPolishProvider: LLMProvider {
|
||||
viewModel.polishSelectableProviders.first { $0.id == viewModel.config.providerId }
|
||||
?? viewModel.polishSelectableProviders.first
|
||||
?? LLMProvider.presets[0]
|
||||
}
|
||||
|
||||
/// Custom dropdown list shown in a popover. SwiftUI's `Menu` label / items
|
||||
/// silently drop bundled (non-SF-Symbol) images on macOS, so we render the
|
||||
/// brand marks in a plain view stack instead.
|
||||
private var providerPickerList: some View {
|
||||
private var currentAsrProvider: LLMProvider {
|
||||
viewModel.asrSelectableProviders.first { $0.id == viewModel.config.asrProviderId }
|
||||
?? viewModel.asrSelectableProviders.first
|
||||
?? LLMProvider.presets[0]
|
||||
}
|
||||
|
||||
private func providerPickerList(
|
||||
providers: [LLMProvider],
|
||||
selectedId: String,
|
||||
onSelect: @escaping (LLMProvider) -> Void
|
||||
) -> some View {
|
||||
VStack(spacing: 0) {
|
||||
ForEach(viewModel.selectableProviders) { provider in
|
||||
ForEach(providers) { provider in
|
||||
Button {
|
||||
viewModel.selectProvider(provider)
|
||||
showProviderPicker = false
|
||||
onSelect(provider)
|
||||
} label: {
|
||||
HStack(spacing: Spacing.sm) {
|
||||
providerLogo(provider.id)
|
||||
Text(provider.name)
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
Spacer(minLength: Spacing.md)
|
||||
if provider.id == currentProvider.id {
|
||||
if provider.id == selectedId {
|
||||
Image(systemName: "checkmark")
|
||||
.font(.system(size: 12, weight: .semibold))
|
||||
.foregroundStyle(palette.accent)
|
||||
@@ -359,6 +485,13 @@ struct MacSettingsView: View {
|
||||
)
|
||||
}
|
||||
|
||||
private var hotkeyTriggerBinding: Binding<String> {
|
||||
Binding(
|
||||
get: { viewModel.hotkeyTrigger.rawValue },
|
||||
set: { viewModel.setHotkeyTrigger(MacHotkeyTrigger(rawValue: $0) ?? .rightOption) }
|
||||
)
|
||||
}
|
||||
|
||||
private var autoPasteBinding: Binding<Bool> {
|
||||
Binding(
|
||||
get: { viewModel.autoPasteEnabled },
|
||||
|
||||
@@ -165,16 +165,69 @@ enum MacSherpaONNXRunner {
|
||||
.filter { !$0.isEmpty }
|
||||
|
||||
for line in lines.reversed() {
|
||||
if line.hasPrefix("{"), let data = line.data(using: .utf8),
|
||||
if line.hasPrefix("{") {
|
||||
// Sherpa's JSON result line (`{"text": ..., "lang": ..., ...}`).
|
||||
// Trust only its `text` field — including when it's empty
|
||||
// (silence/no-speech) — and never fall through to the raw
|
||||
// JSON below, or the JSON blob itself gets inserted as text.
|
||||
if let data = line.data(using: .utf8),
|
||||
let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
|
||||
let text = object["text"] as? String {
|
||||
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if !trimmed.isEmpty { return trimmed }
|
||||
return sanitizeTranscript(text)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if isMetadataNoiseLine(line) { continue }
|
||||
if !line.hasPrefix("/"), !line.hasPrefix("--"), line.count > 1 {
|
||||
return line
|
||||
let cleaned = sanitizeTranscript(line)
|
||||
if !cleaned.isEmpty { return cleaned }
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
/// Qwen3-ASR (via sherpa-onnx) often prefixes the transcript with a
|
||||
/// scaffold such as `language Chinese<asr_text>…`. Older runtimes leave
|
||||
/// that intact in `result.text`; incomplete generations can even stop at
|
||||
/// the bare word `language`. Strip the scaffold so only spoken text remains.
|
||||
private static func sanitizeTranscript(_ raw: String) -> String {
|
||||
var text = raw.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if text.isEmpty { return "" }
|
||||
|
||||
// Prefer the payload after the last `<asr_text>` marker.
|
||||
if let marker = text.range(of: "<asr_text>", options: .backwards) {
|
||||
text = String(text[marker.upperBound...])
|
||||
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
} else if let match = text.range(
|
||||
of: #"^language\s+\S+\s*"#,
|
||||
options: [.regularExpression, .caseInsensitive]
|
||||
) {
|
||||
// Fallback when the marker token was lost but the language prefix remains.
|
||||
text = String(text[match.upperBound...])
|
||||
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
|
||||
// Drop leftover control tokens / bare scaffold words.
|
||||
if isMetadataNoiseLine(text) { return "" }
|
||||
return text
|
||||
}
|
||||
|
||||
/// Lines that are sherpa/Qwen metadata rather than spoken content.
|
||||
private static func isMetadataNoiseLine(_ line: String) -> Bool {
|
||||
let lowered = line.lowercased()
|
||||
switch lowered {
|
||||
case "language", "emotion", "event", "text",
|
||||
"<asr_text>", "</asr_text>", "<|im_end|>":
|
||||
return true
|
||||
default:
|
||||
// Exact scaffold with no spoken payload, e.g. "language Chinese".
|
||||
if lowered.range(
|
||||
of: #"^language(\s+\S+)?$"#,
|
||||
options: .regularExpression
|
||||
) != nil {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,10 +9,65 @@ import Foundation
|
||||
import Speech
|
||||
|
||||
enum MacSpeechLocalASR {
|
||||
static func transcribe(samples: [Float], locale: Locale) async throws -> String {
|
||||
/// Shared resume-once state for one recognition run. The recognizer
|
||||
/// callback (delivered on an arbitrary Speech queue) and the timeout task
|
||||
/// race to finish, and a `CheckedContinuation` must resume exactly once,
|
||||
/// so both go through this lock-guarded gate. It also retains the
|
||||
/// `SFSpeechRecognitionTask` so the losing/failing path can cancel it.
|
||||
private final class RecognitionSession: @unchecked Sendable {
|
||||
private let lock = NSLock()
|
||||
private var isResumed = false
|
||||
private var task: SFSpeechRecognitionTask?
|
||||
private var timeoutTask: Task<Void, Never>?
|
||||
|
||||
func retain(_ task: SFSpeechRecognitionTask) {
|
||||
lock.lock()
|
||||
self.task = task
|
||||
let alreadyResumed = isResumed
|
||||
lock.unlock()
|
||||
// Timeout won the race before the task handle was stored.
|
||||
if alreadyResumed { task.cancel() }
|
||||
}
|
||||
|
||||
func retainTimeout(_ task: Task<Void, Never>) {
|
||||
lock.lock()
|
||||
timeoutTask = task
|
||||
let alreadyResumed = isResumed
|
||||
lock.unlock()
|
||||
// Recognition finished before the handle landed — stop the timer.
|
||||
if alreadyResumed { task.cancel() }
|
||||
}
|
||||
|
||||
/// Returns `true` exactly once across all callers; the winner may
|
||||
/// resume the continuation. Pass `cancellingTask: true` on failure
|
||||
/// paths so the in-flight recognition stops doing work. The winner
|
||||
/// also cancels the timeout task so it doesn't keep the session (and
|
||||
/// continuation captures) alive for the rest of its sleep.
|
||||
func claimResume(cancellingTask: Bool) -> Bool {
|
||||
lock.lock()
|
||||
guard !isResumed else {
|
||||
lock.unlock()
|
||||
return false
|
||||
}
|
||||
isResumed = true
|
||||
let task = self.task
|
||||
let timeout = timeoutTask
|
||||
lock.unlock()
|
||||
if cancellingTask { task?.cancel() }
|
||||
timeout?.cancel()
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
static func transcribe(samples: [Float], locale: Locale, bias: LocalASRBiasPayload? = nil) async throws -> String {
|
||||
let auth = await requestAuthorization()
|
||||
guard auth == .authorized else { throw MacLocalASRError.speechDenied }
|
||||
|
||||
if Self.isChineseLocale(locale) {
|
||||
CustomLanguageModelManager.shared.prepareInBackgroundIfNeeded()
|
||||
_ = try? await CustomLanguageModelManager.shared.prepareIfNeeded()
|
||||
}
|
||||
|
||||
let wavURL = try writeTemporaryWAV(samples: samples, sampleRate: 16_000)
|
||||
defer { try? FileManager.default.removeItem(at: wavURL) }
|
||||
|
||||
@@ -20,18 +75,43 @@ enum MacSpeechLocalASR {
|
||||
guard let recognizer, recognizer.isAvailable else {
|
||||
throw MacLocalASRError.speechFailed("Speech recognizer unavailable")
|
||||
}
|
||||
// The request below sets `requiresOnDeviceRecognition = true`, which
|
||||
// fails (or worse, never produces a final result) when the on-device
|
||||
// model for the locale is missing — fail fast with a clear error.
|
||||
guard recognizer.supportsOnDeviceRecognition else {
|
||||
throw MacLocalASRError.speechFailed(
|
||||
"On-device speech recognition is not available for \(recognizer.locale.identifier). Download the language in System Settings → Keyboard → Dictation."
|
||||
)
|
||||
}
|
||||
|
||||
return try await withCheckedThrowingContinuation { continuation in
|
||||
// Overall deadline: recognition of a file is normally much faster than
|
||||
// realtime, so 2× audio length with a 30 s floor is generous. Without
|
||||
// it, empty audio / cancellation / a missing model can leave the
|
||||
// callback silent forever and the continuation never resumes.
|
||||
let audioSeconds = Double(samples.count) / 16_000
|
||||
let timeoutSeconds = max(30.0, audioSeconds * 2)
|
||||
let session = RecognitionSession()
|
||||
|
||||
return try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<String, Error>) in
|
||||
let request = SFSpeechURLRecognitionRequest(url: wavURL)
|
||||
request.shouldReportPartialResults = false
|
||||
request.requiresOnDeviceRecognition = true
|
||||
CustomLanguageModelManager.applyCustomLanguageModel(
|
||||
to: request,
|
||||
locale: locale,
|
||||
bias: bias
|
||||
)
|
||||
|
||||
recognizer.recognitionTask(with: request) { result, error in
|
||||
let task = recognizer.recognitionTask(with: request) { result, error in
|
||||
if let error {
|
||||
if session.claimResume(cancellingTask: true) {
|
||||
continuation.resume(throwing: MacLocalASRError.speechFailed(error.localizedDescription))
|
||||
}
|
||||
return
|
||||
}
|
||||
// Non-final callbacks carry no usable transcript yet; if a
|
||||
// final result never arrives, the timeout below resumes us.
|
||||
guard let result, result.isFinal else { return }
|
||||
guard session.claimResume(cancellingTask: false) else { return }
|
||||
let text = result.bestTranscription.formattedString
|
||||
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if text.isEmpty {
|
||||
@@ -40,6 +120,16 @@ enum MacSpeechLocalASR {
|
||||
continuation.resume(returning: text)
|
||||
}
|
||||
}
|
||||
session.retain(task)
|
||||
|
||||
let timeout = Task {
|
||||
try? await Task.sleep(for: .seconds(timeoutSeconds))
|
||||
guard !Task.isCancelled else { return }
|
||||
if session.claimResume(cancellingTask: true) {
|
||||
continuation.resume(throwing: MacLocalASRError.speechFailed("Speech recognition timed out"))
|
||||
}
|
||||
}
|
||||
session.retainTimeout(timeout)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -58,4 +148,8 @@ enum MacSpeechLocalASR {
|
||||
try wav.write(to: url)
|
||||
return url
|
||||
}
|
||||
|
||||
private static func isChineseLocale(_ locale: Locale) -> Bool {
|
||||
locale.identifier(.bcp47).lowercased().hasPrefix("zh")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
// MacTextInsertionService.swift
|
||||
// OSGKeyboard · Mac
|
||||
//
|
||||
// Inserts transcribed text into the frontmost app: clipboard first, then
|
||||
// Inserts transcribed text into the target app: clipboard first, then
|
||||
// a synthetic ⌘V (SayIt / Typeless-style). Requires Accessibility trust.
|
||||
// Re-activates the app the user was dictating into (the popover steals
|
||||
// focus) and restores the original clipboard once the paste has landed.
|
||||
|
||||
import AppKit
|
||||
@preconcurrency import ApplicationServices
|
||||
@@ -30,32 +32,158 @@ enum MacTextInsertionService {
|
||||
return AXIsProcessTrustedWithOptions(options)
|
||||
}
|
||||
|
||||
/// Copy to pasteboard and optionally simulate ⌘V in the front app.
|
||||
// MARK: - Paste-target tracking
|
||||
|
||||
/// Start observing app activations early (app launch) so a paste target
|
||||
/// can still be recovered while OSGKeyboard itself is frontmost — e.g.
|
||||
/// when a recording is started from the menu-bar popover.
|
||||
@MainActor
|
||||
static func beginTrackingFrontmostApp() {
|
||||
_ = FrontmostAppTracker.shared
|
||||
}
|
||||
|
||||
/// The app a synthesized ⌘V should land in: the current frontmost app,
|
||||
/// or — when OSGKeyboard is frontmost because the popover has key
|
||||
/// focus — the app that was active immediately before it.
|
||||
@MainActor
|
||||
static func captureTargetApplication() -> NSRunningApplication? {
|
||||
let selfPid = NSRunningApplication.current.processIdentifier
|
||||
if let front = NSWorkspace.shared.frontmostApplication,
|
||||
front.processIdentifier != selfPid {
|
||||
return front
|
||||
}
|
||||
return FrontmostAppTracker.shared.lastExternalApp
|
||||
}
|
||||
|
||||
// MARK: - Insertion
|
||||
|
||||
/// Copy to pasteboard and optionally simulate ⌘V in the target app.
|
||||
/// Returns `true` only if the ⌘V event was actually synthesized. After a
|
||||
/// successful paste the user's original clipboard is put back, so
|
||||
/// dictation never permanently clobbers it.
|
||||
@MainActor
|
||||
static func insert(
|
||||
_ text: String,
|
||||
autoPaste: Bool
|
||||
) throws -> Bool {
|
||||
autoPaste: Bool,
|
||||
targetApp: NSRunningApplication? = nil
|
||||
) async throws -> Bool {
|
||||
guard !text.isEmpty else { return false }
|
||||
let pasteboard = NSPasteboard.general
|
||||
let snapshot = snapshotItems(of: pasteboard)
|
||||
pasteboard.clearContents()
|
||||
pasteboard.setString(text, forType: .string)
|
||||
|
||||
guard autoPaste else { return false }
|
||||
guard AXIsProcessTrusted() else { throw InsertionError.accessibilityNotGranted }
|
||||
|
||||
Thread.sleep(forTimeInterval: 0.08)
|
||||
postCommandV()
|
||||
// Make sure ⌘V lands in the app the user was dictating into, not in
|
||||
// OSGKeyboard's own popover / window.
|
||||
if let targetApp { await activate(targetApp) }
|
||||
try? await Task.sleep(nanoseconds: 80_000_000)
|
||||
guard postCommandV() else { return false }
|
||||
|
||||
// Give the target app time to read the transcript off the
|
||||
// pasteboard, then restore whatever the user had on it.
|
||||
try? await Task.sleep(nanoseconds: 300_000_000)
|
||||
restoreItems(snapshot, to: pasteboard)
|
||||
return true
|
||||
}
|
||||
|
||||
private static func postCommandV() {
|
||||
/// Brings `app` forward and waits (up to ~1 s) until it is frontmost so
|
||||
/// the synthesized keystroke isn't swallowed mid-switch.
|
||||
@MainActor
|
||||
private static func activate(_ app: NSRunningApplication) async {
|
||||
func isFront() -> Bool {
|
||||
NSWorkspace.shared.frontmostApplication?.processIdentifier == app.processIdentifier
|
||||
}
|
||||
guard !isFront() else { return }
|
||||
app.activate()
|
||||
var attempts = 0
|
||||
while !isFront(), attempts < 20 {
|
||||
try? await Task.sleep(nanoseconds: 50_000_000)
|
||||
attempts += 1
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Pasteboard preservation
|
||||
|
||||
/// Every representation of every pasteboard item, so restore round-trips
|
||||
/// rich content (images, files, multiple flavours) losslessly.
|
||||
private static func snapshotItems(
|
||||
of pasteboard: NSPasteboard
|
||||
) -> [[NSPasteboard.PasteboardType: Data]] {
|
||||
(pasteboard.pasteboardItems ?? []).map { item in
|
||||
item.types.reduce(into: [NSPasteboard.PasteboardType: Data]()) { flavours, type in
|
||||
flavours[type] = item.data(forType: type)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static func restoreItems(
|
||||
_ items: [[NSPasteboard.PasteboardType: Data]],
|
||||
to pasteboard: NSPasteboard
|
||||
) {
|
||||
guard !items.isEmpty else { return }
|
||||
pasteboard.clearContents()
|
||||
pasteboard.writeObjects(items.map { flavours in
|
||||
let item = NSPasteboardItem()
|
||||
for (type, data) in flavours { item.setData(data, forType: type) }
|
||||
return item
|
||||
})
|
||||
}
|
||||
|
||||
/// Returns `false` when the CGEvents could not be created — in that case
|
||||
/// nothing was pasted and callers must not report success.
|
||||
private static func postCommandV() -> Bool {
|
||||
let source = CGEventSource(stateID: .combinedSessionState)
|
||||
let keyCode = CGKeyCode(kVK_ANSI_V)
|
||||
let keyDown = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: true)
|
||||
guard
|
||||
let keyDown = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: true),
|
||||
let keyUp = CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: false)
|
||||
keyDown?.flags = CGEventFlags.maskCommand
|
||||
keyUp?.flags = CGEventFlags.maskCommand
|
||||
keyDown?.post(tap: CGEventTapLocation.cghidEventTap)
|
||||
keyUp?.post(tap: CGEventTapLocation.cghidEventTap)
|
||||
else { return false }
|
||||
keyDown.flags = CGEventFlags.maskCommand
|
||||
keyUp.flags = CGEventFlags.maskCommand
|
||||
keyDown.post(tap: CGEventTapLocation.cghidEventTap)
|
||||
keyUp.post(tap: CGEventTapLocation.cghidEventTap)
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Frontmost-app tracker
|
||||
|
||||
/// Remembers the most recent non-OSGKeyboard frontmost app. Needed because
|
||||
/// the menu-bar popover activates OSGKeyboard, hiding the real paste target
|
||||
/// from `NSWorkspace.frontmostApplication`.
|
||||
@MainActor
|
||||
private final class FrontmostAppTracker: NSObject {
|
||||
static let shared = FrontmostAppTracker()
|
||||
|
||||
private(set) var lastExternalApp: NSRunningApplication?
|
||||
|
||||
private override init() {
|
||||
super.init()
|
||||
// Seed with whatever is frontmost now (usually not us at launch).
|
||||
if let front = NSWorkspace.shared.frontmostApplication,
|
||||
front.processIdentifier != NSRunningApplication.current.processIdentifier {
|
||||
lastExternalApp = front
|
||||
}
|
||||
NSWorkspace.shared.notificationCenter.addObserver(
|
||||
self,
|
||||
selector: #selector(appDidActivate(_:)),
|
||||
name: NSWorkspace.didActivateApplicationNotification,
|
||||
object: nil
|
||||
)
|
||||
}
|
||||
|
||||
deinit {
|
||||
NSWorkspace.shared.notificationCenter.removeObserver(self)
|
||||
}
|
||||
|
||||
@objc private func appDidActivate(_ notification: Notification) {
|
||||
guard
|
||||
let app = notification.userInfo?[NSWorkspace.applicationUserInfoKey] as? NSRunningApplication,
|
||||
app.processIdentifier != NSRunningApplication.current.processIdentifier
|
||||
else { return }
|
||||
lastExternalApp = app
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,7 +6,8 @@
|
||||
// semantic colours, resolved to a concrete value for the *active* appearance.
|
||||
// The brand green is kept only as the accent. Light mode uses a warm,
|
||||
// iOS-matched surface set (the default `windowBackgroundColor` reads cold
|
||||
// grey on macOS); Dark mode keeps the native AppKit semantic colours.
|
||||
// grey on macOS); Dark mode uses stepped elevated greys so cards stay
|
||||
// readable against the page background.
|
||||
|
||||
import AppKit
|
||||
import SwiftUI
|
||||
@@ -33,8 +34,8 @@ enum MacSystemPalette {
|
||||
surfaceMuted: resolved(dark ? darkMuted : warmMuted, dark: dark),
|
||||
|
||||
accent: Palette.accent,
|
||||
accentMuted: Palette.accent.opacity(0.16),
|
||||
accentGlow: Palette.accent.opacity(0.35),
|
||||
accentMuted: Palette.accent.opacity(dark ? 0.22 : 0.14),
|
||||
accentGlow: Palette.accent.opacity(dark ? 0.40 : 0.32),
|
||||
|
||||
danger: resolved(.systemRed, dark: dark),
|
||||
success: Palette.accent,
|
||||
@@ -45,8 +46,12 @@ enum MacSystemPalette {
|
||||
textTertiary: resolved(.tertiaryLabelColor, dark: dark),
|
||||
textOnAccent: Color.white,
|
||||
|
||||
divider: resolved(.separatorColor, dark: dark),
|
||||
dividerStrong: resolved(.separatorColor, dark: dark),
|
||||
divider: dark
|
||||
? Color.white.opacity(0.08)
|
||||
: Color.black.opacity(0.06),
|
||||
dividerStrong: dark
|
||||
? Color.white.opacity(0.12)
|
||||
: Color.black.opacity(0.10),
|
||||
|
||||
recordRed: resolved(.systemRed, dark: dark)
|
||||
)
|
||||
|
||||
@@ -57,7 +57,9 @@ struct OSGKeyboardMacApp: App {
|
||||
// very top, matching macOS System Settings.
|
||||
.windowStyle(.hiddenTitleBar)
|
||||
.windowResizability(.contentMinSize)
|
||||
.defaultSize(width: 1_024, height: 720)
|
||||
// Open at the minimum size — same as `MacMetrics.windowMin*`, so the
|
||||
// first launch already matches the smallest allowed window.
|
||||
.defaultSize(width: MacMetrics.windowMinWidth, height: MacMetrics.windowMinHeight)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -94,8 +96,11 @@ final class MacAppDelegate: NSObject, NSApplicationDelegate {
|
||||
|
||||
func applicationDidFinishLaunching(_ notification: Notification) {
|
||||
MacAppearancePreference.applyToApp(.current)
|
||||
MacTextInsertionService.beginTrackingFrontmostApp()
|
||||
configurePopover()
|
||||
configureStatusItem()
|
||||
MacDictationOverlayController.shared.start(observing: MacDictationViewModel.shared)
|
||||
CustomLanguageModelManager.shared.prepareInBackgroundIfNeeded()
|
||||
|
||||
// The menu bar always follows the *system* appearance, so the status
|
||||
// item must ignore the app's forced light/dark override. Re-pin the
|
||||
|
||||
@@ -13,10 +13,18 @@ import Foundation
|
||||
/// Keep this protocol narrow: only what the shared pipeline needs today.
|
||||
/// Platform-specific settings UI and iCloud sync stay on concrete stores.
|
||||
public protocol ConfigurationStore: Sendable {
|
||||
/// Polish / LLM provider id.
|
||||
var providerId: String { get }
|
||||
var baseURL: String { get }
|
||||
var apiKey: String { get }
|
||||
var model: String { get }
|
||||
|
||||
/// Cloud ASR provider id — independent from polish when `engineMode == "cloud"`.
|
||||
var asrProviderId: String { get }
|
||||
var asrBaseURL: String { get }
|
||||
var asrApiKey: String { get }
|
||||
var asrModel: String { get }
|
||||
|
||||
var engineMode: String { get }
|
||||
var polishIntensity: PolishIntensity { get }
|
||||
var personalDictionary: PersonalDictionary { get }
|
||||
|
||||
@@ -180,6 +180,8 @@ public enum TypeStyle {
|
||||
public static let title3 = Font.system(size: 20, weight: .semibold)
|
||||
public static let title2 = Font.system(size: 22, weight: .bold)
|
||||
public static let title = Font.system(size: 28, weight: .bold)
|
||||
/// Home brand line + History / Dictionary / Settings page titles.
|
||||
public static let pageTitle = Font.system(size: 30, weight: .semibold)
|
||||
public static let largeTitle = Font.system(size: 34, weight: .bold)
|
||||
/// Subtle status line under the brand mark (home header).
|
||||
public static let status = Font.system(size: 13, weight: .regular)
|
||||
|
||||
@@ -16,6 +16,10 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
/// Legacy plaintext slot — migrated to Keychain on first read.
|
||||
public static let apiKeyLegacy = "config.apiKey"
|
||||
public static let model = "config.model"
|
||||
/// Cloud ASR provider — independent from polish `providerId`.
|
||||
public static let asrProviderId = "config.asrProviderId"
|
||||
public static let asrBaseURL = "config.asrBaseURL"
|
||||
public static let asrModel = "config.asrModel"
|
||||
public static let modeId = "config.modeId"
|
||||
public static let localeId = "config.localeId"
|
||||
public static let engineMode = "config.engineMode"
|
||||
@@ -51,6 +55,10 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
public var providerId: String
|
||||
public var baseURL: String
|
||||
public var model: String
|
||||
/// Cloud-engine speech-to-text provider (OpenLess-style split from polish).
|
||||
public var asrProviderId: String
|
||||
public var asrBaseURL: String
|
||||
public var asrModel: String
|
||||
public var modeId: String
|
||||
public var localeId: String
|
||||
public var engineMode: String
|
||||
@@ -95,18 +103,32 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
: .polish
|
||||
}
|
||||
|
||||
/// Local engine pins the LLM step to DeepSeek; cloud uses the user's provider.
|
||||
public var polishProviderIdOverride: String? {
|
||||
engineMode == "local" ? "deepseek" : nil
|
||||
}
|
||||
/// Polish LLM provider. Local engine no longer pins DeepSeek — user picks in Settings.
|
||||
public var polishProviderIdOverride: String? { nil }
|
||||
|
||||
public var isCloudAPIKeyMissingForVoiceInput: Bool {
|
||||
public var isCloudLLMKeyMissing: Bool {
|
||||
guard engineMode == "cloud" else { return false }
|
||||
return apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
|
||||
}
|
||||
|
||||
/// API key lives in the Keychain (cross-process, encrypted at rest).
|
||||
/// When settings iCloud sync is on, reads synchronizable Keychain items first.
|
||||
public var isCloudASRKeyMissing: Bool {
|
||||
guard engineMode == "cloud" else { return false }
|
||||
return asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
|
||||
}
|
||||
|
||||
public var isPolishKeyMissing: Bool {
|
||||
if !apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
|
||||
return false
|
||||
}
|
||||
return !PreconfiguredKeys.isDeepseekConfigured
|
||||
}
|
||||
|
||||
public var isCloudAPIKeyMissingForVoiceInput: Bool {
|
||||
guard engineMode == "cloud" else { return false }
|
||||
return isCloudASRKeyMissing || isCloudLLMKeyMissing
|
||||
}
|
||||
|
||||
/// Polish LLM uses `providerId` + Keychain `provider.<id>`.
|
||||
public var apiKey: String {
|
||||
Self.resolveAPIKey(
|
||||
defaults: nil,
|
||||
@@ -115,6 +137,15 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
)
|
||||
}
|
||||
|
||||
/// Cloud ASR uses `asrProviderId` + Keychain `asr.<id>` (falls back to legacy `provider.<id>`).
|
||||
public var asrApiKey: String {
|
||||
Self.resolveASRAPIKey(
|
||||
defaults: nil,
|
||||
providerId: asrProviderId,
|
||||
preferICloudSync: settingsICloudSyncEnabled
|
||||
)
|
||||
}
|
||||
|
||||
public func makeClient() -> LLMClient {
|
||||
OpenAICompatibleClient(
|
||||
baseURL: baseURL,
|
||||
@@ -123,6 +154,20 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
)
|
||||
}
|
||||
|
||||
/// Resolved cloud ASR model — user override or catalog default.
|
||||
public var resolvedASRModel: String {
|
||||
let trimmed = asrModel.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if !trimmed.isEmpty { return trimmed }
|
||||
return CloudASRModelCatalog.defaultModel(for: asrProviderId)
|
||||
}
|
||||
|
||||
/// Resolved cloud ASR base URL for prompt-style providers.
|
||||
public var resolvedASRBaseURL: String {
|
||||
let trimmed = asrBaseURL.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if !trimmed.isEmpty { return trimmed }
|
||||
return LLMProvider.provider(id: asrProviderId).defaultBaseURL
|
||||
}
|
||||
|
||||
// MARK: - Detected app context
|
||||
|
||||
public func detectedAppContext(from defaults: UserDefaults) -> (context: AppContext, observedAt: Date)? {
|
||||
@@ -152,9 +197,18 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
providerId: defaults.string(forKey: Keys.providerId) ?? "openai",
|
||||
baseURL: "",
|
||||
model: "",
|
||||
asrProviderId: defaults.string(forKey: Keys.asrProviderId) ?? "",
|
||||
asrBaseURL: "",
|
||||
asrModel: "",
|
||||
modeId: defaults.string(forKey: Keys.modeId) ?? "polish",
|
||||
localeId: defaults.string(forKey: Keys.localeId) ?? "auto",
|
||||
engineMode: defaults.string(forKey: Keys.engineMode) ?? "cloud",
|
||||
// Privacy-critical default: `local` keeps raw audio on-device
|
||||
// (SpeechAnalyzer). The `cloud` engine uploads recorded audio to
|
||||
// the user's configured ASR provider and must stay an explicit,
|
||||
// acknowledged opt-in (see `hasAcknowledgedCloudSharing`) — a
|
||||
// cloud default would contradict every privacy claim the app
|
||||
// makes in its docs, App Store listing, and permission prompts.
|
||||
engineMode: defaults.string(forKey: Keys.engineMode) ?? "local",
|
||||
hasCompletedOnboarding: defaults.bool(forKey: Keys.hasCompletedOnboarding),
|
||||
onboardingPage: {
|
||||
let saved = defaults.integer(forKey: Keys.onboardingPage)
|
||||
@@ -212,6 +266,19 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
config.model = defaults.string(forKey: Keys.model) ?? preset.defaultModel
|
||||
}
|
||||
|
||||
if config.asrProviderId.isEmpty {
|
||||
config.asrProviderId = config.providerId
|
||||
defaults.set(config.asrProviderId, forKey: Keys.asrProviderId)
|
||||
}
|
||||
let asrPreset = LLMProvider.provider(id: config.asrProviderId)
|
||||
if config.asrBaseURL.isEmpty {
|
||||
config.asrBaseURL = defaults.string(forKey: Keys.asrBaseURL) ?? asrPreset.defaultBaseURL
|
||||
}
|
||||
if config.asrModel.isEmpty {
|
||||
config.asrModel = defaults.string(forKey: Keys.asrModel)
|
||||
?? CloudASRModelCatalog.defaultModel(for: config.asrProviderId)
|
||||
}
|
||||
|
||||
// One-shot legacy migration: plaintext apiKey in UserDefaults → Keychain.
|
||||
_ = resolveAPIKey(
|
||||
defaults: defaults,
|
||||
@@ -219,6 +286,26 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
preferICloudSync: config.settingsICloudSyncEnabled
|
||||
)
|
||||
|
||||
// One-shot default migration for installs that predate an explicit
|
||||
// stored value. The privacy-safe defaults ("local", 30 min TTL) are
|
||||
// for NEW installs only — an existing user who ran on the old
|
||||
// defaults must keep their behavior, both because silently changing
|
||||
// engines under someone is wrong, and because iCloud settings sync
|
||||
// would stamp the flip as a fresh "edit" and propagate it to every
|
||||
// other device, overriding choices made there. Persisting the
|
||||
// resolved value makes the decision stable and sync-invisible.
|
||||
let isExistingInstall = defaults.bool(forKey: Keys.hasCompletedOnboarding)
|
||||
if defaults.string(forKey: Keys.engineMode) == nil {
|
||||
let resolved = isExistingInstall ? "cloud" : "local"
|
||||
config.engineMode = resolved
|
||||
defaults.set(resolved, forKey: Keys.engineMode)
|
||||
}
|
||||
if defaults.string(forKey: Keys.flowInactivityDuration) == nil {
|
||||
let resolved: FlowInactivityDuration = isExistingInstall ? .twelveHours : .default
|
||||
config.flowInactivityDuration = resolved
|
||||
defaults.set(resolved.rawValue, forKey: Keys.flowInactivityDuration)
|
||||
}
|
||||
|
||||
// Cloud no longer exposes off/transcribe; migrate legacy values.
|
||||
if config.engineMode == "cloud", config.modeId != "polish" {
|
||||
config.modeId = "polish"
|
||||
@@ -234,6 +321,15 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
defaults.set(openAI.defaultBaseURL, forKey: Keys.baseURL)
|
||||
defaults.set(openAI.defaultModel, forKey: Keys.model)
|
||||
}
|
||||
if config.engineMode == "cloud", config.asrProviderId == "deepseek" {
|
||||
let openAI = LLMProvider.provider(id: "openai")
|
||||
config.asrProviderId = openAI.id
|
||||
config.asrBaseURL = openAI.defaultBaseURL
|
||||
config.asrModel = CloudASRModelCatalog.defaultModel(for: openAI.id)
|
||||
defaults.set(openAI.id, forKey: Keys.asrProviderId)
|
||||
defaults.set(openAI.defaultBaseURL, forKey: Keys.asrBaseURL)
|
||||
defaults.set(openAI.defaultModel, forKey: Keys.asrModel)
|
||||
}
|
||||
|
||||
return config
|
||||
}
|
||||
@@ -242,6 +338,9 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
defaults.set(providerId, forKey: Keys.providerId)
|
||||
defaults.set(baseURL, forKey: Keys.baseURL)
|
||||
defaults.set(model, forKey: Keys.model)
|
||||
defaults.set(asrProviderId, forKey: Keys.asrProviderId)
|
||||
defaults.set(asrBaseURL, forKey: Keys.asrBaseURL)
|
||||
defaults.set(asrModel, forKey: Keys.asrModel)
|
||||
defaults.set(modeId, forKey: Keys.modeId)
|
||||
defaults.set(localeId, forKey: Keys.localeId)
|
||||
defaults.set(engineMode, forKey: Keys.engineMode)
|
||||
@@ -328,4 +427,16 @@ public struct AppGroupConfiguration: Sendable, Equatable {
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
static func resolveASRAPIKey(
|
||||
defaults: UserDefaults?,
|
||||
providerId: String,
|
||||
preferICloudSync: Bool = false
|
||||
) -> String {
|
||||
if let stored = Keychain.asrApiKey(for: providerId, preferICloudSync: preferICloudSync), !stored.isEmpty {
|
||||
return stored
|
||||
}
|
||||
// Pre-split installs: one shared key under `provider.<id>`.
|
||||
return resolveAPIKey(defaults: defaults, providerId: providerId, preferICloudSync: preferICloudSync)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
// CloudProviderRole.swift
|
||||
// OSGKeyboard · Shared
|
||||
//
|
||||
// Distinguishes cloud ASR credentials from polish LLM credentials
|
||||
// (OpenLess-style split).
|
||||
|
||||
import Foundation
|
||||
|
||||
public enum CloudProviderRole: String, Sendable, Equatable {
|
||||
case asr
|
||||
case polish
|
||||
}
|
||||
@@ -15,7 +15,12 @@ public enum FlowInactivityDuration: String, CaseIterable, Identifiable, Sendable
|
||||
|
||||
public var id: String { rawValue }
|
||||
|
||||
public static let `default`: FlowInactivityDuration = .twelveHours
|
||||
/// 30 minutes, not hours: competitors cap sessions at 5–60 min for a
|
||||
/// reason — a very long TTL keeps advertising "session active" long after
|
||||
/// the host process is likely suspended or dead, amplifying every stale-
|
||||
/// state bug into hours of confusing UI. Users can still opt into longer
|
||||
/// windows explicitly.
|
||||
public static let `default`: FlowInactivityDuration = .thirtyMinutes
|
||||
|
||||
public var timeInterval: TimeInterval {
|
||||
switch self {
|
||||
|
||||
@@ -108,4 +108,11 @@ public struct LLMProvider: Identifiable, Codable, Hashable, Sendable {
|
||||
public static var userSelectablePresets: [LLMProvider] {
|
||||
presets.filter(\.isUserSelectable)
|
||||
}
|
||||
|
||||
/// Cloud ASR presets (excludes providers without a cloud transcription API).
|
||||
public static var asrSelectablePresets: [LLMProvider] {
|
||||
userSelectablePresets.filter {
|
||||
CloudASRModelCatalog.strategy(for: $0.id) != .localFallback
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -75,6 +75,9 @@ public struct LocalASRModelDefinition: Codable, Sendable, Equatable, Identifiabl
|
||||
public let recommendedLocales: [String]
|
||||
public let supportsHotwords: Bool
|
||||
public let hotwordMode: LocalASRHotwordMode
|
||||
/// Optional localization key for a short quality/speed badge
|
||||
/// (e.g. `mac.localASR.badge.fastest`).
|
||||
public let badgeKey: String?
|
||||
public let installKind: LocalASRInstallKind
|
||||
public let installRelativePath: String?
|
||||
public let archiveBaseName: String?
|
||||
|
||||
@@ -9,7 +9,16 @@ import Foundation
|
||||
extension PersonalDictionary {
|
||||
public static let kvsKeyV2 = "personalDictionary.v2"
|
||||
public static let legacyKVSKey = "personalDictionary.v1"
|
||||
public static let tombstoneRetention: TimeInterval = 90 * 24 * 60 * 60
|
||||
/// Tombstones guard against deleted entries "resurrecting" when a
|
||||
/// long-offline device rejoins and re-merges them. A short wall-clock
|
||||
/// retention re-opened that window after only 90 days; a year keeps the
|
||||
/// window closed for any realistically dormant device while staying tiny
|
||||
/// on the wire (a tombstone is ~60 bytes of JSON), and the count cap
|
||||
/// bounds the worst case regardless of clock.
|
||||
public static let tombstoneRetention: TimeInterval = 365 * 24 * 60 * 60
|
||||
/// Hard cap independent of wall clock — the oldest tombstones are
|
||||
/// dropped first once exceeded.
|
||||
public static let maxTombstones = 500
|
||||
|
||||
/// Merges two dictionary snapshots for cross-device sync.
|
||||
///
|
||||
@@ -100,11 +109,19 @@ extension PersonalDictionary {
|
||||
clearedAt: Date?
|
||||
) -> [UUID: Date] {
|
||||
let cutoff = Date().addingTimeInterval(-tombstoneRetention)
|
||||
return tombstones.filter { _, deletedAt in
|
||||
var kept = tombstones.filter { _, deletedAt in
|
||||
if deletedAt < cutoff { return false }
|
||||
if let clearedAt, deletedAt <= clearedAt { return false }
|
||||
return true
|
||||
}
|
||||
// Enforce the count cap that makes the 365-day retention safe on the
|
||||
// KVS byte budget: keep the NEWEST tombstones (dropping an old one
|
||||
// early only re-opens the resurrection window for that one entry).
|
||||
if kept.count > maxTombstones {
|
||||
let newest = kept.sorted { $0.value > $1.value }.prefix(maxTombstones)
|
||||
kept = Dictionary(uniqueKeysWithValues: newest.map { ($0.key, $0.value) })
|
||||
}
|
||||
return kept
|
||||
}
|
||||
|
||||
private static func later(of lhs: Date?, and rhs: Date?) -> Date? {
|
||||
|
||||
@@ -53,6 +53,44 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
persistConfiguration()
|
||||
}
|
||||
}
|
||||
@Published public var asrProviderId: String {
|
||||
didSet {
|
||||
guard !isApplyingConfiguration, asrProviderId != configuration.asrProviderId else { return }
|
||||
configuration.asrProviderId = asrProviderId
|
||||
isSyncingASRProviderAPIKey = true
|
||||
asrApiKey = configuration.asrApiKey
|
||||
isSyncingASRProviderAPIKey = false
|
||||
persistConfiguration()
|
||||
}
|
||||
}
|
||||
@Published public var asrBaseURL: String {
|
||||
didSet {
|
||||
guard !isApplyingConfiguration, asrBaseURL != configuration.asrBaseURL else { return }
|
||||
configuration.asrBaseURL = asrBaseURL
|
||||
persistConfiguration()
|
||||
}
|
||||
}
|
||||
@Published public var asrApiKey: String {
|
||||
didSet {
|
||||
guard oldValue != asrApiKey, !isSyncingASRProviderAPIKey else { return }
|
||||
do {
|
||||
try Keychain.setASRAPIKey(
|
||||
asrApiKey,
|
||||
for: asrProviderId,
|
||||
useICloudSync: configuration.settingsICloudSyncEnabled
|
||||
)
|
||||
} catch {
|
||||
OSGLog.config.warning("ASR Keychain write failed: \(error.localizedDescription, privacy: .public)")
|
||||
}
|
||||
}
|
||||
}
|
||||
@Published public var asrModel: String {
|
||||
didSet {
|
||||
guard !isApplyingConfiguration, asrModel != configuration.asrModel else { return }
|
||||
configuration.asrModel = asrModel
|
||||
persistConfiguration()
|
||||
}
|
||||
}
|
||||
@Published public var modeId: String {
|
||||
didSet {
|
||||
guard !isApplyingConfiguration, modeId != configuration.modeId else { return }
|
||||
@@ -67,8 +105,8 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
persistConfiguration()
|
||||
}
|
||||
}
|
||||
/// "local" → on-device ASR + built-in DeepSeek polish.
|
||||
/// "cloud" → provider cloud ASR (with personal dictionary) + user's cloud LLM polish.
|
||||
/// "local" → on-device ASR + user's LLM polish (or built-in DeepSeek).
|
||||
/// "cloud" → user's cloud ASR + user's cloud LLM polish (independent picks).
|
||||
@Published public var engineMode: String {
|
||||
didSet {
|
||||
guard !isApplyingConfiguration, engineMode != configuration.engineMode else { return }
|
||||
@@ -213,25 +251,38 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
}
|
||||
|
||||
public var isConfigured: Bool {
|
||||
// Local engine uses on-device ASR + built-in DeepSeek polish and
|
||||
// does not need a user API key. Cloud needs base URL, key, and model.
|
||||
if isLocalEngine { return true }
|
||||
return !baseURL.isEmpty && !apiKey.isEmpty && !model.isEmpty
|
||||
if isLocalEngine {
|
||||
return isPolishConfigured
|
||||
}
|
||||
return isASRConfigured && isPolishConfigured
|
||||
}
|
||||
|
||||
public var isPolishConfigured: Bool {
|
||||
if !apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
|
||||
return !baseURL.isEmpty && !model.isEmpty
|
||||
}
|
||||
return PreconfiguredKeys.isDeepseekConfigured
|
||||
}
|
||||
|
||||
public var isASRConfigured: Bool {
|
||||
guard !isLocalEngine else { return true }
|
||||
return !asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
|
||||
&& (!asrBaseURL.isEmpty || CloudASRModelCatalog.strategy(for: asrProviderId) != .prompt)
|
||||
}
|
||||
|
||||
/// On-device ASR only; no cloud API required.
|
||||
public var isLocalEngine: Bool { configuration.isLocalEngine }
|
||||
|
||||
/// Local engine always polishes via the built-in DeepSeek path.
|
||||
public var shouldPolishLocalTranscript: Bool { isLocalEngine }
|
||||
|
||||
/// Cloud engine uses `providerId`. Local engine pins DeepSeek.
|
||||
public var localModeProviderId: String { "deepseek" }
|
||||
/// Built-in DeepSeek path when the user has not supplied their own LLM key.
|
||||
public var localModeProviderId: String {
|
||||
apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty ? "deepseek" : providerId
|
||||
}
|
||||
|
||||
private let defaults: UserDefaults
|
||||
private var configuration: AppGroupConfiguration
|
||||
private var isApplyingConfiguration = false
|
||||
private var isSyncingProviderAPIKey = false
|
||||
private var isSyncingASRProviderAPIKey = false
|
||||
|
||||
public init(defaults: UserDefaults? = nil) {
|
||||
guard let resolvedDefaults = defaults ?? AppGroup.defaultsIfAvailable else {
|
||||
@@ -270,6 +321,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
baseURL = configuration.baseURL
|
||||
apiKey = configuration.apiKey
|
||||
model = configuration.model
|
||||
asrProviderId = configuration.asrProviderId
|
||||
asrBaseURL = configuration.asrBaseURL
|
||||
asrModel = configuration.asrModel
|
||||
modeId = configuration.modeId
|
||||
localeId = configuration.localeId
|
||||
engineMode = configuration.engineMode
|
||||
@@ -284,6 +338,12 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
flowSkipAppSwitch = configuration.flowSkipAppSwitch
|
||||
flowInactivityDuration = configuration.flowInactivityDuration
|
||||
localASRCustomLanguageModelEnabled = configuration.localASRCustomLanguageModelEnabled
|
||||
isSyncingProviderAPIKey = true
|
||||
apiKey = configuration.apiKey
|
||||
isSyncingProviderAPIKey = false
|
||||
isSyncingASRProviderAPIKey = true
|
||||
asrApiKey = configuration.asrApiKey
|
||||
isSyncingASRProviderAPIKey = false
|
||||
isApplyingConfiguration = false
|
||||
}
|
||||
|
||||
@@ -293,6 +353,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
if engineMode == "cloud", providerId == "deepseek" {
|
||||
apply(preset: LLMProvider.provider(id: "openai"))
|
||||
}
|
||||
if engineMode == "cloud", asrProviderId == "deepseek" {
|
||||
applyAsr(preset: LLMProvider.provider(id: "openai"))
|
||||
}
|
||||
}
|
||||
|
||||
private func persistConfiguration(postConfigChanged: Bool = false) {
|
||||
@@ -324,6 +387,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
providerId = fresh.providerId
|
||||
baseURL = fresh.baseURL
|
||||
model = fresh.model
|
||||
asrProviderId = fresh.asrProviderId
|
||||
asrBaseURL = fresh.asrBaseURL
|
||||
asrModel = fresh.asrModel
|
||||
modeId = fresh.modeId
|
||||
localeId = fresh.localeId
|
||||
engineMode = fresh.engineMode
|
||||
@@ -341,6 +407,9 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
isSyncingProviderAPIKey = true
|
||||
apiKey = fresh.apiKey
|
||||
isSyncingProviderAPIKey = false
|
||||
isSyncingASRProviderAPIKey = true
|
||||
asrApiKey = fresh.asrApiKey
|
||||
isSyncingASRProviderAPIKey = false
|
||||
isApplyingConfiguration = false
|
||||
}
|
||||
|
||||
@@ -370,6 +439,23 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
persistConfiguration()
|
||||
}
|
||||
|
||||
public func applyAsr(preset: LLMProvider) {
|
||||
isApplyingConfiguration = true
|
||||
asrProviderId = preset.id
|
||||
if !preset.defaultBaseURL.isEmpty {
|
||||
asrBaseURL = preset.defaultBaseURL
|
||||
}
|
||||
asrModel = CloudASRModelCatalog.defaultModel(for: preset.id)
|
||||
configuration.asrProviderId = asrProviderId
|
||||
configuration.asrBaseURL = asrBaseURL
|
||||
configuration.asrModel = asrModel
|
||||
isSyncingASRProviderAPIKey = true
|
||||
asrApiKey = configuration.asrApiKey
|
||||
isSyncingASRProviderAPIKey = false
|
||||
isApplyingConfiguration = false
|
||||
persistConfiguration()
|
||||
}
|
||||
|
||||
public func reset() {
|
||||
isApplyingConfiguration = true
|
||||
let preset = LLMProvider.provider(id: "openai")
|
||||
@@ -377,12 +463,19 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
baseURL = preset.defaultBaseURL
|
||||
apiKey = ""
|
||||
model = preset.defaultModel
|
||||
asrProviderId = preset.id
|
||||
asrBaseURL = preset.defaultBaseURL
|
||||
asrModel = CloudASRModelCatalog.defaultModel(for: preset.id)
|
||||
asrApiKey = ""
|
||||
handednessPreference = .left
|
||||
localASRCustomLanguageModelEnabled = true
|
||||
hasAcknowledgedCloudSharing = false
|
||||
configuration.providerId = preset.id
|
||||
configuration.baseURL = preset.defaultBaseURL
|
||||
configuration.model = preset.defaultModel
|
||||
configuration.asrProviderId = preset.id
|
||||
configuration.asrBaseURL = preset.defaultBaseURL
|
||||
configuration.asrModel = CloudASRModelCatalog.defaultModel(for: preset.id)
|
||||
configuration.handednessPreference = .left
|
||||
configuration.localASRCustomLanguageModelEnabled = true
|
||||
configuration.hasAcknowledgedCloudSharing = false
|
||||
|
||||
@@ -14,6 +14,10 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
|
||||
public var providerId: SyncedField<String>
|
||||
public var baseURL: SyncedField<String>
|
||||
public var model: SyncedField<String>
|
||||
/// Cloud ASR provider — independent from polish `providerId`.
|
||||
public var asrProviderId: SyncedField<String>
|
||||
public var asrBaseURL: SyncedField<String>
|
||||
public var asrModel: SyncedField<String>
|
||||
public var modeId: SyncedField<String>
|
||||
public var localeId: SyncedField<String>
|
||||
public var engineMode: SyncedField<String>
|
||||
@@ -31,6 +35,9 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
|
||||
providerId: SyncedField<String>,
|
||||
baseURL: SyncedField<String>,
|
||||
model: SyncedField<String>,
|
||||
asrProviderId: SyncedField<String>,
|
||||
asrBaseURL: SyncedField<String>,
|
||||
asrModel: SyncedField<String>,
|
||||
modeId: SyncedField<String>,
|
||||
localeId: SyncedField<String>,
|
||||
engineMode: SyncedField<String>,
|
||||
@@ -47,6 +54,9 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
|
||||
self.providerId = providerId
|
||||
self.baseURL = baseURL
|
||||
self.model = model
|
||||
self.asrProviderId = asrProviderId
|
||||
self.asrBaseURL = asrBaseURL
|
||||
self.asrModel = asrModel
|
||||
self.modeId = modeId
|
||||
self.localeId = localeId
|
||||
self.engineMode = engineMode
|
||||
@@ -60,12 +70,88 @@ public struct SyncedAppSettingsV2: Codable, Equatable, Sendable {
|
||||
self.flowInactivityDuration = flowInactivityDuration
|
||||
}
|
||||
|
||||
private enum CodingKeys: String, CodingKey {
|
||||
case schemaVersion
|
||||
case providerId
|
||||
case baseURL
|
||||
case model
|
||||
case asrProviderId
|
||||
case asrBaseURL
|
||||
case asrModel
|
||||
case modeId
|
||||
case localeId
|
||||
case engineMode
|
||||
case hasAcknowledgedCloudSharing
|
||||
case uiLanguage
|
||||
case translationTargetLocaleId
|
||||
case handednessPreference
|
||||
case cursorDragNavigationEnabled
|
||||
case polishIntensity
|
||||
case flowSkipAppSwitch
|
||||
case flowInactivityDuration
|
||||
}
|
||||
|
||||
public init(from decoder: Decoder) throws {
|
||||
let container = try decoder.container(keyedBy: CodingKeys.self)
|
||||
schemaVersion = try container.decode(Int.self, forKey: .schemaVersion)
|
||||
providerId = try container.decode(SyncedField<String>.self, forKey: .providerId)
|
||||
baseURL = try container.decode(SyncedField<String>.self, forKey: .baseURL)
|
||||
model = try container.decode(SyncedField<String>.self, forKey: .model)
|
||||
modeId = try container.decode(SyncedField<String>.self, forKey: .modeId)
|
||||
localeId = try container.decode(SyncedField<String>.self, forKey: .localeId)
|
||||
engineMode = try container.decode(SyncedField<String>.self, forKey: .engineMode)
|
||||
hasAcknowledgedCloudSharing = try container.decode(SyncedField<Bool>.self, forKey: .hasAcknowledgedCloudSharing)
|
||||
uiLanguage = try container.decode(SyncedField<AppUILanguage>.self, forKey: .uiLanguage)
|
||||
translationTargetLocaleId = try container.decode(
|
||||
SyncedField<String>.self,
|
||||
forKey: .translationTargetLocaleId
|
||||
)
|
||||
handednessPreference = try container.decode(
|
||||
SyncedField<HandednessPreference>.self,
|
||||
forKey: .handednessPreference
|
||||
)
|
||||
cursorDragNavigationEnabled = try container.decode(
|
||||
SyncedField<Bool>.self,
|
||||
forKey: .cursorDragNavigationEnabled
|
||||
)
|
||||
polishIntensity = try container.decode(SyncedField<PolishIntensity>.self, forKey: .polishIntensity)
|
||||
flowSkipAppSwitch = try container.decode(SyncedField<Bool>.self, forKey: .flowSkipAppSwitch)
|
||||
flowInactivityDuration = try container.decode(
|
||||
SyncedField<FlowInactivityDuration>.self,
|
||||
forKey: .flowInactivityDuration
|
||||
)
|
||||
|
||||
if let asrProvider = try container.decodeIfPresent(SyncedField<String>.self, forKey: .asrProviderId) {
|
||||
asrProviderId = asrProvider
|
||||
} else {
|
||||
asrProviderId = providerId
|
||||
}
|
||||
if let asrURL = try container.decodeIfPresent(SyncedField<String>.self, forKey: .asrBaseURL) {
|
||||
asrBaseURL = asrURL
|
||||
} else {
|
||||
asrBaseURL = baseURL
|
||||
}
|
||||
if let asrModelField = try container.decodeIfPresent(SyncedField<String>.self, forKey: .asrModel) {
|
||||
asrModel = asrModelField
|
||||
} else {
|
||||
let fallbackModel = CloudASRModelCatalog.defaultModel(for: providerId.value)
|
||||
asrModel = SyncedField(
|
||||
value: fallbackModel,
|
||||
updatedAt: model.updatedAt,
|
||||
deviceID: model.deviceID
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Monotonic stamp used for `settingsCloudUpdatedAt` bookkeeping.
|
||||
public var latestUpdatedAt: Date {
|
||||
[
|
||||
providerId.updatedAt,
|
||||
baseURL.updatedAt,
|
||||
model.updatedAt,
|
||||
asrProviderId.updatedAt,
|
||||
asrBaseURL.updatedAt,
|
||||
asrModel.updatedAt,
|
||||
modeId.updatedAt,
|
||||
localeId.updatedAt,
|
||||
engineMode.updatedAt,
|
||||
@@ -99,6 +185,9 @@ public extension SyncedAppSettingsV2 {
|
||||
providerId: field(configuration.providerId),
|
||||
baseURL: field(configuration.baseURL),
|
||||
model: field(configuration.model),
|
||||
asrProviderId: field(configuration.asrProviderId),
|
||||
asrBaseURL: field(configuration.asrBaseURL),
|
||||
asrModel: field(configuration.asrModel),
|
||||
modeId: field(configuration.modeId),
|
||||
localeId: field(configuration.localeId),
|
||||
engineMode: field(configuration.engineMode),
|
||||
@@ -119,10 +208,14 @@ public extension SyncedAppSettingsV2 {
|
||||
func field<T>(_ value: T) -> SyncedField<T> {
|
||||
SyncedField(value: value, updatedAt: stamp, deviceID: deviceID)
|
||||
}
|
||||
let provider = field(legacy.providerId)
|
||||
return SyncedAppSettingsV2(
|
||||
providerId: field(legacy.providerId),
|
||||
providerId: provider,
|
||||
baseURL: field(legacy.baseURL),
|
||||
model: field(legacy.model),
|
||||
asrProviderId: provider,
|
||||
asrBaseURL: field(legacy.baseURL),
|
||||
asrModel: field(CloudASRModelCatalog.defaultModel(for: legacy.providerId)),
|
||||
modeId: field(legacy.modeId),
|
||||
localeId: field(legacy.localeId),
|
||||
engineMode: field(legacy.engineMode),
|
||||
@@ -142,6 +235,9 @@ public extension SyncedAppSettingsV2 {
|
||||
providerId: .merge(local: local.providerId, remote: remote.providerId),
|
||||
baseURL: .merge(local: local.baseURL, remote: remote.baseURL),
|
||||
model: .merge(local: local.model, remote: remote.model),
|
||||
asrProviderId: .merge(local: local.asrProviderId, remote: remote.asrProviderId),
|
||||
asrBaseURL: .merge(local: local.asrBaseURL, remote: remote.asrBaseURL),
|
||||
asrModel: .merge(local: local.asrModel, remote: remote.asrModel),
|
||||
modeId: .merge(local: local.modeId, remote: remote.modeId),
|
||||
localeId: .merge(local: local.localeId, remote: remote.localeId),
|
||||
engineMode: .merge(local: local.engineMode, remote: remote.engineMode),
|
||||
@@ -172,6 +268,9 @@ public extension SyncedAppSettingsV2 {
|
||||
configuration.providerId = providerId.value
|
||||
configuration.baseURL = baseURL.value
|
||||
configuration.model = model.value
|
||||
configuration.asrProviderId = asrProviderId.value
|
||||
configuration.asrBaseURL = asrBaseURL.value
|
||||
configuration.asrModel = asrModel.value
|
||||
configuration.modeId = modeId.value
|
||||
configuration.localeId = localeId.value
|
||||
configuration.engineMode = engineMode.value
|
||||
@@ -195,6 +294,9 @@ public extension SyncedAppSettingsV2 {
|
||||
patch(©.providerId, value: configuration.providerId)
|
||||
patch(©.baseURL, value: configuration.baseURL)
|
||||
patch(©.model, value: configuration.model)
|
||||
patch(©.asrProviderId, value: configuration.asrProviderId)
|
||||
patch(©.asrBaseURL, value: configuration.asrBaseURL)
|
||||
patch(©.asrModel, value: configuration.asrModel)
|
||||
patch(©.modeId, value: configuration.modeId)
|
||||
patch(©.localeId, value: configuration.localeId)
|
||||
patch(©.engineMode, value: configuration.engineMode)
|
||||
@@ -221,6 +323,9 @@ public extension SyncedAppSettingsV2 {
|
||||
touch(©.providerId, value: configuration.providerId)
|
||||
touch(©.baseURL, value: configuration.baseURL)
|
||||
touch(©.model, value: configuration.model)
|
||||
touch(©.asrProviderId, value: configuration.asrProviderId)
|
||||
touch(©.asrBaseURL, value: configuration.asrBaseURL)
|
||||
touch(©.asrModel, value: configuration.asrModel)
|
||||
touch(©.modeId, value: configuration.modeId)
|
||||
touch(©.localeId, value: configuration.localeId)
|
||||
touch(©.engineMode, value: configuration.engineMode)
|
||||
|
||||
@@ -16,11 +16,36 @@ public struct SyncedField<T: Codable & Equatable & Sendable>: Codable, Equatable
|
||||
self.deviceID = deviceID
|
||||
}
|
||||
|
||||
/// A remote timestamp may be at most this far in OUR future before we
|
||||
/// stop trusting it. Wall-clock LWW breaks down when one device's clock
|
||||
/// runs fast: its edits would win every merge forever, silently
|
||||
/// discarding later edits from correct-clock devices. Anything beyond
|
||||
/// this skew is a broken clock, not a newer edit.
|
||||
public static var maxTrustedFutureSkew: TimeInterval { 6 * 60 * 60 }
|
||||
|
||||
/// Pick the field with the newer `updatedAt`; ties break lexicographically on `deviceID`.
|
||||
///
|
||||
/// Broken-clock containment: comparing with clamped stamps alone is not
|
||||
/// enough — a far-future stamp stored in the winner would keep beating
|
||||
/// every later genuine edit (whose stamps are merely "now") until that
|
||||
/// wall-clock date actually arrived. So when the winner carries an
|
||||
/// untrusted future stamp, the stamp itself is REWRITTEN to "now" in the
|
||||
/// merged result: from then on any real edit, made later, outranks it.
|
||||
public static func merge(local: SyncedField<T>, remote: SyncedField<T>) -> SyncedField<T> {
|
||||
if remote.updatedAt > local.updatedAt { return remote }
|
||||
if local.updatedAt > remote.updatedAt { return local }
|
||||
return remote.deviceID >= local.deviceID ? remote : local
|
||||
let now = Date()
|
||||
let horizon = now.addingTimeInterval(maxTrustedFutureSkew)
|
||||
let remoteAt = remote.updatedAt > horizon ? now : remote.updatedAt
|
||||
let localAt = local.updatedAt > horizon ? now : local.updatedAt
|
||||
let winner: SyncedField<T>
|
||||
if remoteAt > localAt {
|
||||
winner = remote
|
||||
} else if localAt > remoteAt {
|
||||
winner = local
|
||||
} else {
|
||||
winner = remote.deviceID >= local.deviceID ? remote : local
|
||||
}
|
||||
guard winner.updatedAt > horizon else { return winner }
|
||||
return SyncedField(value: winner.value, updatedAt: now, deviceID: winner.deviceID)
|
||||
}
|
||||
|
||||
public static func make(value: T, deviceID: String) -> SyncedField<T> {
|
||||
|
||||
@@ -12,7 +12,16 @@ public struct SyncedSpeechHistory: Codable, Equatable, Sendable {
|
||||
public static let legacyKVSKey = "speechHistory.v1"
|
||||
public static let maxEntries = 300
|
||||
/// Tombstones older than this window may be pruned during merge.
|
||||
public static let tombstoneRetention: TimeInterval = 90 * 24 * 60 * 60
|
||||
/// Tombstones guard against deleted entries "resurrecting" when a
|
||||
/// long-offline device rejoins and re-merges them. A short wall-clock
|
||||
/// retention re-opened that window after only 90 days; a year keeps the
|
||||
/// window closed for any realistically dormant device while staying tiny
|
||||
/// on the wire (a tombstone is ~60 bytes of JSON), and the count cap
|
||||
/// bounds the worst case regardless of clock.
|
||||
public static let tombstoneRetention: TimeInterval = 365 * 24 * 60 * 60
|
||||
/// Hard cap independent of wall clock — the oldest tombstones are
|
||||
/// dropped first once exceeded.
|
||||
public static let maxTombstones = 500
|
||||
|
||||
public var schemaVersion: Int
|
||||
public var updatedAt: Date
|
||||
@@ -107,15 +116,19 @@ public struct SyncedSpeechHistory: Codable, Equatable, Sendable {
|
||||
clearedAt: Date?
|
||||
) -> [UUID: Date] {
|
||||
let cutoff = Date().addingTimeInterval(-tombstoneRetention)
|
||||
return tombstones.filter { _, deletedAt in
|
||||
if deletedAt < cutoff {
|
||||
return false
|
||||
}
|
||||
if let clearedAt, deletedAt <= clearedAt {
|
||||
return false
|
||||
}
|
||||
var kept = tombstones.filter { _, deletedAt in
|
||||
if deletedAt < cutoff { return false }
|
||||
if let clearedAt, deletedAt <= clearedAt { return false }
|
||||
return true
|
||||
}
|
||||
// Enforce the count cap that makes the 365-day retention safe on the
|
||||
// KVS byte budget: keep the NEWEST tombstones (dropping an old one
|
||||
// early only re-opens the resurrection window for that one entry).
|
||||
if kept.count > maxTombstones {
|
||||
let newest = kept.sorted { $0.value > $1.value }.prefix(maxTombstones)
|
||||
kept = Dictionary(uniqueKeysWithValues: newest.map { ($0.key, $0.value) })
|
||||
}
|
||||
return kept
|
||||
}
|
||||
|
||||
private static func later(of lhs: Date?, and rhs: Date?) -> Date? {
|
||||
|
||||
@@ -45,6 +45,7 @@
|
||||
"recommendedLocales": ["zh-CN", "en-US"],
|
||||
"supportsHotwords": true,
|
||||
"hotwordMode": "recognizerScoped",
|
||||
"badgeKey": "mac.localASR.badge.balanced",
|
||||
"installKind": "archive",
|
||||
"installRelativePath": "models/sherpa-qwen3-0.6b-int8",
|
||||
"archiveBaseName": "sherpa-onnx-qwen3-asr-0.6B-int8-2026-03-25",
|
||||
@@ -71,6 +72,7 @@
|
||||
"recommendedLocales": ["zh-CN", "en-US"],
|
||||
"supportsHotwords": true,
|
||||
"hotwordMode": "recognizerScoped",
|
||||
"badgeKey": "mac.localASR.badge.quality",
|
||||
"installKind": "repository",
|
||||
"installRelativePath": "models/sherpa-qwen3-1.7b-int8",
|
||||
"archiveBaseName": "sherpa-onnx-qwen3-asr-1.7B-int8",
|
||||
@@ -113,30 +115,6 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sherpa-paraformer-zh-int8",
|
||||
"displayName": "Paraformer Large",
|
||||
"backend": "sherpaParaformer",
|
||||
"runtimePlatform": "macos",
|
||||
"sizeBytes": 220000000,
|
||||
"recommendedLocales": ["zh-CN", "en-US"],
|
||||
"supportsHotwords": false,
|
||||
"hotwordMode": "none",
|
||||
"installKind": "archive",
|
||||
"installRelativePath": "models/sherpa-paraformer-zh-int8",
|
||||
"archiveBaseName": "sherpa-onnx-paraformer-zh-int8-2025-10-07",
|
||||
"layout": {
|
||||
"paraformerModel": "model.int8.onnx",
|
||||
"tokens": "tokens.txt"
|
||||
},
|
||||
"sources": [
|
||||
{
|
||||
"type": "github",
|
||||
"priority": 1,
|
||||
"url": "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-paraformer-zh-int8-2025-10-07.tar.bz2"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sherpa-sensevoice-small-int8",
|
||||
"displayName": "SenseVoice Small",
|
||||
@@ -146,6 +124,7 @@
|
||||
"recommendedLocales": ["zh-CN", "en-US", "ja-JP", "ko-KR"],
|
||||
"supportsHotwords": false,
|
||||
"hotwordMode": "none",
|
||||
"badgeKey": "mac.localASR.badge.fastest",
|
||||
"installKind": "archive",
|
||||
"installRelativePath": "models/sherpa-sensevoice-small-int8",
|
||||
"archiveBaseName": "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17",
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
// ASRChunkTranscribing.swift
|
||||
// OSGKeyboard · Shared
|
||||
//
|
||||
// Minimal ASR surface for pipelined utterance chunking. Keeps
|
||||
// `ChunkedUtterancePipeline` independent of iOS-only `SpeechAnalyzer`.
|
||||
|
||||
import Foundation
|
||||
|
||||
public enum ASRChunkResult: Sendable, Equatable {
|
||||
case success(String)
|
||||
case failure(String)
|
||||
case cancelled
|
||||
}
|
||||
|
||||
/// One-shot chunk transcription used by `ChunkedUtterancePipeline`.
|
||||
public protocol ASRChunkTranscribing: Sendable {
|
||||
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult
|
||||
func cancel()
|
||||
func resetForNewUtterance()
|
||||
}
|
||||
|
||||
extension ASRChunkTranscribing {
|
||||
public func cancel() {}
|
||||
public func resetForNewUtterance() {}
|
||||
}
|
||||
@@ -30,7 +30,7 @@ extension AVAudioPCMBuffer: @unchecked @retroactive Sendable {}
|
||||
|
||||
// MARK: - Protocol
|
||||
|
||||
public protocol ASRService: Sendable {
|
||||
public protocol ASRService: ASRChunkTranscribing, Sendable {
|
||||
/// Start a transcription session. The returned stream emits `.partial`
|
||||
/// updates and exactly one `.final` (or `.error`) before finishing.
|
||||
/// `SpeechAnalyzer` is always fully on-device, so there is no
|
||||
@@ -54,12 +54,6 @@ public protocol ASRService: Sendable {
|
||||
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult
|
||||
}
|
||||
|
||||
public enum ASRChunkResult: Sendable, Equatable {
|
||||
case success(String)
|
||||
case failure(String)
|
||||
case cancelled
|
||||
}
|
||||
|
||||
extension ASRService {
|
||||
public func resetForNewUtterance() {}
|
||||
|
||||
|
||||
@@ -53,6 +53,10 @@ public struct AppGroupStore: @unchecked Sendable {
|
||||
public var baseURL: String { configuration.baseURL }
|
||||
public var apiKey: String { configuration.apiKey }
|
||||
public var model: String { configuration.model }
|
||||
public var asrProviderId: String { configuration.asrProviderId }
|
||||
public var asrBaseURL: String { configuration.resolvedASRBaseURL }
|
||||
public var asrApiKey: String { configuration.asrApiKey }
|
||||
public var asrModel: String { configuration.resolvedASRModel }
|
||||
public var modeId: String { configuration.modeId }
|
||||
public var localeId: String { configuration.localeId }
|
||||
public var engineMode: String { configuration.engineMode }
|
||||
@@ -91,6 +95,12 @@ public struct AppGroupStore: @unchecked Sendable {
|
||||
config.baseURL = openAI.defaultBaseURL
|
||||
config.model = openAI.defaultModel
|
||||
}
|
||||
if mode == "cloud", config.asrProviderId == "deepseek" {
|
||||
let openAI = LLMProvider.provider(id: "openai")
|
||||
config.asrProviderId = openAI.id
|
||||
config.asrBaseURL = openAI.defaultBaseURL
|
||||
config.asrModel = CloudASRModelCatalog.defaultModel(for: openAI.id)
|
||||
}
|
||||
}
|
||||
AppGroupConfigDarwin.postConfigChanged()
|
||||
}
|
||||
|
||||
@@ -67,13 +67,13 @@ private actor ChunkWorkQueue {
|
||||
}
|
||||
|
||||
public actor ChunkedUtterancePipeline {
|
||||
private let asr: ASRService
|
||||
private let asr: any ASRChunkTranscribing
|
||||
private let locale: Locale
|
||||
private let config: FlowUtteranceChunkConfig
|
||||
private var cancelled = false
|
||||
|
||||
public init(
|
||||
asr: ASRService,
|
||||
asr: any ASRChunkTranscribing,
|
||||
locale: Locale,
|
||||
config: FlowUtteranceChunkConfig = .flowDefault
|
||||
) {
|
||||
|
||||
@@ -17,31 +17,35 @@ public protocol CloudASRTranscribing: Sendable {
|
||||
|
||||
public enum CloudASRClientFactory {
|
||||
public static func make(store: any ConfigurationStore, session: URLSession = .shared) -> CloudASRTranscribing {
|
||||
let strategy = CloudASRModelCatalog.strategy(for: store.providerId)
|
||||
let providerId = store.asrProviderId
|
||||
let strategy = CloudASRModelCatalog.strategy(for: providerId)
|
||||
let asrModel = store.asrModel.isEmpty
|
||||
? CloudASRModelCatalog.defaultModel(for: providerId)
|
||||
: store.asrModel
|
||||
switch strategy {
|
||||
case .zhipuHotwords:
|
||||
return ZhipuCloudASRClient(
|
||||
apiKey: store.apiKey,
|
||||
model: CloudASRModelCatalog.defaultModel(for: store.providerId),
|
||||
apiKey: store.asrApiKey,
|
||||
model: asrModel,
|
||||
session: session
|
||||
)
|
||||
case .alibabaVocabulary:
|
||||
return AlibabaFunASRClient(
|
||||
apiKey: store.apiKey,
|
||||
model: CloudASRModelCatalog.defaultModel(for: store.providerId),
|
||||
apiKey: store.asrApiKey,
|
||||
model: asrModel,
|
||||
persistence: store.cloudASRPersistence,
|
||||
session: session
|
||||
)
|
||||
case .prompt:
|
||||
return PromptCloudASRClient(
|
||||
providerId: store.providerId,
|
||||
baseURL: store.baseURL,
|
||||
apiKey: store.apiKey,
|
||||
model: CloudASRModelCatalog.defaultModel(for: store.providerId),
|
||||
providerId: providerId,
|
||||
baseURL: store.asrBaseURL,
|
||||
apiKey: store.asrApiKey,
|
||||
model: asrModel,
|
||||
session: session
|
||||
)
|
||||
case .localFallback:
|
||||
return UnsupportedCloudASRClient(providerId: store.providerId)
|
||||
return UnsupportedCloudASRClient(providerId: providerId)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -133,7 +133,7 @@ public final class CloudASRService: ASRService, @unchecked Sendable {
|
||||
}
|
||||
|
||||
private func bindClientIfNeeded() {
|
||||
let providerId = store.providerId
|
||||
let providerId = store.asrProviderId
|
||||
let strategy = CloudASRModelCatalog.strategy(for: providerId)
|
||||
lock.withLock {
|
||||
guard boundProviderId != providerId else { return }
|
||||
|
||||
@@ -70,7 +70,9 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
/// Fire-and-forget preparation for the host app. Safe to call repeatedly.
|
||||
/// Retries after exponential backoff when a prior attempt failed.
|
||||
public func prepareInBackgroundIfNeeded() {
|
||||
#if os(iOS)
|
||||
guard AppGroup.isAvailable else { return }
|
||||
#endif
|
||||
|
||||
let shouldStart = lock.withLock { () -> Bool in
|
||||
if case .preparing = state { return false }
|
||||
@@ -167,8 +169,8 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
throw PrepareError.missingPreparedArtifacts
|
||||
}
|
||||
|
||||
AppGroup.defaultsIfAvailable?.set(fingerprint, forKey: Storage.fingerprintKey)
|
||||
AppGroup.defaultsIfAvailable?.set(Date().timeIntervalSince1970, forKey: Storage.preparedAtKey)
|
||||
Self.persistenceDefaults.set(fingerprint, forKey: Storage.fingerprintKey)
|
||||
Self.persistenceDefaults.set(Date().timeIntervalSince1970, forKey: Storage.preparedAtKey)
|
||||
Self.clearRetryState()
|
||||
|
||||
lock.withLock {
|
||||
@@ -180,8 +182,9 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
return configuration
|
||||
}
|
||||
|
||||
// MARK: - DictationTranscriber factory
|
||||
// MARK: - DictationTranscriber factory (iOS host app)
|
||||
|
||||
#if os(iOS)
|
||||
public static func makeDictationTranscriber(
|
||||
locale: Locale,
|
||||
lmConfiguration: SFSpeechLanguageModel.Configuration?
|
||||
@@ -202,6 +205,34 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
attributeOptions: preset.attributeOptions
|
||||
)
|
||||
}
|
||||
#endif
|
||||
|
||||
// MARK: - Legacy Speech request (macOS Apple Speech fallback)
|
||||
|
||||
/// Up to 100 short phrases for `SFSpeechRecognitionRequest.contextualStrings`.
|
||||
public static func contextualStringsForRecognition(
|
||||
bias: LocalASRBiasPayload?,
|
||||
maxCount: Int = 100
|
||||
) -> [String] {
|
||||
guard let bias, !bias.hardHotwords.isEmpty else { return [] }
|
||||
return Array(bias.hardHotwords.prefix(max(1, maxCount)))
|
||||
}
|
||||
|
||||
/// Applies bundled CLM + optional contextual strings to a legacy on-device request.
|
||||
public static func applyCustomLanguageModel(
|
||||
to request: SFSpeechURLRecognitionRequest,
|
||||
locale: Locale,
|
||||
bias: LocalASRBiasPayload?
|
||||
) {
|
||||
request.requiresOnDeviceRecognition = true
|
||||
if let configuration = shared.configurationForTranscription(locale: locale) {
|
||||
request.customizedLanguageModel = configuration
|
||||
}
|
||||
let phrases = contextualStringsForRecognition(bias: bias)
|
||||
if !phrases.isEmpty {
|
||||
request.contextualStrings = phrases
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Bundle / disk helpers
|
||||
|
||||
@@ -238,15 +269,29 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
}
|
||||
|
||||
static func preparedDirectoryURL() -> URL? {
|
||||
guard let container = FileManager.default.containerURL(
|
||||
if let container = FileManager.default.containerURL(
|
||||
forSecurityApplicationGroupIdentifier: AppGroup.identifier
|
||||
) else {
|
||||
return nil
|
||||
}
|
||||
) {
|
||||
let directory = container.appendingPathComponent(Storage.subdirectory, isDirectory: true)
|
||||
try? FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
|
||||
return directory
|
||||
}
|
||||
#if os(macOS)
|
||||
guard let appSupport = FileManager.default.urls(
|
||||
for: .applicationSupportDirectory,
|
||||
in: .userDomainMask
|
||||
).first else {
|
||||
return nil
|
||||
}
|
||||
let directory = appSupport
|
||||
.appendingPathComponent("OSGKeyboard", isDirectory: true)
|
||||
.appendingPathComponent(Storage.subdirectory, isDirectory: true)
|
||||
try? FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
|
||||
return directory
|
||||
#else
|
||||
return nil
|
||||
#endif
|
||||
}
|
||||
|
||||
static func loadCachedConfigurationFromDisk() -> SFSpeechLanguageModel.Configuration? {
|
||||
guard let manifest = bundledManifest(),
|
||||
@@ -279,7 +324,7 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
}
|
||||
|
||||
private static func storedFingerprint() -> String? {
|
||||
AppGroup.defaultsIfAvailable?.string(forKey: Storage.fingerprintKey)
|
||||
persistenceDefaults.string(forKey: Storage.fingerprintKey)
|
||||
}
|
||||
|
||||
private static func removeItemIfExists(at url: URL) throws {
|
||||
@@ -310,26 +355,28 @@ public final class CustomLanguageModelManager: @unchecked Sendable {
|
||||
|
||||
// MARK: - Retry / backoff
|
||||
|
||||
private static var persistenceDefaults: UserDefaults {
|
||||
AppGroup.defaultsIfAvailable ?? .standard
|
||||
}
|
||||
|
||||
private static func storedAttemptCount() -> Int {
|
||||
AppGroup.defaultsIfAvailable?.integer(forKey: Storage.attemptCountKey) ?? 0
|
||||
persistenceDefaults.integer(forKey: Storage.attemptCountKey)
|
||||
}
|
||||
|
||||
private static func storedLastFailureAt() -> TimeInterval? {
|
||||
let value = AppGroup.defaultsIfAvailable?.double(forKey: Storage.lastFailureAtKey) ?? 0
|
||||
let value = persistenceDefaults.double(forKey: Storage.lastFailureAtKey)
|
||||
return value > 0 ? value : nil
|
||||
}
|
||||
|
||||
private static func recordFailure() {
|
||||
guard let defaults = AppGroup.defaultsIfAvailable else { return }
|
||||
let nextAttempt = storedAttemptCount() + 1
|
||||
defaults.set(nextAttempt, forKey: Storage.attemptCountKey)
|
||||
defaults.set(Date().timeIntervalSince1970, forKey: Storage.lastFailureAtKey)
|
||||
persistenceDefaults.set(nextAttempt, forKey: Storage.attemptCountKey)
|
||||
persistenceDefaults.set(Date().timeIntervalSince1970, forKey: Storage.lastFailureAtKey)
|
||||
}
|
||||
|
||||
private static func clearRetryState() {
|
||||
guard let defaults = AppGroup.defaultsIfAvailable else { return }
|
||||
defaults.removeObject(forKey: Storage.attemptCountKey)
|
||||
defaults.removeObject(forKey: Storage.lastFailureAtKey)
|
||||
persistenceDefaults.removeObject(forKey: Storage.attemptCountKey)
|
||||
persistenceDefaults.removeObject(forKey: Storage.lastFailureAtKey)
|
||||
}
|
||||
|
||||
/// Returns false when retry budget is exhausted or backoff has not elapsed.
|
||||
|
||||
@@ -159,28 +159,78 @@ private final class FlowAudioProofStore: @unchecked Sendable {
|
||||
/// incoming buffer's format actually changes, so downsampling to the ASR target
|
||||
/// rate is always valid regardless of route churn.
|
||||
private final class AdaptiveDownsampler: @unchecked Sendable {
|
||||
// `AVAudioConverter` / `AVAudioFormat` are not `Sendable`, so the state and
|
||||
// the returned converter are guarded manually via the unchecked lock APIs.
|
||||
private let lock = OSAllocatedUnfairLock<(converter: AVAudioConverter, source: AVAudioFormat)?>(uncheckedState: nil)
|
||||
// `AVAudioConverter` / `AVAudioFormat` / `AVAudioPCMBuffer` are not
|
||||
// `Sendable`, so the state is guarded manually via the unchecked lock
|
||||
// APIs. The scratch output buffer is REUSED across tap callbacks —
|
||||
// allocating on the realtime audio thread risks priority inversion, and
|
||||
// taps on one bus are serialized, so a single scratch is safe as long as
|
||||
// callers copy its contents out before returning (AudioBufferSnapshot
|
||||
// does exactly that).
|
||||
private struct State {
|
||||
var converter: AVAudioConverter
|
||||
var source: AVAudioFormat
|
||||
var scratch: AVAudioPCMBuffer
|
||||
}
|
||||
|
||||
private let lock = OSAllocatedUnfairLock<State?>(uncheckedState: nil)
|
||||
let targetFormat: AVAudioFormat
|
||||
|
||||
/// Frame headroom for the reusable output buffer. Taps deliver ≤4096
|
||||
/// input frames; output frames = input × (16k / hardwareRate), which
|
||||
/// exceeds input only for sub-16 kHz hardware (rare telephony routes),
|
||||
/// so 2× the tap size covers every realistic ratio.
|
||||
private static let scratchCapacity: AVAudioFrameCount = 8_192
|
||||
|
||||
init(targetFormat: AVAudioFormat) {
|
||||
self.targetFormat = targetFormat
|
||||
}
|
||||
|
||||
/// Returns a converter valid for `sourceFormat`, rebuilding it lazily when
|
||||
/// the hardware route (and thus the buffer format) changes.
|
||||
func converter(for sourceFormat: AVAudioFormat) -> AVAudioConverter? {
|
||||
lock.withLockUnchecked { state in
|
||||
if let state, state.source == sourceFormat {
|
||||
return state.converter
|
||||
}
|
||||
guard let converter = AVAudioConverter(from: sourceFormat, to: targetFormat) else {
|
||||
/// Downsamples `buffer` into the reusable scratch buffer and returns it,
|
||||
/// rebuilding the converter lazily when the hardware route (and thus the
|
||||
/// source format) changes. The returned buffer is only valid until the
|
||||
/// next call — copy its samples out synchronously.
|
||||
func convertReusingScratch(_ buffer: AVAudioPCMBuffer) -> AVAudioPCMBuffer? {
|
||||
let sourceFormat = buffer.format
|
||||
guard sourceFormat.sampleRate > 0 else { return nil }
|
||||
return lock.withLockUnchecked { state -> AVAudioPCMBuffer? in
|
||||
if state == nil || state!.source != sourceFormat {
|
||||
guard let converter = AVAudioConverter(from: sourceFormat, to: targetFormat),
|
||||
let scratch = AVAudioPCMBuffer(
|
||||
pcmFormat: targetFormat,
|
||||
frameCapacity: Self.scratchCapacity
|
||||
) else {
|
||||
state = nil
|
||||
return nil
|
||||
}
|
||||
state = (converter, sourceFormat)
|
||||
return converter
|
||||
state = State(converter: converter, source: sourceFormat, scratch: scratch)
|
||||
}
|
||||
guard let current = state else { return nil }
|
||||
|
||||
let wanted = AVAudioFrameCount(
|
||||
Double(buffer.frameLength) * targetFormat.sampleRate / sourceFormat.sampleRate
|
||||
)
|
||||
guard wanted > 0, wanted <= current.scratch.frameCapacity else { return nil }
|
||||
current.scratch.frameLength = 0
|
||||
|
||||
// ONE-SHOT input: the converter keeps pulling until the output
|
||||
// buffer's frameCapacity is full, and the scratch is deliberately
|
||||
// oversized — feeding the same tap buffer on every pull would
|
||||
// duplicate the audio ~6× (stuttering ASR input). After the
|
||||
// single feed we report "ran dry", so the expected status is
|
||||
// `.inputRanDry` (output not full), not `.haveData`.
|
||||
var provided = false
|
||||
var error: NSError?
|
||||
let status = current.converter.convert(to: current.scratch, error: &error) { _, outStatus in
|
||||
if provided {
|
||||
outStatus.pointee = .noDataNow
|
||||
return nil
|
||||
}
|
||||
provided = true
|
||||
outStatus.pointee = .haveData
|
||||
return buffer
|
||||
}
|
||||
guard status != .error, error == nil, current.scratch.frameLength > 0 else { return nil }
|
||||
return current.scratch
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -240,6 +290,10 @@ public final class FlowContinuousCapture {
|
||||
private var didInstallTap = false
|
||||
private var isRunning = false
|
||||
private var isRebuilding = false
|
||||
private var interrupted = false
|
||||
/// When the engine last (re)activated — a freshly started engine has
|
||||
/// produced no frames yet and must not be misclassified as a zombie.
|
||||
private var lastActivationAt = Date.distantPast
|
||||
|
||||
private var routeObserver: NSObjectProtocol?
|
||||
private var interruptionObserver: NSObjectProtocol?
|
||||
@@ -250,6 +304,11 @@ public final class FlowContinuousCapture {
|
||||
|
||||
public var running: Bool { isRunning }
|
||||
|
||||
/// True between interruption `.began` and `.ended` (phone call, Siri).
|
||||
/// While set, `setActive(true)` is guaranteed to fail — owners should
|
||||
/// wait for `.ended` (which rebuilds the engine) instead of retrying.
|
||||
public var isInterrupted: Bool { interrupted }
|
||||
|
||||
/// True when the capture session flag, tap, and audio engine are all live.
|
||||
public var engineIsLive: Bool {
|
||||
isRunning && didInstallTap && audioEngine.isRunning
|
||||
@@ -264,9 +323,33 @@ public final class FlowContinuousCapture {
|
||||
/// Called on the main actor when `engineIsLive` may have changed.
|
||||
public var onEngineLiveChanged: ((Bool) -> Void)?
|
||||
|
||||
/// Called on the main actor when the system interrupted capture (phone
|
||||
/// call, Siri). The session owner should fail any mic-open utterance —
|
||||
/// audio frames stop arriving, so continuing to "record" only captures
|
||||
/// a silence gap the user cannot see.
|
||||
public var onInterruptionBegan: (() -> Void)?
|
||||
|
||||
/// Configure `.playAndRecord`, install a permanent input tap, start the engine.
|
||||
///
|
||||
/// Idempotent: "already running and healthy" is a warm-start fast path,
|
||||
/// while "already running but producing no audio" is a zombie state
|
||||
/// (force-quit relaunch, failed cold start, mediaserverd reset) that is
|
||||
/// torn down and rebuilt in place. It must never be a silent no-op —
|
||||
/// a `guard !isRunning` early-return here turned every cold-start retry
|
||||
/// into a guaranteed audio-proof timeout.
|
||||
public func start() throws {
|
||||
guard !isRunning else { return }
|
||||
if isRunning {
|
||||
let startedMomentsAgo = Date().timeIntervalSince(lastActivationAt) < 2
|
||||
if engineIsLive && (engineHasRecentAudio(maxAge: 2) || startedMomentsAgo) {
|
||||
// Healthy warm engine — or one so fresh it simply hasn't
|
||||
// produced its first frame yet (interleaved start attempts
|
||||
// land here; rebuilding a 100 ms-old engine only multiplies
|
||||
// audio-session churn in the fragile post-relaunch window).
|
||||
return
|
||||
}
|
||||
log.info("start(): zombie engine detected (running but no live audio) — forcing rebuild")
|
||||
stop()
|
||||
}
|
||||
audioProofStore.reset()
|
||||
try activateEngine()
|
||||
isRunning = true
|
||||
@@ -353,6 +436,7 @@ public final class FlowContinuousCapture {
|
||||
} catch {
|
||||
throw StartError.engineStartFailed(error.localizedDescription)
|
||||
}
|
||||
lastActivationAt = Date()
|
||||
}
|
||||
|
||||
/// Tear down the engine and release the audio session.
|
||||
@@ -371,6 +455,7 @@ public final class FlowContinuousCapture {
|
||||
audioEngine.stop()
|
||||
}
|
||||
isRunning = false
|
||||
interrupted = false
|
||||
audioProofStore.reset()
|
||||
downsampler = nil
|
||||
targetFormat = nil
|
||||
@@ -384,6 +469,13 @@ public final class FlowContinuousCapture {
|
||||
|
||||
/// Re-activate capture after returning from background without
|
||||
/// reinstalling the tap (iOS may deactivate the audio session).
|
||||
///
|
||||
/// Doubles as the interruption-recovery probe: `setActive(true)` FAILS
|
||||
/// while a call/Siri interruption is live and succeeds once it ends, so a
|
||||
/// successful reassert proves the interruption is over. iOS does not
|
||||
/// guarantee delivery of `.ended` (commonly dropped when the app was
|
||||
/// suspended during the call), so this is the only reliable way to clear
|
||||
/// the `interrupted` latch in that case.
|
||||
@discardableResult
|
||||
public func reassertIfRunning() -> Bool {
|
||||
guard isRunning else { return false }
|
||||
@@ -395,6 +487,7 @@ public final class FlowContinuousCapture {
|
||||
options: [.defaultToSpeaker, .allowBluetoothHFP, .mixWithOthers]
|
||||
)
|
||||
try session.setActive(true, options: .notifyOthersOnDeactivation)
|
||||
interrupted = false
|
||||
if !audioEngine.isRunning {
|
||||
try audioEngine.start()
|
||||
}
|
||||
@@ -415,7 +508,14 @@ public final class FlowContinuousCapture {
|
||||
if engineHasRecentAudio(maxAge: recentFrameMaxAge) {
|
||||
return true
|
||||
}
|
||||
try? await Task.sleep(nanoseconds: 50_000_000)
|
||||
do {
|
||||
try await Task.sleep(nanoseconds: 50_000_000)
|
||||
} catch {
|
||||
// Cancelled — bail out instead of busy-spinning the main
|
||||
// actor for the rest of the window (a cancelled Task.sleep
|
||||
// returns immediately, starving concurrent start attempts).
|
||||
return false
|
||||
}
|
||||
}
|
||||
return engineHasRecentAudio(maxAge: recentFrameMaxAge)
|
||||
}
|
||||
@@ -497,8 +597,11 @@ public final class FlowContinuousCapture {
|
||||
switch type {
|
||||
case .began:
|
||||
log.info("Audio interruption began")
|
||||
interrupted = true
|
||||
notifyEngineLiveChanged()
|
||||
onInterruptionBegan?()
|
||||
case .ended:
|
||||
interrupted = false
|
||||
guard isRunning else { return }
|
||||
let shouldResume: Bool
|
||||
if let optionsRaw {
|
||||
@@ -627,26 +730,12 @@ public final class FlowContinuousCapture {
|
||||
audioProofStore.markFrameReceived()
|
||||
levelStore.update(from: buffer, barCount: FlowCaptureConstants.levelBarCount)
|
||||
|
||||
// Derive the converter from the *live* buffer format so a mid-session
|
||||
// route change (e.g. 48 kHz → 24 kHz) is handled transparently.
|
||||
let sourceFormat = buffer.format
|
||||
let targetFormat = downsampler.targetFormat
|
||||
guard sourceFormat.sampleRate > 0,
|
||||
let converter = downsampler.converter(for: sourceFormat) else { return }
|
||||
|
||||
let outFrames = AVAudioFrameCount(
|
||||
Double(buffer.frameLength) * targetFormat.sampleRate / sourceFormat.sampleRate
|
||||
)
|
||||
guard outFrames > 0,
|
||||
let outBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: outFrames)
|
||||
else { return }
|
||||
|
||||
var error: NSError?
|
||||
let status = converter.convert(to: outBuffer, error: &error) { _, outStatus in
|
||||
outStatus.pointee = .haveData
|
||||
return buffer
|
||||
}
|
||||
guard status == .haveData, error == nil, outBuffer.frameLength > 0 else { return }
|
||||
// The downsampler derives its converter from the *live* buffer
|
||||
// format (mid-session route changes handled transparently) and
|
||||
// returns a REUSED scratch buffer — no per-callback allocation
|
||||
// on the realtime thread. The snapshot below copies the samples
|
||||
// out before the next tap callback can overwrite the scratch.
|
||||
guard let outBuffer = downsampler.convertReusingScratch(buffer) else { return }
|
||||
|
||||
let snapshot = AudioBufferSnapshot(buffer: outBuffer)
|
||||
guard !snapshot.samples.isEmpty else { return }
|
||||
|
||||
@@ -130,6 +130,12 @@ public struct FlowReadySnapshot: Codable, Equatable, Sendable {
|
||||
public let localeId: String
|
||||
public let busyUtteranceId: UUID?
|
||||
public let sessionExpiresAt: TimeInterval?
|
||||
/// Host process generation that wrote this snapshot. A snapshot whose
|
||||
/// generation no longer matches `FlowSessionKeys.hostGeneration` was
|
||||
/// written by a dead process and is void immediately — no need to wait
|
||||
/// out the heartbeat-zombie window. Optional for wire compatibility with
|
||||
/// snapshots written before this field existed.
|
||||
public let hostGeneration: String?
|
||||
|
||||
public init(
|
||||
protocolVersion: Int = 1,
|
||||
@@ -142,7 +148,8 @@ public struct FlowReadySnapshot: Codable, Equatable, Sendable {
|
||||
engineMode: String,
|
||||
localeId: String,
|
||||
busyUtteranceId: UUID? = nil,
|
||||
sessionExpiresAt: TimeInterval? = nil
|
||||
sessionExpiresAt: TimeInterval? = nil,
|
||||
hostGeneration: String? = nil
|
||||
) {
|
||||
self.protocolVersion = protocolVersion
|
||||
self.sessionId = sessionId
|
||||
@@ -155,6 +162,7 @@ public struct FlowReadySnapshot: Codable, Equatable, Sendable {
|
||||
self.localeId = localeId
|
||||
self.busyUtteranceId = busyUtteranceId
|
||||
self.sessionExpiresAt = sessionExpiresAt
|
||||
self.hostGeneration = hostGeneration
|
||||
}
|
||||
}
|
||||
|
||||
@@ -266,13 +274,27 @@ public enum FlowSessionBridge {
|
||||
store.set(readyAt, forKey: FlowSessionKeys.flowHostReadyAt)
|
||||
}
|
||||
} else {
|
||||
// Keep the not-ready payload. The keyboard needs `reason`
|
||||
// (recording / processing / waitingForAudioProof / …) to tell
|
||||
// "host is busy" apart from "host is still starting". Deleting
|
||||
// the payload here forced every mid-utterance ready=false into
|
||||
// a permanent orange `preparingSession` state.
|
||||
clearHostReady(defaults: store, notify: false)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowReadyPayload)
|
||||
}
|
||||
if let expires = snapshot.sessionExpiresAt {
|
||||
store.set(expires, forKey: FlowSessionKeys.flowSessionExpires)
|
||||
}
|
||||
// Only a genuinely live host — ready, or actively serving an
|
||||
// utterance — may refresh the heartbeat here. A host stuck in a
|
||||
// failed cold start would otherwise keep "reviving" itself on every
|
||||
// engine-state flap, flickering the keyboard between reachable and
|
||||
// dead and postponing zombie-state cleanup indefinitely.
|
||||
let provesHostAlive = snapshot.ready
|
||||
|| snapshot.reason == .recording
|
||||
|| snapshot.reason == .processing
|
||||
if provesHostAlive {
|
||||
store.set(snapshot.heartbeatAt, forKey: FlowSessionKeys.flowHeartbeat)
|
||||
}
|
||||
flush(store)
|
||||
FlowSessionDarwin.postHostReadyChanged()
|
||||
}
|
||||
@@ -309,7 +331,8 @@ public enum FlowSessionBridge {
|
||||
heartbeatAt: now,
|
||||
engineMode: AppGroupConfiguration.load(fromAvailable: store).engineMode,
|
||||
localeId: AppGroupConfiguration.load(fromAvailable: store).localeId,
|
||||
sessionExpiresAt: expires
|
||||
sessionExpiresAt: expires,
|
||||
hostGeneration: store.string(forKey: FlowSessionKeys.hostGeneration)
|
||||
)
|
||||
if let data = encode(snapshot) {
|
||||
store.set(data, forKey: FlowSessionKeys.flowReadyPayload)
|
||||
@@ -418,6 +441,54 @@ public enum FlowSessionBridge {
|
||||
return staleness <= FlowSessionKeys.heartbeatStaleInterval
|
||||
}
|
||||
|
||||
// MARK: - Host process generation
|
||||
|
||||
/// Host app: rotate the per-process generation token. Call exactly once,
|
||||
/// as early as possible in the host launch path. Returns the previous
|
||||
/// generation (nil on first-ever launch) so the caller can log it.
|
||||
///
|
||||
/// Rationale: `applicationWillTerminate` is best-effort — it never runs
|
||||
/// when a *suspended* app is force-quit (the common case after a failed
|
||||
/// cold start). Instead of anchoring cleanup on a termination callback
|
||||
/// that may not fire, each launch proves the previous process is dead and
|
||||
/// voids whatever session state it left behind.
|
||||
@discardableResult
|
||||
public static func rotateHostGeneration(defaults: UserDefaults? = nil) -> String? {
|
||||
let store = resolvedDefaults(defaults)
|
||||
let previous = store.string(forKey: FlowSessionKeys.hostGeneration)
|
||||
store.set(UUID().uuidString, forKey: FlowSessionKeys.hostGeneration)
|
||||
flush(store)
|
||||
return previous
|
||||
}
|
||||
|
||||
public static func currentHostGeneration(defaults: UserDefaults? = nil) -> String? {
|
||||
let store = resolvedDefaults(defaults)
|
||||
return store.string(forKey: FlowSessionKeys.hostGeneration)
|
||||
}
|
||||
|
||||
/// Host launch reconciliation: clear every piece of persisted session
|
||||
/// state a previous (dead) generation left behind. Unlike
|
||||
/// `clearFlowState()` this keeps `pendingHostBundleId` — on a keyboard
|
||||
/// `startflow` cold launch the scene delegate stores the host bundle id
|
||||
/// *before* the SwiftUI hierarchy (and thus the session manager) exists,
|
||||
/// and wiping it here would break the return-to-host affordance.
|
||||
public static func clearFlowStateOnHostLaunch(defaults: UserDefaults? = nil) {
|
||||
let store = resolvedDefaults(defaults)
|
||||
store.set(false, forKey: FlowSessionKeys.flowSessionActive)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowSessionExpires)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowHeartbeat)
|
||||
store.removeObject(forKey: FlowSessionKeys.keyboardRecordingState)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowCommandPayload)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowResultPayload)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowAckPayload)
|
||||
store.removeObject(forKey: FlowSessionKeys.flowReadyPayload)
|
||||
clearTranscription(defaults: store)
|
||||
store.removeObject(forKey: FlowSessionKeys.audioLevels)
|
||||
store.removeObject(forKey: FlowSessionKeys.lastActivityAt)
|
||||
clearHostReady(defaults: store, notify: false)
|
||||
flush(store)
|
||||
}
|
||||
|
||||
// MARK: - Host ready contract (host app → keyboard)
|
||||
|
||||
/// Host app: publish whether Flow can accept a new utterance right now.
|
||||
@@ -446,6 +517,13 @@ public enum FlowSessionBridge {
|
||||
let store = resolvedDefaults(defaults)
|
||||
if let snapshot = readySnapshot(defaults: store) {
|
||||
guard snapshot.ready else { return false }
|
||||
// Snapshot written by a dead host generation → void immediately,
|
||||
// without waiting out the heartbeat-zombie window.
|
||||
if let snapshotGeneration = snapshot.hostGeneration,
|
||||
let currentGeneration = store.string(forKey: FlowSessionKeys.hostGeneration),
|
||||
snapshotGeneration != currentGeneration {
|
||||
return false
|
||||
}
|
||||
guard isHostReachable(defaults: store) else { return false }
|
||||
if let readyAt = snapshot.readyAt {
|
||||
let skew = abs(snapshot.heartbeatAt - readyAt)
|
||||
|
||||
@@ -33,6 +33,11 @@ public enum FlowSessionKeys {
|
||||
public static let pendingHostBundleId = "flow.pendingHostBundleId"
|
||||
/// Wall-clock timestamp of the last utterance completion or session start.
|
||||
public static let lastActivityAt = "flow.lastActivityAt"
|
||||
/// One-shot token rotated by every host-process launch. State written by
|
||||
/// a previous generation is void by definition — a fresh launch proves the
|
||||
/// previous process is dead, whether or not its `applicationWillTerminate`
|
||||
/// cleanup ever ran (it does NOT run when a suspended app is force-quit).
|
||||
public static let hostGeneration = "flow.hostGeneration.v1"
|
||||
|
||||
/// Heartbeat older than this → host is not actively reachable for recording.
|
||||
public static let heartbeatStaleInterval: TimeInterval = 3
|
||||
@@ -59,14 +64,23 @@ public enum FlowSessionKeys {
|
||||
public static let localASRWaitTimeout: TimeInterval = 120
|
||||
public static let cloudASRWaitTimeout: TimeInterval = 120
|
||||
|
||||
/// Keyboard watchdog after the user stops recording (not utterance max length).
|
||||
/// Must cover worst-case post-stop backlog: remaining SpeechAnalyzer chunks
|
||||
/// plus cloud LLM polish (see `PolishingService.effectiveTimeout` cap).
|
||||
/// Hard cap on a single LLM polish request. `PolishingService`'s scaled
|
||||
/// per-request timeout clamps to this value, so it participates in the
|
||||
/// keyboard-watchdog budget below.
|
||||
public static let maxPolishTimeout: TimeInterval = 120
|
||||
|
||||
/// Extra slack for result serialization, cross-process propagation, and
|
||||
/// the host's own polling cadence.
|
||||
public static let resultDeliveryMargin: TimeInterval = 20
|
||||
|
||||
/// Keyboard watchdog after the user stops recording (not utterance max
|
||||
/// length). Derived from the host-side budget so it always outlasts the
|
||||
/// host's worst case (ASR drain wait + polish cap + margin) — hand-tuned
|
||||
/// constants drifted below the real host maximum, making the keyboard
|
||||
/// report a timeout for transcriptions that were still going to succeed.
|
||||
public static func keyboardResultTimeout(engineMode: String) -> TimeInterval {
|
||||
if engineMode == "local" {
|
||||
return 180
|
||||
}
|
||||
return 240
|
||||
let asrWait = engineMode == "local" ? localASRWaitTimeout : cloudASRWaitTimeout
|
||||
return asrWait + maxPolishTimeout + resultDeliveryMargin
|
||||
}
|
||||
|
||||
public enum RecordingState: String, Sendable, Equatable {
|
||||
|
||||
@@ -39,20 +39,48 @@ public final class AppCloudSync {
|
||||
?? SpeechHistoryCloudSync(kvs: kvs, makeStore: makeStore, historyDefaults: historyDefaults)
|
||||
}
|
||||
|
||||
/// Serializes external-change pulls: KVS posts change notifications in
|
||||
/// bursts (one per key at times), and overlapping pull-merge-apply runs
|
||||
/// can interleave their read/write phases. `wantsAnotherPull` coalesces
|
||||
/// every burst into at most one trailing re-pull.
|
||||
private var isPulling = false
|
||||
private var wantsAnotherPull = false
|
||||
|
||||
public func startObservingExternalChanges() {
|
||||
guard externalChangeObserver == nil else { return }
|
||||
externalChangeObserver = NotificationCenter.default.addObserver(
|
||||
forName: NSUbiquitousKeyValueStore.didChangeExternallyNotification,
|
||||
object: nil,
|
||||
queue: .main
|
||||
) { [weak self] _ in
|
||||
) { [weak self] note in
|
||||
guard let self else { return }
|
||||
// Distinguish WHY the store changed. `.accountChange` means the
|
||||
// user switched iCloud accounts — the incoming values belong to a
|
||||
// DIFFERENT account and must not be merged into this one's data
|
||||
// (deleted-entry resurrection, foreign history, wrong settings).
|
||||
let reason = note.userInfo?[NSUbiquitousKeyValueStoreChangeReasonKey] as? Int
|
||||
if reason == NSUbiquitousKeyValueStoreAccountChange {
|
||||
return
|
||||
}
|
||||
Task { @MainActor in
|
||||
await self.pullAllIfEnabled()
|
||||
await self.pullAllCoalesced()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func pullAllCoalesced() async {
|
||||
guard !isPulling else {
|
||||
wantsAnotherPull = true
|
||||
return
|
||||
}
|
||||
isPulling = true
|
||||
defer { isPulling = false }
|
||||
repeat {
|
||||
wantsAnotherPull = false
|
||||
await pullAllIfEnabled()
|
||||
} while wantsAnotherPull
|
||||
}
|
||||
|
||||
public func stopObservingExternalChanges() {
|
||||
if let externalChangeObserver {
|
||||
NotificationCenter.default.removeObserver(externalChangeObserver)
|
||||
@@ -79,18 +107,27 @@ public final class AppCloudSync {
|
||||
}
|
||||
|
||||
/// Low-risk manual sync: pull remote changes, merge, then push local state.
|
||||
/// Each push runs independently — one payload failing must not abort the
|
||||
/// others (a too-large history would otherwise also kill the dictionary
|
||||
/// push). The first error is rethrown after every push has been tried.
|
||||
public func syncNow() async throws {
|
||||
let store = makeStore()
|
||||
await pullAllIfEnabled()
|
||||
|
||||
var firstError: Error?
|
||||
func attempt(_ body: () async throws -> Void) async {
|
||||
do { try await body() } catch { if firstError == nil { firstError = error } }
|
||||
}
|
||||
|
||||
if store.settingsICloudSyncEnabled {
|
||||
try await settingsSync.pushLocalIfEnabled()
|
||||
try await usageStatisticsSync.pushLocalIfEnabled()
|
||||
try await speechHistorySync.pushLocalIfEnabled()
|
||||
await attempt { try await settingsSync.pushLocalIfEnabled() }
|
||||
await attempt { try await usageStatisticsSync.pushLocalIfEnabled() }
|
||||
await attempt { try await speechHistorySync.pushLocalIfEnabled() }
|
||||
}
|
||||
if store.personalDictionaryICloudSyncEnabled {
|
||||
try await dictionarySync.pushLocalIfEnabled(store.personalDictionary)
|
||||
await attempt { try await dictionarySync.pushLocalIfEnabled(store.personalDictionary) }
|
||||
}
|
||||
if let firstError { throw firstError }
|
||||
}
|
||||
|
||||
public var settingsSyncService: SettingsCloudSync { settingsSync }
|
||||
|
||||
@@ -24,8 +24,12 @@ public final class SpeechHistoryCloudSync {
|
||||
|
||||
public static let kvsKey = SyncedSpeechHistory.kvsKey
|
||||
public static let legacyKVSKey = SyncedSpeechHistory.legacyKVSKey
|
||||
/// Stay below the ~1 MB per-key KVS limit.
|
||||
public static let maxPayloadBytes = 900_000
|
||||
/// The 1 MB iCloud KVS quota is for the WHOLE store, not per key.
|
||||
/// History and the personal dictionary must fit together (plus settings
|
||||
/// and usage stats) — once the store exceeds 1 MB, KVS rejects writes
|
||||
/// for ALL keys with `QuotaViolation` and every sync silently stops.
|
||||
/// Budget: ~400 KB history + ~400 KB dictionary + headroom for the rest.
|
||||
public static let maxPayloadBytes = 400_000
|
||||
|
||||
private let kvs: UbiquitousKeyValueStoreing
|
||||
private let makeStore: () -> AppGroupStore
|
||||
@@ -50,8 +54,16 @@ public final class SpeechHistoryCloudSync {
|
||||
public func pushLocalIfEnabled() async throws {
|
||||
let store = makeStore()
|
||||
guard store.settingsICloudSyncEnabled else { return }
|
||||
let local = SpeechHistoryStorage.load(from: historyDefaults())
|
||||
try push(local)
|
||||
// Read-merge-write: pushing the local view verbatim would overwrite
|
||||
// entries another device added since our last pull (KVS is
|
||||
// last-writer-wins with no server-side merge).
|
||||
let defaults = historyDefaults()
|
||||
let local = SpeechHistoryStorage.load(from: defaults)
|
||||
let merged = loadRemote().map { SyncedSpeechHistory.merge(local: local, remote: $0) } ?? local
|
||||
if merged != local {
|
||||
apply(merged, to: defaults, postNotification: true)
|
||||
}
|
||||
try push(merged)
|
||||
}
|
||||
|
||||
/// Called when settings sync is first enabled to union local + remote history.
|
||||
@@ -81,11 +93,34 @@ public final class SpeechHistoryCloudSync {
|
||||
}
|
||||
|
||||
public func push(_ history: SyncedSpeechHistory) throws {
|
||||
let data = try encode(history)
|
||||
let data = try encodeFittingBudget(history)
|
||||
kvs.set(data, forKey: Self.kvsKey)
|
||||
_ = kvs.synchronize()
|
||||
}
|
||||
|
||||
/// Encode, dropping the oldest entries until the payload fits the KVS
|
||||
/// budget. Without this, a history that once fit under the old 900 KB
|
||||
/// cap (300 long dictations easily exceed 400 KB) would make EVERY push
|
||||
/// throw forever — automatic pushes are fire-and-forget, so sync would
|
||||
/// just silently die with no way back short of clearing all history.
|
||||
/// Only the *uploaded* copy is trimmed; local history keeps its full
|
||||
/// 300 entries.
|
||||
func encodeFittingBudget(_ history: SyncedSpeechHistory) throws -> Data {
|
||||
var payload = history
|
||||
while true {
|
||||
do {
|
||||
return try encode(payload)
|
||||
} catch SpeechHistoryCloudSyncError.payloadTooLarge {
|
||||
guard payload.entries.count > 1 else { throw SpeechHistoryCloudSyncError.payloadTooLarge(byteCount: 0) }
|
||||
// Drop the oldest ~10% per pass; entries are kept
|
||||
// newest-first by the store, so trim from the tail.
|
||||
let sorted = payload.entries.sorted { $0.createdAt > $1.createdAt }
|
||||
let keep = max(1, sorted.count - max(1, sorted.count / 10))
|
||||
payload.entries = Array(sorted.prefix(keep))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public func loadRemote() -> SyncedSpeechHistory? {
|
||||
if let data = kvs.data(forKey: Self.kvsKey) {
|
||||
return try? decode(data)
|
||||
|
||||
@@ -45,8 +45,16 @@ public final class UsageStatisticsCloudSync {
|
||||
public func pushLocalIfEnabled() async throws {
|
||||
let store = makeStore()
|
||||
guard store.settingsICloudSyncEnabled else { return }
|
||||
// Read-merge-write: this fires after every utterance, so pushing the
|
||||
// local view verbatim would clobber counter slices another device
|
||||
// advanced since our last pull (KVS is last-writer-wins). The
|
||||
// G-Counter merge makes the push commutative instead.
|
||||
let local = SyncedUsageStatisticsStorage.load(from: store.defaults)
|
||||
try push(local)
|
||||
let merged = loadRemote().map { SyncedUsageStatisticsV2.merge(local: local, remote: $0) } ?? local
|
||||
if merged != local {
|
||||
apply(merged, to: store.defaults, postNotification: true)
|
||||
}
|
||||
try push(merged)
|
||||
}
|
||||
|
||||
/// Called when settings sync is first enabled to union local + remote totals.
|
||||
|
||||
@@ -91,8 +91,11 @@ public final class KeyboardState: ObservableObject {
|
||||
@Published public var micDisabled: Bool = false
|
||||
/// One-line helper shown above the mic while `micDisabled == true`.
|
||||
@Published public var micDisabledHint: String = ""
|
||||
/// "local" → on-device ASR only. "cloud" → ASR + LLM polish.
|
||||
@Published public var engineMode: String = "cloud"
|
||||
/// "local" → on-device ASR only. "cloud" → cloud ASR + LLM polish.
|
||||
/// Boot value must match the privacy-safe app default (`local`) so the
|
||||
/// keyboard never assumes the audio-uploading engine before the App
|
||||
/// Group config has been read.
|
||||
@Published public var engineMode: String = "local"
|
||||
/// v0.2.1 follow-up: derived — translation is on iff a target
|
||||
/// locale has been selected (mirrors `ProviderConfig.translationEnabled`
|
||||
/// so the chip / pipeline read the same source of truth).
|
||||
@@ -148,6 +151,46 @@ public final class KeyboardState: ObservableObject {
|
||||
case openSettings
|
||||
}
|
||||
|
||||
// MARK: - Temporary Flow debug (remove after orange-mic investigation)
|
||||
|
||||
/// Mirrored from `KeyboardFlowCoordinator` for the on-screen debug panel.
|
||||
@Published public var debugPendingFlowStart: Bool = false
|
||||
@Published public var debugFlowRecording: Bool = false
|
||||
@Published public var debugAwaitingFlowResult: Bool = false
|
||||
@Published public var debugHasFullAccess: Bool = false
|
||||
|
||||
/// Snapshot for the keyboard debug panel.
|
||||
public func makeFlowDebugRows(hasFullAccess: Bool) -> [FlowDebugRow] {
|
||||
debugHasFullAccess = hasFullAccess
|
||||
let micLabel: String = {
|
||||
switch micVoiceAvailability {
|
||||
case .ready: return "ready"
|
||||
case .recording: return "recording"
|
||||
case .processing: return "processing"
|
||||
case .unavailable(let reason):
|
||||
switch reason {
|
||||
case .hostNotReady: return "unavailable(hostNotReady)"
|
||||
case .preparingSession: return "unavailable(preparingSession)"
|
||||
case .noFullAccess: return "unavailable(noFullAccess)"
|
||||
case .appGroupUnavailable: return "unavailable(appGroupUnavailable)"
|
||||
case .missingAPIKey: return "unavailable(missingAPIKey)"
|
||||
}
|
||||
}
|
||||
}()
|
||||
let localRows: [FlowDebugRow] = [
|
||||
FlowDebugRow("mic", micLabel),
|
||||
FlowDebugRow("phase", String(describing: phase)),
|
||||
FlowDebugRow("pendingStart", debugPendingFlowStart ? "1" : "0"),
|
||||
FlowDebugRow("kb.recording", debugFlowRecording ? "1" : "0"),
|
||||
FlowDebugRow("kb.awaiting", debugAwaitingFlowResult ? "1" : "0"),
|
||||
FlowDebugRow("fullAccess", hasFullAccess ? "1" : "0"),
|
||||
FlowDebugRow("micDisabled", micDisabled ? "1" : "0"),
|
||||
FlowDebugRow("flowSessionPub", flowSessionActive ? "1" : "0"),
|
||||
FlowDebugRow("engine", engineMode)
|
||||
]
|
||||
return localRows + FlowDebugAppGroupSnapshot.rows()
|
||||
}
|
||||
|
||||
// Action hooks — injected by the view controller at install time.
|
||||
public var beginRecording: () -> Void = {}
|
||||
public var endRecording: () -> Void = {}
|
||||
|
||||
@@ -22,12 +22,29 @@ public enum Keychain: @unchecked Sendable {
|
||||
private static let legacyAccount = "current"
|
||||
private static let defaultProviderId = "openai"
|
||||
|
||||
private static func account(for providerId: String) -> String {
|
||||
private static func normalizedProviderId(_ providerId: String) -> String {
|
||||
let trimmed = providerId.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
let normalized = trimmed.isEmpty ? defaultProviderId : trimmed.lowercased()
|
||||
return "provider.\(normalized)"
|
||||
return trimmed.isEmpty ? defaultProviderId : trimmed.lowercased()
|
||||
}
|
||||
|
||||
/// LLM polish credentials (`provider.<id>`).
|
||||
private static func account(for providerId: String) -> String {
|
||||
"provider.\(normalizedProviderId(providerId))"
|
||||
}
|
||||
|
||||
/// Cloud ASR credentials (`asr.<id>`), independent from polish keys.
|
||||
private static func asrAccount(for providerId: String) -> String {
|
||||
"asr.\(normalizedProviderId(providerId))"
|
||||
}
|
||||
|
||||
// NOTE on kSecAttrAccessGroup: we deliberately rely on the DEFAULT
|
||||
// access group (the first entry in each target's keychain-access-groups,
|
||||
// which project.yml pins to `$(AppIdentifierPrefix)com.osgkeyboard.shared`
|
||||
// for every target). Setting the attribute explicitly would require the
|
||||
// team-prefixed string at runtime, which is not portably available
|
||||
// without injecting TeamID through the build system. If a SECOND access
|
||||
// group is ever added to any target, revisit this — reordered groups
|
||||
// would silently change which store these queries hit.
|
||||
private static func baseQuery(providerId: String, synchronizable: Bool) -> [String: Any] {
|
||||
var query: [String: Any] = [
|
||||
kSecClass as String: kSecClassGenericPassword,
|
||||
@@ -43,6 +60,130 @@ public enum Keychain: @unchecked Sendable {
|
||||
|
||||
// MARK: - Read
|
||||
|
||||
// MARK: - ASR keys
|
||||
|
||||
public static func asrApiKey(for providerId: String, preferICloudSync: Bool = false) -> String? {
|
||||
if preferICloudSync, let synced = readASRKey(providerId: providerId, synchronizable: true) {
|
||||
return synced
|
||||
}
|
||||
if let local = readASRKey(providerId: providerId, synchronizable: false) {
|
||||
return local
|
||||
}
|
||||
if preferICloudSync {
|
||||
return readASRKey(providerId: providerId, synchronizable: true)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
public static func asrApiKeyOutcome(
|
||||
for providerId: String,
|
||||
preferICloudSync: Bool = false
|
||||
) -> ReadOutcome {
|
||||
let first = readASRKeyOutcome(providerId: providerId, synchronizable: preferICloudSync)
|
||||
if case .found = first { return first }
|
||||
let second = readASRKeyOutcome(providerId: providerId, synchronizable: !preferICloudSync)
|
||||
if case .found = second { return second }
|
||||
if case .unavailable = first { return first }
|
||||
if case .unavailable = second { return second }
|
||||
return .notFound
|
||||
}
|
||||
|
||||
public static func setASRAPIKey(_ key: String, for providerId: String, useICloudSync: Bool = false) throws {
|
||||
if key.isEmpty {
|
||||
try deleteASRAPIKey(for: providerId, useICloudSync: useICloudSync)
|
||||
return
|
||||
}
|
||||
if useICloudSync {
|
||||
try writeASRKey(key, providerId: providerId, synchronizable: true)
|
||||
try? deleteASRKey(providerId: providerId, synchronizable: false)
|
||||
} else {
|
||||
try writeASRKey(key, providerId: providerId, synchronizable: false)
|
||||
}
|
||||
}
|
||||
|
||||
public static func deleteASRAPIKey(for providerId: String, useICloudSync: Bool = false) throws {
|
||||
try deleteASRKey(providerId: providerId, synchronizable: false)
|
||||
if useICloudSync {
|
||||
try deleteASRKey(providerId: providerId, synchronizable: true)
|
||||
}
|
||||
}
|
||||
|
||||
private static func readASRKey(providerId: String, synchronizable: Bool) -> String? {
|
||||
if case .found(let value) = readASRKeyOutcome(providerId: providerId, synchronizable: synchronizable) {
|
||||
return value
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
private static func readASRKeyOutcome(providerId: String, synchronizable: Bool) -> ReadOutcome {
|
||||
var query = baseASRQuery(providerId: providerId, synchronizable: synchronizable)
|
||||
query[kSecReturnData as String] = true
|
||||
query[kSecMatchLimit as String] = kSecMatchLimitOne
|
||||
var result: CFTypeRef?
|
||||
let status = SecItemCopyMatching(query as CFDictionary, &result)
|
||||
switch status {
|
||||
case errSecSuccess:
|
||||
guard let data = result as? Data,
|
||||
let str = String(data: data, encoding: .utf8) else {
|
||||
return .notFound
|
||||
}
|
||||
return .found(str)
|
||||
case errSecItemNotFound:
|
||||
// Pre-split installs stored one key under `provider.<id>` for both stages.
|
||||
return readKeyOutcome(providerId: providerId, synchronizable: synchronizable)
|
||||
default:
|
||||
#if DEBUG
|
||||
print("⚠️ [OSGKeyboard] ASR Keychain read returned OSStatus \(status); reporting unavailable.")
|
||||
#endif
|
||||
return .unavailable(status)
|
||||
}
|
||||
}
|
||||
|
||||
private static func baseASRQuery(providerId: String, synchronizable: Bool) -> [String: Any] {
|
||||
var query: [String: Any] = [
|
||||
kSecClass as String: kSecClassGenericPassword,
|
||||
kSecAttrService as String: service,
|
||||
kSecAttrAccount as String: asrAccount(for: providerId),
|
||||
kSecAttrSynchronizable as String: synchronizable ? kCFBooleanTrue! : kCFBooleanFalse!,
|
||||
]
|
||||
#if os(macOS)
|
||||
query[kSecUseDataProtectionKeychain as String] = true
|
||||
#endif
|
||||
return query
|
||||
}
|
||||
|
||||
private static func writeASRKey(_ key: String, providerId: String, synchronizable: Bool) throws {
|
||||
let data = Data(key.utf8)
|
||||
var baseQuery = baseASRQuery(providerId: providerId, synchronizable: synchronizable)
|
||||
let updateAttrs: [String: Any] = [kSecValueData as String: data]
|
||||
let updateStatus = SecItemUpdate(baseQuery as CFDictionary, updateAttrs as CFDictionary)
|
||||
switch updateStatus {
|
||||
case errSecSuccess:
|
||||
return
|
||||
case errSecItemNotFound:
|
||||
baseQuery[kSecValueData as String] = data
|
||||
baseQuery[kSecAttrAccessible as String] = synchronizable
|
||||
? kSecAttrAccessibleAfterFirstUnlock
|
||||
: kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly
|
||||
let addStatus = SecItemAdd(baseQuery as CFDictionary, nil)
|
||||
if addStatus != errSecSuccess {
|
||||
throw KeychainError.unexpectedStatus(addStatus)
|
||||
}
|
||||
default:
|
||||
throw KeychainError.unexpectedStatus(updateStatus)
|
||||
}
|
||||
}
|
||||
|
||||
private static func deleteASRKey(providerId: String, synchronizable: Bool) throws {
|
||||
let query = baseASRQuery(providerId: providerId, synchronizable: synchronizable)
|
||||
let status = SecItemDelete(query as CFDictionary)
|
||||
if status != errSecSuccess, status != errSecItemNotFound {
|
||||
throw KeychainError.unexpectedStatus(status)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - LLM keys
|
||||
|
||||
public static func apiKey(for providerId: String, preferICloudSync: Bool = false) -> String? {
|
||||
if preferICloudSync, let synced = readKey(providerId: providerId, synchronizable: true) {
|
||||
return synced
|
||||
@@ -60,7 +201,43 @@ public enum Keychain: @unchecked Sendable {
|
||||
apiKey(for: defaultProviderId)
|
||||
}
|
||||
|
||||
/// Distinguishes "no key stored" from "keychain temporarily unreadable".
|
||||
public enum ReadOutcome: Equatable {
|
||||
case found(String)
|
||||
case notFound
|
||||
/// The keychain could not be read (e.g. `errSecInteractionNotAllowed`
|
||||
/// while the device is locked before first unlock). NOT the same as
|
||||
/// "no key configured" — telling the user to re-enter their key in
|
||||
/// this state would be wrong; the read succeeds once unlocked.
|
||||
case unavailable(OSStatus)
|
||||
}
|
||||
|
||||
/// Like `apiKey(for:)`, but reports WHY a key was not returned so
|
||||
/// callers can distinguish a missing key (user action needed) from a
|
||||
/// transiently locked keychain (retry later).
|
||||
public static func apiKeyOutcome(
|
||||
for providerId: String,
|
||||
preferICloudSync: Bool = false
|
||||
) -> ReadOutcome {
|
||||
let first = readKeyOutcome(providerId: providerId, synchronizable: preferICloudSync)
|
||||
if case .found = first { return first }
|
||||
let second = readKeyOutcome(providerId: providerId, synchronizable: !preferICloudSync)
|
||||
if case .found = second { return second }
|
||||
// Neither store had it: surface "unavailable" when either read was
|
||||
// blocked, since the key may well exist behind the lock.
|
||||
if case .unavailable = first { return first }
|
||||
if case .unavailable = second { return second }
|
||||
return .notFound
|
||||
}
|
||||
|
||||
private static func readKey(providerId: String, synchronizable: Bool) -> String? {
|
||||
if case .found(let value) = readKeyOutcome(providerId: providerId, synchronizable: synchronizable) {
|
||||
return value
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
private static func readKeyOutcome(providerId: String, synchronizable: Bool) -> ReadOutcome {
|
||||
var query = baseQuery(providerId: providerId, synchronizable: synchronizable)
|
||||
query[kSecReturnData as String] = true
|
||||
query[kSecMatchLimit as String] = kSecMatchLimitOne
|
||||
@@ -70,16 +247,16 @@ public enum Keychain: @unchecked Sendable {
|
||||
case errSecSuccess:
|
||||
guard let data = result as? Data,
|
||||
let str = String(data: data, encoding: .utf8) else {
|
||||
return nil
|
||||
return .notFound
|
||||
}
|
||||
return str
|
||||
return .found(str)
|
||||
case errSecItemNotFound:
|
||||
return nil
|
||||
return .notFound
|
||||
default:
|
||||
#if DEBUG
|
||||
print("⚠️ [OSGKeyboard] Keychain read returned OSStatus \(status); treating as no key.")
|
||||
print("⚠️ [OSGKeyboard] Keychain read returned OSStatus \(status); reporting unavailable.")
|
||||
#endif
|
||||
return nil
|
||||
return .unavailable(status)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -193,6 +370,13 @@ public enum Keychain: @unchecked Sendable {
|
||||
try? writeKey(local, providerId: provider.id, synchronizable: true)
|
||||
try? deleteKey(providerId: provider.id, synchronizable: false)
|
||||
}
|
||||
for provider in LLMProvider.asrSelectablePresets {
|
||||
guard let local = readASRKey(providerId: provider.id, synchronizable: false), !local.isEmpty else {
|
||||
continue
|
||||
}
|
||||
try? writeASRKey(local, providerId: provider.id, synchronizable: true)
|
||||
try? deleteASRKey(providerId: provider.id, synchronizable: false)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Onboarding completion (reboot-durable flag)
|
||||
|
||||
+14
-3
@@ -26,8 +26,12 @@ public final class PersonalDictionaryCloudSync {
|
||||
|
||||
public static let kvsKey = PersonalDictionary.kvsKeyV2
|
||||
public static let legacyKVSKey = PersonalDictionary.legacyKVSKey
|
||||
/// Stay below the ~1 MB per-key KVS limit.
|
||||
public static let maxPayloadBytes = 900_000
|
||||
/// The 1 MB iCloud KVS quota covers the WHOLE store, not one key —
|
||||
/// this payload shares it with speech history, settings, and usage
|
||||
/// stats. Exceeding the total quota makes KVS reject writes for ALL
|
||||
/// keys (`QuotaViolation`), silently stopping every sync.
|
||||
/// Budget: ~400 KB dictionary + ~400 KB history + headroom.
|
||||
public static let maxPayloadBytes = 400_000
|
||||
|
||||
private let kvs: UbiquitousKeyValueStoreing
|
||||
private let makeStore: () -> AppGroupStore
|
||||
@@ -75,7 +79,14 @@ public final class PersonalDictionaryCloudSync {
|
||||
public func pushLocalIfEnabled(_ dictionary: PersonalDictionary) async throws {
|
||||
let store = makeStore()
|
||||
guard store.personalDictionaryICloudSyncEnabled else { return }
|
||||
try push(dictionary)
|
||||
// Read-merge-write: KVS is last-writer-wins; uploading the local
|
||||
// view verbatim would drop entries another device added since our
|
||||
// last pull. Tombstones in `merge` keep deletions intact.
|
||||
let merged = loadRemote().map { PersonalDictionary.merge(local: dictionary, remote: $0) } ?? dictionary
|
||||
if merged != dictionary {
|
||||
store.setPersonalDictionary(merged)
|
||||
}
|
||||
try push(merged)
|
||||
}
|
||||
|
||||
/// Enable sync: merge local + remote, persist locally, then upload.
|
||||
|
||||
@@ -10,8 +10,8 @@
|
||||
// English dictation while halving the network round-trip.
|
||||
//
|
||||
// Engine matrix:
|
||||
// - `engineMode == "cloud"` → provider cloud ASR + user's cloud LLM
|
||||
// - `engineMode == "local"` → on-device ASR + built-in DeepSeek
|
||||
// - `engineMode == "cloud"` → user's cloud ASR + user's cloud LLM (independent)
|
||||
// - `engineMode == "local"` → on-device ASR + user's LLM (or built-in DeepSeek)
|
||||
// - Ultra-short, structure-free utterances skip the LLM entirely
|
||||
// - Cloud without API key → raw + `.missingAPIKey` warning
|
||||
// - Local without build key → raw + `.missingAPIKey` warning
|
||||
@@ -37,6 +37,10 @@ public actor PolishingService {
|
||||
/// Local engine DeepSeek step: `PreconfiguredKeys.deepseek` is
|
||||
/// still the repo placeholder, or cloud engine Keychain is empty.
|
||||
case missingAPIKey
|
||||
/// The keychain was unreadable (device locked before first unlock)
|
||||
/// — the key likely EXISTS; treat as transient, never as "please
|
||||
/// re-enter your API key".
|
||||
case keychainLocked
|
||||
}
|
||||
|
||||
/// v0.2.1: what the LLM should do with the raw transcript. The
|
||||
@@ -95,8 +99,13 @@ public actor PolishingService {
|
||||
return TranscriptPostProcessor.localClean(trimmed)
|
||||
}
|
||||
|
||||
if store.engineMode == "cloud", injectedClient == nil {
|
||||
guard !store.apiKey.isEmpty else {
|
||||
if injectedClient == nil {
|
||||
let providerId = Self.resolvedProviderId(store: store, providerIdOverride: providerIdOverride)
|
||||
let hasPolishKey = Self.hasPolishAPIKey(store: store, providerId: providerId)
|
||||
guard hasPolishKey else {
|
||||
if case .unavailable = Keychain.apiKeyOutcome(for: providerId, preferICloudSync: true) {
|
||||
throw PolishError.keychainLocked
|
||||
}
|
||||
throw PolishError.missingAPIKey
|
||||
}
|
||||
}
|
||||
@@ -150,10 +159,14 @@ public actor PolishingService {
|
||||
)
|
||||
let apiKey: String
|
||||
if effectiveProviderId == "deepseek" {
|
||||
guard PreconfiguredKeys.isDeepseekConfigured else {
|
||||
let userKey = store.apiKey.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if !userKey.isEmpty {
|
||||
apiKey = userKey
|
||||
} else if PreconfiguredKeys.isDeepseekConfigured {
|
||||
apiKey = PreconfiguredKeys.deepseek
|
||||
} else {
|
||||
throw PolishError.missingAPIKey
|
||||
}
|
||||
apiKey = PreconfiguredKeys.deepseek
|
||||
} else {
|
||||
apiKey = store.apiKey
|
||||
}
|
||||
@@ -356,7 +369,11 @@ public actor PolishingService {
|
||||
/// (unpolished, unsegmented) ASR text.
|
||||
internal func effectiveTimeout(for text: String) -> TimeInterval {
|
||||
let scaled = timeout + (Double(text.count) / 100.0) * 10.0
|
||||
return min(max(scaled, timeout), 120)
|
||||
// The cap participates in the keyboard-watchdog budget — see
|
||||
// `FlowSessionKeys.keyboardResultTimeout`. Raising it here without
|
||||
// going through that constant would silently break the invariant
|
||||
// "keyboard timeout > host worst case".
|
||||
return min(max(scaled, timeout), FlowSessionKeys.maxPolishTimeout)
|
||||
}
|
||||
|
||||
internal static func resolvedProviderId(
|
||||
@@ -366,11 +383,25 @@ public actor PolishingService {
|
||||
if let providerIdOverride {
|
||||
return providerIdOverride
|
||||
}
|
||||
if store.engineMode == "local" {
|
||||
let id = store.providerId
|
||||
// Local installs without a user LLM key keep using the built-in DeepSeek path.
|
||||
if store.engineMode == "local",
|
||||
id != "deepseek",
|
||||
store.apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty,
|
||||
PreconfiguredKeys.isDeepseekConfigured {
|
||||
return "deepseek"
|
||||
}
|
||||
let id = store.providerId
|
||||
return id == "deepseek" ? "openai" : id
|
||||
return id == "deepseek" && store.engineMode == "cloud" ? "openai" : id
|
||||
}
|
||||
|
||||
internal static func hasPolishAPIKey(store: any ConfigurationStore, providerId: String) -> Bool {
|
||||
if !store.apiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
|
||||
return true
|
||||
}
|
||||
if providerId == "deepseek", PreconfiguredKeys.isDeepseekConfigured {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
internal static func resolveLLMEndpoint(
|
||||
@@ -396,6 +427,8 @@ extension PolishingService.PolishError: LocalizedError {
|
||||
return "LLM polish timed out."
|
||||
case .missingAPIKey:
|
||||
return "Missing API key (cloud: Settings API key; local: build configuration)."
|
||||
case .keychainLocked:
|
||||
return "API key unavailable while the device is locked — will work after unlock."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -34,6 +34,7 @@ public final class SpeechHistoryStore: ObservableObject {
|
||||
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty else { return }
|
||||
|
||||
rebaseOnPersistedStateBeforeMutation()
|
||||
let entry = SpeechHistoryEntry(text: trimmed, engineMode: engineMode)
|
||||
payload.entries.insert(entry, at: 0)
|
||||
payload.trimEntries()
|
||||
@@ -42,6 +43,7 @@ public final class SpeechHistoryStore: ObservableObject {
|
||||
}
|
||||
|
||||
public func delete(id: UUID) {
|
||||
rebaseOnPersistedStateBeforeMutation()
|
||||
guard payload.entries.contains(where: { $0.id == id }) else { return }
|
||||
payload.deletedEntryIDs[id] = Date()
|
||||
payload.entries.removeAll { $0.id == id }
|
||||
@@ -51,12 +53,24 @@ public final class SpeechHistoryStore: ObservableObject {
|
||||
}
|
||||
|
||||
public func clearAll() {
|
||||
rebaseOnPersistedStateBeforeMutation()
|
||||
payload.recordClearAll()
|
||||
payload.updatedAt = Date()
|
||||
payload.pruneTombstonesIfNeeded()
|
||||
applyPayload(postCloudPush: true)
|
||||
}
|
||||
|
||||
/// Cloud pulls write the merged history to disk but only *schedule* the
|
||||
/// in-memory reload (the notification observer hops through a Task).
|
||||
/// Mutating a stale snapshot and saving it wholesale would erase whatever
|
||||
/// that merge just brought in — always rebase on the persisted state
|
||||
/// before mutating.
|
||||
private func rebaseOnPersistedStateBeforeMutation() {
|
||||
let disk = SpeechHistoryStorage.load(from: defaults)
|
||||
guard disk != payload else { return }
|
||||
payload = SyncedSpeechHistory.merge(local: payload, remote: disk)
|
||||
}
|
||||
|
||||
public func snapshot() -> SyncedSpeechHistory {
|
||||
payload
|
||||
}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
// TranscriptionPolishFallback.swift
|
||||
// OSGKeyboard · Shared
|
||||
//
|
||||
// Shared polish-failure handling: conservative raw ASR cleanup plus
|
||||
// bilingual user-visible warnings (iOS Flow + macOS dictation).
|
||||
|
||||
import Foundation
|
||||
|
||||
public enum TranscriptionPolishFallback: Sendable {
|
||||
|
||||
public static func makeDelivery(
|
||||
rawText: String,
|
||||
error: Error,
|
||||
engineMode: String,
|
||||
chunkWarning: String?
|
||||
) -> TranscriptionDelivery {
|
||||
let fallbackText = TranscriptPostProcessor.cleanRawASRFallback(rawText)
|
||||
let warning = warning(for: error, engineMode: engineMode)
|
||||
?? degradedWarning()
|
||||
?? chunkWarning
|
||||
return TranscriptionDelivery(text: fallbackText, polishWarning: warning)
|
||||
}
|
||||
|
||||
public static func warning(for error: Error, engineMode: String) -> String? {
|
||||
if let polishError = error as? PolishingService.PolishError {
|
||||
switch polishError {
|
||||
case .missingAPIKey:
|
||||
if engineMode == "local" {
|
||||
return SharedL10n.string("flow.warning.localPolishUnavailable")
|
||||
}
|
||||
return SharedL10n.string("flow.warning.cloudPolishMissingKey")
|
||||
case .timeout, .keychainLocked:
|
||||
return degradedWarning()
|
||||
case .noTranscript:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if error is LLMError {
|
||||
return degradedWarning()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
public static func degradedWarning() -> String? {
|
||||
SharedL10n.string("flow.warning.polishDegraded")
|
||||
}
|
||||
}
|
||||
@@ -60,6 +60,27 @@ public enum DictationTextComposer {
|
||||
return isCJK(last) && isCJK(first)
|
||||
}
|
||||
|
||||
/// Separator to place between existing document text and an inserted
|
||||
/// transcript. Inserting at a cursor that sits right after "Hello" must
|
||||
/// produce "Hello world", not "Helloworld" — but CJK, whitespace, and
|
||||
/// opening-punctuation boundaries take no space.
|
||||
public static func insertionSeparator(previousContext: String?, insertion: String) -> String {
|
||||
guard let previousContext,
|
||||
let last = previousContext.unicodeScalars.last,
|
||||
let first = insertion.unicodeScalars.first else {
|
||||
return ""
|
||||
}
|
||||
if CharacterSet.whitespacesAndNewlines.contains(last) { return "" }
|
||||
if isCJK(last) || isCJK(first) { return "" }
|
||||
// No space after opening brackets/quotes ("(", "[", "「", """…).
|
||||
if CharacterSet(charactersIn: "([{\u{201C}\u{2018}\u{300C}\u{300E}\u{3010}\u{FF08}").contains(last) {
|
||||
return ""
|
||||
}
|
||||
// No space before closing/clause punctuation (".", ",", ")", "!"…).
|
||||
if CharacterSet.punctuationCharacters.contains(first) { return "" }
|
||||
return " "
|
||||
}
|
||||
|
||||
static func normalizeForOverlap(_ text: String) -> String {
|
||||
text.unicodeScalars.filter {
|
||||
!CharacterSet.whitespacesAndNewlines.contains($0)
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
// FlowDebugPanel.swift
|
||||
// OSGKeyboard · Shared
|
||||
//
|
||||
// TEMPORARY debug overlay for cross-process Flow state. Remove after the
|
||||
// orange-mic investigation. Shows the same App Group contract fields on both
|
||||
// the host app and the keyboard extension so we can see where they diverge.
|
||||
|
||||
import SwiftUI
|
||||
|
||||
/// One labeled row in the temporary Flow debug panel.
|
||||
public struct FlowDebugRow: Equatable, Sendable {
|
||||
public let label: String
|
||||
public let value: String
|
||||
|
||||
public init(_ label: String, _ value: String) {
|
||||
self.label = label
|
||||
self.value = value
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds the App Group half of the debug snapshot (readable from both processes).
|
||||
public enum FlowDebugAppGroupSnapshot {
|
||||
public static func rows(defaults: UserDefaults? = nil) -> [FlowDebugRow] {
|
||||
FlowSessionBridge.reloadFromDisk(defaults: defaults)
|
||||
let snapshot = FlowSessionBridge.readySnapshot(defaults: defaults)
|
||||
let staleness = FlowSessionBridge.heartbeatStaleness(defaults: defaults)
|
||||
let generation = FlowSessionBridge.currentHostGeneration(defaults: defaults)
|
||||
let shortGen: String = {
|
||||
guard let generation, generation.count >= 8 else { return generation ?? "nil" }
|
||||
return String(generation.prefix(8))
|
||||
}()
|
||||
let snapGen: String = {
|
||||
guard let g = snapshot?.hostGeneration, g.count >= 8 else {
|
||||
return snapshot?.hostGeneration ?? "nil"
|
||||
}
|
||||
return String(g.prefix(8))
|
||||
}()
|
||||
let expires: String = {
|
||||
guard let ts = FlowSessionBridge.sessionExpiresAt(defaults: defaults) else { return "nil" }
|
||||
let remaining = ts - Date().timeIntervalSince1970
|
||||
return String(format: "%.0fs", remaining)
|
||||
}()
|
||||
|
||||
return [
|
||||
FlowDebugRow("sessionActive", FlowSessionBridge.isSessionActive(defaults: defaults) ? "1" : "0"),
|
||||
FlowDebugRow("expiresIn", expires),
|
||||
FlowDebugRow("hostReachable", FlowSessionBridge.isHostReachable(defaults: defaults) ? "1" : "0"),
|
||||
FlowDebugRow("hostReady", FlowSessionBridge.isHostReady(defaults: defaults) ? "1" : "0"),
|
||||
FlowDebugRow("hostStale", FlowSessionBridge.isHostStale(defaults: defaults) ? "1" : "0"),
|
||||
FlowDebugRow("hbStale", staleness.map { String(format: "%.1fs", $0) } ?? "nil"),
|
||||
FlowDebugRow("snap.ready", snapshot.map { $0.ready ? "1" : "0" } ?? "nil"),
|
||||
FlowDebugRow("snap.reason", snapshot?.reason.rawValue ?? "nil"),
|
||||
FlowDebugRow("snap.session", shortUUID(snapshot?.sessionId)),
|
||||
FlowDebugRow("gen.now", shortGen),
|
||||
FlowDebugRow("gen.snap", snapGen),
|
||||
FlowDebugRow("gen.match", {
|
||||
guard let a = snapshot?.hostGeneration,
|
||||
let b = generation else { return "n/a" }
|
||||
return a == b ? "1" : "0"
|
||||
}()),
|
||||
FlowDebugRow("pendingHost", FlowSessionBridge.pendingHostBundleId(defaults: defaults) ?? "nil"),
|
||||
FlowDebugRow("recState", FlowSessionBridge.recordingState(defaults: defaults).rawValue),
|
||||
FlowDebugRow("appGroup", AppGroup.isAvailable ? "1" : "0")
|
||||
]
|
||||
}
|
||||
|
||||
private static func shortUUID(_ id: UUID?) -> String {
|
||||
guard let id else { return "nil" }
|
||||
return String(id.uuidString.prefix(8))
|
||||
}
|
||||
}
|
||||
|
||||
/// Collapsible monospaced status panel. Temporary — for investigation only.
|
||||
public struct FlowDebugPanel: View {
|
||||
public let title: String
|
||||
public let rows: [FlowDebugRow]
|
||||
@Binding public var isExpanded: Bool
|
||||
public var maxContentHeight: CGFloat
|
||||
|
||||
public init(
|
||||
title: String,
|
||||
rows: [FlowDebugRow],
|
||||
isExpanded: Binding<Bool>,
|
||||
maxContentHeight: CGFloat = 180
|
||||
) {
|
||||
self.title = title
|
||||
self.rows = rows
|
||||
self._isExpanded = isExpanded
|
||||
self.maxContentHeight = maxContentHeight
|
||||
}
|
||||
|
||||
public var body: some View {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Button {
|
||||
isExpanded.toggle()
|
||||
} label: {
|
||||
HStack(spacing: 6) {
|
||||
Text(isExpanded ? "▼" : "▶")
|
||||
.font(.system(size: 10, weight: .bold, design: .monospaced))
|
||||
Text(title)
|
||||
.font(.system(size: 11, weight: .semibold, design: .monospaced))
|
||||
Spacer(minLength: 0)
|
||||
Text(summaryChip)
|
||||
.font(.system(size: 10, weight: .bold, design: .monospaced))
|
||||
.foregroundStyle(summaryColor)
|
||||
}
|
||||
.foregroundStyle(Color.primary)
|
||||
.contentShape(Rectangle())
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
|
||||
if isExpanded {
|
||||
ScrollView {
|
||||
LazyVStack(alignment: .leading, spacing: 2) {
|
||||
ForEach(Array(rows.enumerated()), id: \.offset) { _, row in
|
||||
HStack(alignment: .top, spacing: 6) {
|
||||
Text(row.label)
|
||||
.font(.system(size: 10, weight: .medium, design: .monospaced))
|
||||
.foregroundStyle(Color.secondary)
|
||||
.frame(width: 92, alignment: .leading)
|
||||
Text(row.value)
|
||||
.font(.system(size: 10, weight: .regular, design: .monospaced))
|
||||
.foregroundStyle(Color.primary)
|
||||
.textSelection(.enabled)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
.frame(maxHeight: maxContentHeight)
|
||||
}
|
||||
}
|
||||
.padding(8)
|
||||
.background(
|
||||
RoundedRectangle(cornerRadius: 8, style: .continuous)
|
||||
.fill(.ultraThinMaterial)
|
||||
)
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: 8, style: .continuous)
|
||||
.stroke(Color.orange.opacity(0.7), lineWidth: 1)
|
||||
)
|
||||
}
|
||||
|
||||
private var summaryChip: String {
|
||||
let hostReady = rows.first(where: { $0.label == "hostReady" })?.value
|
||||
?? rows.first(where: { $0.label == "bridgeReady" })?.value
|
||||
?? "?"
|
||||
let mic = rows.first(where: { $0.label == "mic" })?.value
|
||||
if let mic {
|
||||
return "mic=\(shortMic(mic)) hr=\(hostReady)"
|
||||
}
|
||||
let active = rows.first(where: { $0.label == "isActive" })?.value ?? "?"
|
||||
return "active=\(active) hr=\(hostReady)"
|
||||
}
|
||||
|
||||
private var summaryColor: Color {
|
||||
let hostReady = rows.first(where: { $0.label == "hostReady" })?.value
|
||||
?? rows.first(where: { $0.label == "bridgeReady" })?.value
|
||||
if hostReady == "1" { return .green }
|
||||
return .orange
|
||||
}
|
||||
|
||||
private func shortMic(_ value: String) -> String {
|
||||
if value.hasPrefix("ready") { return "ready" }
|
||||
if value.contains("preparing") { return "prep" }
|
||||
if value.contains("hostNotReady") { return "notReady" }
|
||||
if value.contains("recording") { return "rec" }
|
||||
if value.contains("processing") { return "proc" }
|
||||
if value.contains("noFullAccess") { return "noFA" }
|
||||
if value.contains("appGroup") { return "noAG" }
|
||||
if value.contains("missingAPIKey") { return "noKey" }
|
||||
return String(value.prefix(12))
|
||||
}
|
||||
}
|
||||
@@ -101,18 +101,25 @@
|
||||
"keyboard.translation.a11yHint" = "Toggle translation or change the target language.";
|
||||
|
||||
/* macOS app */
|
||||
"mac.section.dashboard" = "Dashboard";
|
||||
"mac.section.dashboard" = "Home";
|
||||
"mac.section.history" = "History";
|
||||
"mac.section.dictionary" = "Personal Dictionary";
|
||||
"mac.section.dictionary" = "Dictionary";
|
||||
"mac.section.settings" = "Settings";
|
||||
"mac.brand.subtitle" = "AI DICTATION";
|
||||
"mac.brand.tagline" = "Speak it. It’s typed.";
|
||||
"mac.brand.tagline.subtitle" = "Local-first · Cross-device · One-tap dictation";
|
||||
"mac.devices" = "Devices";
|
||||
"mac.status.ready" = "Ready to dictate…";
|
||||
"mac.page.history.subtitle" = "Every dictation, kept in order.";
|
||||
"mac.page.dictionary.subtitle" = "Words that teach recognition your voice.";
|
||||
"mac.page.settings.subtitle" = "Engine, shortcuts, and appearance.";
|
||||
"mac.status.ready" = "Ready when you are…";
|
||||
"mac.status.listening" = "Listening…";
|
||||
"mac.status.transcribing" = "Transcribing…";
|
||||
"mac.status.polishing" = "Polishing…";
|
||||
"mac.status.copied" = "Copied to clipboard";
|
||||
"mac.status.pasted" = "Inserted into front app";
|
||||
"mac.status.copiedAndPasted" = "Copied and inserted";
|
||||
"mac.status.deliveryWithNote" = "%@ — %@";
|
||||
"mac.stat.dictationTime" = "Dictation Time";
|
||||
"mac.stat.words" = "Dictation Chars";
|
||||
"mac.stat.translation" = "Translation Chars";
|
||||
@@ -123,6 +130,12 @@
|
||||
"mac.stat.customTerms" = "Custom terms";
|
||||
"mac.status.chipReady" = "Ready";
|
||||
"mac.status.chipProcessing" = "Processing";
|
||||
"mac.overlay.listening" = "Listening";
|
||||
"mac.overlay.preparing" = "Starting…";
|
||||
"mac.overlay.transcribing" = "Transcribing";
|
||||
"mac.overlay.polishing" = "Polishing";
|
||||
"mac.overlay.live" = "Live";
|
||||
"mac.overlay.done" = "Done";
|
||||
"mac.record.start" = "Record";
|
||||
"mac.record.stop" = "Stop";
|
||||
"mac.record.pressStop" = "Press Stop";
|
||||
@@ -133,8 +146,8 @@
|
||||
"mac.mode.local" = "Local Mode";
|
||||
"mac.connected" = "Connected";
|
||||
"mac.offline" = "Offline";
|
||||
"mac.history.recent" = "Recent";
|
||||
"mac.history.empty" = "No voice transcripts yet.";
|
||||
"mac.history.empty" = "No dictations yet";
|
||||
"mac.history.emptyBody" = "Hold Option anywhere to speak — transcripts land here.";
|
||||
"mac.history.select" = "Select a dictation";
|
||||
"mac.history.clearTitle" = "Clear all history?";
|
||||
"mac.history.clearMessage" = "This cannot be undone.";
|
||||
@@ -151,6 +164,12 @@
|
||||
"mac.dict.deleteMessage" = "This cannot be undone.";
|
||||
"mac.hint.holdOption" = "Hold Option to dictate";
|
||||
"mac.settings.cloudProvider" = "CLOUD PROVIDER";
|
||||
"mac.settings.polishProvider" = "TEXT POLISH (LLM)";
|
||||
"mac.settings.asrProvider" = "SPEECH RECOGNITION (ASR)";
|
||||
"mac.settings.asrService" = "ASR service";
|
||||
"mac.settings.asrApiKey" = "ASR API key";
|
||||
"mac.settings.asrModel" = "ASR model";
|
||||
"mac.settings.baseURL" = "Base URL";
|
||||
"mac.settings.service" = "Service";
|
||||
"mac.settings.apiKey" = "API Key";
|
||||
"mac.settings.model" = "Model";
|
||||
@@ -174,6 +193,15 @@
|
||||
"mac.settings.autoPasteDesc" = "Simulate ⌘V in the front app after transcription (requires Accessibility).";
|
||||
"mac.settings.hotkey" = "Global shortcut";
|
||||
"mac.settings.hotkeyDesc" = "Hold Option (⌥) to dictate from any app.";
|
||||
"mac.settings.hotkeyTrigger" = "Shortcut key";
|
||||
"mac.settings.hotkeyTriggerDesc" = "Which Option (⌥) key starts dictation when held.";
|
||||
"mac.hotkeyTrigger.rightOption" = "Right Option (⌥)";
|
||||
"mac.hotkeyTrigger.leftOption" = "Left Option (⌥)";
|
||||
"mac.hotkeyTrigger.eitherOption" = "Either Option key";
|
||||
"mac.hint.hold.rightOption" = "Hold right Option (⌥) to dictate";
|
||||
"mac.hint.hold.leftOption" = "Hold left Option (⌥) to dictate";
|
||||
"mac.hint.hold.eitherOption" = "Hold either Option (⌥) to dictate";
|
||||
"mac.hint.holdOption" = "Hold Option to dictate";
|
||||
"mac.settings.qwen3Model" = "Qwen3 model folder";
|
||||
"mac.settings.qwen3ModelDesc" = "Folder with config.json, model.safetensors, vocab.json, and merges.txt.";
|
||||
"mac.settings.qwen3Browse" = "Choose folder…";
|
||||
@@ -241,6 +269,9 @@
|
||||
"mac.localASR.installed" = "Installed";
|
||||
"mac.localASR.notInstalled" = "Not installed";
|
||||
"mac.localASR.personalDictionaryTag" = "Personal dictionary";
|
||||
"mac.localASR.badge.fastest" = "Fastest";
|
||||
"mac.localASR.badge.balanced" = "Most balanced";
|
||||
"mac.localASR.badge.quality" = "Best quality";
|
||||
"mac.localASR.hotwordsYes" = "Hotwords";
|
||||
"mac.localASR.hotwordsNo" = "No hotwords";
|
||||
"mac.localASR.catalogMissing" = "Local ASR catalog is missing from the app bundle.";
|
||||
|
||||
@@ -101,18 +101,25 @@
|
||||
"keyboard.translation.a11yHint" = "切换翻译或更改目标语言。";
|
||||
|
||||
/* macOS 应用 */
|
||||
"mac.section.dashboard" = "仪表盘";
|
||||
"mac.section.dashboard" = "首页";
|
||||
"mac.section.history" = "历史";
|
||||
"mac.section.dictionary" = "个性词库";
|
||||
"mac.section.dictionary" = "词库";
|
||||
"mac.section.settings" = "设置";
|
||||
"mac.brand.subtitle" = "AI 听写";
|
||||
"mac.brand.tagline" = "开口即文字。";
|
||||
"mac.brand.tagline.subtitle" = "本地优先 · 跨端同步 · 一键听写";
|
||||
"mac.devices" = "设备";
|
||||
"mac.status.ready" = "准备听写…";
|
||||
"mac.status.listening" = "录音中…";
|
||||
"mac.page.history.subtitle" = "每一次听写,按时间妥善保存。";
|
||||
"mac.page.dictionary.subtitle" = "让识别更懂你的用词。";
|
||||
"mac.page.settings.subtitle" = "引擎、快捷键与外观。";
|
||||
"mac.status.ready" = "准备好了,随时开口…";
|
||||
"mac.status.listening" = "正在聆听…";
|
||||
"mac.status.transcribing" = "识别中…";
|
||||
"mac.status.polishing" = "润色中…";
|
||||
"mac.status.copied" = "已复制到剪贴板";
|
||||
"mac.status.pasted" = "已插入前台应用";
|
||||
"mac.status.copiedAndPasted" = "已复制并插入";
|
||||
"mac.status.deliveryWithNote" = "%@ — %@";
|
||||
"mac.stat.dictationTime" = "听写时长";
|
||||
"mac.stat.words" = "听写字数";
|
||||
"mac.stat.translation" = "翻译字数";
|
||||
@@ -123,6 +130,12 @@
|
||||
"mac.stat.customTerms" = "自定义词条";
|
||||
"mac.status.chipReady" = "就绪";
|
||||
"mac.status.chipProcessing" = "处理中";
|
||||
"mac.overlay.listening" = "聆听中";
|
||||
"mac.overlay.preparing" = "启动中…";
|
||||
"mac.overlay.transcribing" = "识别中";
|
||||
"mac.overlay.polishing" = "润色中";
|
||||
"mac.overlay.live" = "实时";
|
||||
"mac.overlay.done" = "已完成";
|
||||
"mac.record.start" = "开始录音";
|
||||
"mac.record.stop" = "停止";
|
||||
"mac.record.pressStop" = "点击停止";
|
||||
@@ -133,8 +146,8 @@
|
||||
"mac.mode.local" = "本地模式";
|
||||
"mac.connected" = "已连接";
|
||||
"mac.offline" = "离线";
|
||||
"mac.history.recent" = "最近";
|
||||
"mac.history.empty" = "还没有语音识别记录。";
|
||||
"mac.history.empty" = "还没有听写记录";
|
||||
"mac.history.emptyBody" = "在任意应用按住 Option 开口说话,记录会出现在这里。";
|
||||
"mac.history.select" = "选择一条记录";
|
||||
"mac.history.clearTitle" = "清空全部历史?";
|
||||
"mac.history.clearMessage" = "此操作无法撤销。";
|
||||
@@ -151,6 +164,12 @@
|
||||
"mac.dict.deleteMessage" = "此操作无法撤销。";
|
||||
"mac.hint.holdOption" = "长按 Option 开始听写";
|
||||
"mac.settings.cloudProvider" = "云端服务商";
|
||||
"mac.settings.polishProvider" = "文本润色(LLM)";
|
||||
"mac.settings.asrProvider" = "语音转写(ASR)";
|
||||
"mac.settings.asrService" = "转写服务";
|
||||
"mac.settings.asrApiKey" = "转写 API 密钥";
|
||||
"mac.settings.asrModel" = "转写模型";
|
||||
"mac.settings.baseURL" = "接口地址";
|
||||
"mac.settings.service" = "服务商";
|
||||
"mac.settings.apiKey" = "API 密钥";
|
||||
"mac.settings.model" = "模型";
|
||||
@@ -174,6 +193,15 @@
|
||||
"mac.settings.autoPasteDesc" = "转写完成后向前台应用模拟 ⌘V(需辅助功能权限)。";
|
||||
"mac.settings.hotkey" = "全局快捷键";
|
||||
"mac.settings.hotkeyDesc" = "按住 Option (⌥) 键即可从任意应用开始听写。";
|
||||
"mac.settings.hotkeyTrigger" = "快捷键按键";
|
||||
"mac.settings.hotkeyTriggerDesc" = "按住哪个 Option (⌥) 键开始听写。";
|
||||
"mac.hotkeyTrigger.rightOption" = "右 Option (⌥)";
|
||||
"mac.hotkeyTrigger.leftOption" = "左 Option (⌥)";
|
||||
"mac.hotkeyTrigger.eitherOption" = "任一 Option 键";
|
||||
"mac.hint.hold.rightOption" = "长按右 Option(⌥)开始听写";
|
||||
"mac.hint.hold.leftOption" = "长按左 Option(⌥)开始听写";
|
||||
"mac.hint.hold.eitherOption" = "长按任一 Option(⌥)开始听写";
|
||||
"mac.hint.holdOption" = "长按 Option 开始听写";
|
||||
"mac.settings.qwen3Model" = "Qwen3 模型目录";
|
||||
"mac.settings.qwen3ModelDesc" = "需包含 config.json、model.safetensors、vocab.json 与 merges.txt。";
|
||||
"mac.settings.qwen3Browse" = "选择文件夹…";
|
||||
@@ -241,6 +269,9 @@
|
||||
"mac.localASR.installed" = "已安装";
|
||||
"mac.localASR.notInstalled" = "未安装";
|
||||
"mac.localASR.personalDictionaryTag" = "个性词库";
|
||||
"mac.localASR.badge.fastest" = "速度最快";
|
||||
"mac.localASR.badge.balanced" = "最平衡";
|
||||
"mac.localASR.badge.quality" = "质量最好";
|
||||
"mac.localASR.hotwordsYes" = "支持热词";
|
||||
"mac.localASR.hotwordsNo" = "无热词";
|
||||
"mac.localASR.catalogMissing" = "应用包内缺少本地 ASR 模型目录。";
|
||||
|
||||
@@ -20,7 +20,8 @@ final class AppGroupConfigurationTests: XCTestCase {
|
||||
XCTAssertEqual(config.providerId, "openai")
|
||||
XCTAssertEqual(config.modeId, "polish")
|
||||
XCTAssertEqual(config.localeId, "auto")
|
||||
XCTAssertEqual(config.engineMode, "cloud")
|
||||
// Privacy-critical: the default engine must keep audio on-device.
|
||||
XCTAssertEqual(config.engineMode, "local")
|
||||
XCTAssertFalse(config.hasCompletedOnboarding)
|
||||
XCTAssertEqual(config.onboardingPage, 0)
|
||||
XCTAssertFalse(config.hasAcknowledgedCloudSharing)
|
||||
@@ -31,7 +32,7 @@ final class AppGroupConfigurationTests: XCTestCase {
|
||||
XCTAssertEqual(config.polishIntensity, .default)
|
||||
XCTAssertTrue(config.personalDictionary.entries.isEmpty)
|
||||
XCTAssertTrue(config.flowSkipAppSwitch)
|
||||
XCTAssertEqual(config.flowInactivityDuration, .twelveHours)
|
||||
XCTAssertEqual(config.flowInactivityDuration, .thirtyMinutes)
|
||||
}
|
||||
|
||||
func testSaveAndLoadRoundTrip() {
|
||||
@@ -40,9 +41,13 @@ final class AppGroupConfigurationTests: XCTestCase {
|
||||
config.providerId = "anthropic"
|
||||
config.baseURL = "https://example.com/v1"
|
||||
config.model = "claude-test"
|
||||
config.asrProviderId = "zhipu"
|
||||
config.asrBaseURL = "https://open.bigmodel.cn/api/paas/v4"
|
||||
config.asrModel = "glm-asr-2512"
|
||||
config.modeId = "polish"
|
||||
config.localeId = "zh-Hans"
|
||||
config.engineMode = "local"
|
||||
// Non-default value so the round-trip proves persistence.
|
||||
config.engineMode = "cloud"
|
||||
config.hasCompletedOnboarding = true
|
||||
config.onboardingPage = 2
|
||||
config.hasAcknowledgedCloudSharing = true
|
||||
@@ -52,15 +57,19 @@ final class AppGroupConfigurationTests: XCTestCase {
|
||||
config.cursorDragNavigationEnabled = false
|
||||
config.polishIntensity = .light
|
||||
config.flowSkipAppSwitch = false
|
||||
config.flowInactivityDuration = .thirtyMinutes
|
||||
// Use a non-default value so the round-trip actually proves persistence.
|
||||
config.flowInactivityDuration = .threeHours
|
||||
config.save(to: defaults)
|
||||
|
||||
let loaded = AppGroupConfiguration.load(fromAvailable: defaults)
|
||||
XCTAssertEqual(loaded.providerId, "anthropic")
|
||||
XCTAssertEqual(loaded.baseURL, "https://example.com/v1")
|
||||
XCTAssertEqual(loaded.model, "claude-test")
|
||||
XCTAssertEqual(loaded.asrProviderId, "zhipu")
|
||||
XCTAssertEqual(loaded.asrBaseURL, "https://open.bigmodel.cn/api/paas/v4")
|
||||
XCTAssertEqual(loaded.asrModel, "glm-asr-2512")
|
||||
XCTAssertEqual(loaded.localeId, "zh-Hans")
|
||||
XCTAssertEqual(loaded.engineMode, "local")
|
||||
XCTAssertEqual(loaded.engineMode, "cloud")
|
||||
XCTAssertTrue(loaded.hasCompletedOnboarding)
|
||||
XCTAssertEqual(loaded.onboardingPage, 2)
|
||||
XCTAssertTrue(loaded.hasAcknowledgedCloudSharing)
|
||||
@@ -71,7 +80,37 @@ final class AppGroupConfigurationTests: XCTestCase {
|
||||
XCTAssertFalse(loaded.cursorDragNavigationEnabled)
|
||||
XCTAssertEqual(loaded.polishIntensity, .light)
|
||||
XCTAssertFalse(loaded.flowSkipAppSwitch)
|
||||
XCTAssertEqual(loaded.flowInactivityDuration, .thirtyMinutes)
|
||||
XCTAssertEqual(loaded.flowInactivityDuration, .threeHours)
|
||||
}
|
||||
|
||||
/// Existing installs (onboarding completed, no explicit engineMode key)
|
||||
/// ran on the old "cloud"/12h defaults — a silent flip to the new
|
||||
/// privacy defaults would change their engine under them AND propagate
|
||||
/// through settings sync as a fake fresh edit to their other devices.
|
||||
func testDefaultMigrationPreservesExistingInstallBehavior() {
|
||||
let defaults = makeDefaults()
|
||||
defaults.set(true, forKey: AppGroupConfiguration.Keys.hasCompletedOnboarding)
|
||||
|
||||
let config = AppGroupConfiguration.load(fromAvailable: defaults)
|
||||
|
||||
XCTAssertEqual(config.engineMode, "cloud", "pre-picker installs stay on their old default")
|
||||
XCTAssertEqual(config.flowInactivityDuration, .twelveHours)
|
||||
// The resolution is persisted so it is stable and sync-invisible.
|
||||
XCTAssertEqual(defaults.string(forKey: AppGroupConfiguration.Keys.engineMode), "cloud")
|
||||
XCTAssertEqual(
|
||||
defaults.string(forKey: AppGroupConfiguration.Keys.flowInactivityDuration),
|
||||
FlowInactivityDuration.twelveHours.rawValue
|
||||
)
|
||||
}
|
||||
|
||||
func testDefaultMigrationGivesFreshInstallPrivacyDefaults() {
|
||||
let defaults = makeDefaults()
|
||||
|
||||
let config = AppGroupConfiguration.load(fromAvailable: defaults)
|
||||
|
||||
XCTAssertEqual(config.engineMode, "local")
|
||||
XCTAssertEqual(config.flowInactivityDuration, .thirtyMinutes)
|
||||
XCTAssertEqual(defaults.string(forKey: AppGroupConfiguration.Keys.engineMode), "local")
|
||||
}
|
||||
|
||||
func testTranslationEnabledDerivedFromTargetLocale() {
|
||||
|
||||
@@ -41,4 +41,30 @@ final class ConfigurationStoreTests: XCTestCase {
|
||||
let service = ASRServiceFactory.make(store: store as any ConfigurationStore)
|
||||
XCTAssertTrue(service is SpeechAnalyzerASR)
|
||||
}
|
||||
|
||||
func testASRAndPolishProvidersAreIndependent() throws {
|
||||
try Keychain.setAPIKey("sk-llm", for: "openai", useICloudSync: false)
|
||||
try Keychain.setASRAPIKey("sk-asr", for: "zhipu", useICloudSync: false)
|
||||
|
||||
var config = AppGroupConfiguration.load(fromAvailable: defaults)
|
||||
config.engineMode = "cloud"
|
||||
config.providerId = "openai"
|
||||
config.asrProviderId = "zhipu"
|
||||
config.save(to: defaults)
|
||||
|
||||
let loaded = AppGroupStore(defaults: defaults)
|
||||
XCTAssertEqual(loaded.providerId, "openai")
|
||||
XCTAssertEqual(loaded.asrProviderId, "zhipu")
|
||||
XCTAssertEqual(loaded.apiKey, "sk-llm")
|
||||
XCTAssertEqual(loaded.asrApiKey, "sk-asr")
|
||||
|
||||
let asrClient = CloudASRClientFactory.make(store: loaded)
|
||||
XCTAssertTrue(asrClient is ZhipuCloudASRClient)
|
||||
}
|
||||
|
||||
func testLegacyInstallCopiesProviderIdToAsrProviderId() {
|
||||
defaults.set("qwen", forKey: AppGroupConfiguration.Keys.providerId)
|
||||
let config = AppGroupConfiguration.load(fromAvailable: defaults)
|
||||
XCTAssertEqual(config.asrProviderId, "qwen")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
// FlowBudgetAndMergeTests.swift
|
||||
// OSGKeyboardTests
|
||||
//
|
||||
// Guards the cross-cutting invariants introduced by the reliability
|
||||
// overhaul: timeout budgets derived from a single source, LWW clock
|
||||
// clamping, and mutation-rebase for the speech history store.
|
||||
|
||||
import XCTest
|
||||
@testable import OSGKeyboardShared
|
||||
|
||||
final class FlowBudgetAndMergeTests: XCTestCase {
|
||||
|
||||
// MARK: - Timeout budget invariant
|
||||
|
||||
/// The keyboard's post-stop watchdog must outlast the host's worst case
|
||||
/// (ASR drain wait + LLM polish cap) with real margin — otherwise the
|
||||
/// keyboard reports a timeout for transcriptions that are still going
|
||||
/// to succeed, and hand-tuned constants have drifted below the host
|
||||
/// maximum before.
|
||||
func testKeyboardResultTimeoutOutlastsHostWorstCase() {
|
||||
for engineMode in ["local", "cloud"] {
|
||||
let hostWorstCase = (engineMode == "local"
|
||||
? FlowSessionKeys.localASRWaitTimeout
|
||||
: FlowSessionKeys.cloudASRWaitTimeout)
|
||||
+ FlowSessionKeys.maxPolishTimeout
|
||||
let keyboardTimeout = FlowSessionKeys.keyboardResultTimeout(engineMode: engineMode)
|
||||
XCTAssertGreaterThanOrEqual(
|
||||
keyboardTimeout,
|
||||
hostWorstCase + 10,
|
||||
"keyboard watchdog (\(engineMode)) must exceed host worst case with margin"
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - SyncedField future-clock clamping
|
||||
|
||||
func testMergePrefersGenuinelyNewerRemote() {
|
||||
let older = SyncedField(value: "a", updatedAt: Date(timeIntervalSinceNow: -100), deviceID: "A")
|
||||
let newer = SyncedField(value: "b", updatedAt: Date(timeIntervalSinceNow: -10), deviceID: "B")
|
||||
XCTAssertEqual(SyncedField.merge(local: older, remote: newer).value, "b")
|
||||
XCTAssertEqual(SyncedField.merge(local: newer, remote: older).value, "b")
|
||||
}
|
||||
|
||||
/// A device with a clock years in the future must not win every merge
|
||||
/// forever: its timestamp is clamped to "now" for comparison, so an
|
||||
/// edit carrying a trusted (within-skew) later stamp still beats it —
|
||||
/// with unclamped LWW the year-ahead stamp would win against everything
|
||||
/// until that wall-clock date actually arrived.
|
||||
func testMergeClampsAbsurdFutureRemoteTimestamp() {
|
||||
let farFuture = Date().addingTimeInterval(365 * 24 * 3600)
|
||||
let brokenClock = SyncedField(value: "broken", updatedAt: farFuture, deviceID: "B")
|
||||
// Sane edit one minute ahead of now: inside the trusted skew window,
|
||||
// so it is NOT clamped — while the broken stamp collapses to ~now.
|
||||
let local = SyncedField(value: "sane", updatedAt: Date().addingTimeInterval(60), deviceID: "A")
|
||||
XCTAssertEqual(
|
||||
SyncedField.merge(local: local, remote: brokenClock).value,
|
||||
"sane",
|
||||
"a year-ahead stamp must lose to a trusted, genuinely newer edit"
|
||||
)
|
||||
XCTAssertEqual(
|
||||
SyncedField.merge(local: brokenClock, remote: local).value,
|
||||
"sane",
|
||||
"clamping must be symmetric regardless of which side is remote"
|
||||
)
|
||||
}
|
||||
|
||||
/// The winner's untrusted future stamp must be REWRITTEN to now in the
|
||||
/// merged result — otherwise the stored far-future stamp keeps beating
|
||||
/// every later genuine edit until that wall-clock date arrives.
|
||||
func testMergeFlattensUntrustedWinnerStamp() {
|
||||
let farFuture = Date().addingTimeInterval(365 * 24 * 3600)
|
||||
let broken = SyncedField(value: "broken", updatedAt: farFuture, deviceID: "B")
|
||||
let old = SyncedField(value: "old", updatedAt: Date(timeIntervalSinceNow: -9999), deviceID: "A")
|
||||
let merged = SyncedField.merge(local: old, remote: broken)
|
||||
XCTAssertEqual(merged.value, "broken", "newer (clamped) edit still wins this merge")
|
||||
XCTAssertLessThan(
|
||||
merged.updatedAt.timeIntervalSinceNow, 60,
|
||||
"the far-future stamp must be flattened so later real edits can outrank it"
|
||||
)
|
||||
}
|
||||
|
||||
func testMergeTrustsModestFutureSkew() {
|
||||
// Small forward skew (minutes) is normal clock drift and stays trusted.
|
||||
let slightlyAhead = SyncedField(value: "ahead", updatedAt: Date().addingTimeInterval(120), deviceID: "A")
|
||||
let past = SyncedField(value: "past", updatedAt: Date(timeIntervalSinceNow: -3600), deviceID: "B")
|
||||
XCTAssertEqual(SyncedField.merge(local: past, remote: slightlyAhead).value, "ahead")
|
||||
}
|
||||
|
||||
// MARK: - History push byte budget
|
||||
|
||||
/// A history that outgrew the KVS byte budget must be trimmed (oldest
|
||||
/// entries first) for upload, not fail forever — automatic pushes are
|
||||
/// fire-and-forget, so a throwing encode would silently kill sync with
|
||||
/// no recovery path short of clearing all history.
|
||||
@MainActor
|
||||
func testOversizedHistoryPushTrimsOldestEntriesToFitBudget() throws {
|
||||
let sync = SpeechHistoryCloudSync(
|
||||
kvs: FakeUbiquitousKeyValueStore(),
|
||||
makeStore: { AppGroupStore(defaults: self.makeDefaults()) },
|
||||
historyDefaults: { self.makeDefaults() }
|
||||
)
|
||||
|
||||
// ~300 entries × ~2.4 KB ≈ 720 KB encoded — over the 400 KB budget.
|
||||
let filler = String(repeating: "很长的听写内容 long dictation text ", count: 80)
|
||||
let now = Date()
|
||||
var history = SyncedSpeechHistory.empty
|
||||
history.entries = (0..<300).map { index in
|
||||
SpeechHistoryEntry(
|
||||
text: "\(filler)#\(index)",
|
||||
createdAt: now.addingTimeInterval(TimeInterval(-index)),
|
||||
engineMode: "local"
|
||||
)
|
||||
}
|
||||
|
||||
let data = try sync.encodeFittingBudget(history)
|
||||
XCTAssertLessThanOrEqual(data.count, SpeechHistoryCloudSync.maxPayloadBytes)
|
||||
|
||||
let decoded = try sync.decode(data)
|
||||
XCTAssertFalse(decoded.entries.isEmpty)
|
||||
// Newest entries must survive the trim.
|
||||
XCTAssertTrue(decoded.entries.contains { $0.text.hasSuffix("#0") })
|
||||
XCTAssertFalse(decoded.entries.contains { $0.text.hasSuffix("#299") })
|
||||
}
|
||||
|
||||
// MARK: - Insertion word-boundary hygiene
|
||||
|
||||
func testInsertionSeparatorAddsSpaceBetweenLatinWords() {
|
||||
XCTAssertEqual(
|
||||
DictationTextComposer.insertionSeparator(previousContext: "Hello", insertion: "world"),
|
||||
" "
|
||||
)
|
||||
XCTAssertEqual(
|
||||
DictationTextComposer.insertionSeparator(previousContext: "version 2", insertion: "is out"),
|
||||
" "
|
||||
)
|
||||
}
|
||||
|
||||
func testInsertionSeparatorSkipsWhitespaceCJKAndPunctuationBoundaries() {
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "Hello ", insertion: "world"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "line\n", insertion: "next"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "你好", insertion: "世界"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "说英文", insertion: "now"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "see (", insertion: "note"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "wait", insertion: ", then go"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: nil, insertion: "fresh"), "")
|
||||
XCTAssertEqual(DictationTextComposer.insertionSeparator(previousContext: "", insertion: "fresh"), "")
|
||||
}
|
||||
|
||||
// MARK: - SpeechHistoryStore rebase-before-mutation
|
||||
|
||||
private func makeDefaults() -> UserDefaults {
|
||||
let suite = "group.com.osgkeyboard.shared.tests.history.\(UUID().uuidString)"
|
||||
let defaults = UserDefaults(suiteName: suite)!
|
||||
defaults.removePersistentDomain(forName: suite)
|
||||
return defaults
|
||||
}
|
||||
|
||||
/// Cloud pulls write merged history to disk and only *schedule* the
|
||||
/// in-memory reload. A mutation racing that reload must not wipe what
|
||||
/// the merge brought in.
|
||||
@MainActor
|
||||
func testAppendDoesNotEraseEntriesMergedToDiskBehindItsBack() {
|
||||
let defaults = makeDefaults()
|
||||
let store = SpeechHistoryStore(defaults: defaults)
|
||||
|
||||
store.append(text: "本地第一条", engineMode: "local")
|
||||
XCTAssertEqual(store.entries.count, 1)
|
||||
|
||||
// Simulate a cloud merge landing on disk without the store's
|
||||
// in-memory payload being reloaded yet.
|
||||
var onDisk = SpeechHistoryStorage.load(from: defaults)
|
||||
let remoteEntry = SpeechHistoryEntry(text: "远端合并进来的一条", engineMode: "cloud")
|
||||
onDisk.entries.append(remoteEntry)
|
||||
onDisk.updatedAt = Date()
|
||||
SpeechHistoryStorage.save(onDisk, to: defaults)
|
||||
|
||||
// Mutate through the store — pre-fix this overwrote the disk state
|
||||
// with the stale in-memory payload, deleting the remote entry.
|
||||
store.append(text: "本地第二条", engineMode: "local")
|
||||
|
||||
let persisted = SpeechHistoryStorage.load(from: defaults)
|
||||
XCTAssertTrue(
|
||||
persisted.entries.contains { $0.id == remoteEntry.id },
|
||||
"append must rebase on the persisted state instead of clobbering the cloud merge"
|
||||
)
|
||||
XCTAssertTrue(persisted.entries.contains { $0.text == "本地第二条" })
|
||||
XCTAssertTrue(persisted.entries.contains { $0.text == "本地第一条" })
|
||||
}
|
||||
|
||||
@MainActor
|
||||
func testDeleteAfterExternalDiskMergeStillTombstones() {
|
||||
let defaults = makeDefaults()
|
||||
let store = SpeechHistoryStore(defaults: defaults)
|
||||
store.append(text: "要删除的一条", engineMode: "local")
|
||||
guard let target = store.entries.first else {
|
||||
return XCTFail("expected an entry")
|
||||
}
|
||||
|
||||
// External merge adds an unrelated entry on disk.
|
||||
var onDisk = SpeechHistoryStorage.load(from: defaults)
|
||||
onDisk.entries.append(SpeechHistoryEntry(text: "外部条目", engineMode: "cloud"))
|
||||
onDisk.updatedAt = Date()
|
||||
SpeechHistoryStorage.save(onDisk, to: defaults)
|
||||
|
||||
store.delete(id: target.id)
|
||||
|
||||
let persisted = SpeechHistoryStorage.load(from: defaults)
|
||||
XCTAssertFalse(persisted.entries.contains { $0.id == target.id })
|
||||
XCTAssertNotNil(persisted.deletedEntryIDs[target.id], "delete must record a tombstone")
|
||||
XCTAssertTrue(persisted.entries.contains { $0.text == "外部条目" }, "external entry must survive")
|
||||
}
|
||||
}
|
||||
@@ -238,6 +238,142 @@ final class FlowSessionBridgeTests: XCTestCase {
|
||||
XCTAssertEqual(FlowSessionBridge.latestAck(defaults: defaults), ack)
|
||||
}
|
||||
|
||||
func testNotReadySnapshotDoesNotRefreshHeartbeat() {
|
||||
let defaults = makeDefaults()
|
||||
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
|
||||
let zombieHeartbeat = Date().timeIntervalSince1970 - 120
|
||||
defaults.set(zombieHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
|
||||
|
||||
// A host stuck in a failed cold start writes not-ready snapshots on
|
||||
// every engine flap; those must NOT revive the heartbeat, or zombie
|
||||
// detection is postponed forever.
|
||||
FlowSessionBridge.writeReadySnapshot(
|
||||
FlowReadySnapshot(
|
||||
sessionId: UUID(),
|
||||
ready: false,
|
||||
reason: .waitingForAudioProof,
|
||||
engineMode: "local",
|
||||
localeId: "zh-Hans"
|
||||
),
|
||||
defaults: defaults
|
||||
)
|
||||
|
||||
XCTAssertTrue(FlowSessionBridge.isHostStale(defaults: defaults))
|
||||
}
|
||||
|
||||
func testBusySnapshotStillRefreshesHeartbeat() {
|
||||
let defaults = makeDefaults()
|
||||
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
|
||||
let staleHeartbeat = Date().timeIntervalSince1970 - 10
|
||||
defaults.set(staleHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
|
||||
|
||||
// Recording/processing proves the host is alive even though the
|
||||
// snapshot is not "ready" — the heartbeat must keep flowing so the
|
||||
// keyboard does not declare a mid-utterance host dead.
|
||||
let sessionId = UUID()
|
||||
FlowSessionBridge.writeReadySnapshot(
|
||||
FlowReadySnapshot(
|
||||
sessionId: sessionId,
|
||||
ready: false,
|
||||
reason: .recording,
|
||||
engineMode: "local",
|
||||
localeId: "zh-Hans",
|
||||
busyUtteranceId: UUID()
|
||||
),
|
||||
defaults: defaults
|
||||
)
|
||||
|
||||
XCTAssertTrue(FlowSessionBridge.isHostReachable(defaults: defaults))
|
||||
// Not-ready busy snapshots must remain readable so the keyboard can
|
||||
// distinguish "host is recording" from "host is still starting".
|
||||
let snap = FlowSessionBridge.readySnapshot(defaults: defaults)
|
||||
XCTAssertEqual(snap?.reason, .recording)
|
||||
XCTAssertEqual(snap?.ready, false)
|
||||
XCTAssertEqual(snap?.sessionId, sessionId)
|
||||
}
|
||||
|
||||
func testNotReadyStartingSnapshotIsRetainedWithoutRevivingHeartbeat() {
|
||||
let defaults = makeDefaults()
|
||||
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
|
||||
let zombieHeartbeat = Date().timeIntervalSince1970 - 120
|
||||
defaults.set(zombieHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
|
||||
|
||||
FlowSessionBridge.writeReadySnapshot(
|
||||
FlowReadySnapshot(
|
||||
sessionId: UUID(),
|
||||
ready: false,
|
||||
reason: .waitingForAudioProof,
|
||||
engineMode: "local",
|
||||
localeId: "zh-Hans"
|
||||
),
|
||||
defaults: defaults
|
||||
)
|
||||
|
||||
XCTAssertTrue(FlowSessionBridge.isHostStale(defaults: defaults))
|
||||
XCTAssertEqual(
|
||||
FlowSessionBridge.readySnapshot(defaults: defaults)?.reason,
|
||||
.waitingForAudioProof
|
||||
)
|
||||
}
|
||||
|
||||
func testStaleGenerationSnapshotIsNotReady() {
|
||||
let defaults = makeDefaults()
|
||||
let sessionId = UUID()
|
||||
let now = Date().timeIntervalSince1970
|
||||
FlowSessionBridge.rotateHostGeneration(defaults: defaults)
|
||||
let liveGeneration = FlowSessionBridge.currentHostGeneration(defaults: defaults)
|
||||
FlowSessionBridge.markSessionActive(duration: 60, sessionId: sessionId, defaults: defaults)
|
||||
FlowSessionBridge.writeReadySnapshot(
|
||||
FlowReadySnapshot(
|
||||
sessionId: sessionId,
|
||||
ready: true,
|
||||
reason: .ready,
|
||||
heartbeatAt: now,
|
||||
readyAt: now,
|
||||
engineMode: "local",
|
||||
localeId: "zh-Hans",
|
||||
hostGeneration: liveGeneration
|
||||
),
|
||||
defaults: defaults
|
||||
)
|
||||
XCTAssertTrue(FlowSessionBridge.isHostReady(defaults: defaults))
|
||||
|
||||
// Host relaunches (force-quit path) → new generation. The old ready
|
||||
// snapshot must be void instantly, without waiting out the 60 s
|
||||
// heartbeat-zombie window.
|
||||
FlowSessionBridge.rotateHostGeneration(defaults: defaults)
|
||||
XCTAssertFalse(FlowSessionBridge.isHostReady(defaults: defaults))
|
||||
}
|
||||
|
||||
func testClearFlowStateOnHostLaunchPreservesPendingHost() {
|
||||
let defaults = makeDefaults()
|
||||
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
|
||||
FlowSessionBridge.setHostReady(true, defaults: defaults)
|
||||
FlowSessionBridge.setPendingHostBundleId("com.example.host", defaults: defaults)
|
||||
|
||||
FlowSessionBridge.clearFlowStateOnHostLaunch(defaults: defaults)
|
||||
|
||||
XCTAssertFalse(FlowSessionBridge.isSessionActive(defaults: defaults))
|
||||
XCTAssertFalse(FlowSessionBridge.isHostReady(defaults: defaults))
|
||||
// The startflow scene-delegate write happens before the session
|
||||
// manager exists — launch reconciliation must not eat it.
|
||||
XCTAssertEqual(
|
||||
FlowSessionBridge.pendingHostBundleId(defaults: defaults),
|
||||
"com.example.host"
|
||||
)
|
||||
}
|
||||
|
||||
func testRotateHostGenerationReturnsPreviousToken() {
|
||||
let defaults = makeDefaults()
|
||||
XCTAssertNil(FlowSessionBridge.rotateHostGeneration(defaults: defaults))
|
||||
let first = FlowSessionBridge.currentHostGeneration(defaults: defaults)
|
||||
XCTAssertNotNil(first)
|
||||
|
||||
let previous = FlowSessionBridge.rotateHostGeneration(defaults: defaults)
|
||||
XCTAssertEqual(previous, first)
|
||||
XCTAssertNotEqual(FlowSessionBridge.currentHostGeneration(defaults: defaults), first)
|
||||
}
|
||||
|
||||
func testReadySnapshotDrivesHostReady() {
|
||||
let defaults = makeDefaults()
|
||||
let sessionId = UUID()
|
||||
|
||||
@@ -17,10 +17,10 @@ final class FlowSessionPolicyTests: XCTestCase {
|
||||
XCTAssertTrue(FlowSessionPolicy.skipAppSwitch(defaults: defaults))
|
||||
}
|
||||
|
||||
func testInactivityDurationDefaultsToTwelveHours() {
|
||||
func testInactivityDurationDefaultsToThirtyMinutes() {
|
||||
let defaults = makeDefaults()
|
||||
XCTAssertEqual(FlowSessionPolicy.inactivityDuration(defaults: defaults), .twelveHours)
|
||||
XCTAssertEqual(FlowSessionPolicy.sessionDuration(defaults: defaults), 12 * 60 * 60)
|
||||
XCTAssertEqual(FlowSessionPolicy.inactivityDuration(defaults: defaults), .thirtyMinutes)
|
||||
XCTAssertEqual(FlowSessionPolicy.sessionDuration(defaults: defaults), 30 * 60)
|
||||
}
|
||||
|
||||
func testTouchLastActivityExtendsExpiry() {
|
||||
|
||||
@@ -340,7 +340,7 @@ final class IntelligentPolishTests: XCTestCase {
|
||||
|
||||
@MainActor
|
||||
func testFlowFallbackDeliveryCleansTextAndCarriesWeakNetworkWarning() {
|
||||
let delivery = FlowSessionManager.makeFallbackDelivery(
|
||||
let delivery = TranscriptionPolishFallback.makeDelivery(
|
||||
rawText: " 你 是不是 已经 解决了 这个 问题 ? ",
|
||||
error: LLMError.transport("offline"),
|
||||
engineMode: "cloud",
|
||||
@@ -352,6 +352,20 @@ final class IntelligentPolishTests: XCTestCase {
|
||||
XCTAssertEqual(delivery.polishWarning, SharedL10n.string("flow.warning.polishDegraded"))
|
||||
}
|
||||
|
||||
func testTranscriptionPolishFallbackLocalMissingKeyWarning() {
|
||||
let delivery = TranscriptionPolishFallback.makeDelivery(
|
||||
rawText: "测试文本",
|
||||
error: PolishingService.PolishError.missingAPIKey,
|
||||
engineMode: "local",
|
||||
chunkWarning: nil
|
||||
)
|
||||
XCTAssertEqual(delivery.text, "测试文本")
|
||||
XCTAssertEqual(
|
||||
delivery.polishWarning,
|
||||
SharedL10n.string("flow.warning.localPolishUnavailable")
|
||||
)
|
||||
}
|
||||
|
||||
func testHasStructureSignalDetectsChineseEnumeration() {
|
||||
XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "首先测试其次上线"))
|
||||
XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "第一点修复"))
|
||||
|
||||
@@ -13,7 +13,20 @@ final class LocalASRModelCatalogTests: XCTestCase {
|
||||
XCTAssertFalse(catalog.models.contains { $0.id == "qwen3-mlx-1.7b" })
|
||||
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-qwen3-0.6b-int8" })
|
||||
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-qwen3-1.7b-int8" })
|
||||
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-paraformer-zh-int8" })
|
||||
XCTAssertTrue(catalog.models.contains { $0.id == "sherpa-sensevoice-small-int8" })
|
||||
XCTAssertFalse(catalog.models.contains { $0.id == "sherpa-paraformer-zh-int8" })
|
||||
XCTAssertEqual(
|
||||
LocalASRModelCatalog.model("sherpa-sensevoice-small-int8", in: catalog)?.badgeKey,
|
||||
"mac.localASR.badge.fastest"
|
||||
)
|
||||
XCTAssertEqual(
|
||||
LocalASRModelCatalog.model("sherpa-qwen3-0.6b-int8", in: catalog)?.badgeKey,
|
||||
"mac.localASR.badge.balanced"
|
||||
)
|
||||
XCTAssertEqual(
|
||||
LocalASRModelCatalog.model("sherpa-qwen3-1.7b-int8", in: catalog)?.badgeKey,
|
||||
"mac.localASR.badge.quality"
|
||||
)
|
||||
}
|
||||
|
||||
func testSherpaQwen317BUsesRepositoryInstall() throws {
|
||||
@@ -32,9 +45,9 @@ final class LocalASRModelCatalogTests: XCTestCase {
|
||||
XCTAssertTrue(model.supportsHotwords)
|
||||
}
|
||||
|
||||
func testCapabilitiesForParaformer() throws {
|
||||
func testCapabilitiesForSenseVoice() throws {
|
||||
let catalog = try LocalASRModelCatalog.loadBundled()
|
||||
let model = try XCTUnwrap(LocalASRModelCatalog.model("sherpa-paraformer-zh-int8", in: catalog))
|
||||
let model = try XCTUnwrap(LocalASRModelCatalog.model("sherpa-sensevoice-small-int8", in: catalog))
|
||||
let caps = LocalASRModelCatalog.capabilities(for: model)
|
||||
XCTAssertEqual(caps.hotwordMode, .none)
|
||||
XCTAssertFalse(model.supportsHotwords)
|
||||
|
||||
@@ -44,6 +44,9 @@ final class SettingsCloudSyncTests: XCTestCase {
|
||||
providerId: SyncedField(value: "openai", updatedAt: stampA, deviceID: deviceA),
|
||||
baseURL: SyncedField(value: "https://local.example", updatedAt: stampA, deviceID: deviceA),
|
||||
model: SyncedField(value: "gpt-local", updatedAt: stampA, deviceID: deviceA),
|
||||
asrProviderId: SyncedField(value: "zhipu", updatedAt: stampA, deviceID: deviceA),
|
||||
asrBaseURL: SyncedField(value: "https://asr-local.example", updatedAt: stampA, deviceID: deviceA),
|
||||
asrModel: SyncedField(value: "glm-asr-local", updatedAt: stampA, deviceID: deviceA),
|
||||
modeId: SyncedField(value: "polish", updatedAt: stampA, deviceID: deviceA),
|
||||
localeId: SyncedField(value: "auto", updatedAt: stampA, deviceID: deviceA),
|
||||
engineMode: SyncedField(value: "cloud", updatedAt: stampA, deviceID: deviceA),
|
||||
@@ -64,6 +67,9 @@ final class SettingsCloudSyncTests: XCTestCase {
|
||||
providerId: SyncedField(value: "openai", updatedAt: stampA, deviceID: deviceB),
|
||||
baseURL: SyncedField(value: "https://remote.example", updatedAt: stampB, deviceID: deviceB),
|
||||
model: SyncedField(value: "gpt-remote", updatedAt: stampB, deviceID: deviceB),
|
||||
asrProviderId: SyncedField(value: "qwen", updatedAt: stampB, deviceID: deviceB),
|
||||
asrBaseURL: SyncedField(value: "https://asr-remote.example", updatedAt: stampB, deviceID: deviceB),
|
||||
asrModel: SyncedField(value: "fun-asr-remote", updatedAt: stampB, deviceID: deviceB),
|
||||
modeId: SyncedField(value: "polish", updatedAt: stampA, deviceID: deviceB),
|
||||
localeId: SyncedField(value: "ja", updatedAt: stampB, deviceID: deviceB),
|
||||
engineMode: SyncedField(value: "local", updatedAt: stampB, deviceID: deviceB),
|
||||
@@ -80,6 +86,7 @@ final class SettingsCloudSyncTests: XCTestCase {
|
||||
let merged = SyncedAppSettingsV2.merge(local: local, remote: remote)
|
||||
|
||||
XCTAssertEqual(merged.baseURL.value, "https://remote.example")
|
||||
XCTAssertEqual(merged.asrProviderId.value, "qwen")
|
||||
XCTAssertEqual(merged.localeId.value, "ja")
|
||||
XCTAssertEqual(merged.engineMode.value, "local")
|
||||
}
|
||||
|
||||
@@ -21,7 +21,7 @@ OSGKeyboard is a free, source-available alternative to commercial voice-input to
|
||||
2. Speak naturally (up to 3.5 minutes / 210 seconds per take)
|
||||
3. Tap again to stop — the AI polishes your words into clean text and inserts at the cursor
|
||||
|
||||
Audio is transcribed **on-device** by Apple's `SpeechAnalyzer` + `DictationTranscriber` (iOS 26+). Only the **polished transcript** is sent to your chosen cloud LLM. **No audio ever leaves your phone.**
|
||||
By default, audio is transcribed **on-device** by Apple's `SpeechAnalyzer` + `DictationTranscriber` (iOS 26+) — **no audio leaves your phone** unless you say so. If polish is enabled, only the transcript text goes to your chosen LLM. Optionally, you can switch to a **cloud ASR engine** (explicit opt-in with a confirmation): in that mode your recordings are uploaded to the ASR provider you configure.
|
||||
|
||||
Under the hood, OSGKeyboard uses a **Flow session model**: a long-lived audio session runs in the host app, the keyboard extension writes tiny "start / stop" signals to the App Group, and the polished text is delivered back to the keyboard for insertion. You do not need to jump back to the host app between recordings.
|
||||
|
||||
@@ -34,7 +34,7 @@ Under the hood, OSGKeyboard uses a **Flow session model**: a long-lived audio se
|
||||
- ✍️ **AI polishing** — adds structure, punctuation, fixes grammar, optionally produces lists
|
||||
- 🧩 **Local + cloud polish toggle** — local engine is ASR-only by default; opt into a post-ASR cloud polish step (DeepSeek by default) when the iOS speech recognition isn't strong enough for your environment (noisy far-field audio, strong accents, etc.)
|
||||
- 🔌 **Bring-your-own API** — works with any OpenAI-compatible endpoint (OpenAI, DeepSeek, Qwen DashScope, Moonshot, Zhipu, your own self-hosted server, …)
|
||||
- 🔒 **Privacy first** — audio never leaves your device; only the final transcript is sent to the LLM you choose
|
||||
- 🔒 **Privacy first** — on-device ASR by default, so audio never leaves your device unless you explicitly opt into the cloud engine; polish sends only the transcript to the LLM you choose
|
||||
- 🎨 **Native SwiftUI** — dark theme, frosted glass, pure Swift 6, ~3,600 lines of code
|
||||
- 🪶 **Zero dependencies** — no SwiftPM packages, no CocoaPods, no Carthage
|
||||
- 🔁 **Flow session** — keep recording across multiple takes without bouncing back to the host app
|
||||
@@ -46,7 +46,7 @@ Under the hood, OSGKeyboard uses a **Flow session model**: a long-lived audio se
|
||||
### Requirements
|
||||
|
||||
- macOS with **Xcode 26** (matches `project.yml` deployment target iOS 26)
|
||||
- iPhone running **iOS 26.0+**
|
||||
- iPhone or iPad running **iOS 26.0+** (iPad fully supported: Split View / Stage Manager, adaptive layout)
|
||||
- [XcodeGen](https://github.com/yonaskolb/XcodeGen): `brew install xcodegen`
|
||||
- An OpenAI-compatible API key (e.g. from [OpenAI](https://platform.openai.com/api-keys), [DeepSeek](https://platform.deepseek.com/api_keys), or [Qwen DashScope](https://dashscope.console.aliyun.com/apiKey)). Not needed if you stay on the "local ASR only" engine.
|
||||
|
||||
@@ -65,6 +65,17 @@ xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
|
||||
> `project.yml` by `Scripts/generate-xcodeproj.sh`. Always re-run the script
|
||||
> after `git pull` if `project.yml` has changed.
|
||||
|
||||
### macOS distribution (decision)
|
||||
|
||||
The macOS menu-bar app ships via **Developer ID direct distribution** (notarized,
|
||||
non-sandboxed), NOT the Mac App Store. This is deliberate: its core features —
|
||||
global hold-to-talk hotkey, Accessibility-based text insertion into other apps,
|
||||
and synthesized ⌘V — are incompatible with the Mac App Store sandbox, and a
|
||||
sandboxed Accessibility grant also tends to reset after every app update.
|
||||
`OSGKeyboardMac.entitlements` therefore keeps `com.apple.security.app-sandbox`
|
||||
set to `false`; do not flip it back on without redesigning the insertion path.
|
||||
(The iOS app targets the iOS App Store as usual — see `AUDIT_APPSTORE.md`.)
|
||||
|
||||
### Enable the keyboard in iOS
|
||||
|
||||
The host app walks you through a **5-step onboarding**:
|
||||
@@ -130,9 +141,9 @@ OSGKeyboard/
|
||||
|
||||
**Engine modes:**
|
||||
|
||||
- `cloud` (default) — on-device ASR via `SpeechAnalyzer`, transcript is sent to your configured LLM for polish.
|
||||
- `local` — on-device ASR via `SpeechAnalyzer` only; transcript is inserted as-is. No network round-trip.
|
||||
- `local` + "Cloud polish after ASR" toggle (Settings → Engine) — same on-device ASR, but the transcript is routed through your configured LLM before insertion. Useful when iOS speech recognition isn't accurate enough in your environment.
|
||||
- `local` (default) — on-device ASR via `SpeechAnalyzer`; transcript is inserted as-is. No network round-trip.
|
||||
- `local` + "Cloud polish after ASR" toggle (Settings → Engine) — same on-device ASR, but the transcript (text only) is routed through your configured LLM before insertion. Useful when iOS speech recognition isn't accurate enough in your environment.
|
||||
- `cloud` (opt-in, requires an explicit confirmation) — **your voice recordings are uploaded** to the ASR provider you configure (e.g. OpenAI `/audio/transcriptions`, DashScope, Zhipu), and the resulting transcript is sent to your LLM for polish. Choose this only when you accept your provider's privacy terms.
|
||||
|
||||
**Cross-process plumbing (host app ↔ keyboard extension):**
|
||||
|
||||
|
||||
+5
-5
@@ -20,7 +20,7 @@ OSGKeyboard 是商业语音输入工具的免费、源码可见替代方案。
|
||||
2. 自由说话(单次上限 3.5 分钟 / 210 秒)
|
||||
3. 再按一下结束 —— AI 自动整理成干净的文字并插入光标
|
||||
|
||||
**音频始终在设备本地转写**(iOS 26+ 的 `SpeechAnalyzer` + `DictationTranscriber`),**只有润色后的文本** 会发到你选择的云端 LLM。**音频永不离开你的手机。**
|
||||
默认情况下,**音频在设备本地转写**(iOS 26+ 的 `SpeechAnalyzer` + `DictationTranscriber`)——**除非你主动选择,音频不会离开你的手机**;开启润色时也只有文本会发到你选择的 LLM。你也可以显式切换到**云端识别引擎**(需二次确认的 opt-in):该模式下你的录音会上传到你配置的识别服务商。
|
||||
|
||||
项目内部采用 **Flow 会话模型**:主 App 维护一个长生命周期的音频会话,键盘扩展只通过 App Group 写入"开始 / 停止"等轻量信号,润色后的文本再由主 App 回传给键盘插入。**多次录音之间无需反复跳回主 App**。
|
||||
|
||||
@@ -33,7 +33,7 @@ OSGKeyboard 是商业语音输入工具的免费、源码可见替代方案。
|
||||
- ✍️ **AI 润色** —— 自动加结构、补标点、修正语法、可生成列表
|
||||
- 🧩 **本地 + 云端润色开关** —— 本地模式默认仅在设备上识别;若 iOS 语音识别效果不理想(远场、噪声、方言),可开启「识别后云端润色」,默认走 DeepSeek
|
||||
- 🔌 **自带 API 接入** —— 兼容任何 OpenAI 兼容协议端点(OpenAI / DeepSeek / Qwen DashScope / Moonshot / 智谱 / 自建服务器 ……)
|
||||
- 🔒 **隐私优先** —— 音频不离开设备;只有润色文本会发给你选择的 LLM
|
||||
- 🔒 **隐私优先** —— 默认端侧识别,音频不离开设备(除非显式开启云端识别引擎);润色只发送文本给你选择的 LLM
|
||||
- 🎨 **原生 SwiftUI** —— 暗色主题、毛玻璃、纯 Swift 6 实现,约 3,600 行代码
|
||||
- 🪶 **零依赖** —— 无 SwiftPM 包、无 CocoaPods、无 Carthage
|
||||
- 🔁 **Flow 会话** —— 多次录音无需跳回主 App,会话自动维持心跳与续期
|
||||
@@ -127,9 +127,9 @@ OSGKeyboard/
|
||||
|
||||
**引擎模式:**
|
||||
|
||||
- `cloud`(默认)—— 端侧 `SpeechAnalyzer` 识别,文本发到云端 LLM 润色。
|
||||
- `local` —— 仅端侧 `SpeechAnalyzer` 识别,原始文本直接插入,不联网。
|
||||
- `local` + 「识别后云端润色」开关(设置 → 引擎)—— 同样走端侧 ASR,但识别完成后送 LLM 润色再插入。适用于 iOS 识别效果不理想的场景。
|
||||
- `local`(默认)—— 仅端侧 `SpeechAnalyzer` 识别,原始文本直接插入,不联网。
|
||||
- `local` + 「识别后云端润色」开关(设置 → 引擎)—— 同样走端侧 ASR,但识别完成后送 LLM 润色(仅文本)再插入。适用于 iOS 识别效果不理想的场景。
|
||||
- `cloud`(opt-in,需显式确认)—— **你的语音录音会上传**到你配置的识别服务商(如 OpenAI `/audio/transcriptions`、DashScope、智谱),识别文本再送 LLM 润色。请在接受服务商隐私条款的前提下选用。
|
||||
|
||||
**跨进程管道(主 App ↔ 键盘扩展):**
|
||||
|
||||
|
||||
@@ -50,10 +50,12 @@
|
||||
OSGKeyboard is a free, open-source custom keyboard for iOS 26 that turns
|
||||
your voice into clean, AI-polished text — in any app.
|
||||
|
||||
Hold the mic key, speak naturally, release. The keyboard transcribes
|
||||
your voice entirely on-device (Apple's iOS 26 SpeechAnalyzer +
|
||||
DictationTranscriber), and only the final text is sent to the AI you
|
||||
choose to polish it. Your audio never leaves your iPhone.
|
||||
Hold the mic key, speak naturally, release. By default the keyboard
|
||||
transcribes your voice entirely on-device (Apple's iOS 26
|
||||
SpeechAnalyzer + DictationTranscriber), and only the final text is
|
||||
sent to the AI you choose to polish it — your audio never leaves your
|
||||
device unless you explicitly opt into the cloud ASR engine, which
|
||||
uploads recordings to the provider you configure.
|
||||
|
||||
WHY OSGKEYBOARD
|
||||
|
||||
@@ -62,8 +64,9 @@ WHY OSGKEYBOARD
|
||||
types for you.
|
||||
• Push-to-talk, the way voice should work. No more "Hey Siri" mode that
|
||||
listens to the whole room.
|
||||
• On-device speech recognition. Powered by Apple's iOS 26 speech
|
||||
pipeline — no cloud ASR, no audio upload.
|
||||
• On-device speech recognition by default. Powered by Apple's iOS 26
|
||||
speech pipeline — no audio upload unless you explicitly enable the
|
||||
optional cloud ASR engine (confirmation required).
|
||||
• Bring-your-own AI. Connect any OpenAI-compatible endpoint (OpenAI,
|
||||
DeepSeek, Qwen DashScope, Moonshot, Zhipu, your own self-hosted
|
||||
server). Your API key stays in the iOS Keychain.
|
||||
@@ -76,12 +79,13 @@ WHY OSGKEYBOARD
|
||||
• Zero dependencies. No trackers, no analytics, no crash reporters.
|
||||
The whole project is ~8,700 lines of Swift you can audit in an
|
||||
afternoon.
|
||||
• Privacy first. PrivacyInfo.xcprivacy declares zero collected data;
|
||||
we don't run a server.
|
||||
• Privacy first. PrivacyInfo.xcprivacy declares exactly what the app
|
||||
touches (voice audio + transcripts, on-device by default, never
|
||||
linked or tracked); we don't run a server.
|
||||
|
||||
BUILT FOR
|
||||
|
||||
• iOS 26 and later, iPhone only.
|
||||
• iOS 26 and later, iPhone and iPad.
|
||||
• Anyone who types more than 100 words a day on their phone.
|
||||
• Developers, writers, students, and translators who want voice input
|
||||
that respects their privacy.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user