feat: iCloud settings sync and cold-start return redesign

- Add iCloud key-value settings sync (engine/language/polish/Flow prefs);
  API keys stay on-device. New "Sync settings via iCloud" toggle.
- Redesign cold-start handoff: bottom-bar left-to-right swipe guidance,
  auto-dismiss on app switch, tap-anywhere to close, retained return link.
- Harden keyboard->app handoff with host-disconnected hint.
- Include prior Unreleased ASR fixes (route-change crash, fallback warning,
  multi-utterance recognition, local ASR diagnostics).

Release 0.5.0 (build 18).
This commit is contained in:
Rocky
2026-07-07 17:56:01 +08:00
parent bf844caa7f
commit 128aab1b02
42 changed files with 1818 additions and 337 deletions
+15
View File
@@ -7,6 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [0.5.0] - 2026-07-07
### Added
- **iCloud settings sync**: engine, language, polish, and Flow preferences now stay in sync across your devices via iCloud (key-value store); a new "Sync settings via iCloud" toggle in Settings controls it. API keys stay on each device and are never uploaded. / **iCloud 设置同步**:引擎、语言、润色与 Flow 偏好现可通过 iCloud(键值存储)在多设备间自动同步;设置中新增「通过 iCloud 同步设置」开关控制此功能。API 密钥仅保留在各自设备本地,绝不上传。
### Changed
- **Cold-start return guidance**: the handoff screen now points to the bottom Home indicator with a left-to-right swipe animation (instead of the misleading "swipe up"), auto-dismisses once you switch back to your previous app, and closes on a tap anywhere; a "Return to [App]" link is still offered when available. / **冷启动返回引导**:交接页改为指向底部横条并配左向右滑动动画(不再是易误解的「向上滑」),切换回上一个 App 后自动消失,点按任意处即可关闭;可用时仍保留「返回 [App]」文本按钮。
- **Voice session handoff robustness**: hardened the keyboard→app cold-start flow, including a clearer "Voice session disconnected" hint when the host session drops. / **语音会话交接健壮性**:强化键盘→主 App 的冷启动链路,宿主会话断开时给出更清晰的「语音会话已断开」提示。
### Fixed
- **Local ASR route-change crash**: dictating with the on-device engine no longer terminates the app with `Failed to create tap due to format mismatch`. On-device `SpeechAnalyzer` warmup reconfigures the shared audio session, which triggers a route change; the tap was reinstalled with a stale hardware format (48 kHz) that no longer matched the live node (24 kHz). The tap now binds to the node's live format (`installTap(format: nil)`) and the downsampling converter rebuilds itself from the actual buffer format, so route churn is handled without crashing. / **本地识别路由切换崩溃**:使用端侧引擎听写不再以 `Failed to create tap due to format mismatch` 崩溃退出。端侧 `SpeechAnalyzer` 预热会重配共享音频会话并触发路由变化,此前重装 tap 时用了过期的硬件采样率(48 kHz),与真实节点(24 kHz)不匹配。现在 tap 绑定节点实时格式(`installTap(format: nil)`),降采样转换器按实际缓冲区格式自适应重建,路由抖动不再导致崩溃。
- **ASR fallback warning showed raw key**: the weak-network / missing-key fallback hint displayed its localization key (e.g. `flow.warning.polishDegraded`) instead of the translated sentence. These keys live in the `Shared.strings` table but were looked up via the main-app `Localizable` table; they are now resolved through `SharedL10n`. / **ASR 兜底提示显示变量名**:弱网 / 缺少密钥的兜底提示此前显示本地化键名(如 `flow.warning.polishDegraded`)而非译文。这些键位于 `Shared.strings`,却被按主 App 的 `Localizable` 表查找;现改为经 `SharedL10n` 解析。
- **Flow multi-utterance recognition**: the second (and later) dictation in a session no longer returns "no speech". The session-long downsampling `AVAudioConverter` was being permanently locked by an `.endOfStream` tail-flush, starving every utterance after the first (both on-device and cloud). Trailing speech is still preserved by the live drain loop. Also added recovery from `mediaServicesWereReset`. / **Flow 连续多句识别**:同一会话内第二句及之后不再提示「未识别到语音」。此前会话级降采样 `AVAudioConverter` 被尾音冲刷的 `.endOfStream` 永久锁死,导致首句之后每句都拿不到音频(端侧与云端均受影响)。尾音仍由实时排空环节保留。并新增媒体服务重置(`mediaServicesWereReset`)后的自愈重建。
- **Local ASR diagnostics**: add chunk-level `SpeechAnalyzer` logs and a settings switch to bypass the custom language model, so local recognition failures can be isolated between Apple assets, CLM attachment, and empty analyzer results. / **本地识别诊断**:新增 `SpeechAnalyzer` 分块级日志,并在设置中加入跳过自定义语言模型的诊断开关,用于区分 Apple 端侧资源、自定义语言模型挂载、以及分析器空结果三类问题。
## [0.4.1] - 2026-07-06 ## [0.4.1] - 2026-07-06
### Added ### Added
+85 -14
View File
@@ -1,29 +1,100 @@
// AppURLHandler.swift // AppURLHandler.swift
// OSGKeyboard · Main App // OSGKeyboard · Main App
// //
// Captures `sourceApplication` from UIKit open-URL options (scheme D). // iOS 26 URL handling via the UIScene lifecycle. `application(_:open:options:)`
// and `UIApplication.OpenURLOptionsKey.sourceApplication` are deprecated in
// iOS 26; the only supported way to read `sourceApplication` (scheme D
// host-return whitelist) is `UIOpenURLContext.options.sourceApplication` from a
// scene delegate SwiftUI's `.onOpenURL` does not expose it.
import UIKit import UIKit
import OSGKeyboardShared import OSGKeyboardShared
extension Notification.Name { /// Buffers launch/open URLs until the SwiftUI root registers a handler.
static let osgKeyboardOpenURL = Notification.Name("osgkeyboard.openURL") ///
/// On a cold launch the scene delivers the URL in `scene(_:willConnectTo:)`,
/// which fires *before* the SwiftUI view hierarchy is on screen. Without
/// buffering, that first `osgkeyboard://startflow` (the keyboard app
/// handoff) would be dropped.
@MainActor
final class AppOpenURLRouter {
static let shared = AppOpenURLRouter()
private var handler: ((URL) -> Void)?
private var pending: [URL] = []
private init() {}
/// Register the live handler and flush anything buffered before launch.
func register(_ handler: @escaping (URL) -> Void) {
self.handler = handler
let buffered = pending
pending.removeAll()
buffered.forEach(handler)
}
func route(_ url: URL) {
if let handler {
handler(url)
} else {
pending.append(url)
}
}
} }
final class AppURLHandler: NSObject, UIApplicationDelegate { final class AppURLHandler: NSObject, UIApplicationDelegate {
/// SwiftUI `@main` apps get no scene delegate by default. Attach ours so
/// scene-based URL delivery the only iOS 26 path to `sourceApplication`
/// reaches `AppSceneDelegate`. We deliberately do NOT create a window here;
/// SwiftUI's `WindowGroup` still owns the UI.
func application( func application(
_ application: UIApplication, _ application: UIApplication,
open url: URL, configurationForConnecting connectingSceneSession: UISceneSession,
options: [UIApplication.OpenURLOptionsKey: Any] = [:] options: UIScene.ConnectionOptions
) -> Bool { ) -> UISceneConfiguration {
if let source = options[.sourceApplication] as? String { let configuration = UISceneConfiguration(
FlowSessionBridge.setPendingHostBundleId(source) name: nil,
} sessionRole: connectingSceneSession.role
NotificationCenter.default.post(
name: .osgKeyboardOpenURL,
object: nil,
userInfo: ["url": url]
) )
return true configuration.delegateClass = AppSceneDelegate.self
return configuration
}
}
final class AppSceneDelegate: NSObject, UIWindowSceneDelegate {
/// Cold launch: the URL arrives in the connection options.
func scene(
_ scene: UIScene,
willConnectTo session: UISceneSession,
options connectionOptions: UIScene.ConnectionOptions
) {
handle(connectionOptions.urlContexts)
}
/// Warm open while the app is already running or suspended in memory.
func scene(_ scene: UIScene, openURLContexts URLContexts: Set<UIOpenURLContext>) {
handle(URLContexts)
}
private func handle(_ contexts: Set<UIOpenURLContext>) {
// Extract Sendable primitives up front so we never hop a
// non-Sendable `UIOpenURLContext` across the actor boundary.
let items: [(url: URL, source: String?)] = contexts.map {
($0.url, $0.options.sourceApplication)
}
guard !items.isEmpty else { return }
// Scene delegate callbacks are delivered on the main thread.
MainActor.assumeIsolated {
for item in items {
// `sourceApplication` is only non-nil when the caller belongs to
// the same Apple Developer Team (our own keyboard extension)
// exactly what the host-return whitelist relies on.
if let source = item.source {
FlowSessionBridge.setPendingHostBundleId(source)
}
AppOpenURLRouter.shared.route(item.url)
}
}
} }
} }
+95 -15
View File
@@ -49,6 +49,8 @@ final class FlowSessionManager: ObservableObject {
private var expiryTask: Task<Void, Never>? private var expiryTask: Task<Void, Never>?
private var levelTask: Task<Void, Never>? private var levelTask: Task<Void, Never>?
private var startTask: Task<Void, Never>? private var startTask: Task<Void, Never>?
/// Last recording state the poll loop observed logs only on transition.
private var lastObservedRecordingState: FlowSessionKeys.RecordingState = .idle
private var isUtteranceRecording = false private var isUtteranceRecording = false
/// True from `stopped` until the result/error is written back to App Group. /// True from `stopped` until the result/error is written back to App Group.
private var isUtteranceProcessing = false private var isUtteranceProcessing = false
@@ -86,6 +88,8 @@ final class FlowSessionManager: ObservableObject {
isColdStartHandoff = true isColdStartHandoff = true
} }
reconcilePersistedFlowStateBeforeStart()
if isActive { if isActive {
extendSession(duration: duration) extendSession(duration: duration)
if coldStart { if coldStart {
@@ -105,6 +109,33 @@ final class FlowSessionManager: ObservableObject {
} }
} }
/// Clears App Group Flow state left behind when the host process was killed
/// or the device rebooted while the session flag was still set.
private func reconcilePersistedFlowStateBeforeStart() {
if FlowSessionBridge.isHostStale() {
if isActive {
endSession()
} else {
FlowSessionBridge.clearFlowState()
FlowLiveActivityController.endSession()
}
debug("reconciled zombie persisted Flow state")
return
}
guard !isActive else { return }
let orphaned = FlowSessionBridge.recordingState()
switch orphaned {
case .recording, .stopped, .processing:
FlowSessionBridge.setRecordingState(.idle)
FlowSessionBridge.clearPendingTranscription()
debug("cleared orphaned keyboard recording state: \(orphaned.rawValue)")
case .idle, .aborted:
break
}
}
/// Auto-start (or renew) the Flow session on every app foreground when /// Auto-start (or renew) the Flow session on every app foreground when
/// permissions allow the "always auto-open, no off switch" policy. Also /// permissions allow the "always auto-open, no off switch" policy. Also
/// clears any orphaned Live Activity a previously force-quit process left /// clears any orphaned Live Activity a previously force-quit process left
@@ -196,6 +227,9 @@ final class FlowSessionManager: ObservableObject {
writeHeartbeatIfActive() writeHeartbeatIfActive()
case .background: case .background:
setAppForeground(false) setAppForeground(false)
if coldStartContext != nil {
dismissColdStartOverlay()
}
beginBackgroundKeepAlive() beginBackgroundKeepAlive()
@unknown default: @unknown default:
break break
@@ -328,16 +362,13 @@ final class FlowSessionManager: ObservableObject {
return return
} }
coldStartContext = FlowColdStartContext( coldStartContext = FlowColdStartContext(hostEntry: hostEntry)
hostEntry: hostEntry,
showReturnAlert: hostEntry != nil
)
} }
private func bindSessionASRIfNeeded(force: Bool = false) { private func bindSessionASRIfNeeded(force: Bool = false) {
let engineMode = store.engineMode let engineMode = store.engineMode
if !force, if !force,
let sessionASR, sessionASR != nil,
sessionASREngineMode == engineMode { sessionASREngineMode == engineMode {
return return
} }
@@ -374,6 +405,11 @@ final class FlowSessionManager: ObservableObject {
private func startPolling() { private func startPolling() {
pollingTask?.cancel() pollingTask?.cancel()
lastObservedRecordingState = FlowSessionBridge.recordingState()
FlowDiagnostics.log(
"polling started: initialRecordingState=\(lastObservedRecordingState.rawValue) " +
"container=\(AppGroup.containerPathForDiagnostics)"
)
pollingTask = Task { @MainActor [weak self] in pollingTask = Task { @MainActor [weak self] in
while !Task.isCancelled { while !Task.isCancelled {
self?.handleKeyboardSignal() self?.handleKeyboardSignal()
@@ -383,7 +419,17 @@ final class FlowSessionManager: ObservableObject {
} }
private func handleKeyboardSignal() { private func handleKeyboardSignal() {
switch FlowSessionBridge.recordingState() { let signal = FlowSessionBridge.recordingState()
if signal != lastObservedRecordingState {
// The single most important cross-process signal: proves whether the
// host actually SEES the keyboard's recording state writes.
FlowDiagnostics.log(
"poll observed recordingState \(lastObservedRecordingState.rawValue)\(signal.rawValue) " +
"[rec=\(isUtteranceRecording) proc=\(isUtteranceProcessing) fg=\(isAppForeground)]"
)
lastObservedRecordingState = signal
}
switch signal {
case .recording: case .recording:
guard !isUtteranceRecording, !isUtteranceProcessing else { return } guard !isUtteranceRecording, !isUtteranceProcessing else { return }
beginUtterance() beginUtterance()
@@ -431,7 +477,9 @@ final class FlowSessionManager: ObservableObject {
FlowLiveActivityController.update(phase: .recording) FlowLiveActivityController.update(phase: .recording)
FlowDiagnostics.log( FlowDiagnostics.log(
"beginUtterance engine=\(store.engineMode) " + "beginUtterance engine=\(store.engineMode) " +
"asrType=\(type(of: asr)) pipelined=true max=\(Int(FlowSessionKeys.maxUtteranceDuration))s" "asrType=\(type(of: asr)) pipelined=true " +
"localCustomLM=\(store.localASRCustomLanguageModelEnabled) " +
"max=\(Int(FlowSessionKeys.maxUtteranceDuration))s"
) )
asrTask = Task.detached(priority: .userInitiated) { [weak manager = self] in asrTask = Task.detached(priority: .userInitiated) { [weak manager = self] in
@@ -638,12 +686,18 @@ final class FlowSessionManager: ObservableObject {
// so the keyboard can show the "fill in your key" hint // so the keyboard can show the "fill in your key" hint
// inline rather than a generic failure message. The raw // inline rather than a generic failure message. The raw
// transcript is still delivered no data loss. // transcript is still delivered no data loss.
let warning = Self.warningFromPolishError(error, engineMode: engineMode) ?? chunkNote let fallback = Self.makeFallbackDelivery(
rawText: text,
error: error,
engineMode: engineMode,
chunkWarning: chunkNote
)
FlowDiagnostics.log( FlowDiagnostics.log(
"polish failed after \(String(format: "%.1f", Date().timeIntervalSince(polishStarted)))s: " + "polish failed after \(String(format: "%.1f", Date().timeIntervalSince(polishStarted)))s: " +
"\(error.localizedDescription)" "\(error.localizedDescription)"
) )
FlowSessionBridge.storeTranscriptionResult(text, polishWarning: warning) delivered = fallback.text
FlowSessionBridge.storeTranscriptionResult(fallback.text, polishWarning: fallback.polishWarning)
} }
SpeechHistoryStore.shared.recordUtterance( SpeechHistoryStore.shared.recordUtterance(
@@ -684,15 +738,41 @@ final class FlowSessionManager: ObservableObject {
/// v0.2.0: surface the local-mode cloud-polish error path with a /// v0.2.0: surface the local-mode cloud-polish error path with a
/// localised hint ("please fill in your DeepSeek key in Settings") /// localised hint ("please fill in your DeepSeek key in Settings")
/// rather than letting the keyboard show a generic network error. /// rather than letting the keyboard show a generic network error.
static func makeFallbackDelivery(
rawText: String,
error: Error,
engineMode: String,
chunkWarning: String?
) -> TranscriptionDelivery {
let fallbackText = TranscriptPostProcessor.cleanRawASRFallback(rawText)
let warning = warningFromPolishError(error, engineMode: engineMode)
?? polishDegradedWarning()
?? chunkWarning
return TranscriptionDelivery(text: fallbackText, polishWarning: warning)
}
private static func warningFromPolishError(_ error: Error, engineMode: String) -> String? { private static func warningFromPolishError(_ error: Error, engineMode: String) -> String? {
guard let polishError = error as? PolishingService.PolishError, if let polishError = error as? PolishingService.PolishError {
polishError == .missingAPIKey else { switch polishError {
return nil case .missingAPIKey:
if engineMode == "local" {
return SharedL10n.string("flow.warning.localPolishUnavailable")
}
return SharedL10n.string("flow.warning.cloudPolishMissingKey")
case .timeout:
return polishDegradedWarning()
case .noTranscript:
return nil
}
} }
if engineMode == "local" { if error is LLMError {
return AppL10n.string("flow.warning.localPolishUnavailable") return polishDegradedWarning()
} }
return AppL10n.string("flow.warning.cloudPolishMissingKey") return nil
}
private static func polishDegradedWarning() -> String? {
SharedL10n.string("flow.warning.polishDegraded")
} }
private func asrWaitTimeout() -> TimeInterval { private func asrWaitTimeout() -> TimeInterval {
@@ -62,6 +62,12 @@ final class SpeechHistoryStore: ObservableObject {
) )
} }
func delete(id: UUID) {
guard entries.contains(where: { $0.id == id }) else { return }
entries.removeAll { $0.id == id }
persist()
}
func clearAll() { func clearAll() {
entries.removeAll() entries.removeAll()
persist() persist()
+73 -67
View File
@@ -1,14 +1,13 @@
// FlowColdStartOverlay.swift // FlowColdStartOverlay.swift
// OSGKeyboard · Main App // OSGKeyboard · Main App
// //
// Minimal cold-start handoff UI: swipe-back guidance and optional return alert. // Minimal cold-start handoff UI: bottom-bar swipe guidance and optional return link.
import SwiftUI import SwiftUI
import OSGKeyboardShared import OSGKeyboardShared
struct FlowColdStartContext: Equatable { struct FlowColdStartContext: Equatable {
let hostEntry: HostAppEntry? let hostEntry: HostAppEntry?
var showReturnAlert: Bool
} }
struct FlowColdStartOverlay: View { struct FlowColdStartOverlay: View {
@@ -18,78 +17,62 @@ struct FlowColdStartOverlay: View {
let onReturnToHost: () -> Void let onReturnToHost: () -> Void
let onDismiss: () -> Void let onDismiss: () -> Void
@State private var showAlert: Bool @State private var swipeOffset: CGFloat = 0
init( private let homeBarWidth: CGFloat = 134
context: FlowColdStartContext,
onReturnToHost: @escaping () -> Void,
onDismiss: @escaping () -> Void
) {
self.context = context
self.onReturnToHost = onReturnToHost
self.onDismiss = onDismiss
_showAlert = State(initialValue: context.showReturnAlert)
}
var body: some View { var body: some View {
ZStack { ZStack {
palette.background.opacity(0.96) palette.background.opacity(0.96)
.ignoresSafeArea() .ignoresSafeArea()
VStack(spacing: Spacing.xl) { VStack(spacing: 0) {
Image("OSGBrandMark") Spacer()
.renderingMode(.template)
.resizable()
.aspectRatio(contentMode: .fit)
.frame(width: 64, height: 64)
.foregroundStyle(palette.accent)
.accessibilityHidden(true)
Text("flow.coldStart.title") VStack(spacing: Spacing.xl) {
.font(TypeStyle.title3) Image("OSGBrandMark")
.foregroundStyle(palette.textPrimary) .renderingMode(.template)
.multilineTextAlignment(.center) .resizable()
.aspectRatio(contentMode: .fit)
Text(swipeHintKey) .frame(width: 64, height: 64)
.font(TypeStyle.body)
.foregroundStyle(palette.textSecondary)
.multilineTextAlignment(.center)
.padding(.horizontal, Spacing.lg)
swipeHintAnimation
.padding(.top, Spacing.md)
Button(action: onDismiss) {
Text("flow.coldStart.dismiss")
.font(TypeStyle.body.weight(.semibold))
.foregroundStyle(palette.accent) .foregroundStyle(palette.accent)
.frame(maxWidth: .infinity) .accessibilityHidden(true)
.padding(.vertical, Spacing.md)
Text("flow.coldStart.title")
.font(TypeStyle.title3)
.foregroundStyle(palette.textPrimary)
.multilineTextAlignment(.center)
Text("flow.coldStart.swipeHint")
.font(TypeStyle.body)
.foregroundStyle(palette.textSecondary)
.multilineTextAlignment(.center)
.padding(.horizontal, Spacing.lg)
if context.hostEntry != nil {
Button(action: onReturnToHost) {
Text(returnButtonTitle)
.font(TypeStyle.body.weight(.semibold))
.foregroundStyle(palette.accent)
}
.buttonStyle(.plain)
}
} }
.buttonStyle(.plain)
.padding(.horizontal, Spacing.xl) .padding(.horizontal, Spacing.xl)
.padding(.top, Spacing.lg)
}
.padding(Spacing.xl)
}
.alert(alertTitle, isPresented: $showAlert) {
if context.hostEntry != nil {
Button(returnButtonTitle, action: onReturnToHost)
}
Button("flow.coldStart.dismiss", role: .cancel, action: onDismiss)
} message: {
Text("flow.coldStart.alert.message")
}
}
private var swipeHintKey: LocalizedStringKey { Spacer()
context.hostEntry == nil
? "flow.coldStart.swipeHint"
: "flow.coldStart.swipeHint.withSystemBack"
}
private var alertTitle: String { bottomSwipeGuide
AppL10n.string("flow.coldStart.alert.title") .padding(.bottom, Spacing.md)
Text("flow.coldStart.tapToDismiss")
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
.padding(.bottom, Spacing.xl)
}
}
.contentShape(Rectangle())
.onTapGesture(perform: onDismiss)
} }
private var returnButtonTitle: String { private var returnButtonTitle: String {
@@ -100,15 +83,38 @@ struct FlowColdStartOverlay: View {
return AppL10n.format("flow.coldStart.return.named", appName) return AppL10n.format("flow.coldStart.return.named", appName)
} }
private var swipeHintAnimation: some View { private var bottomSwipeGuide: some View {
VStack(spacing: Spacing.sm) { VStack(spacing: Spacing.sm) {
Image(systemName: "chevron.up") Image(systemName: "arrow.down")
.font(.system(size: 20, weight: .semibold)) .font(.system(size: 14, weight: .semibold))
.foregroundStyle(palette.textTertiary) .foregroundStyle(palette.textTertiary)
RoundedRectangle(cornerRadius: 3, style: .continuous)
.fill(palette.textTertiary.opacity(0.5)) ZStack(alignment: .leading) {
.frame(width: 120, height: 5) RoundedRectangle(cornerRadius: 2.5, style: .continuous)
.fill(palette.textTertiary.opacity(0.35))
.frame(width: homeBarWidth, height: 5)
Circle()
.fill(palette.accent)
.frame(width: 8, height: 8)
.offset(x: swipeOffset)
}
.frame(width: homeBarWidth, height: 16)
HStack(spacing: Spacing.xs) {
Image(systemName: "arrow.left")
.font(.system(size: 12, weight: .semibold))
Image(systemName: "arrow.right")
.font(.system(size: 12, weight: .semibold))
}
.foregroundStyle(palette.textTertiary)
} }
.accessibilityLabel(AppL10n.string("flow.coldStart.swipeAccessibility")) .accessibilityLabel(AppL10n.string("flow.coldStart.swipeAccessibility"))
.onAppear {
swipeOffset = 0
withAnimation(.easeInOut(duration: 1.4).repeatForever(autoreverses: true)) {
swipeOffset = homeBarWidth - 8
}
}
} }
} }
+41 -34
View File
@@ -32,16 +32,7 @@ struct HistoryView: View {
if store.entries.isEmpty { if store.entries.isEmpty {
emptyState emptyState
} else { } else {
ScrollView { list
LazyVStack(alignment: .leading, spacing: Spacing.xl) {
ForEach(store.groupedByDay, id: \.day) { group in
daySection(day: group.day, items: group.items)
}
}
.padding(.horizontal, Spacing.lg)
.padding(.vertical, Spacing.md)
.tabBarScrollBottomPadding()
}
} }
} }
.background(palette.background) .background(palette.background)
@@ -74,6 +65,38 @@ struct HistoryView: View {
} }
} }
// MARK: - List
private var list: some View {
List {
ForEach(store.groupedByDay, id: \.day) { group in
Section {
ForEach(group.items) { entry in
historyRow(entry)
.listRowInsets(EdgeInsets(top: 0, leading: 0, bottom: 0, trailing: 0))
.listRowBackground(palette.surface)
.listRowSeparatorTint(palette.divider)
}
.onDelete { offsets in
delete(items: group.items, at: offsets)
}
} header: {
Text(Self.dayFormatter.string(from: group.day))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.textCase(.uppercase)
.tracking(0.5)
}
}
}
.listStyle(.insetGrouped)
.listSectionSpacing(Spacing.lg)
.scrollContentBackground(.hidden)
.background(palette.background)
.contentMargins(.top, Spacing.md, for: .scrollContent)
.tabBarScrollBottomPadding()
}
private var emptyState: some View { private var emptyState: some View {
VStack(spacing: Spacing.sm) { VStack(spacing: Spacing.sm) {
Spacer() Spacer()
@@ -88,30 +111,6 @@ struct HistoryView: View {
.padding(.horizontal, Spacing.xl) .padding(.horizontal, Spacing.xl)
} }
private func daySection(day: Date, items: [SpeechHistoryEntry]) -> some View {
VStack(alignment: .leading, spacing: Spacing.sm) {
Text(Self.dayFormatter.string(from: day))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.textCase(.uppercase)
.tracking(0.5)
VStack(spacing: 0) {
ForEach(Array(items.enumerated()), id: \.element.id) { index, entry in
historyRow(entry)
if index < items.count - 1 {
Divider().background(palette.divider)
}
}
}
.background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.xl, style: .continuous))
.overlay(
RoundedRectangle(cornerRadius: Radius.xl, style: .continuous)
.stroke(palette.divider, lineWidth: 0.5)
)
}
}
private func historyRow(_ entry: SpeechHistoryEntry) -> some View { private func historyRow(_ entry: SpeechHistoryEntry) -> some View {
VStack(alignment: .leading, spacing: Spacing.xxs) { VStack(alignment: .leading, spacing: Spacing.xxs) {
Text(Self.timeFormatter.string(from: entry.createdAt)) Text(Self.timeFormatter.string(from: entry.createdAt))
@@ -126,4 +125,12 @@ struct HistoryView: View {
.padding(Spacing.md) .padding(Spacing.md)
.frame(maxWidth: .infinity, alignment: .leading) .frame(maxWidth: .infinity, alignment: .leading)
} }
// MARK: - Mutations
private func delete(items: [SpeechHistoryEntry], at offsets: IndexSet) {
for index in offsets {
store.delete(id: items[index].id)
}
}
} }
@@ -25,6 +25,8 @@ struct LocalModelsGroup: View {
speechRow speechRow
Divider().background(palette.divider) Divider().background(palette.divider)
polishRow polishRow
Divider().background(palette.divider)
customLanguageModelDiagnosticRow
} }
.background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous)) .background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous))
.overlay( .overlay(
@@ -63,6 +65,25 @@ struct LocalModelsGroup: View {
.frame(minHeight: SettingsListMetrics.singleLineMinHeight) .frame(minHeight: SettingsListMetrics.singleLineMinHeight)
} }
// MARK: Custom language model diagnostic row
private var customLanguageModelDiagnosticRow: some View {
Toggle(isOn: $config.localASRCustomLanguageModelEnabled) {
VStack(alignment: .leading, spacing: Spacing.xxs) {
Text("settings.localModels.customLM.title")
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
Text("settings.localModels.customLM.subtitle")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
}
}
.tint(palette.accent)
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.sm)
.frame(minHeight: SettingsListMetrics.singleLineMinHeight)
}
// MARK: Helpers // MARK: Helpers
/// Accent badge naming the engine that backs each local-mode row /// Accent badge naming the engine that backs each local-mode row
+11 -6
View File
@@ -38,14 +38,19 @@ struct MainAppRoot: View {
.onAppear { .onAppear {
flowManager.setAppForeground(scenePhase == .active) flowManager.setAppForeground(scenePhase == .active)
flowManager.activateOnForeground() flowManager.activateOnForeground()
PersonalDictionaryCloudSync.shared.startObservingExternalChanges() AppCloudSync.shared.startObservingExternalChanges()
// Registering here also flushes any URL buffered during a cold
// launch (the keyboard app `startflow` handoff arrives via the
// scene delegate before this view is on screen).
AppOpenURLRouter.shared.register { url in
handleIncomingURL(url)
}
Task { Task {
await PersonalDictionaryCloudSync.shared.pullAndMergeIfEnabled() await AppCloudSync.shared.pullAllIfEnabled()
} }
} }
.onReceive(NotificationCenter.default.publisher(for: .osgKeyboardOpenURL)) { notification in .onReceive(NotificationCenter.default.publisher(for: .settingsDidSyncFromCloud)) { _ in
guard let url = notification.userInfo?["url"] as? URL else { return } config.reloadFromPersistedStorage()
handleIncomingURL(url)
} }
.onChange(of: config.hasCompletedOnboarding) { _, done in .onChange(of: config.hasCompletedOnboarding) { _, done in
if done { if done {
@@ -59,7 +64,7 @@ struct MainAppRoot: View {
flowManager.activateOnForeground() flowManager.activateOnForeground()
} }
Task { Task {
await PersonalDictionaryCloudSync.shared.pullAndMergeIfEnabled() await AppCloudSync.shared.pullAllIfEnabled()
} }
} }
} }
@@ -0,0 +1,103 @@
// SettingsICloudSyncRow.swift
// OSGKeyboard · Main App
//
// Settings-row toggle for mirroring user preferences through iCloud KVS.
// API keys remain in Keychain and are never uploaded.
import SwiftUI
import OSGKeyboardShared
@MainActor
struct SettingsICloudSyncRow: View {
@Environment(\.themePalette) private var palette: ThemePalette
@State private var isEnabled: Bool = AppGroupStore().settingsICloudSyncEnabled
@State private var syncErrorMessage: String?
@State private var isApplyingToggle = false
private let store = AppGroupStore()
var body: some View {
VStack(alignment: .leading, spacing: Spacing.xxs) {
Toggle(isOn: toggleBinding) {
VStack(alignment: .leading, spacing: Spacing.xxs) {
Text("settings.appSettings.iCloudSync.title")
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
Text("settings.appSettings.iCloudSync.subtitle")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.fixedSize(horizontal: false, vertical: true)
}
}
.tint(palette.accent)
.disabled(isApplyingToggle)
if let syncErrorMessage {
Text(syncErrorMessage)
.font(TypeStyle.caption2)
.foregroundStyle(.red)
.fixedSize(horizontal: false, vertical: true)
.padding(.top, Spacing.xxs)
}
}
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.sm)
.frame(minHeight: SettingsListMetrics.singleLineMinHeight, alignment: .leading)
.onAppear { reloadFromStore() }
.onReceive(
NotificationCenter.default.publisher(for: .settingsDidSyncFromCloud)
) { _ in
reloadFromStore()
}
}
private var toggleBinding: Binding<Bool> {
Binding(
get: { isEnabled },
set: { newValue in
guard newValue != isEnabled else { return }
if newValue {
enableSync()
} else {
disableSync()
}
}
)
}
private func reloadFromStore() {
isEnabled = store.settingsICloudSyncEnabled
}
private func enableSync() {
isApplyingToggle = true
syncErrorMessage = nil
Task {
do {
try await SettingsCloudSync.shared.enableSync()
reloadFromStore()
} catch let error as SettingsCloudSyncError {
isEnabled = false
syncErrorMessage = localizedSyncError(error)
} catch {
isEnabled = false
syncErrorMessage = error.localizedDescription
}
isApplyingToggle = false
}
}
private func disableSync() {
SettingsCloudSync.shared.disableSync()
isEnabled = false
syncErrorMessage = nil
}
private func localizedSyncError(_ error: SettingsCloudSyncError) -> String {
switch error {
case .encodeFailed, .decodeFailed:
return AppL10n.string("settings.appSettings.iCloudSync.error.generic")
}
}
}
+4
View File
@@ -182,6 +182,10 @@ struct SettingsView: View {
Divider().background(palette.divider) Divider().background(palette.divider)
cursorDragNavigationToggleRow cursorDragNavigationToggleRow
Divider().background(palette.divider)
SettingsICloudSyncRow()
} }
.background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.xl, style: .continuous)) .background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.xl, style: .continuous))
.overlay( .overlay(
+10 -6
View File
@@ -368,6 +368,11 @@
"settings.personalDictionary.iCloudSync.error.tooLarge" = "Dictionary is too large to sync via iCloud. Remove some entries and try again."; "settings.personalDictionary.iCloudSync.error.tooLarge" = "Dictionary is too large to sync via iCloud. Remove some entries and try again.";
"settings.personalDictionary.iCloudSync.error.generic" = "Could not sync your dictionary with iCloud. Try again later."; "settings.personalDictionary.iCloudSync.error.generic" = "Could not sync your dictionary with iCloud. Try again later.";
"settings.iCloudSync.title" = "iCloud Sync";
"settings.appSettings.iCloudSync.title" = "Sync settings via iCloud";
"settings.appSettings.iCloudSync.subtitle" = "Keep engine, language, polish, and Flow preferences in sync across your devices. API keys stay on each device.";
"settings.appSettings.iCloudSync.error.generic" = "Could not sync settings with iCloud. Try again later.";
/* v0.3.0: Polish intensity */ /* v0.3.0: Polish intensity */
"settings.polishIntensity.title" = "Polish intensity"; "settings.polishIntensity.title" = "Polish intensity";
@@ -381,15 +386,14 @@
"settings.flow.inactivity.3h" = "3 hours"; "settings.flow.inactivity.3h" = "3 hours";
"settings.flow.inactivity.12h" = "12 hours"; "settings.flow.inactivity.12h" = "12 hours";
"settings.flow.inactivity.24h" = "24 hours"; "settings.flow.inactivity.24h" = "24 hours";
"settings.localModels.customLM.title" = "Use custom language model";
"settings.localModels.customLM.subtitle" = "Diagnostic switch. Turn off to test pure Apple on-device recognition if local ASR gets stuck or returns no speech.";
/* Cold-start handoff (scheme B) */ /* Cold-start handoff (scheme B) */
"flow.coldStart.title" = "Voice is ready"; "flow.coldStart.title" = "Voice is ready";
"flow.coldStart.swipeHint" = "Swipe up from the bottom edge to return to your previous app."; "flow.coldStart.swipeHint" = "Swipe right along the bar at the bottom to return to your previous app.";
"flow.coldStart.swipeHint.withSystemBack" = "Tap in the top-left or swipe up from the bottom to return."; "flow.coldStart.swipeAccessibility" = "Swipe right along the bottom bar to return";
"flow.coldStart.dismiss" = "Got it"; "flow.coldStart.tapToDismiss" = "Tap anywhere to close";
"flow.coldStart.swipeAccessibility" = "Swipe up from the bottom to return";
"flow.coldStart.alert.title" = "Voice is ready";
"flow.coldStart.alert.message" = "You can return to continue typing, or swipe up from the bottom.";
"flow.coldStart.return.named" = "Return to %@"; "flow.coldStart.return.named" = "Return to %@";
"flow.coldStart.return.generic" = "Return to app"; "flow.coldStart.return.generic" = "Return to app";
+10 -6
View File
@@ -367,6 +367,11 @@
"settings.personalDictionary.iCloudSync.error.tooLarge" = "词库过大,无法通过 iCloud 同步。请删除部分词条后重试。"; "settings.personalDictionary.iCloudSync.error.tooLarge" = "词库过大,无法通过 iCloud 同步。请删除部分词条后重试。";
"settings.personalDictionary.iCloudSync.error.generic" = "无法与 iCloud 同步词库,请稍后重试。"; "settings.personalDictionary.iCloudSync.error.generic" = "无法与 iCloud 同步词库,请稍后重试。";
"settings.iCloudSync.title" = "iCloud 同步";
"settings.appSettings.iCloudSync.title" = "设置 iCloud 同步";
"settings.appSettings.iCloudSync.subtitle" = "在多台设备间同步引擎、语言、润色和 Flow 偏好。API 密钥仍保留在各设备本地。";
"settings.appSettings.iCloudSync.error.generic" = "无法与 iCloud 同步设置,请稍后重试。";
/* v0.3.0: 润色强度 */ /* v0.3.0: 润色强度 */
"settings.polishIntensity.title" = "润色强度"; "settings.polishIntensity.title" = "润色强度";
@@ -380,15 +385,14 @@
"settings.flow.inactivity.3h" = "3 小时"; "settings.flow.inactivity.3h" = "3 小时";
"settings.flow.inactivity.12h" = "12 小时"; "settings.flow.inactivity.12h" = "12 小时";
"settings.flow.inactivity.24h" = "24 小时"; "settings.flow.inactivity.24h" = "24 小时";
"settings.localModels.customLM.title" = "使用自定义语言模型";
"settings.localModels.customLM.subtitle" = "诊断开关。本地识别卡住或提示未识别时,可关闭它测试纯 Apple 端侧识别。";
/* 冷启动兜底(方案 B */ /* 冷启动兜底(方案 B */
"flow.coldStart.title" = "语音已就绪"; "flow.coldStart.title" = "语音已就绪";
"flow.coldStart.swipeHint" = "屏幕底部边缘向上滑动,返回上一个 App。"; "flow.coldStart.swipeHint" = "沿屏幕底部的横条,从左向右滑动,返回上一个 App。";
"flow.coldStart.swipeHint.withSystemBack" = "点左上角 或从底部向上滑动返回"; "flow.coldStart.swipeAccessibility" = "沿底部横条从左向右滑动返回";
"flow.coldStart.dismiss" = "知道了"; "flow.coldStart.tapToDismiss" = "点按屏幕关闭";
"flow.coldStart.swipeAccessibility" = "从底部向上滑动返回";
"flow.coldStart.alert.title" = "语音已就绪";
"flow.coldStart.alert.message" = "可返回继续输入,或从底部向上滑动返回。";
"flow.coldStart.return.named" = "返回%@"; "flow.coldStart.return.named" = "返回%@";
"flow.coldStart.return.generic" = "返回 App"; "flow.coldStart.return.generic" = "返回 App";
@@ -72,7 +72,10 @@ final class KeyboardConfigSync {
func syncOnboardingStateFromAppGroup() { func syncOnboardingStateFromAppGroup() {
let store = AppGroupStore() let store = AppGroupStore()
state.hasCompletedOnboarding = store.hasCompletedOnboarding // Fall back to the reboot-durable Keychain marker so a device restart
// does not resurrect the in-keyboard onboarding overlay when the App
// Group value transiently reads empty.
state.hasCompletedOnboarding = store.hasCompletedOnboarding || Keychain.hasCompletedOnboarding()
state.onboardingPage = store.onboardingPage state.onboardingPage = store.onboardingPage
} }
@@ -36,6 +36,8 @@ final class KeyboardFlowCoordinator {
private var wasFlowSessionActive = false private var wasFlowSessionActive = false
private var flowSessionMonitorTask: Task<Void, Never>? private var flowSessionMonitorTask: Task<Void, Never>?
private var isAwaitingFlowResult = false private var isAwaitingFlowResult = false
private var lastFlowAutoStartAttempt: TimeInterval = 0
private static let flowAutoStartCooldown: TimeInterval = 20
init( init(
state: KeyboardState, state: KeyboardState,
@@ -87,10 +89,16 @@ final class KeyboardFlowCoordinator {
refreshFlowPartialIfNeeded() refreshFlowPartialIfNeeded()
consumePendingFlowDeliveryIfNeeded() consumePendingFlowDeliveryIfNeeded()
let active = FlowSessionBridge.isSessionActive() recoverFromDeadHostIfNeeded()
state.flowSessionActive = active
if wasFlowSessionActive && !active && !isFlowRecording && !isPendingFlowStart { if FlowSessionBridge.clearIfHostStale() {
debug("cleared zombie Flow session from App Group")
}
let reachable = FlowSessionBridge.isHostReachable()
state.flowSessionActive = reachable
if wasFlowSessionActive && !reachable && !isFlowRecording && !isPendingFlowStart {
switch state.phase { switch state.phase {
case .recording, .processing: case .recording, .processing:
break break
@@ -98,7 +106,9 @@ final class KeyboardFlowCoordinator {
showFlowSessionExpiredHint() showFlowSessionExpiredHint()
} }
} }
wasFlowSessionActive = active wasFlowSessionActive = reachable
maybeAutoStartFlowSession()
} }
func toggleRecording() { func toggleRecording() {
@@ -135,7 +145,13 @@ final class KeyboardFlowCoordinator {
detectAndStoreAppContext() detectAndStoreAppContext()
if FlowSessionBridge.isSessionActive() { let reachable = FlowSessionBridge.isHostReachable()
debug(
"pressBegan hostReachable=\(reachable) " +
"staleness=\(FlowSessionBridge.heartbeatStaleness().map { String(format: "%.1f", $0) } ?? "nil") " +
"container=\(AppGroup.containerPathForDiagnostics)"
)
if reachable {
startFlowRecording() startFlowRecording()
} else { } else {
beginFlowStart() beginFlowStart()
@@ -153,6 +169,7 @@ final class KeyboardFlowCoordinator {
stopUtteranceCountdown() stopUtteranceCountdown()
ExtensionScreenWakeLock.release() ExtensionScreenWakeLock.release()
FlowSessionBridge.setRecordingState(.stopped) FlowSessionBridge.setRecordingState(.stopped)
debug("pressEnded wrote .stopped (readback=\(FlowSessionBridge.recordingState().rawValue))")
state.phase = .processing state.phase = .processing
state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing")
startFlowResultWatchdog() startFlowResultWatchdog()
@@ -225,11 +242,63 @@ final class KeyboardFlowCoordinator {
} }
} }
if isPendingFlowStart, FlowSessionBridge.isSessionActive() { if isPendingFlowStart, FlowSessionBridge.isHostReachable() {
completeFlowStartHandoff() completeFlowStartHandoff()
} }
} }
/// When the host process died mid-utterance, abort local recording / waiting
/// so the user is not stuck until the long result watchdog fires.
private func recoverFromDeadHostIfNeeded() {
guard FlowSessionBridge.isHostStale() else { return }
if isFlowRecording {
isFlowRecording = false
stopUtteranceCountdown()
ExtensionScreenWakeLock.release()
FlowSessionBridge.setRecordingState(.aborted)
stopFlowWatchdog()
state.level = 0
state.phase = .idle
state.lastTranscript = ""
debug("aborted recording — host heartbeat zombie")
return
}
if isAwaitingFlowResult {
failHostDisconnected()
}
}
/// Restores the pre-ABCD behaviour: when the keyboard appears and the host
/// is not reachable, automatically jump to the main app to start Flow.
private func maybeAutoStartFlowSession() {
guard !FlowSessionBridge.isHostReachable() else { return }
guard !isPendingFlowStart, !isFlowRecording, !isAwaitingFlowResult else { return }
guard hasFullAccess(), AppGroup.isAvailable else { return }
guard case .idle = state.phase else { return }
let now = Date().timeIntervalSince1970
guard now - lastFlowAutoStartAttempt >= Self.flowAutoStartCooldown else { return }
lastFlowAutoStartAttempt = now
beginFlowStart()
}
private func failHostDisconnected() {
isAwaitingFlowResult = false
isFlowRecording = false
isPendingFlowStart = false
stopUtteranceCountdown()
ExtensionScreenWakeLock.release()
FlowSessionBridge.setRecordingState(.aborted)
stopFlowWatchdog()
state.level = 0
let message = ExtL10n.string("keyboard.flow.hostDisconnected")
state.phase = .error(.flowSessionExpired, message: message)
scheduleAutoClearError()
debug("host disconnected while awaiting Flow result")
}
private func showFlowSessionExpiredHint() { private func showFlowSessionExpiredHint() {
let message = ExtL10n.string("keyboard.flow.sessionExpired") let message = ExtL10n.string("keyboard.flow.sessionExpired")
state.phase = .error(.flowSessionExpired, message: message) state.phase = .error(.flowSessionExpired, message: message)
@@ -252,6 +321,10 @@ final class KeyboardFlowCoordinator {
} }
private func startFlowRecording() { private func startFlowRecording() {
guard FlowSessionBridge.isHostReachable() else {
beginFlowStart()
return
}
isPendingFlowStart = false isPendingFlowStart = false
flowStartDeadline = 0 flowStartDeadline = 0
stopFlowWatchdog() stopFlowWatchdog()
@@ -266,7 +339,9 @@ final class KeyboardFlowCoordinator {
} }
startUtteranceCountdown() startUtteranceCountdown()
startFlowLevelWatchdog() startFlowLevelWatchdog()
debug("startFlowRecording") // Read back in-process to confirm the write landed before we rely on
// the host polling it out cross-process.
debug("startFlowRecording wrote .recording (readback=\(FlowSessionBridge.recordingState().rawValue))")
} }
private func startUtteranceCountdown() { private func startUtteranceCountdown() {
@@ -305,7 +380,7 @@ final class KeyboardFlowCoordinator {
stopFlowWatchdog() stopFlowWatchdog()
flowWatchdogTask = Task { @MainActor [weak self] in flowWatchdogTask = Task { @MainActor [weak self] in
while let self, !Task.isCancelled, self.isPendingFlowStart { while let self, !Task.isCancelled, self.isPendingFlowStart {
if FlowSessionBridge.isSessionActive() { if FlowSessionBridge.isHostReachable() {
self.completeFlowStartHandoff() self.completeFlowStartHandoff()
return return
} }
@@ -362,17 +437,20 @@ final class KeyboardFlowCoordinator {
isAwaitingFlowResult = true isAwaitingFlowResult = true
let startedAt = Date().timeIntervalSince1970 let startedAt = Date().timeIntervalSince1970
let resultTimeout = FlowWatchdog.resultTimeout(engineMode: state.engineMode) let resultTimeout = FlowWatchdog.resultTimeout(engineMode: state.engineMode)
debug("resultWatchdog started timeout=\(Int(resultTimeout))s engine=\(state.engineMode)")
flowWatchdogTask = Task { @MainActor [weak self] in flowWatchdogTask = Task { @MainActor [weak self] in
while let self, !Task.isCancelled { while let self, !Task.isCancelled {
if let delivery = FlowSessionBridge.consumeTranscriptionDelivery() { if let delivery = FlowSessionBridge.consumeTranscriptionDelivery() {
self.isAwaitingFlowResult = false self.isAwaitingFlowResult = false
self.stopFlowWatchdog() self.stopFlowWatchdog()
self.debug("resultWatchdog consumed delivery len=\(delivery.text.count)")
self.textInserter.handleFlowTranscript(delivery) self.textInserter.handleFlowTranscript(delivery)
return return
} }
if let error = FlowSessionBridge.consumeTranscriptionError() { if let error = FlowSessionBridge.consumeTranscriptionError() {
self.isAwaitingFlowResult = false self.isAwaitingFlowResult = false
self.stopFlowWatchdog() self.stopFlowWatchdog()
self.debug("resultWatchdog consumed error kind=\(error.kind.rawValue)")
self.state.phase = .error( self.state.phase = .error(
.fromFlowTranscription(error), .fromFlowTranscription(error),
message: error.message message: error.message
@@ -382,9 +460,22 @@ final class KeyboardFlowCoordinator {
} }
self.refreshFlowPartialIfNeeded() self.refreshFlowPartialIfNeeded()
let now = Date().timeIntervalSince1970 let now = Date().timeIntervalSince1970
let staleness = FlowSessionBridge.heartbeatStaleness() ?? .infinity
if staleness > FlowSessionKeys.heartbeatZombieInterval {
self.debug("resultWatchdog: host heartbeat zombie (staleness=\(String(format: "%.1f", staleness))s)")
self.failHostDisconnected()
return
}
if !FlowSessionBridge.isHostReachable(),
now - startedAt > FlowSessionKeys.keyboardHostDisconnectFailFast {
self.debug("resultWatchdog: host unreachable after \(String(format: "%.1f", now - startedAt))s")
self.failHostDisconnected()
return
}
if now - startedAt > resultTimeout { if now - startedAt > resultTimeout {
self.isAwaitingFlowResult = false self.isAwaitingFlowResult = false
self.stopFlowWatchdog() self.stopFlowWatchdog()
self.debug("resultWatchdog TIMEOUT after \(Int(resultTimeout))s — no result from host")
let msg = ExtL10n.string("keyboard.flow.resultTimeout") let msg = ExtL10n.string("keyboard.flow.resultTimeout")
self.state.phase = .error(.flowResultTimeout, message: msg) self.state.phase = .error(.flowResultTimeout, message: msg)
self.scheduleAutoClearError() self.scheduleAutoClearError()
+1
View File
@@ -139,6 +139,7 @@
"keyboard.flow.startingSession" = "Starting voice session…"; "keyboard.flow.startingSession" = "Starting voice session…";
"keyboard.flow.transcribing" = "Transcribing…"; "keyboard.flow.transcribing" = "Transcribing…";
"keyboard.flow.resultTimeout" = "Timed out waiting for transcription. Try again."; "keyboard.flow.resultTimeout" = "Timed out waiting for transcription. Try again.";
"keyboard.flow.hostDisconnected" = "Voice session disconnected. Open OSGKeyboard to restart.";
"keyboard.flow.manualOpenHost" = "Could not auto-open the app. Open OSGKeyboard from the Home Screen, then return."; "keyboard.flow.manualOpenHost" = "Could not auto-open the app. Open OSGKeyboard from the Home Screen, then return.";
/* Keyboard status badges */ /* Keyboard status badges */
@@ -139,6 +139,7 @@
"keyboard.flow.startingSession" = "正在启动语音会话…"; "keyboard.flow.startingSession" = "正在启动语音会话…";
"keyboard.flow.transcribing" = "识别中…"; "keyboard.flow.transcribing" = "识别中…";
"keyboard.flow.resultTimeout" = "等待识别结果超时,请重试"; "keyboard.flow.resultTimeout" = "等待识别结果超时,请重试";
"keyboard.flow.hostDisconnected" = "语音会话已断开,请打开 OSGKeyboard 重新启动";
"keyboard.flow.manualOpenHost" = "无法自动跳转,请从主屏幕打开 OSGKeyboard 后返回"; "keyboard.flow.manualOpenHost" = "无法自动跳转,请从主屏幕打开 OSGKeyboard 后返回";
/* Keyboard status badges */ /* Keyboard status badges */
@@ -23,6 +23,15 @@ public enum AppGroup {
) != nil ) != nil
}() }()
/// On-disk container path (or `"nil"`) for diagnostics. When this reads
/// `nil` in a process, that process cannot share App Group state the
/// `CFPrefsPlistSource Container: (null)` console warning comes from here.
public static var containerPathForDiagnostics: String {
FileManager.default.containerURL(
forSecurityApplicationGroupIdentifier: identifier
)?.path ?? "nil"
}
/// Shared UserDefaults when the App Group suite is available; `nil` otherwise. /// Shared UserDefaults when the App Group suite is available; `nil` otherwise.
/// ///
/// Prefer this in Release builds and in the keyboard extension so callers /// Prefer this in Release builds and in the keyboard extension so callers
@@ -32,10 +32,16 @@ public struct AppGroupConfiguration: Sendable, Equatable {
public static let personalDictionary = "config.personalDictionary.v1" public static let personalDictionary = "config.personalDictionary.v1"
/// When true, the main app mirrors the personal dictionary via iCloud KVS. /// When true, the main app mirrors the personal dictionary via iCloud KVS.
public static let personalDictionaryICloudSyncEnabled = "config.personalDictionary.iCloudSyncEnabled" public static let personalDictionaryICloudSyncEnabled = "config.personalDictionary.iCloudSyncEnabled"
/// When true, the main app mirrors user settings via iCloud KVS.
public static let settingsICloudSyncEnabled = "config.settings.iCloudSyncEnabled"
/// Wall-clock stamp of the last settings blob applied from iCloud KVS.
public static let settingsCloudUpdatedAt = "config.settings.cloudUpdatedAt"
/// When true, the host app auto-returns to the source app after a cold-start handoff. /// When true, the host app auto-returns to the source app after a cold-start handoff.
public static let flowSkipAppSwitch = "config.flowSkipAppSwitch" public static let flowSkipAppSwitch = "config.flowSkipAppSwitch"
/// Raw `FlowInactivityDuration` value; session expires after this idle window. /// Raw `FlowInactivityDuration` value; session expires after this idle window.
public static let flowInactivityDuration = "config.flowInactivityDuration" public static let flowInactivityDuration = "config.flowInactivityDuration"
/// Diagnostic switch: when false, local ASR skips the custom language model.
public static let localASRCustomLanguageModelEnabled = "config.localASR.customLanguageModelEnabled"
} }
// MARK: - Stored fields // MARK: - Stored fields
@@ -57,10 +63,14 @@ public struct AppGroupConfiguration: Sendable, Equatable {
public var personalDictionary: PersonalDictionary public var personalDictionary: PersonalDictionary
/// Opt-in iCloud KVS sync for the personal dictionary (main app only). /// Opt-in iCloud KVS sync for the personal dictionary (main app only).
public var personalDictionaryICloudSyncEnabled: Bool public var personalDictionaryICloudSyncEnabled: Bool
/// Opt-in iCloud KVS sync for user settings (main app only).
public var settingsICloudSyncEnabled: Bool
/// Auto-return to the host app after `startflow` cold start (default on). /// Auto-return to the host app after `startflow` cold start (default on).
public var flowSkipAppSwitch: Bool public var flowSkipAppSwitch: Bool
/// Idle timeout before the Flow session ends; resets on each utterance. /// Idle timeout before the Flow session ends; resets on each utterance.
public var flowInactivityDuration: FlowInactivityDuration public var flowInactivityDuration: FlowInactivityDuration
/// Whether local `SpeechAnalyzer` should attach the prepared custom language model.
public var localASRCustomLanguageModelEnabled: Bool
// MARK: - Derived // MARK: - Derived
@@ -164,6 +174,12 @@ public struct AppGroupConfiguration: Sendable, Equatable {
} }
return defaults.bool(forKey: Keys.personalDictionaryICloudSyncEnabled) return defaults.bool(forKey: Keys.personalDictionaryICloudSyncEnabled)
}(), }(),
settingsICloudSyncEnabled: {
if defaults.object(forKey: Keys.settingsICloudSyncEnabled) == nil {
return true
}
return defaults.bool(forKey: Keys.settingsICloudSyncEnabled)
}(),
flowSkipAppSwitch: { flowSkipAppSwitch: {
if defaults.object(forKey: Keys.flowSkipAppSwitch) == nil { if defaults.object(forKey: Keys.flowSkipAppSwitch) == nil {
return true return true
@@ -172,7 +188,13 @@ public struct AppGroupConfiguration: Sendable, Equatable {
}(), }(),
flowInactivityDuration: FlowInactivityDuration.fromStored( flowInactivityDuration: FlowInactivityDuration.fromStored(
defaults.string(forKey: Keys.flowInactivityDuration) defaults.string(forKey: Keys.flowInactivityDuration)
) ),
localASRCustomLanguageModelEnabled: {
if defaults.object(forKey: Keys.localASRCustomLanguageModelEnabled) == nil {
return true
}
return defaults.bool(forKey: Keys.localASRCustomLanguageModelEnabled)
}()
) )
let preset = LLMProvider.provider(id: config.providerId) let preset = LLMProvider.provider(id: config.providerId)
@@ -222,7 +244,9 @@ public struct AppGroupConfiguration: Sendable, Equatable {
defaults.set(polishIntensity.rawValue, forKey: Keys.polishIntensity) defaults.set(polishIntensity.rawValue, forKey: Keys.polishIntensity)
defaults.set(flowSkipAppSwitch, forKey: Keys.flowSkipAppSwitch) defaults.set(flowSkipAppSwitch, forKey: Keys.flowSkipAppSwitch)
defaults.set(flowInactivityDuration.rawValue, forKey: Keys.flowInactivityDuration) defaults.set(flowInactivityDuration.rawValue, forKey: Keys.flowInactivityDuration)
defaults.set(localASRCustomLanguageModelEnabled, forKey: Keys.localASRCustomLanguageModelEnabled)
defaults.set(personalDictionaryICloudSyncEnabled, forKey: Keys.personalDictionaryICloudSyncEnabled) defaults.set(personalDictionaryICloudSyncEnabled, forKey: Keys.personalDictionaryICloudSyncEnabled)
defaults.set(settingsICloudSyncEnabled, forKey: Keys.settingsICloudSyncEnabled)
Self.encodePersonalDictionary(personalDictionary, to: defaults) Self.encodePersonalDictionary(personalDictionary, to: defaults)
} }
@@ -78,6 +78,11 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
guard !isApplyingConfiguration, guard !isApplyingConfiguration,
hasCompletedOnboarding != configuration.hasCompletedOnboarding else { return } hasCompletedOnboarding != configuration.hasCompletedOnboarding else { return }
configuration.hasCompletedOnboarding = hasCompletedOnboarding configuration.hasCompletedOnboarding = hasCompletedOnboarding
// Mirror to the reboot-durable Keychain marker so a device restart
// can never resurrect the onboarding flow (or lose a replay reset).
let newValue = hasCompletedOnboarding
OSGLog.config.info("[onboarding] didSet → \(newValue, privacy: .public), mirroring to Keychain")
Keychain.setOnboardingCompleted(hasCompletedOnboarding)
if hasCompletedOnboarding { if hasCompletedOnboarding {
configuration.onboardingPage = 0 configuration.onboardingPage = 0
onboardingPage = 0 onboardingPage = 0
@@ -190,6 +195,19 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
} }
} }
/// Diagnostic switch: disable to isolate whether the custom language model
/// is causing local SpeechAnalyzer to return empty results.
@Published public var localASRCustomLanguageModelEnabled: Bool {
didSet {
guard !isApplyingConfiguration,
localASRCustomLanguageModelEnabled != configuration.localASRCustomLanguageModelEnabled else {
return
}
configuration.localASRCustomLanguageModelEnabled = localASRCustomLanguageModelEnabled
persistConfiguration()
}
}
public var isConfigured: Bool { public var isConfigured: Bool {
// Local engine uses on-device ASR + built-in DeepSeek polish and // Local engine uses on-device ASR + built-in DeepSeek polish and
// does not need a user API key. Cloud needs base URL, key, and model. // does not need a user API key. Cloud needs base URL, key, and model.
@@ -221,6 +239,28 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
self.defaults = resolvedDefaults self.defaults = resolvedDefaults
self.configuration = AppGroupConfiguration.load(fromAvailable: resolvedDefaults) self.configuration = AppGroupConfiguration.load(fromAvailable: resolvedDefaults)
// Onboarding completion must survive a device reboot. App Group
// UserDefaults can transiently read empty right after boot, which would
// falsely re-show onboarding. Trust the durable Keychain marker when the
// App Group value looks unset, and backfill it once the App Group value
// is confirmed true (covers users onboarded before this safeguard).
let appGroupOnboarding = configuration.hasCompletedOnboarding
let keychainOnboarding = Keychain.hasCompletedOnboarding()
// Distinguish "key absent" (nil plist not loaded / data-protection race)
// from "key present == false" (something actually wrote false).
let rawKeyPresent = resolvedDefaults.object(forKey: AppGroupConfiguration.Keys.hasCompletedOnboarding) != nil
OSGLog.config.info(
"[onboarding] init: appGroup=\(appGroupOnboarding, privacy: .public) (keyPresent=\(rawKeyPresent, privacy: .public)), keychain=\(keychainOnboarding, privacy: .public)"
)
if appGroupOnboarding {
Keychain.setOnboardingCompleted(true)
} else if keychainOnboarding {
configuration.hasCompletedOnboarding = true
OSGLog.config.info("[onboarding] init: App Group read false but Keychain true → restored to true")
}
let finalOnboarding = configuration.hasCompletedOnboarding
OSGLog.config.info("[onboarding] init: final=\(finalOnboarding, privacy: .public)")
isApplyingConfiguration = true isApplyingConfiguration = true
providerId = configuration.providerId providerId = configuration.providerId
baseURL = configuration.baseURL baseURL = configuration.baseURL
@@ -239,6 +279,7 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
polishIntensity = configuration.polishIntensity polishIntensity = configuration.polishIntensity
flowSkipAppSwitch = configuration.flowSkipAppSwitch flowSkipAppSwitch = configuration.flowSkipAppSwitch
flowInactivityDuration = configuration.flowInactivityDuration flowInactivityDuration = configuration.flowInactivityDuration
localASRCustomLanguageModelEnabled = configuration.localASRCustomLanguageModelEnabled
isApplyingConfiguration = false isApplyingConfiguration = false
} }
@@ -255,6 +296,55 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
if postConfigChanged { if postConfigChanged {
AppGroupConfigDarwin.postConfigChanged() AppGroupConfigDarwin.postConfigChanged()
} }
scheduleSettingsCloudPushIfEnabled()
}
/// Re-read App Group defaults after a cloud pull updates the cache.
public func reloadFromPersistedStorage() {
var fresh = AppGroupConfiguration.load(fromAvailable: defaults)
// Keep the reboot-durable onboarding marker authoritative across cloud
// pulls, matching the resilience applied at init.
let freshOnboarding = fresh.hasCompletedOnboarding
let keychainOnboarding = Keychain.hasCompletedOnboarding()
OSGLog.config.info(
"[onboarding] reload: appGroup=\(freshOnboarding, privacy: .public), keychain=\(keychainOnboarding, privacy: .public)"
)
if freshOnboarding {
Keychain.setOnboardingCompleted(true)
} else if keychainOnboarding {
fresh.hasCompletedOnboarding = true
OSGLog.config.info("[onboarding] reload: App Group read false but Keychain true → restored to true")
}
isApplyingConfiguration = true
configuration = fresh
providerId = fresh.providerId
baseURL = fresh.baseURL
model = fresh.model
modeId = fresh.modeId
localeId = fresh.localeId
engineMode = fresh.engineMode
hasCompletedOnboarding = fresh.hasCompletedOnboarding
onboardingPage = fresh.onboardingPage
hasAcknowledgedCloudSharing = fresh.hasAcknowledgedCloudSharing
uiLanguage = fresh.uiLanguage
translationTargetLocaleId = fresh.translationTargetLocaleId
handednessPreference = fresh.handednessPreference
cursorDragNavigationEnabled = fresh.cursorDragNavigationEnabled
polishIntensity = fresh.polishIntensity
flowSkipAppSwitch = fresh.flowSkipAppSwitch
flowInactivityDuration = fresh.flowInactivityDuration
localASRCustomLanguageModelEnabled = fresh.localASRCustomLanguageModelEnabled
isSyncingProviderAPIKey = true
apiKey = fresh.apiKey
isSyncingProviderAPIKey = false
isApplyingConfiguration = false
}
private func scheduleSettingsCloudPushIfEnabled() {
guard configuration.settingsICloudSyncEnabled else { return }
Task { @MainActor in
try? await SettingsCloudSync.shared.pushLocalIfEnabled()
}
} }
public func apply(preset: LLMProvider) { public func apply(preset: LLMProvider) {
@@ -284,11 +374,13 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
apiKey = "" apiKey = ""
model = preset.defaultModel model = preset.defaultModel
handednessPreference = .left handednessPreference = .left
localASRCustomLanguageModelEnabled = true
hasAcknowledgedCloudSharing = false hasAcknowledgedCloudSharing = false
configuration.providerId = preset.id configuration.providerId = preset.id
configuration.baseURL = preset.defaultBaseURL configuration.baseURL = preset.defaultBaseURL
configuration.model = preset.defaultModel configuration.model = preset.defaultModel
configuration.handednessPreference = .left configuration.handednessPreference = .left
configuration.localASRCustomLanguageModelEnabled = true
configuration.hasAcknowledgedCloudSharing = false configuration.hasAcknowledgedCloudSharing = false
isApplyingConfiguration = false isApplyingConfiguration = false
persistConfiguration() persistConfiguration()
@@ -0,0 +1,107 @@
// SyncedAppSettings.swift
// OSGKeyboard · Shared
//
// User-facing app settings mirrored through iCloud KVS. Excludes
// device-local state (onboarding progress, detected app context,
// personal dictionary blob, and API keys in Keychain).
import Foundation
public struct SyncedAppSettings: Codable, Sendable, Equatable {
public var updatedAt: Date
public var providerId: String
public var baseURL: String
public var model: String
public var modeId: String
public var localeId: String
public var engineMode: String
public var hasAcknowledgedCloudSharing: Bool
public var uiLanguage: AppUILanguage
public var translationTargetLocaleId: String
public var handednessPreference: HandednessPreference
public var cursorDragNavigationEnabled: Bool
public var polishIntensity: PolishIntensity
public var flowSkipAppSwitch: Bool
public var flowInactivityDuration: FlowInactivityDuration
public init(
updatedAt: Date = Date(),
providerId: String,
baseURL: String,
model: String,
modeId: String,
localeId: String,
engineMode: String,
hasAcknowledgedCloudSharing: Bool,
uiLanguage: AppUILanguage,
translationTargetLocaleId: String,
handednessPreference: HandednessPreference,
cursorDragNavigationEnabled: Bool,
polishIntensity: PolishIntensity,
flowSkipAppSwitch: Bool,
flowInactivityDuration: FlowInactivityDuration
) {
self.updatedAt = updatedAt
self.providerId = providerId
self.baseURL = baseURL
self.model = model
self.modeId = modeId
self.localeId = localeId
self.engineMode = engineMode
self.hasAcknowledgedCloudSharing = hasAcknowledgedCloudSharing
self.uiLanguage = uiLanguage
self.translationTargetLocaleId = translationTargetLocaleId
self.handednessPreference = handednessPreference
self.cursorDragNavigationEnabled = cursorDragNavigationEnabled
self.polishIntensity = polishIntensity
self.flowSkipAppSwitch = flowSkipAppSwitch
self.flowInactivityDuration = flowInactivityDuration
}
}
public extension SyncedAppSettings {
/// Build a cloud payload from the current App Group configuration.
static func from(configuration: AppGroupConfiguration, updatedAt: Date = Date()) -> SyncedAppSettings {
SyncedAppSettings(
updatedAt: updatedAt,
providerId: configuration.providerId,
baseURL: configuration.baseURL,
model: configuration.model,
modeId: configuration.modeId,
localeId: configuration.localeId,
engineMode: configuration.engineMode,
hasAcknowledgedCloudSharing: configuration.hasAcknowledgedCloudSharing,
uiLanguage: configuration.uiLanguage,
translationTargetLocaleId: configuration.translationTargetLocaleId,
handednessPreference: configuration.handednessPreference,
cursorDragNavigationEnabled: configuration.cursorDragNavigationEnabled,
polishIntensity: configuration.polishIntensity,
flowSkipAppSwitch: configuration.flowSkipAppSwitch,
flowInactivityDuration: configuration.flowInactivityDuration
)
}
/// Apply syncable fields onto a configuration, preserving device-local
/// fields such as onboarding progress and the personal dictionary.
func applying(to configuration: inout AppGroupConfiguration) {
configuration.providerId = providerId
configuration.baseURL = baseURL
configuration.model = model
configuration.modeId = modeId
configuration.localeId = localeId
configuration.engineMode = engineMode
configuration.hasAcknowledgedCloudSharing = hasAcknowledgedCloudSharing
configuration.uiLanguage = uiLanguage
configuration.translationTargetLocaleId = translationTargetLocaleId
configuration.handednessPreference = handednessPreference
configuration.cursorDragNavigationEnabled = cursorDragNavigationEnabled
configuration.polishIntensity = polishIntensity
configuration.flowSkipAppSwitch = flowSkipAppSwitch
configuration.flowInactivityDuration = flowInactivityDuration
}
/// Last-write-wins merge for whole settings blobs.
static func merge(local: SyncedAppSettings, remote: SyncedAppSettings) -> SyncedAppSettings {
remote.updatedAt >= local.updatedAt ? remote : local
}
}
+123 -34
View File
@@ -198,28 +198,30 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
func warmup(locale: Locale) async { func warmup(locale: Locale) async {
guard let resolvedLocale = await DictationTranscriber.supportedLocale(equivalentTo: locale) else { guard let resolvedLocale = await DictationTranscriber.supportedLocale(equivalentTo: locale) else {
Self.debug("warmup locale unsupported requested=\(locale.identifier(.bcp47))")
return return
} }
let localeID = resolvedLocale.identifier(.bcp47) let localeID = resolvedLocale.identifier(.bcp47)
let cachedLocaleID = lock.withLock { chunkPreparedLocaleID } let cachedLocaleID = lock.withLock { chunkPreparedLocaleID }
if cachedLocaleID == localeID, lock.withLock({ chunkAnalyzerFormat != nil }) { if cachedLocaleID == localeID, lock.withLock({ chunkAnalyzerFormat != nil }) {
Self.debug("warmup cache hit locale=\(localeID)")
return return
} }
let lmConfiguration = CustomLanguageModelManager.shared.configurationForTranscription( let setup = Self.makeDiagnosticTranscriber(locale: resolvedLocale)
locale: resolvedLocale Self.debug(
) "warmup start locale=\(localeID) customLMEnabled=\(setup.customLanguageModelEnabled) " +
let transcriber = CustomLanguageModelManager.makeDictationTranscriber( "customLMAttached=\(setup.usesCustomLanguageModel) " +
locale: resolvedLocale, "clmState=\(Self.describeCLMState(setup.clmState))"
lmConfiguration: lmConfiguration
) )
do { do {
try await Self.prepareAssetsIfNeeded(for: transcriber, locale: resolvedLocale) try await Self.prepareAssetsIfNeeded(for: setup.transcriber, locale: resolvedLocale)
guard let format = await SpeechAnalyzer.bestAvailableAudioFormat( guard let format = await SpeechAnalyzer.bestAvailableAudioFormat(
compatibleWith: [transcriber], compatibleWith: [setup.transcriber],
considering: Self.captureFormat considering: Self.captureFormat
) else { ) else {
Self.debug("warmup format unsupported locale=\(localeID)")
return return
} }
lock.withLock { lock.withLock {
@@ -236,13 +238,25 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
guard !samples.isEmpty else { return .success("") } guard !samples.isEmpty else { return .success("") }
if Task.isCancelled { return .cancelled } if Task.isCancelled { return .cancelled }
let startedAt = Date()
let rms = Self.rms(of: samples)
Self.debug(
"chunk start samples=\(samples.count) rms=\(String(format: "%.4f", rms)) " +
"locale=\(locale.identifier(.bcp47))"
)
do { do {
let text = try await transcribeSamples(samples, locale: locale, reuseChunkPrep: true) let text = try await transcribeSamples(samples, locale: locale, reuseChunkPrep: true)
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
Self.debug(
"chunk success textLen=\(trimmed.count) elapsed=\(Self.elapsed(startedAt))s " +
"empty=\(trimmed.isEmpty)"
)
return trimmed.isEmpty ? .success("") : .success(trimmed) return trimmed.isEmpty ? .success("") : .success(trimmed)
} catch is CancellationError { } catch is CancellationError {
Self.debug("chunk cancelled elapsed=\(Self.elapsed(startedAt))s")
return .cancelled return .cancelled
} catch { } catch {
Self.debug("chunk failed elapsed=\(Self.elapsed(startedAt))s error=\(error.localizedDescription)")
return .failure(error.localizedDescription) return .failure(error.localizedDescription)
} }
} }
@@ -257,12 +271,11 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
throw ASRChunkError.localeUnsupported throw ASRChunkError.localeUnsupported
} }
let localeID = resolvedLocale.identifier(.bcp47) let localeID = resolvedLocale.identifier(.bcp47)
let lmConfiguration = CustomLanguageModelManager.shared.configurationForTranscription( let setup = Self.makeDiagnosticTranscriber(locale: resolvedLocale)
locale: resolvedLocale Self.debug(
) "chunk setup locale=\(localeID) customLMEnabled=\(setup.customLanguageModelEnabled) " +
let transcriber = CustomLanguageModelManager.makeDictationTranscriber( "customLMAttached=\(setup.usesCustomLanguageModel) " +
locale: resolvedLocale, "clmState=\(Self.describeCLMState(setup.clmState))"
lmConfiguration: lmConfiguration
) )
let analyzerFormat: AVAudioFormat let analyzerFormat: AVAudioFormat
@@ -271,10 +284,14 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
cachedPrep.0 == localeID, cachedPrep.0 == localeID,
let cached = cachedPrep.1 { let cached = cachedPrep.1 {
analyzerFormat = cached analyzerFormat = cached
Self.debug(
"chunk using cached analyzer format sr=\(Int(cached.sampleRate)) " +
"channels=\(cached.channelCount) common=\(cached.commonFormat.rawValue)"
)
} else { } else {
try await Self.prepareAssetsIfNeeded(for: transcriber, locale: resolvedLocale) try await Self.prepareAssetsIfNeeded(for: setup.transcriber, locale: resolvedLocale)
guard let format = await SpeechAnalyzer.bestAvailableAudioFormat( guard let format = await SpeechAnalyzer.bestAvailableAudioFormat(
compatibleWith: [transcriber], compatibleWith: [setup.transcriber],
considering: Self.captureFormat considering: Self.captureFormat
) else { ) else {
throw ASRChunkError.formatUnsupported throw ASRChunkError.formatUnsupported
@@ -284,6 +301,10 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
chunkPreparedLocaleID = localeID chunkPreparedLocaleID = localeID
chunkAnalyzerFormat = format chunkAnalyzerFormat = format
} }
Self.debug(
"chunk prepared analyzer format sr=\(Int(format.sampleRate)) " +
"channels=\(format.channelCount) common=\(format.commonFormat.rawValue)"
)
} }
let snapshot = AudioBufferSnapshot(samples: samples, sampleRate: 16_000) let snapshot = AudioBufferSnapshot(samples: samples, sampleRate: 16_000)
@@ -291,12 +312,12 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
throw ASRChunkError.formatUnsupported throw ASRChunkError.formatUnsupported
} }
let analyzer = SpeechAnalyzer(modules: [transcriber]) let analyzer = SpeechAnalyzer(modules: [setup.transcriber])
try await analyzer.prepareToAnalyze(in: analyzerFormat) try await analyzer.prepareToAnalyze(in: analyzerFormat)
let resultsTask = Task<String, Error> { let resultsTask = Task<String, Error> {
var accumulator = ProgressiveDictationTranscriptAccumulator() var accumulator = ProgressiveDictationTranscriptAccumulator()
for try await result in transcriber.results { for try await result in setup.transcriber.results {
if Task.isCancelled { break } if Task.isCancelled { break }
let text = String(result.text.characters) let text = String(result.text.characters)
_ = accumulator.ingest(range: result.range, text: text) _ = accumulator.ingest(range: result.range, text: text)
@@ -369,15 +390,15 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
} }
// Each pipelined chunk is 30 s; long dictation preset keeps a // Each pipelined chunk is 30 s; long dictation preset keeps a
// single chunk coherent (Flow utterances run up to 3 min). // single chunk coherent (Flow utterances run up to 3 min).
let lmConfiguration = CustomLanguageModelManager.shared.configurationForTranscription( let setup = Self.makeDiagnosticTranscriber(locale: resolvedLocale)
locale: resolvedLocale Self.debug(
) "stream setup locale=\(resolvedLocale.identifier(.bcp47)) " +
let transcriber = CustomLanguageModelManager.makeDictationTranscriber( "customLMEnabled=\(setup.customLanguageModelEnabled) " +
locale: resolvedLocale, "customLMAttached=\(setup.usesCustomLanguageModel) " +
lmConfiguration: lmConfiguration "clmState=\(Self.describeCLMState(setup.clmState))"
) )
do { do {
try await Self.prepareAssetsIfNeeded(for: transcriber, locale: resolvedLocale) try await Self.prepareAssetsIfNeeded(for: setup.transcriber, locale: resolvedLocale)
} catch { } catch {
Self.debug("asset prepare failed: \(error.localizedDescription)") Self.debug("asset prepare failed: \(error.localizedDescription)")
continuation.yield(.error(SharedL10n.string("error.asr.assetsNotReady"))) continuation.yield(.error(SharedL10n.string("error.asr.assetsNotReady")))
@@ -385,11 +406,11 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
return return
} }
let newAnalyzer = SpeechAnalyzer(modules: [transcriber]) let newAnalyzer = SpeechAnalyzer(modules: [setup.transcriber])
self.lock.withLock { self.analyzer = newAnalyzer } self.lock.withLock { self.analyzer = newAnalyzer }
guard let analyzerFormat = await SpeechAnalyzer.bestAvailableAudioFormat( guard let analyzerFormat = await SpeechAnalyzer.bestAvailableAudioFormat(
compatibleWith: [transcriber], compatibleWith: [setup.transcriber],
considering: Self.captureFormat considering: Self.captureFormat
) else { ) else {
continuation.yield(.error(SharedL10n.string("error.asr.formatUnsupported"))) continuation.yield(.error(SharedL10n.string("error.asr.formatUnsupported")))
@@ -405,7 +426,7 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
// while `analyzeSequence` drains the input stream. // while `analyzeSequence` drains the input stream.
let resultsTask = Task<String, Error> { let resultsTask = Task<String, Error> {
var accumulator = ProgressiveDictationTranscriptAccumulator() var accumulator = ProgressiveDictationTranscriptAccumulator()
for try await result in transcriber.results { for try await result in setup.transcriber.results {
if Task.isCancelled { break } if Task.isCancelled { break }
let text = String(result.text.characters) let text = String(result.text.characters)
guard let full = accumulator.ingest(range: result.range, text: text) else { guard let full = accumulator.ingest(range: result.range, text: text) else {
@@ -457,23 +478,91 @@ final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
} }
} }
private struct DiagnosticTranscriber {
let transcriber: DictationTranscriber
let customLanguageModelEnabled: Bool
let usesCustomLanguageModel: Bool
let clmState: CustomLanguageModelManager.PrepareState
}
private static func makeDiagnosticTranscriber(locale: Locale) -> DiagnosticTranscriber {
let defaults = AppGroup.defaultsIfAvailable
let clmKey = AppGroupConfiguration.Keys.localASRCustomLanguageModelEnabled
let clmEnabled = defaults?.object(forKey: clmKey) == nil
? true
: (defaults?.bool(forKey: clmKey) ?? true)
let clmState = CustomLanguageModelManager.shared.currentState()
let lmConfiguration = clmEnabled
? CustomLanguageModelManager.shared.configurationForTranscription(locale: locale)
: nil
let transcriber = CustomLanguageModelManager.makeDictationTranscriber(
locale: locale,
lmConfiguration: lmConfiguration
)
return DiagnosticTranscriber(
transcriber: transcriber,
customLanguageModelEnabled: clmEnabled,
usesCustomLanguageModel: lmConfiguration != nil,
clmState: clmState
)
}
private static func describeCLMState(_ state: CustomLanguageModelManager.PrepareState) -> String {
switch state {
case .idle:
return "idle"
case .preparing:
return "preparing"
case .ready:
return "ready"
case .failed(let message):
return "failed(\(message))"
}
}
private static func rms(of samples: [Float]) -> Float {
guard !samples.isEmpty else { return 0 }
var sum: Float = 0
for sample in samples {
sum += sample * sample
}
return sqrtf(sum / Float(samples.count))
}
private static func elapsed(_ start: Date) -> String {
String(format: "%.2f", Date().timeIntervalSince(start))
}
private static func debug(_ message: String) { private static func debug(_ message: String) {
#if DEBUG OSGLog.asr.info("\(message, privacy: .public)")
print("🎙️[ASRService] \(message)")
#endif
} }
private static func prepareAssetsIfNeeded( private static func prepareAssetsIfNeeded(
for transcriber: DictationTranscriber, for transcriber: DictationTranscriber,
locale: Locale locale: Locale
) async throws { ) async throws {
let localeID = locale.identifier(.bcp47)
let startedAt = Date()
do { do {
_ = try await AssetInventory.reserve(locale: locale) _ = try await AssetInventory.reserve(locale: locale)
Self.debug("asset reserve ok locale=\(localeID)")
} catch { } catch {
// Reservation may already exist or slots are full; continue. // Reservation may already exist or slots are full; continue, but
// log it so local-ASR setup failures are not hidden behind a later
// "no speech" timeout.
Self.debug("asset reserve non-fatal locale=\(localeID) error=\(error.localizedDescription)")
} }
if let request = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) { do {
try await request.downloadAndInstall() if let request = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) {
Self.debug("asset install required locale=\(localeID)")
try await request.downloadAndInstall()
Self.debug("asset install done locale=\(localeID) elapsed=\(elapsed(startedAt))s")
} else {
Self.debug("asset already installed locale=\(localeID) elapsed=\(elapsed(startedAt))s")
}
} catch {
Self.debug("asset prepare failed locale=\(localeID) elapsed=\(elapsed(startedAt))s error=\(error.localizedDescription)")
throw error
} }
} }
@@ -57,6 +57,7 @@ public struct AppGroupStore: @unchecked Sendable {
public var polishModeForPipeline: PolishingService.PolishMode { configuration.polishModeForPipeline } public var polishModeForPipeline: PolishingService.PolishMode { configuration.polishModeForPipeline }
public var polishProviderIdOverride: String? { configuration.polishProviderIdOverride } public var polishProviderIdOverride: String? { configuration.polishProviderIdOverride }
public var isCloudAPIKeyMissingForVoiceInput: Bool { configuration.isCloudAPIKeyMissingForVoiceInput } public var isCloudAPIKeyMissingForVoiceInput: Bool { configuration.isCloudAPIKeyMissingForVoiceInput }
public var localASRCustomLanguageModelEnabled: Bool { configuration.localASRCustomLanguageModelEnabled }
/// Whether the keyboard top-bar translation chip should render. /// Whether the keyboard top-bar translation chip should render.
public var isTranslationChipVisible: Bool { true } public var isTranslationChipVisible: Bool { true }
@@ -112,6 +113,10 @@ public struct AppGroupStore: @unchecked Sendable {
mutateConfiguration { $0.polishIntensity = intensity } mutateConfiguration { $0.polishIntensity = intensity }
} }
public func setLocalASRCustomLanguageModelEnabled(_ enabled: Bool) {
mutateConfiguration { $0.localASRCustomLanguageModelEnabled = enabled }
}
public var hasCompletedOnboarding: Bool { public var hasCompletedOnboarding: Bool {
get { configuration.hasCompletedOnboarding } get { configuration.hasCompletedOnboarding }
set { setHasCompletedOnboarding(newValue) } set { setHasCompletedOnboarding(newValue) }
@@ -129,6 +134,8 @@ public struct AppGroupStore: @unchecked Sendable {
config.onboardingPage = 0 config.onboardingPage = 0
} }
} }
// Mirror to the reboot-durable Keychain marker (keyboard-side completion).
Keychain.setOnboardingCompleted(completed)
} }
public func setOnboardingPage(_ page: Int) { public func setOnboardingPage(_ page: Int) {
@@ -167,6 +174,22 @@ public struct AppGroupStore: @unchecked Sendable {
mutateConfiguration { $0.personalDictionaryICloudSyncEnabled = enabled } mutateConfiguration { $0.personalDictionaryICloudSyncEnabled = enabled }
} }
public var settingsICloudSyncEnabled: Bool {
get { configuration.settingsICloudSyncEnabled }
set { setSettingsICloudSyncEnabled(newValue) }
}
public func setSettingsICloudSyncEnabled(_ enabled: Bool) {
mutateConfiguration { $0.settingsICloudSyncEnabled = enabled }
}
/// Timestamp of the last settings blob applied from iCloud KVS.
public var settingsCloudUpdatedAt: Date? {
let raw = defaults.double(forKey: AppGroupConfiguration.Keys.settingsCloudUpdatedAt)
guard raw > 0 else { return nil }
return Date(timeIntervalSince1970: raw)
}
// MARK: - Client // MARK: - Client
public func makeClient() -> LLMClient { public func makeClient() -> LLMClient {
@@ -123,6 +123,48 @@ private final class FlowLevelStore: @unchecked Sendable {
} }
} }
/// Route-adaptive downsampling converter, safe to call from the realtime tap.
///
/// `AVAudioEngine.installTap(format:)` traps with an **uncatchable** NSException
/// when the format passed to it does not match the input node's *live* format.
/// After an audio-route change which the on-device `SpeechAnalyzer` triggers
/// during warmup by reconfiguring the shared `AVAudioSession` the value
/// returned by `inputNode.outputFormat(forBus:)` can lag behind the real
/// hardware rate (e.g. it reports 48 kHz while the node has already switched to
/// 24 kHz). Installing a tap with that stale explicit format crashes the whole
/// app (`Failed to create tap due to format mismatch`).
///
/// We therefore install the tap with `format: nil` (which always uses the
/// node's live format) and rebuild the sample-rate converter *here* whenever the
/// incoming buffer's format actually changes, so downsampling to the ASR target
/// rate is always valid regardless of route churn.
private final class AdaptiveDownsampler: @unchecked Sendable {
// `AVAudioConverter` / `AVAudioFormat` are not `Sendable`, so the state and
// the returned converter are guarded manually via the unchecked lock APIs.
private let lock = OSAllocatedUnfairLock<(converter: AVAudioConverter, source: AVAudioFormat)?>(uncheckedState: nil)
let targetFormat: AVAudioFormat
init(targetFormat: AVAudioFormat) {
self.targetFormat = targetFormat
}
/// Returns a converter valid for `sourceFormat`, rebuilding it lazily when
/// the hardware route (and thus the buffer format) changes.
func converter(for sourceFormat: AVAudioFormat) -> AVAudioConverter? {
lock.withLockUnchecked { state in
if let state, state.source == sourceFormat {
return state.converter
}
guard let converter = AVAudioConverter(from: sourceFormat, to: targetFormat) else {
state = nil
return nil
}
state = (converter, sourceFormat)
return converter
}
}
}
@MainActor @MainActor
public final class FlowContinuousCapture { public final class FlowContinuousCapture {
@@ -169,16 +211,18 @@ public final class FlowContinuousCapture {
private let drainTracker = FlowCaptureDrainTracker() private let drainTracker = FlowCaptureDrainTracker()
private let tailSampleCounter = OSAllocatedUnfairLock(initialState: 0) private let tailSampleCounter = OSAllocatedUnfairLock(initialState: 0)
private var audioConverter: AVAudioConverter? private var downsampler: AdaptiveDownsampler?
private var targetFormat: AVAudioFormat? private var targetFormat: AVAudioFormat?
private var hwFormat: AVAudioFormat? private var hwFormat: AVAudioFormat?
private var drainPolicy = FlowCaptureTailDrainPolicy.flowDefault private var drainPolicy = FlowCaptureTailDrainPolicy.flowDefault
private var didInstallTap = false private var didInstallTap = false
private var isRunning = false private var isRunning = false
private var isRebuilding = false
private var routeObserver: NSObjectProtocol? private var routeObserver: NSObjectProtocol?
private var interruptionObserver: NSObjectProtocol? private var interruptionObserver: NSObjectProtocol?
private var mediaResetObserver: NSObjectProtocol?
private let log = Logger(subsystem: "com.osgkeyboard.shared", category: "FlowCapture") private let log = Logger(subsystem: "com.osgkeyboard.shared", category: "FlowCapture")
public init() {} public init() {}
@@ -227,11 +271,11 @@ public final class FlowContinuousCapture {
) else { ) else {
throw StartError.formatCreateFailed throw StartError.formatCreateFailed
} }
guard let converter = AVAudioConverter(from: hardwareFormat, to: resolvedTargetFormat) else {
throw StartError.converterCreateFailed
}
audioConverter = converter // Route-adaptive converter: it rebuilds itself from the live buffer
// format inside the tap, so it never assumes a fixed hardware rate.
let downsampler = AdaptiveDownsampler(targetFormat: resolvedTargetFormat)
self.downsampler = downsampler
targetFormat = resolvedTargetFormat targetFormat = resolvedTargetFormat
hwFormat = hardwareFormat hwFormat = hardwareFormat
@@ -249,9 +293,7 @@ public final class FlowContinuousCapture {
let tailCounter = tailSampleCounter let tailCounter = tailSampleCounter
let policy = drainPolicy let policy = drainPolicy
let tap = Self.makeAudioTapBlock( let tap = Self.makeAudioTapBlock(
converter: converter, downsampler: downsampler,
targetFormat: resolvedTargetFormat,
hwFormat: hardwareFormat,
gate: gateLock, gate: gateLock,
levelStore: levels, levelStore: levels,
prerollStore: preroll, prerollStore: preroll,
@@ -260,7 +302,10 @@ public final class FlowContinuousCapture {
tailSampleCounter: tailCounter, tailSampleCounter: tailCounter,
drainPolicy: policy drainPolicy: policy
) )
inputNode.installTap(onBus: 0, bufferSize: 4096, format: hardwareFormat, block: tap) // `format: nil` binds the tap to the input node's *live* format. Passing
// an explicit (possibly stale) format here is what crashed the app on a
// route change (48 kHz client vs 24 kHz hardware); nil can never mismatch.
inputNode.installTap(onBus: 0, bufferSize: 4096, format: nil, block: tap)
didInstallTap = true didInstallTap = true
audioEngine.prepare() audioEngine.prepare()
@@ -287,7 +332,7 @@ public final class FlowContinuousCapture {
audioEngine.stop() audioEngine.stop()
} }
isRunning = false isRunning = false
audioConverter = nil downsampler = nil
targetFormat = nil targetFormat = nil
hwFormat = nil hwFormat = nil
try? AVAudioSession.sharedInstance().setActive( try? AVAudioSession.sharedInstance().setActive(
@@ -339,14 +384,35 @@ public final class FlowContinuousCapture {
} }
} }
} }
// Apple QA1749: when the system media server resets, the engine,
// converter and audio session all become orphaned and must be
// rebuilt from scratch otherwise capture silently produces no
// audio (another cause of "waveform moves but ASR is empty").
if mediaResetObserver == nil {
mediaResetObserver = center.addObserver(
forName: AVAudioSession.mediaServicesWereResetNotification,
object: nil,
queue: .main
) { [weak self] _ in
MainActor.assumeIsolated { self?.handleMediaServicesReset() }
}
}
} }
private func removeSessionObservers() { private func removeSessionObservers() {
let center = NotificationCenter.default let center = NotificationCenter.default
if let routeObserver { center.removeObserver(routeObserver) } if let routeObserver { center.removeObserver(routeObserver) }
if let interruptionObserver { center.removeObserver(interruptionObserver) } if let interruptionObserver { center.removeObserver(interruptionObserver) }
if let mediaResetObserver { center.removeObserver(mediaResetObserver) }
routeObserver = nil routeObserver = nil
interruptionObserver = nil interruptionObserver = nil
mediaResetObserver = nil
}
private func handleMediaServicesReset() {
guard isRunning else { return }
log.info("Media services were reset — rebuilding engine and converter")
rebuildEngine()
} }
private func handleRouteChange(reasonRaw: UInt?) { private func handleRouteChange(reasonRaw: UInt?) {
@@ -388,7 +454,9 @@ public final class FlowContinuousCapture {
/// Stop and rebuild the engine against the current route, keeping /// Stop and rebuild the engine against the current route, keeping
/// `isRunning` intact so the session survives the swap transparently. /// `isRunning` intact so the session survives the swap transparently.
private func rebuildEngine() { private func rebuildEngine() {
guard isRunning else { return } guard isRunning, !isRebuilding else { return }
isRebuilding = true
defer { isRebuilding = false }
if audioEngine.isRunning { if audioEngine.isRunning {
audioEngine.stop() audioEngine.stop()
} }
@@ -436,9 +504,15 @@ public final class FlowContinuousCapture {
try? await Task.sleep(nanoseconds: FlowCaptureConstants.drainPollIntervalNs) try? await Task.sleep(nanoseconds: FlowCaptureConstants.drainPollIntervalNs)
} }
let flushSamples = flushConverterTailToStream() // NOTE: We intentionally do NOT signal `.endOfStream` to the shared
tailSampleCounter.withLock { $0 += flushSamples } // downsampling converter here. `AVAudioConverter` is stateful: once its
// input block returns `.endOfStream`, the converter is permanently
// finished and every subsequent `.haveData` conversion (from the live
// tap) returns no data which silently starved every utterance after
// the first (Apple docs + AVAudioConverter reuse guidance). Trailing
// speech is already preserved by the live `.draining` forwarding loop
// above; the converter's sub-millisecond internal filter tail is not
// worth poisoning a session-long converter for.
streamRelay.finish() streamRelay.finish()
gate.withLock { $0 = .idle } gate.withLock { $0 = .idle }
@@ -466,58 +540,10 @@ public final class FlowContinuousCapture {
levelStore.snapshot() levelStore.snapshot()
} }
// MARK: - Converter flush
@discardableResult
private func flushConverterTailToStream() -> Int {
guard let converter = audioConverter,
let targetFormat,
let hwFormat else {
return 0
}
var flushedSamples = 0
let capacity = AVAudioFrameCount(max(512, hwFormat.sampleRate / 20))
guard let outBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: capacity) else {
return 0
}
var endOfStreamSignaled = false
while true {
outBuffer.frameLength = 0
var error: NSError?
let status = converter.convert(to: outBuffer, error: &error) { _, outStatus in
if endOfStreamSignaled {
outStatus.pointee = .noDataNow
return nil
}
endOfStreamSignaled = true
outStatus.pointee = .endOfStream
return nil
}
if status == .error || error != nil {
break
}
guard status == .haveData, outBuffer.frameLength > 0 else {
break
}
let snapshot = AudioBufferSnapshot(buffer: outBuffer)
guard !snapshot.samples.isEmpty else { break }
streamRelay.yield(snapshot)
flushedSamples += snapshot.samples.count
}
return flushedSamples
}
// MARK: - Audio tap (nonisolated runs on realtime thread) // MARK: - Audio tap (nonisolated runs on realtime thread)
private nonisolated static func makeAudioTapBlock( private nonisolated static func makeAudioTapBlock(
converter: AVAudioConverter, downsampler: AdaptiveDownsampler,
targetFormat: AVAudioFormat,
hwFormat: AVAudioFormat,
gate: OSAllocatedUnfairLock<UtteranceGatePhase>, gate: OSAllocatedUnfairLock<UtteranceGatePhase>,
levelStore: FlowLevelStore, levelStore: FlowLevelStore,
prerollStore: FlowPrerollStore, prerollStore: FlowPrerollStore,
@@ -529,8 +555,15 @@ public final class FlowContinuousCapture {
return { buffer, _ in return { buffer, _ in
levelStore.update(from: buffer, barCount: FlowCaptureConstants.levelBarCount) levelStore.update(from: buffer, barCount: FlowCaptureConstants.levelBarCount)
// Derive the converter from the *live* buffer format so a mid-session
// route change (e.g. 48 kHz 24 kHz) is handled transparently.
let sourceFormat = buffer.format
let targetFormat = downsampler.targetFormat
guard sourceFormat.sampleRate > 0,
let converter = downsampler.converter(for: sourceFormat) else { return }
let outFrames = AVAudioFrameCount( let outFrames = AVAudioFrameCount(
Double(buffer.frameLength) * targetFormat.sampleRate / hwFormat.sampleRate Double(buffer.frameLength) * targetFormat.sampleRate / sourceFormat.sampleRate
) )
guard outFrames > 0, guard outFrames > 0,
let outBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: outFrames) let outBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: outFrames)
@@ -117,9 +117,9 @@ public enum FlowSessionBridge {
// MARK: - Session validity (keyboard) // MARK: - Session validity (keyboard)
/// True when the session contract is still valid (not expired). /// True when the App Group session contract is still valid (not expired).
/// Does not require a fresh heartbeat the host may be suspended in /// Does **not** mean the host process is alive use `isHostReachable()` for
/// background while the continuous audio session is frozen. /// recording gates and "session ready" UI.
public static func isSessionActive(defaults: UserDefaults? = nil) -> Bool { public static func isSessionActive(defaults: UserDefaults? = nil) -> Bool {
let store = resolvedDefaults(defaults) let store = resolvedDefaults(defaults)
guard store.bool(forKey: FlowSessionKeys.flowSessionActive) else { return false } guard store.bool(forKey: FlowSessionKeys.flowSessionActive) else { return false }
@@ -128,19 +128,47 @@ public enum FlowSessionBridge {
return expires > Date().timeIntervalSince1970 return expires > Date().timeIntervalSince1970
} }
/// Seconds since the host last wrote `flowHeartbeat`; nil when never written.
public static func heartbeatStaleness(defaults: UserDefaults? = nil) -> TimeInterval? {
let store = resolvedDefaults(defaults)
let heartbeat = store.double(forKey: FlowSessionKeys.flowHeartbeat)
guard heartbeat > 0 else { return nil }
return Date().timeIntervalSince1970 - heartbeat
}
/// True when the host app recently wrote a heartbeat (foreground or /// True when the host app recently wrote a heartbeat (foreground or
/// actively processing). Used for auto-start heuristics, not gating record. /// actively processing). Gating record / "session ready" UI must use this,
/// not `isSessionActive()` alone.
public static func isHostReachable(defaults: UserDefaults? = nil) -> Bool { public static func isHostReachable(defaults: UserDefaults? = nil) -> Bool {
let store = resolvedDefaults(defaults) let store = resolvedDefaults(defaults)
guard isSessionActive(defaults: store) else { return false } guard isSessionActive(defaults: store) else { return false }
guard let staleness = heartbeatStaleness(defaults: store) else { return false }
let heartbeat = store.double(forKey: FlowSessionKeys.flowHeartbeat)
guard heartbeat > 0 else { return false }
let staleness = Date().timeIntervalSince1970 - heartbeat
return staleness <= FlowSessionKeys.heartbeatStaleInterval return staleness <= FlowSessionKeys.heartbeatStaleInterval
} }
/// True when the session contract flag is still set but the host heartbeat
/// proves the process is gone (reboot, force-quit, long suspend).
public static func isHostStale(
staleAfter: TimeInterval = FlowSessionKeys.heartbeatZombieInterval,
defaults: UserDefaults? = nil
) -> Bool {
let store = resolvedDefaults(defaults)
guard isSessionActive(defaults: store) else { return false }
guard let staleness = heartbeatStaleness(defaults: store) else { return true }
return staleness > staleAfter
}
/// Clears orphaned App Group Flow state when the host is provably dead.
@discardableResult
public static func clearIfHostStale(
staleAfter: TimeInterval = FlowSessionKeys.heartbeatZombieInterval,
defaults: UserDefaults? = nil
) -> Bool {
guard isHostStale(staleAfter: staleAfter, defaults: defaults) else { return false }
clearFlowState(defaults: defaults)
return true
}
public static func sessionExpiresAt(defaults: UserDefaults? = nil) -> TimeInterval? { public static func sessionExpiresAt(defaults: UserDefaults? = nil) -> TimeInterval? {
let store = resolvedDefaults(defaults) let store = resolvedDefaults(defaults)
let expires = store.double(forKey: FlowSessionKeys.flowSessionExpires) let expires = store.double(forKey: FlowSessionKeys.flowSessionExpires)
@@ -26,9 +26,16 @@ public enum FlowSessionKeys {
/// Wall-clock timestamp of the last utterance completion or session start. /// Wall-clock timestamp of the last utterance completion or session start.
public static let lastActivityAt = "flow.lastActivityAt" public static let lastActivityAt = "flow.lastActivityAt"
/// Heartbeat older than this while the host is foreground likely killed. /// Heartbeat older than this host is not actively reachable for recording.
public static let heartbeatStaleInterval: TimeInterval = 3 public static let heartbeatStaleInterval: TimeInterval = 3
/// Session flag still set but heartbeat older than this host process is
/// dead (force-quit, reboot). Keyboard / host should clear persisted state.
public static let heartbeatZombieInterval: TimeInterval = 60
/// After mic stop, fail fast when the host heartbeat is gone longer than this.
public static let keyboardHostDisconnectFailFast: TimeInterval = 15
/// Legacy fixed session length prefer `FlowSessionPolicy.sessionDuration()`. /// Legacy fixed session length prefer `FlowSessionPolicy.sessionDuration()`.
public static let defaultSessionDuration: TimeInterval = 480 public static let defaultSessionDuration: TimeInterval = 480
@@ -0,0 +1,70 @@
// AppCloudSync.swift
// OSGKeyboard · Shared
//
// Single entry point for iCloud KVS sync in the main app: preferences
// toggles, settings payload, and personal dictionary.
import Foundation
@MainActor
public final class AppCloudSync {
public static let shared = AppCloudSync()
private let kvs: UbiquitousKeyValueStoreing
private let makeStore: () -> AppGroupStore
private let settingsSync: SettingsCloudSync
private let dictionarySync: PersonalDictionaryCloudSync
private var externalChangeObserver: NSObjectProtocol?
public init(
kvs: UbiquitousKeyValueStoreing = NSUbiquitousKeyValueStore.default,
makeStore: @escaping () -> AppGroupStore = { AppGroupStore() },
settingsSync: SettingsCloudSync? = nil,
dictionarySync: PersonalDictionaryCloudSync? = nil
) {
self.kvs = kvs
self.makeStore = makeStore
self.settingsSync = settingsSync ?? SettingsCloudSync(kvs: kvs, makeStore: makeStore)
self.dictionarySync = dictionarySync ?? PersonalDictionaryCloudSync(kvs: kvs, makeStore: makeStore)
}
public func startObservingExternalChanges() {
guard externalChangeObserver == nil else { return }
externalChangeObserver = NotificationCenter.default.addObserver(
forName: NSUbiquitousKeyValueStore.didChangeExternallyNotification,
object: nil,
queue: .main
) { [weak self] _ in
guard let self else { return }
Task { @MainActor in
await self.pullAllIfEnabled()
}
}
}
public func stopObservingExternalChanges() {
if let externalChangeObserver {
NotificationCenter.default.removeObserver(externalChangeObserver)
self.externalChangeObserver = nil
}
}
/// Launch / foreground: refresh KVS toggles, then pull payloads.
public func pullAllIfEnabled() async {
let store = makeStore()
ICloudSyncPreferences.migrateLegacyTogglesIfNeeded(kvs: kvs, store: store)
let toggles = ICloudSyncPreferences.load(from: kvs, store: store)
ICloudSyncPreferences.cacheToAppGroup(
settingsEnabled: toggles.settings,
dictionaryEnabled: toggles.dictionary,
store: store
)
await settingsSync.pullAndMergeIfEnabled()
await dictionarySync.pullAndMergeIfEnabled()
}
public var settingsSyncService: SettingsCloudSync { settingsSync }
public var dictionarySyncService: PersonalDictionaryCloudSync { dictionarySync }
}
@@ -0,0 +1,53 @@
// ICloudSyncPreferences.swift
// OSGKeyboard · Shared
//
// iCloud KVS is the source of truth for cross-device sync toggles
// (scheme A). App Group UserDefaults keeps a local cache so the
// keyboard extension and offline UI can read the last-known state.
import Foundation
public enum ICloudSyncPreferences {
public static let settingsEnabledKey = "iCloudSync.settingsEnabled"
public static let dictionaryEnabledKey = "personalDictionary.syncEnabled"
/// Read sync toggles from KVS, falling back to the App Group cache
/// when a key has not been uploaded yet.
public static func load(from kvs: UbiquitousKeyValueStoreing, store: AppGroupStore) -> (settings: Bool, dictionary: Bool) {
let settings = kvs.object(forKey: settingsEnabledKey) as? Bool
?? store.settingsICloudSyncEnabled
let dictionary = kvs.object(forKey: dictionaryEnabledKey) as? Bool
?? store.personalDictionaryICloudSyncEnabled
return (settings, dictionary)
}
/// Mirror KVS toggles into the App Group cache.
public static func cacheToAppGroup(
settingsEnabled: Bool,
dictionaryEnabled: Bool,
store: AppGroupStore
) {
store.setSettingsICloudSyncEnabled(settingsEnabled)
store.setPersonalDictionaryICloudSyncEnabled(dictionaryEnabled)
}
public static func pushSettingsEnabled(_ enabled: Bool, kvs: UbiquitousKeyValueStoreing) {
kvs.set(enabled, forKey: settingsEnabledKey)
_ = kvs.synchronize()
}
public static func pushDictionaryEnabled(_ enabled: Bool, kvs: UbiquitousKeyValueStoreing) {
kvs.set(enabled, forKey: dictionaryEnabledKey)
_ = kvs.synchronize()
}
/// One-time migration: upload locally cached toggles when KVS has no value yet.
public static func migrateLegacyTogglesIfNeeded(kvs: UbiquitousKeyValueStoreing, store: AppGroupStore) {
if kvs.object(forKey: dictionaryEnabledKey) == nil {
pushDictionaryEnabled(store.personalDictionaryICloudSyncEnabled, kvs: kvs)
}
if kvs.object(forKey: settingsEnabledKey) == nil {
pushSettingsEnabled(store.settingsICloudSyncEnabled, kvs: kvs)
}
}
}
@@ -0,0 +1,137 @@
// SettingsCloudSync.swift
// OSGKeyboard · Shared
//
// Mirrors user-facing app settings through iCloud KVS. API keys stay
// in Keychain and are never uploaded.
import Foundation
public extension Notification.Name {
/// Posted after remote settings are applied to the App Group cache.
static let settingsDidSyncFromCloud = Notification.Name(
"com.osgkeyboard.settings.didSyncFromCloud"
)
}
public enum SettingsCloudSyncError: Error, Equatable, Sendable {
case encodeFailed
case decodeFailed
}
@MainActor
public final class SettingsCloudSync {
public static let shared = SettingsCloudSync()
public static let kvsKey = "appSettings.v1"
private let kvs: UbiquitousKeyValueStoreing
private let makeStore: () -> AppGroupStore
public init(
kvs: UbiquitousKeyValueStoreing = NSUbiquitousKeyValueStore.default,
makeStore: @escaping () -> AppGroupStore = { AppGroupStore() }
) {
self.kvs = kvs
self.makeStore = makeStore
}
public func pullAndMergeIfEnabled() async {
let store = makeStore()
guard store.settingsICloudSyncEnabled else { return }
await pullAndMerge(store: store)
}
public func pushLocalIfEnabled() async throws {
let store = makeStore()
guard store.settingsICloudSyncEnabled else { return }
let local = SyncedAppSettings.from(configuration: store.configurationSnapshot())
try push(local)
}
public func enableSync() async throws {
let store = makeStore()
ICloudSyncPreferences.pushSettingsEnabled(true, kvs: kvs)
ICloudSyncPreferences.cacheToAppGroup(
settingsEnabled: true,
dictionaryEnabled: store.personalDictionaryICloudSyncEnabled,
store: store
)
let local = SyncedAppSettings.from(configuration: store.configurationSnapshot())
let remote = loadRemote() ?? local
let merged = SyncedAppSettings.merge(local: local, remote: remote)
apply(merged, to: store, postNotification: false)
try push(merged)
NotificationCenter.default.post(name: .settingsDidSyncFromCloud, object: nil)
}
public func disableSync() {
let store = makeStore()
ICloudSyncPreferences.pushSettingsEnabled(false, kvs: kvs)
store.setSettingsICloudSyncEnabled(false)
}
public func pullAndMerge(store: AppGroupStore) async {
guard store.settingsICloudSyncEnabled else { return }
guard let remote = loadRemote() else { return }
let local = SyncedAppSettings.from(
configuration: store.configurationSnapshot(),
updatedAt: store.settingsCloudUpdatedAt ?? .distantPast
)
let merged = SyncedAppSettings.merge(local: local, remote: remote)
guard merged != local else { return }
apply(merged, to: store, postNotification: true)
}
public func push(_ settings: SyncedAppSettings) throws {
let encoder = JSONEncoder()
encoder.dateEncodingStrategy = .iso8601
guard let data = try? encoder.encode(settings) else {
throw SettingsCloudSyncError.encodeFailed
}
kvs.set(data, forKey: Self.kvsKey)
_ = kvs.synchronize()
}
public func loadRemote() -> SyncedAppSettings? {
guard let data = kvs.data(forKey: Self.kvsKey) else { return nil }
return try? decode(data)
}
public func decode(_ data: Data) throws -> SyncedAppSettings {
let decoder = JSONDecoder()
decoder.dateDecodingStrategy = .iso8601
guard let settings = try? decoder.decode(SyncedAppSettings.self, from: data) else {
throw SettingsCloudSyncError.decodeFailed
}
return settings
}
private func apply(
_ settings: SyncedAppSettings,
to store: AppGroupStore,
postNotification: Bool
) {
var config = store.configurationSnapshot()
settings.applying(to: &config)
store.saveConfiguration(config, settingsCloudUpdatedAt: settings.updatedAt)
if postNotification {
AppGroupConfigDarwin.postConfigChanged()
NotificationCenter.default.post(name: .settingsDidSyncFromCloud, object: nil)
}
}
}
private extension AppGroupStore {
func configurationSnapshot() -> AppGroupConfiguration {
AppGroupConfiguration.load(fromAvailable: defaults)
}
func saveConfiguration(_ configuration: AppGroupConfiguration, settingsCloudUpdatedAt: Date) {
let config = configuration
config.save(to: defaults)
defaults.set(settingsCloudUpdatedAt.timeIntervalSince1970, forKey: AppGroupConfiguration.Keys.settingsCloudUpdatedAt)
}
}
@@ -80,7 +80,8 @@ public final class KeyboardState: ObservableObject {
@Published public var onDeviceSupported: Bool = false @Published public var onDeviceSupported: Bool = false
/// Seconds remaining in the current utterance (Flow tap-to-talk). /// Seconds remaining in the current utterance (Flow tap-to-talk).
@Published public var utteranceRemainingSeconds: Int = Int(FlowSessionKeys.maxUtteranceDuration) @Published public var utteranceRemainingSeconds: Int = Int(FlowSessionKeys.maxUtteranceDuration)
/// Whether the host app's Flow voice session is currently valid. /// Whether the host app's Flow voice session is live and reachable (fresh
/// heartbeat). Do not use the App Group session flag alone for UI gating.
@Published public var flowSessionActive: Bool = false @Published public var flowSessionActive: Bool = false
/// When true, the mic is intentionally disabled (e.g. cloud engine /// When true, the mic is intentionally disabled (e.g. cloud engine
/// selected but the provider-specific API key is missing). /// selected but the provider-specific API key is missing).
+72
View File
@@ -180,4 +180,76 @@ public enum Keychain: @unchecked Sendable {
throw KeychainError.unexpectedStatus(status) throw KeychainError.unexpectedStatus(status)
} }
} }
// MARK: - Onboarding completion (reboot-durable flag)
// App Group UserDefaults can transiently read empty right after a device
// reboot (data protection / `cfprefsd` not warmed), which made the app
// falsely re-show onboarding. This Keychain marker uses the same
// `AfterFirstUnlockThisDeviceOnly` class reliably readable once the app
// can run, device-local, never synced so it stays a trustworthy fallback
// that survives the App Group read race.
private static let onboardingService = "com.osgkeyboard.onboarding"
private static let onboardingAccount = "hasCompletedOnboarding"
/// Durable "user finished onboarding" marker. `false` when unset or unreadable.
public static func hasCompletedOnboarding() -> Bool {
let query: [String: Any] = [
kSecClass as String: kSecClassGenericPassword,
kSecAttrService as String: onboardingService,
kSecAttrAccount as String: onboardingAccount,
kSecReturnData as String: true,
kSecMatchLimit as String: kSecMatchLimitOne,
]
var result: CFTypeRef?
let status = SecItemCopyMatching(query as CFDictionary, &result)
guard status == errSecSuccess,
let data = result as? Data,
let str = String(data: data, encoding: .utf8) else {
OSGLog.config.info("[onboarding] Keychain read: status=\(status, privacy: .public) → false")
return false
}
let completed = str == "1"
OSGLog.config.info(
"[onboarding] Keychain read: status=ok value=\(str, privacy: .public)\(completed, privacy: .public)"
)
return completed
}
/// Mirror the onboarding-completed flag. Best-effort and idempotent a
/// no-op when the stored value already matches, so it can be called from
/// frequently-saved config paths without Keychain churn.
public static func setOnboardingCompleted(_ completed: Bool) {
guard hasCompletedOnboarding() != completed else {
OSGLog.config.info("[onboarding] Keychain write skipped (already \(completed, privacy: .public))")
return
}
let baseQuery: [String: Any] = [
kSecClass as String: kSecClassGenericPassword,
kSecAttrService as String: onboardingService,
kSecAttrAccount as String: onboardingAccount,
]
guard completed else {
let delStatus = SecItemDelete(baseQuery as CFDictionary)
OSGLog.config.info("[onboarding] Keychain delete: status=\(delStatus, privacy: .public)")
return
}
let data = Data("1".utf8)
let updateStatus = SecItemUpdate(
baseQuery as CFDictionary,
[kSecValueData as String: data] as CFDictionary
)
if updateStatus == errSecItemNotFound {
var addQuery = baseQuery
addQuery[kSecValueData as String] = data
addQuery[kSecAttrAccessible as String] = kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly
let addStatus = SecItemAdd(addQuery as CFDictionary, nil)
OSGLog.config.info("[onboarding] Keychain add: status=\(addStatus, privacy: .public)")
} else {
OSGLog.config.info("[onboarding] Keychain update: status=\(updateStatus, privacy: .public)")
}
}
} }
@@ -565,7 +565,12 @@ public final class LiveDictationController: ObservableObject {
try? await Task.sleep(nanoseconds: 20_000_000) try? await Task.sleep(nanoseconds: 20_000_000)
} }
_ = flushConverterTailToStream() // Trailing speech is preserved by the live `.draining` forwarding
// loop above. We deliberately do NOT signal `.endOfStream` to the
// converter to squeeze its internal filter tail: that both races the
// still-running audio-thread tap on the same non-thread-safe converter
// and (in reused-converter paths) permanently locks it. The dropped
// tail is sub-millisecond and inaudible.
streamRelay.finish() streamRelay.finish()
teardownCaptureEngine() teardownCaptureEngine()
captureGate.withLock { $0 = .idle } captureGate.withLock { $0 = .idle }
@@ -580,50 +585,6 @@ public final class LiveDictationController: ObservableObject {
) )
} }
@discardableResult
private func flushConverterTailToStream() -> Int {
guard let converter = audioConverter,
let targetFormat,
let hwFormat else {
return 0
}
var flushedSamples = 0
let capacity = AVAudioFrameCount(max(512, hwFormat.sampleRate / 20))
guard let outBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: capacity) else {
return 0
}
var endOfStreamSignaled = false
while true {
outBuffer.frameLength = 0
var error: NSError?
let status = converter.convert(to: outBuffer, error: &error) { _, outStatus in
if endOfStreamSignaled {
outStatus.pointee = .noDataNow
return nil
}
endOfStreamSignaled = true
outStatus.pointee = .endOfStream
return nil
}
if status == .error || error != nil {
break
}
guard status == .haveData, outBuffer.frameLength > 0 else {
break
}
let snapshot = AudioBufferSnapshot(buffer: outBuffer)
guard !snapshot.samples.isEmpty else { break }
streamRelay.yield(snapshot)
flushedSamples += snapshot.samples.count
}
return flushedSamples
}
private func teardownCaptureEngine() { private func teardownCaptureEngine() {
if didInstallTap { if didInstallTap {
audioEngine.inputNode.removeTap(onBus: 0) audioEngine.inputNode.removeTap(onBus: 0)
@@ -80,7 +80,12 @@ public final class PersonalDictionaryCloudSync {
/// Enable sync: merge local + remote, persist locally, then upload. /// Enable sync: merge local + remote, persist locally, then upload.
public func enableSync() async throws { public func enableSync() async throws {
let store = makeStore() let store = makeStore()
store.setPersonalDictionaryICloudSyncEnabled(true) ICloudSyncPreferences.pushDictionaryEnabled(true, kvs: kvs)
ICloudSyncPreferences.cacheToAppGroup(
settingsEnabled: store.settingsICloudSyncEnabled,
dictionaryEnabled: true,
store: store
)
let local = store.personalDictionary let local = store.personalDictionary
let remote = loadRemote() ?? .empty let remote = loadRemote() ?? .empty
@@ -90,7 +95,9 @@ public final class PersonalDictionaryCloudSync {
} }
public func disableSync() { public func disableSync() {
makeStore().setPersonalDictionaryICloudSyncEnabled(false) let store = makeStore()
ICloudSyncPreferences.pushDictionaryEnabled(false, kvs: kvs)
store.setPersonalDictionaryICloudSyncEnabled(false)
} }
// MARK: - Core operations // MARK: - Core operations
@@ -8,6 +8,8 @@ import Foundation
public protocol UbiquitousKeyValueStoreing: AnyObject { public protocol UbiquitousKeyValueStoreing: AnyObject {
func data(forKey key: String) -> Data? func data(forKey key: String) -> Data?
func set(_ value: Data?, forKey key: String) func set(_ value: Data?, forKey key: String)
func object(forKey key: String) -> Any?
func set(_ value: Any?, forKey key: String)
@discardableResult @discardableResult
func synchronize() -> Bool func synchronize() -> Bool
} }
@@ -43,6 +43,19 @@ public enum TranscriptPostProcessor: Sendable {
text.trimmingCharacters(in: .whitespacesAndNewlines) text.trimmingCharacters(in: .whitespacesAndNewlines)
} }
/// Conservative cleanup for raw ASR fallback delivery. This is used when
/// polish/translation cannot run, so it must not rewrite meaning or invent
/// punctuation; it only removes formatting artifacts that ASR/chunking can
/// introduce.
public static func cleanRawASRFallback(_ text: String) -> String {
var result = text.trimmingCharacters(in: .whitespacesAndNewlines)
result = repairMidSentenceLineBreaks(result)
result = collapseHorizontalWhitespace(result)
result = removeCJKBoundarySpaces(result)
result = normalizePunctuationSpacing(result)
return normalizeWhitespaceAndPunctuation(result)
}
// MARK: - Post-LLM pipeline // MARK: - Post-LLM pipeline
/// Apply deterministic cleanup and quality gate to LLM output. /// Apply deterministic cleanup and quality gate to LLM output.
@@ -220,6 +233,53 @@ public enum TranscriptPostProcessor: Sendable {
return result.trimmingCharacters(in: .whitespacesAndNewlines) return result.trimmingCharacters(in: .whitespacesAndNewlines)
} }
private static func collapseHorizontalWhitespace(_ text: String) -> String {
text.replacingOccurrences(
of: #"[^\S\r\n]+"#,
with: " ",
options: .regularExpression
)
}
private static func removeCJKBoundarySpaces(_ text: String) -> String {
var output = ""
let characters = Array(text)
for index in characters.indices {
let current = characters[index]
if current.isWhitespace,
let previous = previousNonWhitespace(in: characters, before: index),
let next = nextNonWhitespace(in: characters, after: index),
shouldDropSpaceBetween(previous: previous, next: next) {
continue
}
output.append(current)
}
return output
}
private static func normalizePunctuationSpacing(_ text: String) -> String {
var output = ""
let characters = Array(text)
for index in characters.indices {
let current = characters[index]
if current.isWhitespace,
let next = nextNonWhitespace(in: characters, after: index),
isClosingPunctuation(next) {
continue
}
if isOpeningPunctuation(current),
let next = nextNonWhitespace(in: characters, after: index),
next.isWhitespace {
output.append(current)
continue
}
output.append(current)
}
return output
.replacingOccurrences(of: #"([(「“])\s+"#, with: "$1", options: .regularExpression)
.replacingOccurrences(of: #"\s+([,。!?;:、,.!?;:])"#, with: "$1", options: .regularExpression)
}
// MARK: - Prefix / quote cleanup // MARK: - Prefix / quote cleanup
public static func stripExplanatoryPrefix(from text: String) -> String { public static func stripExplanatoryPrefix(from text: String) -> String {
@@ -271,6 +331,41 @@ public enum TranscriptPostProcessor: Sendable {
return (pAscii && nAscii) ? " " : "" return (pAscii && nAscii) ? " " : ""
} }
private static func previousNonWhitespace(in characters: [Character], before index: Int) -> Character? {
guard index > characters.startIndex else { return nil }
for i in stride(from: index - 1, through: characters.startIndex, by: -1) {
if !characters[i].isWhitespace { return characters[i] }
}
return nil
}
private static func nextNonWhitespace(in characters: [Character], after index: Int) -> Character? {
let nextIndex = index + 1
guard nextIndex < characters.endIndex else { return nil }
for i in nextIndex..<characters.endIndex {
if !characters[i].isWhitespace { return characters[i] }
}
return nil
}
private static func shouldDropSpaceBetween(previous: Character, next: Character) -> Bool {
(isCJKCharacter(previous) && isCJKCharacter(next)) || isClosingPunctuation(next)
}
private static func isCJKCharacter(_ character: Character) -> Bool {
character.unicodeScalars.contains(where: isCJKScalar)
}
private static func isClosingPunctuation(_ character: Character) -> Bool {
let closing: Set<Character> = ["", "", "", "", "", "", "", ",", ".", "!", "?", ";", ":"]
return closing.contains(character)
}
private static func isOpeningPunctuation(_ character: Character) -> Bool {
let opening: Set<Character> = ["", "", ""]
return opening.contains(character)
}
private static func extractEmojis(from text: String) -> [String] { private static func extractEmojis(from text: String) -> [String] {
text.unicodeScalars.filter(isEmojiScalar).map { String($0) } text.unicodeScalars.filter(isEmojiScalar).map { String($0) }
} }
@@ -7,6 +7,7 @@
/* v0.2.0: flow-level warnings surfaced alongside the final transcript. */ /* v0.2.0: flow-level warnings surfaced alongside the final transcript. */
"flow.warning.cloudPolishMissingKey" = "Cloud polish needs an API key in Settings. Inserted raw ASR text."; "flow.warning.cloudPolishMissingKey" = "Cloud polish needs an API key in Settings. Inserted raw ASR text.";
"flow.warning.localPolishUnavailable" = "Built-in polish is unavailable. Inserted raw ASR text."; "flow.warning.localPolishUnavailable" = "Built-in polish is unavailable. Inserted raw ASR text.";
"flow.warning.polishDegraded" = "Weak network — inserted raw ASR text without polish.";
/* LLM providers */ /* LLM providers */
"provider.openai" = "OpenAI"; "provider.openai" = "OpenAI";
@@ -7,6 +7,7 @@
/* v0.2.0: flow-level warnings surfaced alongside the final transcript. */ /* v0.2.0: flow-level warnings surfaced alongside the final transcript. */
"flow.warning.cloudPolishMissingKey" = "云端润色需要先在设置中填写 API Key,本次已插入原始识别结果。"; "flow.warning.cloudPolishMissingKey" = "云端润色需要先在设置中填写 API Key,本次已插入原始识别结果。";
"flow.warning.localPolishUnavailable" = "内置润色暂不可用,本次已插入原始识别结果。"; "flow.warning.localPolishUnavailable" = "内置润色暂不可用,本次已插入原始识别结果。";
"flow.warning.polishDegraded" = "弱网识别,本次未润色,已插入原始识别结果。";
/* LLM providers */ /* LLM providers */
"provider.openai" = "OpenAI"; "provider.openai" = "OpenAI";
@@ -0,0 +1,37 @@
// FakeUbiquitousKeyValueStore.swift
// OSGKeyboardTests
//
// In-memory KVS fake for hermetic iCloud sync tests.
import Foundation
@testable import OSGKeyboardShared
final class FakeUbiquitousKeyValueStore: UbiquitousKeyValueStoreing, @unchecked Sendable {
private var storage: [String: Any] = [:]
func data(forKey key: String) -> Data? {
storage[key] as? Data
}
func set(_ value: Data?, forKey key: String) {
if let value {
storage[key] = value
} else {
storage.removeValue(forKey: key)
}
}
func object(forKey key: String) -> Any? {
storage[key]
}
func set(_ value: Any?, forKey key: String) {
if let value {
storage[key] = value
} else {
storage.removeValue(forKey: key)
}
}
func synchronize() -> Bool { true }
}
@@ -23,6 +23,40 @@ final class FlowSessionBridgeTests: XCTestCase {
defaults.set(staleHeartbeat, forKey: FlowSessionKeys.flowHeartbeat) defaults.set(staleHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
XCTAssertTrue(FlowSessionBridge.isSessionActive(defaults: defaults)) XCTAssertTrue(FlowSessionBridge.isSessionActive(defaults: defaults))
XCTAssertFalse(FlowSessionBridge.isHostReachable(defaults: defaults)) XCTAssertFalse(FlowSessionBridge.isHostReachable(defaults: defaults))
XCTAssertFalse(FlowSessionBridge.isHostStale(defaults: defaults))
}
func testHostStaleWhenHeartbeatVeryOld() {
let defaults = makeDefaults()
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
let zombieHeartbeat = Date().timeIntervalSince1970 - 120
defaults.set(zombieHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
XCTAssertTrue(FlowSessionBridge.isSessionActive(defaults: defaults))
XCTAssertFalse(FlowSessionBridge.isHostReachable(defaults: defaults))
XCTAssertTrue(FlowSessionBridge.isHostStale(defaults: defaults))
}
func testClearIfHostStaleRemovesZombieSession() {
let defaults = makeDefaults()
FlowSessionBridge.markSessionActive(duration: 3_600, defaults: defaults)
FlowSessionBridge.setRecordingState(.stopped, defaults: defaults)
let zombieHeartbeat = Date().timeIntervalSince1970 - 120
defaults.set(zombieHeartbeat, forKey: FlowSessionKeys.flowHeartbeat)
XCTAssertTrue(FlowSessionBridge.clearIfHostStale(defaults: defaults))
XCTAssertFalse(FlowSessionBridge.isSessionActive(defaults: defaults))
XCTAssertEqual(FlowSessionBridge.recordingState(defaults: defaults), .idle)
}
func testHostStaleWhenSessionActiveButNoHeartbeat() {
let defaults = makeDefaults()
defaults.set(true, forKey: FlowSessionKeys.flowSessionActive)
defaults.set(Date().timeIntervalSince1970 + 3_600, forKey: FlowSessionKeys.flowSessionExpires)
XCTAssertTrue(FlowSessionBridge.isHostStale(defaults: defaults))
XCTAssertTrue(FlowSessionBridge.clearIfHostStale(defaults: defaults))
XCTAssertFalse(FlowSessionBridge.isSessionActive(defaults: defaults))
} }
func testSessionInactiveWhenExpired() { func testSessionInactiveWhenExpired() {
@@ -326,6 +326,32 @@ final class IntelligentPolishTests: XCTestCase {
XCTAssertEqual(output, "this is a broken sentence") XCTAssertEqual(output, "this is a broken sentence")
} }
func testCleanRawASRFallbackRemovesChineseInteriorSpaces() {
let input = " 你 是不是 已经 解决了 这个 问题 ? "
let output = TranscriptPostProcessor.cleanRawASRFallback(input)
XCTAssertEqual(output, "你是不是已经解决了这个问题?")
}
func testCleanRawASRFallbackPreservesEnglishAndMixedSpaces() {
let input = "iOS 版本 uses Swift UI"
let output = TranscriptPostProcessor.cleanRawASRFallback(input)
XCTAssertEqual(output, "iOS 版本 uses Swift UI")
}
@MainActor
func testFlowFallbackDeliveryCleansTextAndCarriesWeakNetworkWarning() {
let delivery = FlowSessionManager.makeFallbackDelivery(
rawText: " 你 是不是 已经 解决了 这个 问题 ? ",
error: LLMError.transport("offline"),
engineMode: "cloud",
chunkWarning: nil
)
XCTAssertEqual(delivery.text, "你是不是已经解决了这个问题?")
XCTAssertFalse(delivery.text.contains("未润色"))
XCTAssertEqual(delivery.polishWarning, SharedL10n.string("flow.warning.polishDegraded"))
}
func testHasStructureSignalDetectsChineseEnumeration() { func testHasStructureSignalDetectsChineseEnumeration() {
XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "首先测试其次上线")) XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "首先测试其次上线"))
XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "第一点修复")) XCTAssertTrue(TranscriptPostProcessor.hasStructureSignal(in: "第一点修复"))
@@ -8,23 +8,7 @@ import XCTest
// MARK: - Fake KVS // MARK: - Fake KVS
private final class FakeUbiquitousKeyValueStore: UbiquitousKeyValueStoreing, @unchecked Sendable { // See FakeUbiquitousKeyValueStore.swift
private var storage: [String: Data] = [:]
func data(forKey key: String) -> Data? {
storage[key]
}
func set(_ value: Data?, forKey key: String) {
if let value {
storage[key] = value
} else {
storage.removeValue(forKey: key)
}
}
func synchronize() -> Bool { true }
}
// MARK: - Tests // MARK: - Tests
@@ -170,6 +154,7 @@ final class PersonalDictionaryCloudSyncTests: XCTestCase {
try await sync.enableSync() try await sync.enableSync()
XCTAssertTrue(store.personalDictionaryICloudSyncEnabled) XCTAssertTrue(store.personalDictionaryICloudSyncEnabled)
XCTAssertEqual(kvs.object(forKey: ICloudSyncPreferences.dictionaryEnabledKey) as? Bool, true)
XCTAssertEqual(Set(store.personalDictionary.entries.map(\.term)), Set(["LocalTerm", "RemoteTerm"])) XCTAssertEqual(Set(store.personalDictionary.entries.map(\.term)), Set(["LocalTerm", "RemoteTerm"]))
XCTAssertNotNil(sync.loadRemote()) XCTAssertNotNil(sync.loadRemote())
} }
@@ -0,0 +1,157 @@
// SettingsCloudSyncTests.swift
// OSGKeyboardTests
//
// Hermetic tests for iCloud KVS settings sync + preference toggles.
import XCTest
@testable import OSGKeyboardShared
@MainActor
final class SettingsCloudSyncTests: XCTestCase {
private var suiteName: String!
private var defaults: UserDefaults!
private var store: AppGroupStore!
private var kvs: FakeUbiquitousKeyValueStore!
private var settingsSync: SettingsCloudSync!
override func setUp() {
super.setUp()
suiteName = "group.com.osgkeyboard.shared.tests.settings.\(UUID().uuidString)"
defaults = UserDefaults(suiteName: suiteName)!
defaults.removePersistentDomain(forName: suiteName)
store = AppGroupStore(defaults: defaults)
kvs = FakeUbiquitousKeyValueStore()
settingsSync = SettingsCloudSync(kvs: kvs) { [unowned self] in store }
}
override func tearDown() {
defaults.removePersistentDomain(forName: suiteName)
super.tearDown()
}
func testMergePrefersNewerUpdatedAt() {
let older = SyncedAppSettings(
updatedAt: Date(timeIntervalSince1970: 100),
providerId: "openai",
baseURL: "https://old.example",
model: "gpt-old",
modeId: "polish",
localeId: "auto",
engineMode: "cloud",
hasAcknowledgedCloudSharing: false,
uiLanguage: .english,
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId,
handednessPreference: .left,
cursorDragNavigationEnabled: true,
polishIntensity: .medium,
flowSkipAppSwitch: true,
flowInactivityDuration: .twelveHours
)
let newer = SyncedAppSettings(
updatedAt: Date(timeIntervalSince1970: 200),
providerId: "openai",
baseURL: "https://new.example",
model: "gpt-new",
modeId: "polish",
localeId: "zh-Hans",
engineMode: "local",
hasAcknowledgedCloudSharing: true,
uiLanguage: .chinese,
translationTargetLocaleId: "en",
handednessPreference: .right,
cursorDragNavigationEnabled: false,
polishIntensity: .light,
flowSkipAppSwitch: false,
flowInactivityDuration: .threeHours
)
let merged = SyncedAppSettings.merge(local: older, remote: newer)
XCTAssertEqual(merged.model, "gpt-new")
XCTAssertEqual(merged.localeId, "zh-Hans")
XCTAssertEqual(merged.engineMode, "local")
}
func testEnableSyncUploadsMergedSettingsAndToggle() async throws {
store.setModeId("polish")
store.setLocaleId("zh-Hans")
let remote = SyncedAppSettings(
updatedAt: Date().addingTimeInterval(3600),
providerId: "openai",
baseURL: "https://remote.example",
model: "remote-model",
modeId: "polish",
localeId: "en",
engineMode: "cloud",
hasAcknowledgedCloudSharing: true,
uiLanguage: .english,
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId,
handednessPreference: .right,
cursorDragNavigationEnabled: true,
polishIntensity: .medium,
flowSkipAppSwitch: true,
flowInactivityDuration: .twelveHours
)
try settingsSync.push(remote)
try await settingsSync.enableSync()
XCTAssertTrue(store.settingsICloudSyncEnabled)
XCTAssertEqual(kvs.object(forKey: ICloudSyncPreferences.settingsEnabledKey) as? Bool, true)
XCTAssertEqual(settingsSync.loadRemote()?.localeId, "en")
}
func testPullAndMergeAppliesRemoteSettingsToAppGroup() async throws {
store.setSettingsICloudSyncEnabled(true)
store.setLocaleId("auto")
let remote = SyncedAppSettings(
updatedAt: Date(timeIntervalSince1970: 900),
providerId: "openai",
baseURL: "https://remote.example",
model: "remote-model",
modeId: "polish",
localeId: "ja",
engineMode: "cloud",
hasAcknowledgedCloudSharing: false,
uiLanguage: .english,
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId,
handednessPreference: .left,
cursorDragNavigationEnabled: true,
polishIntensity: .medium,
flowSkipAppSwitch: true,
flowInactivityDuration: .twelveHours
)
try settingsSync.push(remote)
await settingsSync.pullAndMerge(store: store)
XCTAssertEqual(store.localeId, "ja")
XCTAssertEqual(store.settingsCloudUpdatedAt?.timeIntervalSince1970, 900, accuracy: 1)
}
func testPushLocalIfEnabledSkipsWhenDisabled() async throws {
store.setSettingsICloudSyncEnabled(false)
store.setLocaleId("ko")
try await settingsSync.pushLocalIfEnabled()
XCTAssertNil(settingsSync.loadRemote())
}
func testICloudSyncPreferencesMigrateLegacyToggles() {
store.setSettingsICloudSyncEnabled(false)
store.setPersonalDictionaryICloudSyncEnabled(true)
ICloudSyncPreferences.migrateLegacyTogglesIfNeeded(kvs: kvs, store: store)
XCTAssertEqual(kvs.object(forKey: ICloudSyncPreferences.settingsEnabledKey) as? Bool, false)
XCTAssertEqual(kvs.object(forKey: ICloudSyncPreferences.dictionaryEnabledKey) as? Bool, true)
}
func testAppGroupConfigurationDefaultsSettingsICloudSyncToOn() {
let config = AppGroupConfiguration.load(fromAvailable: defaults)
XCTAssertTrue(config.settingsICloudSyncEnabled)
}
}
+14 -6
View File
@@ -35,9 +35,15 @@ settings:
SWIFT_STRICT_CONCURRENCY: complete SWIFT_STRICT_CONCURRENCY: complete
GENERATE_INFOPLIST_FILE: NO GENERATE_INFOPLIST_FILE: NO
ENABLE_MODULE_VERIFIER: YES ENABLE_MODULE_VERIFIER: YES
MODULE_VERIFIER_SUPPORTED_LANGUAGES: "objective-c objective-c++"
MODULE_VERIFIER_SUPPORTED_LANGUAGE_STANDARDS: "gnu17 gnu++17"
CLANG_ANALYZER_LOCALIZABILITY_NONLOCALIZED: YES
ASSETCATALOG_COMPILER_GENERATE_ASSET_SYMBOLS: YES
ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES
STRING_CATALOG_GENERATE_SYMBOLS: YES
CLANG_CXX_LANGUAGE_STANDARD: c++17 CLANG_CXX_LANGUAGE_STANDARD: c++17
MARKETING_VERSION: "0.4.1" MARKETING_VERSION: "0.5.0"
CURRENT_PROJECT_VERSION: "17" CURRENT_PROJECT_VERSION: "18"
# 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖) # 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖)
# 项目级签名 xcconfig,适用于所有 target # 项目级签名 xcconfig,适用于所有 target
@@ -325,10 +331,12 @@ targets:
DYLIB_INSTALL_NAME_BASE: "@rpath" DYLIB_INSTALL_NAME_BASE: "@rpath"
APPLICATION_EXTENSION_API_ONLY: YES APPLICATION_EXTENSION_API_ONLY: YES
ENABLE_MODULE_VERIFIER: YES ENABLE_MODULE_VERIFIER: YES
# Frameworks inherit the host app's signing identity; no profile # Embedded frameworks are re-signed by the containing app at package
# is required, but pinning the team keeps the build reproducible. # time. Signing this framework as a standalone build product is noisy
CODE_SIGN_STYLE: Automatic # and not recommended by Xcode.
DEVELOPMENT_TEAM: X329MZU23S CODE_SIGNING_ALLOWED: NO
CODE_SIGNING_REQUIRED: NO
CODE_SIGN_IDENTITY: ""
dependencies: dependencies:
- sdk: Speech.framework - sdk: Speech.framework
- sdk: AVFoundation.framework - sdk: AVFoundation.framework