diff --git a/CHANGELOG.md b/CHANGELOG.md index 5100272..a4f30c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,13 +8,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added -- **Clipboard voice command**: long-press the mic after copying eligible text to treat speech as an instruction over the clipboard snapshot and insert the generated result; short press stays dictation. / **剪贴板语音指令**：复制合格文本后长按麦克风，将语音视为对剪贴板材料的指令并插入生成结果；短按仍为听写。 -- **Clipboard record confirm**: show recording UI only after the host confirms capture; enforce a short minimum record window; light-prewarm ASR when clipboard becomes eligible (no idle mic hold). / **剪贴板开录确认**：宿主确认采音后再显示录音态；确认后再保证最短有效录音；剪贴板刚合格时轻预热 ASR（空闲不占麦）。 +- **In-app release notes**: after onboarding, show a What's New sheet when the marketing version changes; Settings version row opens the same remote page (`download.osglab.com`) with current version, language, and theme. / **应用内更新说明**：完成引导后，在 marketing 版本变化时弹出更新说明；设置页版本行打开同一远程页（`download.osglab.com`），并传入当前版本、语言与深浅色。 +- **Typing input settings l10n**: localize schema / fuzzy / resource strings on the Text Input settings page (and related General toggles) via in-app language. / **文本输入设置双语**：文本输入设置页的方案 / 模糊音 / 资源文案（及通用里相关开关）跟随应用内语言。 +- **Clipboard voice command**: long-press the mic when the clipboard has text to treat speech as an instruction over a one-shot clipboard snapshot and insert the result; short press stays dictation; tap again to stop (not release-to-stop). / **剪贴板语音指令**：剪贴板有文字时长按麦克风，将语音视为对该次冻结快照的指令并插入结果；短按仍为听写；再点一下结束录音（非松手结束）。 +- **Clipboard record confirm**: show recording UI only after the host confirms capture; enforce a short minimum record window after confirm. / **剪贴板开录确认**：宿主确认采音后再显示录音态；确认后再保证最短有效录音。 +- **Clipboard recording chrome**: after host-confirmed clipboard recording, the mic disc animates red→blue and shows side captions (“指令录制中” / “点按结束处理”); dictation stays red. / **剪贴板录音视觉**：宿主确认剪贴板录音后，麦克风圆盘红→蓝过渡，两侧显示「指令录制中」「点按结束处理」；普通听写仍为红色。 +- **Clipboard failure tips**: when long-press cannot start (paste denied, empty/short/code-like material, secure field, no Full Access), show a brief reason above the mic. / **剪贴板失败提示**：长按无法开始时（拒贴、空/过短/验证码类、密码框、无完全访问）在麦克风上方短暂显示原因。 ### Removed - **Keep-alive settings & Live Activity**: remove Settings keep-alive mode picker and its note; delete the Dynamic Island Live Activity extension and all ActivityKit session code. Voice sessions stay on silent low-profile PiP only, with user-facing copy that never names Picture in Picture. / **保活设置与灵动岛**：移除设置中的保活方式选项及说明；删除灵动岛 Live Activity 扩展与全部 ActivityKit 会话代码。语音会话仅保留静默低感知 PiP，用户可见文案不再出现「画中画」。 +- **Clipboard 30s window & continuous rewrite**: drop the copy-time eligibility clock and multi-turn rewrite session; each long-press is an independent round. / **剪贴板 30 秒窗与连续改写**：去掉复制后资格计时与多轮改写会话；每次长按为独立一轮。 ### Fixed +- **Clipboard prepare stuck**: after the first Allow Paste, do not double-`startRecording`; recover or cancel when host never confirms capture so UI cannot remain on「准备录音…」. / **剪贴板卡在准备录音**：首次「允许粘贴」后不再重复 `startRecording`；宿主未确认采音时及时恢复或取消，避免 UI 一直停在「准备录音…」。 +- **Clipboard prepare decision tests**: unit-cover restore/dedupe/adopt/abort matrices (including a 20-round stress case) so paste-alert reopen cannot regress into a second start. / **剪贴板准备态决策单测**：覆盖恢复/去重/领养/中止矩阵（含 20 轮加压），防止粘贴弹窗重开再次双发 start。 +- **Clipboard cold-start handoff**: when the host app is not alive, long-press stores a sticky voice+snapshot and opens `startflow` without entering fake「准备录音…」; after return (including default typing → forced voice), the user long-presses again to start blue clipboard recording. / **剪贴板冷启动交接**：宿主未活时长按只保留语音+快照粘性并走 `startflow`，不进入假「准备录音…」；返回后（含默认打字→强制语音）再长按才进入蓝色剪贴板录音。 +- **PiP arm re-jump**: keyboard open / Voice-tab appear no longer reopens `startflow` while a session is alive, warming, or within the arm cooldown; force-quit zombies can still arm after an 8s unreachable grace. / **PiP 反复跳转**：会话已存活/预热中或仍在拉起冷却窗内时不再重复 `startflow`；强杀后残留 session 在心跳失联超过 8s 仍可拉起一次。 +- **Clipboard prepare chrome**: while waiting for host capture confirm, the mic stays grey and disabled (not blue); blue + side captions appear only after confirmed recording. / **剪贴板准备态视觉**：等待宿主确认采音时麦克风保持灰色不可点（非蓝色）；确认录音后才显示蓝色与两侧文案。 +- **Cold capture first-press**: host retries a soft-dead / failed engine start once and rejects `running && !engineLive` as success. / **冷启动首次采音**：宿主对软死/失败引擎自动重建一次，且不把 `running 但无 live` 当作成功。 +- **Clipboard paste prompt spam**: idle affordance peeks `hasStrings` only; pasteboard *contents* are read only on long-press, so opening the keyboard no longer repeatedly requests paste permission. / **剪贴板粘贴授权刷屏**：空闲仅探测 `hasStrings`；正文只在长按时读取，打开键盘不再反复申请粘贴权限。 +- **Clipboard paste-alert continuity**: survive the system Allow Paste alert without snapping to the typing grid, keep blue clipboard chrome from prepare onward, and ignore the finger-up that follows long-press so stop stays tap-to-finish. / **剪贴板粘贴弹窗连续性**：系统「允许粘贴」弹窗期间不跳回打字键盘；准备阶段即显示蓝色剪贴板态；忽略长按后的抬手，仍需再点结束。 +- **Clipboard paste-alert voice sticky**: persist a short App Group prefer-voice + snapshot flag (with `synchronize`) across paste-alert dismiss/recreate so reopen stays on the voice keyboard with blue clipboard chrome instead of the default typing grid / red dictation mic. / **剪贴板粘贴弹窗语音粘性**：用 App Group 短时标记（并 `synchronize`）跨弹窗销毁/重建，重开仍进语音键盘并恢复蓝色剪贴板态，而不是默认打字键盘 / 红色听写麦。 - **Custom polish style editor**: open create/edit with `sheet(item:)` so the form always loads the selected pack instead of a blank default template. / **自定义润色风格编辑**：新建/编辑改为 `sheet(item:)` 呈现，表单始终加载所选风格，而不再偶发显示空白默认模板。 ## [1.6.5] - 2026-08-06 diff --git a/OSGKeyboard/AppIcon.icon/Assets/Group 29 3.png b/OSGKeyboard/AppIcon.icon/Assets/Group 29 3.png new file mode 100644 index 0000000..0c372e9 Binary files /dev/null and b/OSGKeyboard/AppIcon.icon/Assets/Group 29 3.png differ diff --git a/OSGKeyboard/AppIcon.icon/icon.json b/OSGKeyboard/AppIcon.icon/icon.json index a30ebb2..7d56cc9 100644 --- a/OSGKeyboard/AppIcon.icon/icon.json +++ b/OSGKeyboard/AppIcon.icon/icon.json @@ -8,6 +8,18 @@ "hidden" : false, "layers" : [ { + "image-name" : "Group 29 3.png", + "name" : "Group 29 3", + "position" : { + "scale" : 0.67, + "translation-in-points" : [ + 0, + 0 + ] + } + }, + { + "hidden" : true, "image-name" : "Group 29.svg", "name" : "Group 29" } diff --git a/OSGKeyboard/Services/FlowSessionManager.swift b/OSGKeyboard/Services/FlowSessionManager.swift index 85fc152..5355154 100644 --- a/OSGKeyboard/Services/FlowSessionManager.swift +++ b/OSGKeyboard/Services/FlowSessionManager.swift @@ -1141,9 +1141,19 @@ final class FlowSessionManager: ObservableObject { if capture.engineHasRecentAudio(maxAge: 2) { micReady = true } else { - micReady = await capture.awaitAudioFlowing( + var flowing = await capture.awaitAudioFlowing( timeout: Self.coldStartAudioProofTimeout ) + if !flowing { + // First cold capture after PiP arm often proves audio late — one rebuild. + debug("PiP audio proof timeout — one capture rebuild before failing") + capture.stop(releaseSession: false) + _ = await startCaptureForPiPUtteranceIfNeeded() + flowing = await capture.awaitAudioFlowing( + timeout: Self.coldStartAudioProofTimeout + ) + } + micReady = flowing } guard !Task.isCancelled, canContinueStart(startToken) else { releaseOrphanedCaptureIfNeeded() @@ -1168,16 +1178,33 @@ final class FlowSessionManager: ObservableObject { } /// Start capture for a PiP utterance without blocking on the first frame. + /// Cold first start after relaunch often needs one rebuild (VPIO / -66635). private func startCaptureForPiPUtteranceIfNeeded() async -> Bool { if capture.engineHasRecentAudio(maxAge: 2) { return true } do { try await capture.start() - return true + if capture.engineIsLive || capture.engineHasRecentAudio(maxAge: 2) { + return true + } + debug("PiP utterance capture soft-dead after start — one rebuild retry") + capture.stop(releaseSession: false) + try? await Task.sleep(nanoseconds: 120_000_000) + try await capture.start() + // Do not accept `running && !engineLive` — that is the silent-mic failure mode. + return capture.engineIsLive || capture.engineHasRecentAudio(maxAge: 2) } catch { - debug("PiP utterance capture start failed: \(error.localizedDescription)") - return false + debug("PiP utterance capture start failed: \(error.localizedDescription) — retry once") + capture.stop(releaseSession: false) + try? await Task.sleep(nanoseconds: 150_000_000) + do { + try await capture.start() + return capture.engineIsLive || capture.engineHasRecentAudio(maxAge: 2) + } catch { + debug("PiP utterance capture retry failed: \(error.localizedDescription)") + return false + } } } diff --git a/OSGKeyboard/Services/LegalLinks.swift b/OSGKeyboard/Services/LegalLinks.swift index 2f751cb..838ebef 100644 --- a/OSGKeyboard/Services/LegalLinks.swift +++ b/OSGKeyboard/Services/LegalLinks.swift @@ -15,4 +15,10 @@ enum LegalLinks { static var supportURL: URL? { URL(string: "https://github.com/hkgood/OSGKeyboard/issues") } + + /// In-app release notes (hosted on download.osglab.com). Prefer + /// `ReleaseNotesStore.pageURL(language:colorScheme:)` so v/lang/theme are set. + static var releaseNotesURL: URL? { + URL(string: ReleaseNotesStore.pageURLString) + } } diff --git a/OSGKeyboard/Services/ReleaseNotesStore.swift b/OSGKeyboard/Services/ReleaseNotesStore.swift new file mode 100644 index 0000000..e5a6574 --- /dev/null +++ b/OSGKeyboard/Services/ReleaseNotesStore.swift @@ -0,0 +1,82 @@ +// ReleaseNotesStore.swift +// OSGKeyboard · Main App +// +// Tracks which marketing version's release notes the user has already seen, +// and builds the remote release-notes URL with v / lang / theme query params. + +import Foundation +import OSGKeyboardShared +import SwiftUI + +enum ReleaseNotesStore { + static let lastSeenKey = "config.lastSeenMarketingVersion" + static let pageURLString = "https://download.osglab.com/osgkeyboardversion.html" + + static var lastSeenMarketingVersion: String? { + get { + UserDefaults.standard.string(forKey: lastSeenKey) + } + set { + if let newValue { + UserDefaults.standard.set(newValue, forKey: lastSeenKey) + } else { + UserDefaults.standard.removeObject(forKey: lastSeenKey) + } + } + } + + /// True when the user has not acknowledged the current marketing version. + static var shouldPresentAutomatically: Bool { + let current = AppVersionDisplay.marketingVersion + guard current != "—" else { return false } + return lastSeenMarketingVersion != current + } + + static func markCurrentVersionSeen() { + let current = AppVersionDisplay.marketingVersion + guard current != "—" else { return } + lastSeenMarketingVersion = current + } + + /// Query language for the remote page (`zh` / `en`). + static func queryLanguage(for uiLanguage: AppUILanguage) -> String { + let code = uiLanguage.resolvedLanguageCode() + return code.hasPrefix("zh") ? "zh" : "en" + } + + /// Query theme for the remote page (`light` / `dark`). + static func queryTheme(for colorScheme: ColorScheme) -> String { + colorScheme == .dark ? "dark" : "light" + } + + static func pageURL( + version: String = AppVersionDisplay.marketingVersion, + language: AppUILanguage, + colorScheme: ColorScheme + ) -> URL? { + guard version != "—", + var components = URLComponents(string: pageURLString) else { return nil } + components.queryItems = [ + URLQueryItem(name: "v", value: version), + URLQueryItem(name: "lang", value: queryLanguage(for: language)), + URLQueryItem(name: "theme", value: queryTheme(for: colorScheme)), + ] + return components.url + } +} + +@MainActor +final class ReleaseNotesController: ObservableObject { + static let shared = ReleaseNotesController() + + @Published var isPresented = false + + func presentIfNeeded(onboardingCompleted: Bool) { + guard onboardingCompleted, ReleaseNotesStore.shouldPresentAutomatically else { return } + isPresented = true + } + + func presentManually() { + isPresented = true + } +} diff --git a/OSGKeyboard/Views/Components/RemoteWebView.swift b/OSGKeyboard/Views/Components/RemoteWebView.swift index c525ab4..0ce0493 100644 --- a/OSGKeyboard/Views/Components/RemoteWebView.swift +++ b/OSGKeyboard/Views/Components/RemoteWebView.swift @@ -19,15 +19,26 @@ struct RemoteWebView: UIViewRepresentable { webView.isOpaque = false webView.backgroundColor = .clear webView.scrollView.backgroundColor = .clear + // Sheet / NavigationStack already lays out inside the safe area. + // Automatic adjustment would add a second bottom inset and leave a dead band. + webView.scrollView.contentInsetAdjustmentBehavior = .never + webView.scrollView.contentInset = .zero + webView.scrollView.scrollIndicatorInsets = .zero webView.navigationDelegate = context.coordinator + context.coordinator.loadedURL = url webView.load(URLRequest(url: url)) return webView } - func updateUIView(_ uiView: WKWebView, context: Context) {} + func updateUIView(_ uiView: WKWebView, context: Context) { + guard context.coordinator.loadedURL != url else { return } + context.coordinator.loadedURL = url + uiView.load(URLRequest(url: url)) + } final class Coordinator: NSObject, WKNavigationDelegate { @Binding var isLoading: Bool + var loadedURL: URL? init(isLoading: Binding) { _isLoading = isLoading diff --git a/OSGKeyboard/Views/MainAppRoot.swift b/OSGKeyboard/Views/MainAppRoot.swift index 57297eb..a3e6713 100644 --- a/OSGKeyboard/Views/MainAppRoot.swift +++ b/OSGKeyboard/Views/MainAppRoot.swift @@ -15,6 +15,7 @@ struct MainAppRoot: View { // Singleton is owned by `ProviderConfig.shared`, not by this view — // `@ObservedObject` keeps subscriptions correct across Settings replay. @ObservedObject private var config = ProviderConfig.shared + @ObservedObject private var releaseNotes = ReleaseNotesController.shared @StateObject private var flowManager = FlowSessionManager() @State private var postOnboardingWarmupTask: Task? @@ -40,6 +41,14 @@ struct MainAppRoot: View { .allowsHitTesting(false) .accessibilityHidden(true) } + .sheet(isPresented: $releaseNotes.isPresented, onDismiss: { + // Auto-present and Settings entry both count as acknowledged. + ReleaseNotesStore.markCurrentVersionSeen() + }) { + // Pass language explicitly; sheet chrome uses AppL10n (in-app override). + ReleaseNotesSheet(language: config.uiLanguage) + .environment(\.locale, config.uiLanguage.swiftUILocale) + } .onAppear { flowManager.setAppForeground(scenePhase == .active) // Register the URL handler BEFORE the foreground auto-start. @@ -66,6 +75,7 @@ struct MainAppRoot: View { // Capture/ASR remain lazy and start only on an actual mic press. flowManager.activateOnForeground(reason: "MainAppRoot.onAppear") schedulePostOnboardingWarmup(reason: "MainAppRoot.onAppear") + releaseNotes.presentIfNeeded(onboardingCompleted: true) } else { OSGDiag.log( "MainAppRoot.onAppear skip Flow/CLM/Rime (onboarding incomplete)", @@ -80,6 +90,7 @@ struct MainAppRoot: View { if done { flowManager.activateOnForeground(reason: "onboardingCompleted") schedulePostOnboardingWarmup(reason: "onboardingCompleted") + releaseNotes.presentIfNeeded(onboardingCompleted: true) } } .onChange(of: scenePhase) { _, phase in @@ -94,6 +105,7 @@ struct MainAppRoot: View { flowManager.activateOnForeground(reason: "scenePhase.active") // Retry deferred Rime/CLM after a jetsam-prone launch. schedulePostOnboardingWarmup(reason: "scenePhase.active.retry") + releaseNotes.presentIfNeeded(onboardingCompleted: true) } Task { await AppCloudSync.shared.pullAllIfEnabled() diff --git a/OSGKeyboard/Views/ReleaseNotesSheet.swift b/OSGKeyboard/Views/ReleaseNotesSheet.swift new file mode 100644 index 0000000..6670de5 --- /dev/null +++ b/OSGKeyboard/Views/ReleaseNotesSheet.swift @@ -0,0 +1,71 @@ +// ReleaseNotesSheet.swift +// OSGKeyboard · Main App +// +// Sheet that loads the remote release-notes HTML for the current version. + +import SwiftUI +import OSGKeyboardShared + +struct ReleaseNotesSheet: View { + @Environment(\.dismiss) private var dismiss + @Environment(\.themePalette) private var palette: ThemePalette + @Environment(\.colorScheme) private var colorScheme + + /// In-app UI language (not system-only) — chrome strings go through `AppL10n`. + let language: AppUILanguage + + @AppStorage(AppearancePreference.storageKey) + private var appearanceRaw = AppearancePreference.system.rawValue + + @State private var isLoading = true + + private var appearance: AppearancePreference { + AppearancePreference.fromStored(appearanceRaw) + } + + /// Theme actually shown in this sheet (honors Settings → Appearance). + private var resolvedColorScheme: ColorScheme { + appearance.colorScheme ?? colorScheme + } + + var body: some View { + NavigationStack { + Group { + if let url = ReleaseNotesStore.pageURL( + language: language, + colorScheme: resolvedColorScheme + ) { + ZStack { + RemoteWebView(url: url, isLoading: $isLoading) + if isLoading { + ProgressView() + .tint(palette.accent) + } + } + } else { + ContentUnavailableView( + AppL10n.string("releaseNotes.unavailable.title", language: language), + systemImage: "wifi.slash", + description: Text( + AppL10n.string("releaseNotes.unavailable.message", language: language) + ) + ) + } + } + .background(palette.background.ignoresSafeArea()) + .navigationTitle(AppL10n.string("releaseNotes.title", language: language)) + .navigationBarTitleDisplayMode(.inline) + .toolbar { + ToolbarItem(placement: .confirmationAction) { + Button(AppL10n.string("common.done", language: language)) { + dismiss() + } + } + } + } + // Sheet can drop WindowGroup environment; re-assert language + appearance. + .environment(\.locale, language.swiftUILocale) + .preferredColorScheme(appearance.colorScheme) + .environment(\.themePalette, resolvedColorScheme == .dark ? Palette.dark : Palette.light) + } +} diff --git a/OSGKeyboard/Views/SettingsPreferenceRows.swift b/OSGKeyboard/Views/SettingsPreferenceRows.swift index dbcc804..f54268d 100644 --- a/OSGKeyboard/Views/SettingsPreferenceRows.swift +++ b/OSGKeyboard/Views/SettingsPreferenceRows.swift @@ -132,15 +132,16 @@ struct PolishIntensityPickerRow: View { struct DefaultTypingInputToggleRow: View { @Environment(\.themePalette) private var palette: ThemePalette + @ObservedObject private var config = ProviderConfig.shared @Binding var isOn: Bool var body: some View { Toggle(isOn: $isOn) { VStack(alignment: .leading, spacing: 3) { - Text("settings.typingInput.default.title") + Text(AppL10n.string("settings.typingInput.default.title", language: config.uiLanguage)) .font(TypeStyle.body) .foregroundStyle(palette.textPrimary) - Text("settings.typingInput.default.description") + Text(AppL10n.string("settings.typingInput.default.description", language: config.uiLanguage)) .font(TypeStyle.caption) .foregroundStyle(palette.textSecondary) .fixedSize(horizontal: false, vertical: true) @@ -153,15 +154,16 @@ struct DefaultTypingInputToggleRow: View { struct RememberLastSurfaceToggleRow: View { @Environment(\.themePalette) private var palette: ThemePalette + @ObservedObject private var config = ProviderConfig.shared @Binding var isOn: Bool var body: some View { Toggle(isOn: $isOn) { VStack(alignment: .leading, spacing: 3) { - Text("settings.typingInput.rememberLast.title") + Text(AppL10n.string("settings.typingInput.rememberLast.title", language: config.uiLanguage)) .font(TypeStyle.body) .foregroundStyle(palette.textPrimary) - Text("settings.typingInput.rememberLast.description") + Text(AppL10n.string("settings.typingInput.rememberLast.description", language: config.uiLanguage)) .font(TypeStyle.caption) .foregroundStyle(palette.textSecondary) .fixedSize(horizontal: false, vertical: true) diff --git a/OSGKeyboard/Views/SettingsSecondaryPages.swift b/OSGKeyboard/Views/SettingsSecondaryPages.swift index 391ef06..b070749 100644 --- a/OSGKeyboard/Views/SettingsSecondaryPages.swift +++ b/OSGKeyboard/Views/SettingsSecondaryPages.swift @@ -13,15 +13,35 @@ import OSGKeyboardShared struct SettingsNavigationRow: View { @Environment(\.themePalette) private var palette: ThemePalette - let title: LocalizedStringKey + private let localizedTitle: LocalizedStringKey? + private let resolvedTitle: String? var subtitle: String? + init(title: LocalizedStringKey, subtitle: String? = nil) { + self.localizedTitle = title + self.resolvedTitle = nil + self.subtitle = subtitle + } + + /// Prefers in-app language via an already-resolved string (`AppL10n`). + init(titleText: String, subtitle: String? = nil) { + self.localizedTitle = nil + self.resolvedTitle = titleText + self.subtitle = subtitle + } + var body: some View { HStack(spacing: Spacing.sm) { VStack(alignment: .leading, spacing: Spacing.xxs) { - Text(title) - .font(TypeStyle.body) - .foregroundStyle(palette.textPrimary) + if let resolvedTitle { + Text(resolvedTitle) + .font(TypeStyle.body) + .foregroundStyle(palette.textPrimary) + } else if let localizedTitle { + Text(localizedTitle) + .font(TypeStyle.body) + .foregroundStyle(palette.textPrimary) + } if let subtitle, !subtitle.isEmpty { Text(subtitle) .font(TypeStyle.caption2) @@ -39,23 +59,33 @@ struct SettingsNavigationRow: View { } } -/// Read-only version / build row for Settings home (below About). +/// Version / build row for Settings home (below About). Opens release notes. struct SettingsVersionRow: View { @Environment(\.themePalette) private var palette: ThemePalette + @ObservedObject private var config = ProviderConfig.shared var body: some View { - HStack(spacing: Spacing.sm) { - Text("settings.version.title") - .font(TypeStyle.body) - .foregroundStyle(palette.textPrimary) - Spacer(minLength: Spacing.xs) - Text(AppVersionDisplay.detailedLabel) - .font(TypeStyle.body) - .foregroundStyle(palette.textSecondary) - .multilineTextAlignment(.trailing) + Button { + ReleaseNotesController.shared.presentManually() + } label: { + HStack(spacing: Spacing.sm) { + Text(AppL10n.string("settings.version.title", language: config.uiLanguage)) + .font(TypeStyle.body) + .foregroundStyle(palette.textPrimary) + Spacer(minLength: Spacing.xs) + Text(AppVersionDisplay.detailedLabel) + .font(TypeStyle.body) + .foregroundStyle(palette.textSecondary) + .multilineTextAlignment(.trailing) + Image(systemName: "chevron.right") + .font(.system(size: 14, weight: .semibold)) + .foregroundStyle(palette.textTertiary) + } + .settingsListRow() + .contentShape(Rectangle()) } - .settingsListRow() - .accessibilityElement(children: .combine) + .buttonStyle(.plain) + .accessibilityHint(AppL10n.string("releaseNotes.openHint", language: config.uiLanguage)) } } @@ -205,7 +235,12 @@ struct GeneralSettingsView: View { NavigationLink { TypingInputSettingsView() } label: { - SettingsNavigationRow(title: "settings.typingInput.title") + SettingsNavigationRow( + titleText: AppL10n.string( + "settings.typingInput.title", + language: config.uiLanguage + ) + ) } .buttonStyle(.plain) diff --git a/OSGKeyboard/Views/SettingsView.swift b/OSGKeyboard/Views/SettingsView.swift index f1d3b8e..35e23fd 100644 --- a/OSGKeyboard/Views/SettingsView.swift +++ b/OSGKeyboard/Views/SettingsView.swift @@ -171,7 +171,7 @@ struct SettingsView: View { Divider().background(palette.divider) - // Read-only version row — not tappable. + // Opens the remote release-notes sheet (same as post-upgrade prompt). SettingsVersionRow() } .surfaceCard() diff --git a/OSGKeyboard/Views/TypingInputSettingsView.swift b/OSGKeyboard/Views/TypingInputSettingsView.swift index 3940fc9..9c71f39 100644 --- a/OSGKeyboard/Views/TypingInputSettingsView.swift +++ b/OSGKeyboard/Views/TypingInputSettingsView.swift @@ -9,6 +9,7 @@ import OSGKeyboardShared struct TypingInputSettingsView: View { @Environment(\.themePalette) private var palette: ThemePalette + @ObservedObject private var config = ProviderConfig.shared @ObservedObject private var configuration = TypingInputConfiguration.shared @State private var isDeploying = false @@ -16,10 +17,14 @@ struct TypingInputSettingsView: View { var body: some View { List { - Section("输入方案") { - Picker("拼音方案", selection: $configuration.schema) { + Section(AppL10n.string("settings.typingInput.schema.section", language: config.uiLanguage)) { + Picker( + AppL10n.string("settings.typingInput.schema.picker", language: config.uiLanguage), + selection: $configuration.schema + ) { ForEach(TypingInputSchema.allCases) { schema in - Text(schema.displayName).tag(schema) + Text(AppL10n.string(schema.labelKey, language: config.uiLanguage)) + .tag(schema) } } .pickerStyle(.inline) @@ -39,14 +44,14 @@ struct TypingInputSettingsView: View { ) } } header: { - Text("模糊音") + Text(AppL10n.string("settings.typingInput.fuzzy.section", language: config.uiLanguage)) } footer: { - Text("默认全部关闭。只开启你需要的组合，避免候选噪音。") + Text(AppL10n.string("settings.typingInput.fuzzy.footer", language: config.uiLanguage)) } - Section("输入法资源") { + Section(AppL10n.string("settings.typingInput.resources.section", language: config.uiLanguage)) { HStack { - Text("状态") + Text(AppL10n.string("settings.typingInput.resources.status", language: config.uiLanguage)) Spacer() if isDeploying { ProgressView() @@ -59,7 +64,7 @@ struct TypingInputSettingsView: View { } } - Button("重新部署输入法资源") { + Button(AppL10n.string("settings.typingInput.resources.redeploy", language: config.uiLanguage)) { deployUpdatedSchemas() } .disabled(isDeploying) @@ -67,13 +72,18 @@ struct TypingInputSettingsView: View { } .scrollContentBackground(.hidden) .background(palette.background) - .navigationTitle("settings.typingInput.title") + .navigationTitle(AppL10n.string("settings.typingInput.title", language: config.uiLanguage)) .navigationBarTitleDisplayMode(.inline) } private var statusText: String { if let deploymentError { return deploymentError } - return RimeResourceInstaller.isReady ? "已就绪" : "待初始化" + return AppL10n.string( + RimeResourceInstaller.isReady + ? "settings.typingInput.resources.ready" + : "settings.typingInput.resources.pending", + language: config.uiLanguage + ) } private func deployUpdatedSchemas() { diff --git a/OSGKeyboard/en.lproj/Localizable.strings b/OSGKeyboard/en.lproj/Localizable.strings index 7018704..8137457 100644 --- a/OSGKeyboard/en.lproj/Localizable.strings +++ b/OSGKeyboard/en.lproj/Localizable.strings @@ -182,6 +182,10 @@ "settings.systemPrompt.hint" = "Used by the cloud LLM when polishing your dictation. Local engine skips this step."; "settings.about.title" = "About"; "settings.version.title" = "Version"; +"releaseNotes.title" = "What's New"; +"releaseNotes.openHint" = "View release notes for this version"; +"releaseNotes.unavailable.title" = "Cannot load page"; +"releaseNotes.unavailable.message" = "Release notes could not be opened. Check your network connection and try again."; "settings.daily.title" = "Daily"; "settings.config.title" = "Configuration"; "settings.general.title" = "General"; @@ -193,6 +197,18 @@ "settings.typingInput.default.description" = "Open the text input keyboard instead of voice input by default"; "settings.typingInput.rememberLast.title" = "Remember Last Choice"; "settings.typingInput.rememberLast.description" = "Reopen on the voice or text surface you left last time"; +"settings.typingInput.schema.section" = "Input Method"; +"settings.typingInput.schema.picker" = "Pinyin Scheme"; +"typing.schema.fullPinyin" = "Full Pinyin"; +"typing.schema.microsoftDoublePinyin" = "Microsoft Shuangpin"; +"typing.schema.sogouDoublePinyin" = "Sogou Shuangpin"; +"settings.typingInput.fuzzy.section" = "Fuzzy Pinyin"; +"settings.typingInput.fuzzy.footer" = "All off by default. Enable only the pairs you need to avoid noisy candidates."; +"settings.typingInput.resources.section" = "Input Resources"; +"settings.typingInput.resources.status" = "Status"; +"settings.typingInput.resources.ready" = "Ready"; +"settings.typingInput.resources.pending" = "Not initialized"; +"settings.typingInput.resources.redeploy" = "Redeploy Input Resources"; "settings.speechRecognition.title" = "Speech Recognition"; "settings.textPolish.title" = "Text Polish"; "settings.preferences.title" = "Preferences"; diff --git a/OSGKeyboard/zh-Hans.lproj/Localizable.strings b/OSGKeyboard/zh-Hans.lproj/Localizable.strings index bd8b723..1ab4133 100644 --- a/OSGKeyboard/zh-Hans.lproj/Localizable.strings +++ b/OSGKeyboard/zh-Hans.lproj/Localizable.strings @@ -182,6 +182,10 @@ "settings.systemPrompt.hint" = "云端润色时使用。本地识别模式不会用到此提示词。"; "settings.about.title" = "关于"; "settings.version.title" = "版本"; +"releaseNotes.title" = "更新说明"; +"releaseNotes.openHint" = "查看本版本更新说明"; +"releaseNotes.unavailable.title" = "无法加载页面"; +"releaseNotes.unavailable.message" = "无法打开更新说明，请检查网络连接后重试。"; "settings.daily.title" = "日常"; "settings.config.title" = "配置"; "settings.general.title" = "通用"; @@ -193,6 +197,18 @@ "settings.typingInput.default.description" = "打开键盘时，默认使用文字输入键盘而非语音输入"; "settings.typingInput.rememberLast.title" = "记住上次选择"; "settings.typingInput.rememberLast.description" = "下次打开键盘时，保持你上次离开时的语音或文字输入界面"; +"settings.typingInput.schema.section" = "输入方案"; +"settings.typingInput.schema.picker" = "拼音方案"; +"typing.schema.fullPinyin" = "全拼"; +"typing.schema.microsoftDoublePinyin" = "微软双拼"; +"typing.schema.sogouDoublePinyin" = "搜狗双拼"; +"settings.typingInput.fuzzy.section" = "模糊音"; +"settings.typingInput.fuzzy.footer" = "默认全部关闭。只开启你需要的组合，避免候选噪音。"; +"settings.typingInput.resources.section" = "输入法资源"; +"settings.typingInput.resources.status" = "状态"; +"settings.typingInput.resources.ready" = "已就绪"; +"settings.typingInput.resources.pending" = "待初始化"; +"settings.typingInput.resources.redeploy" = "重新部署输入法资源"; "settings.speechRecognition.title" = "语音识别配置"; "settings.textPolish.title" = "文本润色配置"; "settings.preferences.title" = "偏好设置"; diff --git a/OSGKeyboardExt/KeyboardViewController.swift b/OSGKeyboardExt/KeyboardViewController.swift index 430a830..51176b0 100644 --- a/OSGKeyboardExt/KeyboardViewController.swift +++ b/OSGKeyboardExt/KeyboardViewController.swift @@ -50,6 +50,16 @@ public final class KeyboardViewController: UIInputViewController { private var hosting: UIHostingController? private var keyboardHeightConstraint: NSLayoutConstraint? private var systemEncapsulatedHeight: CGFloat = 228 + /// Presentation height priming is only valid during the slide-in. After + /// `viewDidAppear` we must keep the constraint at `target` — re-applying + /// the offset (or letting a paste alert interrupt the appear sequence) + /// makes the slot land at `target + encapsulated` and floats the chrome. + private enum HeightPresentationPhase { + case idle + case priming + case presented + } + private var heightPhase: HeightPresentationPhase = .idle private var cancellables = Set() private var textInserter: KeyboardTextInserter! @@ -118,6 +128,7 @@ public final class KeyboardViewController: UIInputViewController { _ = configSync.loadPersistedConfig() configSync.installDarwinObservers() flowCoordinator.refreshSessionState() + flowCoordinator.restoreClipboardCommandIfNeeded() OSGDiag.log( "KVC.viewDidLoad done surface=\(state.surface.rawValue) " + "sessionActive=\(FlowSessionBridge.isSessionActive()) " @@ -133,15 +144,27 @@ public final class KeyboardViewController: UIInputViewController { + "preserve=\(flowCoordinator.preservesLifecycleOnDisappear) \(OSGDiag.memoryTag())", category: "boot" ) + heightPhase = .idle + // Block pasteboard content reads until the next viewDidAppear height lock. + flowCoordinator.setClipboardContentReadsEnabled(false) flowCoordinator.stopSessionMonitor() // Remember what the user left on, then pre-position a reused // extension instance for the next open policy (no first-frame jump). - TypingInputConfiguration.persistLastSurface(state.surface) - prepareSurfaceForNextPresentation() + // Skip snap-to-typing while clipboard paste alert / utterance owns the mic — + // otherwise Allow Paste reopens on the typing grid (visible jump). + let preserve = flowCoordinator.preservesLifecycleOnDisappear + || flowCoordinator.isClipboardCommandActive + || ClipboardCommandResume.shouldPreferVoice() + TypingInputConfiguration.persistLastSurface( + preserve || ClipboardCommandResume.shouldPreferVoice() ? .voice : state.surface + ) + if !preserve { + prepareSurfaceForNextPresentation() + } if state.surface == .typing { typingSession.leaveTypingMode() } - if flowCoordinator.preservesLifecycleOnDisappear { + if preserve { return } ExtensionScreenWakeLock.releaseAll() @@ -159,13 +182,16 @@ public final class KeyboardViewController: UIInputViewController { configureDictationBehavior() KeyboardSetupBridge.markExtensionAppearance(hasFullAccess: hasFullAccess) state.debugHasFullAccess = hasFullAccess + // Do NOT read pasteboard contents here — `refreshSessionState` may peek + // changeCount only while content reads stay disabled until height locks. flowCoordinator.refreshSessionState() - flowCoordinator.refreshClipboardEligibility() flowCoordinator.startSessionMonitor() configSync.syncOnboardingStateFromAppGroup() configSync.refreshConfigFromAppGroup() // Settings may have changed while the extension stayed alive. applyPreferredSurfaceOnOpen() + // After paste-alert reopen: restore clipboard chrome if sticky + host busy. + flowCoordinator.restoreClipboardCommandIfNeeded() // Re-warm Taptic after host app switches: SwiftUI `onAppear` often // skips when the extension process is reused, leaving generators cold. KeyboardHapticFeedback.prepare() @@ -181,9 +207,17 @@ public final class KeyboardViewController: UIInputViewController { public override func viewIsAppearing(_ animated: Bool) { super.viewIsAppearing(animated) - applyPresentationHeightOffset() + if heightPhase == .presented { + // Spurious re-appear while already on screen (e.g. system alert + // lifecycle noise) — never re-run the offset trick. + lockPresentedKeyboardHeight() + } else { + heightPhase = .priming + applyPresentationHeightOffset() + } OSGDiag.log( - "KVC.viewIsAppearing height=\(keyboardHeightConstraint?.constant ?? -1) " + "KVC.viewIsAppearing phase=\(heightPhaseLog) " + + "height=\(keyboardHeightConstraint?.constant ?? -1) " + "\(OSGDiag.memoryTag())", category: "boot" ) @@ -196,12 +230,15 @@ public final class KeyboardViewController: UIInputViewController { category: "boot" ) disableSystemGestureDelays() - keyboardHeightConstraint?.constant = targetKeyboardHeight + heightPhase = .presented + lockPresentedKeyboardHeight() refreshReturnKeyRole() - // Run after the extension is fully presented so UIKit accepts the - // containing-app handoff even when typing mode is the default surface. + // After height is locked: (1) allow pasteboard content reads so the + // paste alert cannot interrupt presentation math; (2) arm PiP handoff. DispatchQueue.main.async { [weak self] in - self?.flowCoordinator.ensurePiPReadyOnKeyboardOpen() + guard let self, self.heightPhase == .presented else { return } + self.flowCoordinator.setClipboardContentReadsEnabled(true) + self.flowCoordinator.ensurePiPReadyOnKeyboardOpen() } OSGDiag.log( "KVC.viewDidAppear done height=\(targetKeyboardHeight) \(OSGDiag.memoryTag())", @@ -240,6 +277,7 @@ public final class KeyboardViewController: UIInputViewController { public override func viewDidLayoutSubviews() { super.viewDidLayoutSubviews() cursorDrag?.layoutChrome() + enforcePresentedKeyboardHeightIfNeeded() } // MARK: - Services @@ -290,9 +328,6 @@ public final class KeyboardViewController: UIInputViewController { state.beginClipboardCommand = { [weak self] in self?.flowCoordinator.clipboardCommandPressBegan() } - state.endClipboardCommand = { [weak self] in - self?.flowCoordinator.clipboardCommandPressEnded() - } state.refreshClipboardEligibility = { [weak self] in self?.flowCoordinator.refreshClipboardEligibility() } @@ -368,13 +403,21 @@ public final class KeyboardViewController: UIInputViewController { private func applyPreferredSurfaceOnOpen() { let preferred = TypingInputConfiguration.preferredSurfaceOnOpen() + let sticky = ClipboardCommandResume.shouldPreferVoice() + let resolved = KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: state.locksTypingSurface, + clipboardCommandActive: flowCoordinator.isClipboardCommandActive, + stickyPreferVoice: sticky, + preferred: preferred + ) OSGDiag.log( "applyPreferredSurfaceOnOpen preferred=\(preferred.rawValue) " - + "remember=\(TypingInputConfiguration.remembersLastSurface()) " - + "defaultTyping=\(TypingInputConfiguration.prefersTypingOnOpen())", + + "resolved=\(resolved.rawValue) stickyVoice=\(sticky ? 1 : 0) " + + "clipboardActive=\(flowCoordinator.isClipboardCommandActive ? 1 : 0) " + + "locksTyping=\(state.locksTypingSurface ? 1 : 0)", category: "boot" ) - applySurface(preferred) + applySurface(resolved) } /// When not remembering, snap to the static open preference while hidden @@ -395,10 +438,42 @@ public final class KeyboardViewController: UIInputViewController { // final assignment in `viewDidAppear`. Avoid synchronous layout here: // this is also called during `viewDidLoad`, where re-entrant layout can // observe partially initialized controller dependencies. + if heightPhase == .presented { + lockPresentedKeyboardHeight() + } else { + keyboardHeightConstraint?.constant = targetKeyboardHeight + view.setNeedsLayout() + } + } + + private var heightPhaseLog: String { + switch heightPhase { + case .idle: return "idle" + case .priming: return "priming" + case .presented: return "presented" + } + } + + private func lockPresentedKeyboardHeight() { keyboardHeightConstraint?.constant = targetKeyboardHeight view.setNeedsLayout() } + /// After presentation, the constraint must stay at `target`. Spurious + /// lifecycle noise must not leave us primed at `target − encapsulated`. + private func enforcePresentedKeyboardHeightIfNeeded() { + guard heightPhase == .presented else { return } + let target = targetKeyboardHeight + guard let constraint = keyboardHeightConstraint else { return } + let was = constraint.constant + guard abs(was - target) > 0.5 else { return } + constraint.constant = target + OSGDiag.log( + "KVC.heightEnforce constraint \(Int(was))→\(Int(target))", + category: "boot" + ) + } + private func refreshReturnKeyRole() { state.returnKeyRole = returnKeyRole(for: textDocumentProxy.returnKeyType ?? .default) // Secure fields must not run English autocomplete / autocorrect / learning. @@ -480,6 +555,8 @@ public final class KeyboardViewController: UIInputViewController { } private func applyPresentationHeightOffset() { + // Only valid while priming the slide-in. Callers must not invoke this + // after `heightPhase == .presented`. if let encapsulated = view.constraints.first(where: { constraint in constraint.firstItem as? UIView === view && constraint.firstAttribute == .height diff --git a/OSGKeyboardExt/Services/ClipboardPasteboardReader.swift b/OSGKeyboardExt/Services/ClipboardPasteboardReader.swift index 16c3429..50db270 100644 --- a/OSGKeyboardExt/Services/ClipboardPasteboardReader.swift +++ b/OSGKeyboardExt/Services/ClipboardPasteboardReader.swift @@ -1,12 +1,27 @@ // ClipboardPasteboardReader.swift // OSGKeyboard · Keyboard Extension // -// Opportunity-read of UIPasteboard for clipboard-command eligibility. +// Pasteboard peeks for clipboard-command UI and long-press snapshot capture. +// +// Idle affordance must use metadata only (`hasStrings` / `changeCount`) so the +// system「允许粘贴」alert never appears while the keyboard is merely open. +// Content reads (`string`) happen only on an explicit long-press. import UIKit import OSGKeyboardShared enum ClipboardPasteboardReader { + /// Metadata-only — does not trigger the paste permission prompt. + static func changeCount() -> Int { + UIPasteboard.general.changeCount + } + + /// Metadata-only — whether the pasteboard currently holds string items. + static func hasStrings() -> Bool { + UIPasteboard.general.hasStrings + } + + /// Content sample for long-press snapshot. May present the system paste alert. static func sample() -> (changeCount: Int, text: String?) { let board = UIPasteboard.general let changeCount = board.changeCount diff --git a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift index 1f696a3..4915655 100644 --- a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift +++ b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift @@ -67,21 +67,26 @@ final class KeyboardFlowCoordinator { /// Ignores single-frame "host dead" samples before allowing a cold-start jump /// from non-press recovery paths. private var coldStartDebouncer = FlowColdStartDebouncer() - /// Opportunity-read eligibility window (copy → 30s). - private var clipboardEligibility: ClipboardCommandEligibility? - /// Active clipboard-command task session (continuous rewrite). - private var clipboardTaskSession: ClipboardCommandTaskSession? - /// True while the live utterance is a clipboard-command hold-to-talk. + /// Frozen clipboard material for the in-flight clipboard-command utterance. + private var clipboardFrozenSnapshot: String? + /// True while the live utterance is a clipboard-command round. private var isClipboardCommandUtterance = false - /// Clipboard start sent; waiting for host `reason=recording` before red UI. + /// True while blocked inside `UIPasteboard.string` (system paste alert). + /// Must preserve extension lifecycle / voice surface across that alert. + private var isAcquiringClipboardPaste = false + /// Ignore the finger-up that follows long-press start (tap-to-stop, not release-to-stop). + private var suppressNextMicToggle = false + /// Clipboard start sent; waiting for host `reason=recording` before confirmed capture UI. private var clipboardAwaitingHostRecordConfirm = false /// Wall time when host confirmed real capture for this clipboard utterance. private var clipboardHostRecordConfirmedAt: TimeInterval? - /// Finger already up — stop after confirm + minimum recording window. + /// User tapped stop — honor after confirm + minimum recording window. private var clipboardStopRequested = false private var clipboardDeferredStopTask: Task? - /// Last pasteboard changeCount we already asked the host to light-prewarm. - private var clipboardPrewarmedChangeCount: Int? + private var clipboardFailureHintTask: Task? + private var clipboardPreparingWatchdogTask: Task? + /// Wall time when we entered clipboard「准备录音…」awaiting host confirm. + private var clipboardPreparingStartedAt: TimeInterval = 0 init( state: KeyboardState, @@ -106,7 +111,19 @@ final class KeyboardFlowCoordinator { } var preservesLifecycleOnDisappear: Bool { - isPendingFlowStart || isFlowRecording || isAwaitingFlowResult + isPendingFlowStart + || isFlowRecording + || isAwaitingFlowResult + || isClipboardCommandUtterance + || isAcquiringClipboardPaste + || ClipboardCommandResume.shouldPreferVoice() + } + + /// True while a clipboard-command round owns the mic (incl. paste-alert acquire). + var isClipboardCommandActive: Bool { + isClipboardCommandUtterance + || isAcquiringClipboardPaste + || ClipboardCommandResume.shouldPreferVoice() } /// Session/transcription changes are pushed in real time by Darwin @@ -135,6 +152,10 @@ final class KeyboardFlowCoordinator { /// keyboard opens directly into Pinyin/English typing mode. The mic stays /// visually ready; if the host contract is missing, one automatic handoff /// prepares PiP so the next press does not need another app switch. + /// + /// Must **not** re-jump when a session is already alive/warming (common after + /// the user returns from startflow and taps the Voice tab — height/appear + /// noise used to call this again while `ready` briefly lagged). func ensurePiPReadyOnKeyboardOpen() { guard FlowHandoffPolicy.allowsProactiveHostAutoLaunch, state.hasCompletedOnboarding, @@ -145,13 +166,24 @@ final class KeyboardFlowCoordinator { !isAwaitingFlowResult else { return } FlowSessionBridge.reloadFromDisk() - if FlowSessionBridge.isHostReady() { return } - - if let reason = FlowSessionBridge.readySnapshot()?.reason, - reason == .recording || reason == .processing || reason == .awaitingDelivery { + let withinReadyGrace = lastHostReadyAt > 0 + && (Date().timeIntervalSince1970 - lastHostReadyAt) <= Self.hostReadyGrace + let shouldArm = FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: FlowSessionBridge.isHostReady(), + snapshotReason: FlowSessionBridge.readySnapshot()?.reason, + sessionActive: FlowSessionBridge.isSessionActive(), + hostReachable: FlowSessionBridge.isHostReachable(), + hostStale: FlowSessionBridge.isHostStale(), + withinReadyGrace: withinReadyGrace, + inCooldown: FlowSessionBridge.isPiPArmInCooldown(), + heartbeatStaleness: FlowSessionBridge.heartbeatStaleness() + ) + guard shouldArm else { + traceState("keyboardOpen.autoArmPiP.skipped", extra: "gate=0") return } + FlowSessionBridge.markPiPArmAttempt() detectAndStoreAppContext() beginFlowStart(recordAfterHandoff: false) traceState("keyboardOpen.autoArmPiP") @@ -182,6 +214,7 @@ final class KeyboardFlowCoordinator { recomputeMicVoiceAvailability() refreshClipboardEligibility() promoteClipboardRecordingFromSnapshotIfNeeded() + recoverClipboardPreparingIfHostMovedOn() startHostReadyWaitIfNeeded() // Proactive host auto-launch is disabled (FlowHandoffPolicy): a single // stale ready snapshot after finalize must never open startflow. @@ -312,6 +345,15 @@ final class KeyboardFlowCoordinator { flowStartDeadline = 0 stopHostReadyWait() isFlowRecording = true + // Paste-alert may have destroyed in-memory clipboard flags; sticky + // snapshot means this busy utterance is still a clipboard round. + if !isClipboardCommandUtterance, + ClipboardCommandResume.shouldPreferVoice() { + if let snap = ClipboardCommandResume.pendingSnapshot(), !snap.isEmpty { + clipboardFrozenSnapshot = snap + } + isClipboardCommandUtterance = true + } if isClipboardCommandUtterance { noteClipboardHostRecordingConfirmed() } else { @@ -320,6 +362,7 @@ final class KeyboardFlowCoordinator { state.lastTranscript = "" } } + publishClipboardUIState() if let view = wakeLockView() { ExtensionScreenWakeLock.acquire(from: view) } @@ -467,22 +510,39 @@ final class KeyboardFlowCoordinator { } func toggleRecording() { + // Finger-up after long-press start must not stop / start dictation. + if suppressNextMicToggle { + suppressNextMicToggle = false + if isClipboardCommandUtterance + || state.phase == .recording + || state.phase == .requestingPermissions { + traceState("clipboard.toggle.suppressed", extra: "reason=postLongPressRelease") + return + } + } switch state.phase { case .recording: - pressEnded() - case .idle, .denied, .error: - // Short press while a clipboard task is open → exit command mode, then dictation. - if clipboardTaskSession != nil { - endClipboardTaskSession() + if isClipboardCommandUtterance { + requestClipboardStop() + } else { + pressEnded() } + case .requestingPermissions: + // Clipboard preparing: tap cancels/stops once host confirms (+ min window). + if isClipboardCommandUtterance { + requestClipboardStop() + } + case .idle, .denied, .error: isClipboardCommandUtterance = false + clipboardFrozenSnapshot = nil + publishClipboardUIState() pressBegan() - case .requestingPermissions, .processing: + case .processing: break } } - /// Long-press reached 0.45s — start clipboard-command hold-to-talk immediately. + /// Long-press reached 0.45s — resolve material, gate on host, then record or cold-start. func clipboardCommandPressBegan() { switch state.phase { case .idle, .denied, .error: @@ -493,46 +553,252 @@ final class KeyboardFlowCoordinator { return } - refreshClipboardEligibility() - let secure = fieldContextProvider()?.isSecureEntry == true - if secure { - endClipboardTaskSession() - publishClipboardUIState() + clearClipboardFailureHint() + + guard hasFullAccess() else { + showClipboardFailure(.noFullAccess) + return + } + if fieldContextProvider()?.isSecureEntry == true { + showClipboardFailure(.secureField) return } - let now = Date().timeIntervalSince1970 - if let session = clipboardTaskSession, session.isActive(at: now) { - isClipboardCommandUtterance = true - pressBegan() - return + // Prefer voice early so cold-start / paste-alert reopen lands on voice, + // even when default open surface is typing. + ClipboardCommandResume.markPreferVoice() + if state.surface != .voice { + state.setSurface(.voice) } + // Finger-up from this long-press must not toggle dictation. + suppressNextMicToggle = true - guard let eligibility = clipboardEligibility, eligibility.isOpen(at: now) else { - // No open window — do not enter command mode. - isClipboardCommandUtterance = false - return - } + guard let snapshot = resolveClipboardSnapshotForCommand() else { return } - let fingerprint = fieldContextProvider()?.deliveryFingerprint - clipboardTaskSession = ClipboardCommandTaskSession( - snapshot: eligibility.snapshot, - expiresAt: now + ClipboardMaterialFilter.sessionDuration, - fieldFingerprint: fingerprint - ) - isClipboardCommandUtterance = true + clipboardFrozenSnapshot = snapshot + ClipboardCommandResume.storeSnapshot(snapshot) publishClipboardUIState() - pressBegan() + + // Host gate *before* claim /「准备录音…」— never fake-record when host is dead. + // Use the *raw* ready contract (not holdReady grace) so the first press after + // cold-start does not claim while audio is still stale. + recomputeMicVoiceAvailability() + let hostReadyRaw = FlowSessionBridge.isHostReady() + let withinReadyGrace = lastHostReadyAt > 0 + && (Date().timeIntervalSince1970 - lastHostReadyAt) <= Self.hostReadyGrace + // Force-quit leaves sessionActive=true; treat unreachable heartbeat as dead + // so clipboard can still open startflow (same grace as proactive PiP arm). + let heartbeatStale = FlowSessionBridge.heartbeatStaleness() ?? .infinity + let sessionEffectivelyDead = !FlowSessionBridge.isHostReachable() + && heartbeatStale >= FlowHandoffPolicy.proactiveUnreachableArmGrace + let sessionActiveForGate = FlowSessionBridge.isSessionActive() && !sessionEffectivelyDead + let micAction: FlowMicPressAction + if hostReadyRaw, state.micVoiceAvailability.isReady { + micAction = .startRecording + } else { + micAction = FlowHandoffPolicy.micPressAction( + availability: hostReadyRaw ? state.micVoiceAvailability : .unavailable(.hostNotReady), + sessionActive: sessionActiveForGate, + hostReachable: FlowSessionBridge.isHostReachable(), + hostStale: FlowSessionBridge.isHostStale() || sessionEffectivelyDead, + withinReadyGrace: withinReadyGrace + ) + } + switch ClipboardPreparingPolicy.hostGateAction(micPressAction: micAction) { + case .startRecordingNow: + beginClipboardRecordingRound() + case .openHostColdStart: + deferClipboardUntilHostWarm(reason: "coldStart") + detectAndStoreAppContext() + if !isPendingFlowStart, !FlowSessionBridge.isPiPArmInCooldown() { + beginFlowStart(recordAfterHandoff: false) + } + showClipboardHostWarmupHint() + case .waitForHost: + deferClipboardUntilHostWarm(reason: "waitHost") + // Do not set recordWhenHostReady — auto-record after warm-up is opt-out. + showClipboardHostWarmupHint() + case .ignore: + // Specific hard gates (API key / full access) already handled above; + // leftover ignore → treat as host not ready. + deferClipboardUntilHostWarm(reason: "gateIgnore") + showClipboardHostWarmupHint() + } } - func clipboardCommandPressEnded() { - guard isClipboardCommandUtterance else { return } - clipboardStopRequested = true - if clipboardAwaitingHostRecordConfirm { - // Finger up before host capture — keep preparing; stop after confirm + min window. - traceState("clipboard.stop.deferred", extra: "reason=awaitingHostConfirm") - return + /// Sticky snapshot (post cold-start) wins; otherwise one pasteboard content read. + private func resolveClipboardSnapshotForCommand() -> String? { + if let existing = clipboardFrozenSnapshot ?? ClipboardCommandResume.pendingSnapshot(), + !existing.isEmpty { + isAcquiringClipboardPaste = false + publishClipboardUIState() + return existing } + + isAcquiringClipboardPaste = true + publishClipboardUIState() + let sample = ClipboardPasteboardReader.sample() + isAcquiringClipboardPaste = false + + guard let raw = sample.text else { + ClipboardCommandResume.clear() + publishClipboardUIState() + showClipboardFailure( + ClipboardPasteboardReader.hasStrings() ? .pasteDenied : .material(.empty) + ) + refreshClipboardEligibility() + return nil + } + + switch ClipboardMaterialFilter.evaluate(raw) { + case .rejected(let reason): + ClipboardCommandResume.clear() + publishClipboardUIState() + showClipboardFailure(.material(reason)) + return nil + case .eligible(let snapshot): + return snapshot + } + } + + /// Host is ready — claim the round and start (grey preparing → blue after confirm). + private func beginClipboardRecordingRound() { + isClipboardCommandUtterance = true + let claimId = UUID() + currentUtteranceId = claimId + ClipboardCommandResume.markStartIssued(claimId) + publishClipboardUIState() + if state.surface != .voice { + state.setSurface(.voice) + } + pressBegan() + traceState("clipboard.round.begin", extra: "utterance=\(claimId.uuidString.prefix(8))") + } + + /// Keep sticky voice + snapshot; do not claim start or show「准备录音…」. + private func deferClipboardUntilHostWarm(reason: String) { + isClipboardCommandUtterance = false + clipboardAwaitingHostRecordConfirm = false + clipboardPreparingWatchdogTask?.cancel() + clipboardPreparingWatchdogTask = nil + // Drop any stale mid-flight claim so restore cannot auto-write startRecording. + if ClipboardCommandResume.hasStartIssued() { + // Re-store snapshot/preferVoice without startIssued. + if let snap = clipboardFrozenSnapshot ?? ClipboardCommandResume.pendingSnapshot() { + ClipboardCommandResume.clear() + ClipboardCommandResume.storeSnapshot(snap) + } else { + ClipboardCommandResume.clear() + ClipboardCommandResume.markPreferVoice() + } + } + publishClipboardUIState() + traceState("clipboard.round.deferred", extra: reason) + } + + /// Soft tip that does **not** clear sticky snapshot / prefer-voice. + private func showClipboardHostWarmupHint() { + state.clipboardFailureHint = ExtL10n.string("keyboard.clipboard.hint.hostStarting") + clipboardFailureHintTask?.cancel() + let duration = ClipboardMaterialFilter.failureHintDuration + clipboardFailureHintTask = Task { @MainActor [weak self] in + try? await Task.sleep(nanoseconds: UInt64(duration * 1_000_000_000)) + guard let self, !Task.isCancelled else { return } + self.clearClipboardFailureHint() + } + } + + /// After paste-alert / cold-start keyboard reopen: force voice; resume mid-flight only. + func restoreClipboardCommandIfNeeded() { + guard ClipboardCommandResume.shouldPreferVoice() else { return } + if state.surface != .voice { + state.setSurface(.voice) + } + + // Snapshot for the next long-press (cold-start return) — not a live round yet. + if let snapshot = ClipboardCommandResume.pendingSnapshot(), !snapshot.isEmpty { + clipboardFrozenSnapshot = snapshot + } + + let hasIssued = ClipboardCommandResume.hasStartIssued() + if hasIssued, + !isClipboardCommandUtterance, + let issued = ClipboardCommandResume.startIssuedUtteranceId() { + // Mid-flight paste-alert recreate — reattach the live clipboard round. + isClipboardCommandUtterance = true + currentUtteranceId = issued + suppressNextMicToggle = true + traceState( + "clipboard.resume.rehydratedLive", + extra: "utterance=\(issued.uuidString.prefix(8))" + ) + } + + // Prefer adopting an in-flight host utterance (keeps one wire start). + FlowSessionBridge.reloadFromDisk() + if let ready = FlowSessionBridge.readySnapshot(), + ready.reason == .recording || ready.reason == .processing { + adoptHostBusyStateIfNeeded(snapshot: ready) + } + + let preparingPhase: ClipboardPreparingPhase = { + switch state.phase { + case .idle: return .idle + case .denied: return .denied + case .error: return .error + case .requestingPermissions: return .requestingPermissions + case .recording: return .recording + case .processing: return .processing + } + }() + + switch ClipboardPreparingPolicy.restoreAction( + hasStartIssued: hasIssued, + phase: preparingPhase + ) { + case .awaitExistingStart: + isClipboardCommandUtterance = true + clipboardAwaitingHostRecordConfirm = true + clipboardPreparingStartedAt = Date().timeIntervalSince1970 + state.phase = .requestingPermissions + state.lastTranscript = ExtL10n.string("keyboard.placeholder.preparingRecording") + publishClipboardUIState() + scheduleClipboardPreparingWatchdog() + ensureClipboardStartCommandWritten() + recoverClipboardPreparingIfHostMovedOn() + traceState("clipboard.resume.awaitExistingStart") + case .preferVoiceOnly: + // Cold-start / deferred host: stay on voice with sticky snapshot; user long-presses again. + isClipboardCommandUtterance = false + clipboardAwaitingHostRecordConfirm = false + publishClipboardUIState() + refreshClipboardEligibility() + traceState( + "clipboard.resume.preferVoiceOnly", + extra: "snapshot=\(clipboardFrozenSnapshot == nil ? 0 : 1)" + ) + case .refreshOnly: + publishClipboardUIState() + if clipboardAwaitingHostRecordConfirm { + ensureClipboardStartCommandWritten() + } + recoverClipboardPreparingIfHostMovedOn() + } + } + + /// Explicit tap-to-stop for an in-flight clipboard-command utterance. + func requestClipboardStop() { + guard isClipboardCommandUtterance else { return } + switch ClipboardPreparingPolicy.stopWhilePreparing( + awaitingHostConfirm: clipboardAwaitingHostRecordConfirm + ) { + case .abortPreparing: + abortClipboardPreparing(reason: "userCancel", message: nil) + return + case .requestStop: + break + } + clipboardStopRequested = true if let confirmedAt = clipboardHostRecordConfirmedAt { let elapsed = Date().timeIntervalSince1970 - confirmedAt let minimum = ClipboardMaterialFilter.minimumRecordingAfterHostConfirm @@ -548,45 +814,79 @@ final class KeyboardFlowCoordinator { pressEnded() } + /// Idle affordance only: metadata `hasStrings`. Never reads pasteboard contents. + func setClipboardContentReadsEnabled(_ enabled: Bool) { + // Height-lock gate retained as a refresh hook after presentation; content + // reads are no longer tied to this flag. + if enabled { + refreshClipboardEligibility() + } + } + func refreshClipboardEligibility() { let secure = fieldContextProvider()?.isSecureEntry == true guard hasFullAccess(), !secure else { - clipboardEligibility = nil - clipboardPrewarmedChangeCount = nil + state.clipboardCommandEligible = false publishClipboardUIState() return } - - let sample = ClipboardPasteboardReader.sample() - let previousChangeCount = clipboardEligibility?.changeCount - clipboardEligibility = ClipboardCommandEligibilityTracker.refresh( - changeCount: sample.changeCount, - rawText: sample.text, - previous: clipboardEligibility - ) - - if let session = clipboardTaskSession, !session.isActive() { - endClipboardTaskSession() - } + // Sticky snapshot (after cold-start) keeps long-press affordance even if + // the pasteboard was cleared while the user was in the host app. + let hasStickyMaterial = !(ClipboardCommandResume.pendingSnapshot() ?? "").isEmpty + || !(clipboardFrozenSnapshot ?? "").isEmpty + state.clipboardCommandEligible = + ClipboardPasteboardReader.hasStrings() || hasStickyMaterial publishClipboardUIState() - requestClipboardLightPrewarmIfNeeded(previousChangeCount: previousChangeCount) } - private func endClipboardTaskSession() { + private func resetClipboardUtteranceState() { clipboardDeferredStopTask?.cancel() clipboardDeferredStopTask = nil - clipboardTaskSession = nil + clipboardPreparingWatchdogTask?.cancel() + clipboardPreparingWatchdogTask = nil + clipboardPreparingStartedAt = 0 + clipboardFrozenSnapshot = nil isClipboardCommandUtterance = false + isAcquiringClipboardPaste = false + suppressNextMicToggle = false clipboardAwaitingHostRecordConfirm = false clipboardHostRecordConfirmedAt = nil clipboardStopRequested = false + ClipboardCommandResume.clear() publishClipboardUIState() } private func publishClipboardUIState() { - let now = Date().timeIntervalSince1970 - state.clipboardCommandEligible = clipboardEligibility?.isOpen(at: now) == true - state.clipboardCommandSessionActive = clipboardTaskSession?.isActive(at: now) == true + state.clipboardCommandUtteranceActive = + isClipboardCommandUtterance || isAcquiringClipboardPaste + // Blue chrome only after host-confirmed capture — preparing stays grey. + state.clipboardCommandRecording = + isClipboardCommandUtterance + && state.phase == .recording + && !clipboardAwaitingHostRecordConfirm + } + + private func showClipboardFailure(_ failure: ClipboardCommandFailure) { + isClipboardCommandUtterance = false + clipboardFrozenSnapshot = nil + publishClipboardUIState() + state.clipboardFailureHint = ExtL10n.string(failure.localizationKey) + traceState("clipboard.rejected", extra: failure.localizationKey) + clipboardFailureHintTask?.cancel() + let duration = ClipboardMaterialFilter.failureHintDuration + clipboardFailureHintTask = Task { @MainActor [weak self] in + try? await Task.sleep(nanoseconds: UInt64(duration * 1_000_000_000)) + guard let self, !Task.isCancelled else { return } + self.clearClipboardFailureHint() + } + } + + private func clearClipboardFailureHint() { + clipboardFailureHintTask?.cancel() + clipboardFailureHintTask = nil + if state.clipboardFailureHint != nil { + state.clipboardFailureHint = nil + } } /// Promote preparing → recording when host publishes real capture; honor deferred stop. @@ -594,6 +894,9 @@ final class KeyboardFlowCoordinator { let now = Date().timeIntervalSince1970 if clipboardAwaitingHostRecordConfirm || clipboardHostRecordConfirmedAt == nil { clipboardAwaitingHostRecordConfirm = false + clipboardPreparingWatchdogTask?.cancel() + clipboardPreparingWatchdogTask = nil + clipboardPreparingStartedAt = 0 if clipboardHostRecordConfirmedAt == nil { clipboardHostRecordConfirmedAt = now } @@ -602,6 +905,7 @@ final class KeyboardFlowCoordinator { || state.lastTranscript == ExtL10n.string("keyboard.placeholder.preparing") { state.lastTranscript = "" } + publishClipboardUIState() KeyboardHapticFeedback.play(role: .action, intensity: state.keyboardHapticIntensity) traceState("clipboard.hostRecording.confirmed") } @@ -629,30 +933,60 @@ final class KeyboardFlowCoordinator { } } - /// Light prewarm: ASR assets only (no mic). Once per eligible pasteboard change. - private func requestClipboardLightPrewarmIfNeeded(previousChangeCount: Int?) { - guard let eligibility = clipboardEligibility, eligibility.isOpen() else { return } - if clipboardPrewarmedChangeCount == eligibility.changeCount { return } - // Only fire when eligibility newly appears or changeCount advances. - if previousChangeCount == eligibility.changeCount { return } - - guard let sessionId = FlowSessionBridge.readySnapshot()?.sessionId ?? activeSessionId else { - return + private func scheduleClipboardPreparingWatchdog() { + clipboardPreparingWatchdogTask?.cancel() + let timeout = ClipboardCommandResume.preparingTimeout + clipboardPreparingWatchdogTask = Task { @MainActor [weak self] in + try? await Task.sleep(nanoseconds: UInt64(timeout * 1_000_000_000)) + guard let self, !Task.isCancelled else { return } + guard self.isClipboardCommandUtterance, self.clipboardAwaitingHostRecordConfirm else { return } + self.abortClipboardPreparing( + reason: "watchdog", + message: ExtL10n.string(ClipboardCommandFailure.prepareFailed.localizationKey) + ) } - clipboardPrewarmedChangeCount = eligibility.changeCount - let command = FlowCommand( - sessionId: sessionId, - utteranceId: UUID(), - commandSeq: nextCommandSeq(), - action: .prewarm, - localeId: state.localeId - ) - FlowSessionBridge.writeCommand(command) - FlowTrace.keyboard( - "command.prewarm", - "seq=\(command.commandSeq) changeCount=\(eligibility.changeCount)" - ) - traceState("clipboard.prewarm.requested", extra: "changeCount=\(eligibility.changeCount)") + } + + /// Leave「准备录音…」without waiting for a host confirm that never arrives. + private func abortClipboardPreparing(reason: String, message: String?) { + guard isClipboardCommandUtterance, clipboardAwaitingHostRecordConfirm else { return } + clipboardPreparingWatchdogTask?.cancel() + clipboardPreparingWatchdogTask = nil + clipboardPreparingStartedAt = 0 + clipboardAwaitingHostRecordConfirm = false + clipboardStopRequested = false + clipboardDeferredStopTask?.cancel() + clipboardDeferredStopTask = nil + + if isFlowRecording { + writeCommand(.abort) + isFlowRecording = false + stopUtteranceCountdown() + ExtensionScreenWakeLock.release() + } + stopFlowWatchdog() + FlowSessionBridge.setPendingKeyboardUtteranceId(nil) + lastStoppedUtteranceId = currentUtteranceId + currentUtteranceId = nil + + resetClipboardUtteranceState() + if let message, !message.isEmpty { + state.clipboardFailureHint = message + clipboardFailureHintTask?.cancel() + let duration = ClipboardMaterialFilter.failureHintDuration + clipboardFailureHintTask = Task { @MainActor [weak self] in + try? await Task.sleep(nanoseconds: UInt64(duration * 1_000_000_000)) + guard let self, !Task.isCancelled else { return } + self.clearClipboardFailureHint() + } + state.phase = .idle + state.lastTranscript = "" + } else { + state.phase = .idle + state.lastTranscript = "" + } + recomputeMicVoiceAvailability() + traceState("clipboard.preparing.aborted", extra: reason) } /// If we already sent start and host is recording our utterance, confirm UI. @@ -665,6 +999,152 @@ final class KeyboardFlowCoordinator { noteClipboardHostRecordingConfirmed() } + /// Host finished/failed/took another utterance while UI still shows「准备录音…」. + private func recoverClipboardPreparingIfHostMovedOn() { + let hostReason: ClipboardHostBusyReason? = { + switch FlowSessionBridge.readySnapshot()?.reason { + case .recording: return .recording + case .processing: return .processing + default: return nil + } + }() + let action = ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: clipboardAwaitingHostRecordConfirm, + currentUtteranceId: currentUtteranceId, + hostBusyUtteranceId: FlowSessionBridge.readySnapshot()?.busyUtteranceId, + hostReason: hostReason, + hasTerminalFailureForCurrent: { + guard let result = matchingResult() else { return false } + return isTerminalFailure(result) + }() + ) + + switch action { + case .none, .wait: + return + case .abortForHostFailure: + if let result = matchingResult(), isTerminalFailure(result) { + FlowSessionBridge.writeAck( + FlowAck( + sessionId: result.sessionId, + utteranceId: result.utteranceId, + commandSeq: result.commandSeq, + hostGeneration: result.hostGeneration, + revision: result.revision + ) + ) + lastConsumedUtteranceId = result.utteranceId + abortClipboardPreparing( + reason: "hostTerminalFailure", + message: result.text + ?? ExtL10n.string(ClipboardCommandFailure.prepareFailed.localizationKey) + ) + } else { + abortClipboardPreparing( + reason: "hostTerminalFailure", + message: ExtL10n.string(ClipboardCommandFailure.prepareFailed.localizationKey) + ) + } + case .confirmRecording: + noteClipboardHostRecordingConfirmed() + case .adoptSibling(let busyId): + currentUtteranceId = busyId + FlowSessionBridge.setPendingKeyboardUtteranceId(busyId) + ClipboardCommandResume.markStartIssued(busyId) + let snapshot = FlowSessionBridge.readySnapshot() + isFlowRecording = snapshot?.reason == .recording + if snapshot?.reason == .recording { + noteClipboardHostRecordingConfirmed() + } else { + clipboardAwaitingHostRecordConfirm = false + clipboardPreparingWatchdogTask?.cancel() + clipboardPreparingWatchdogTask = nil + state.phase = .processing + state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") + publishClipboardUIState() + startFlowResultWatchdog() + } + traceState( + "clipboard.preparing.adoptSibling", + extra: "busy=\(busyId.uuidString.prefix(8)) reason=\(snapshot?.reason.rawValue ?? "nil")" + ) + } + } + + /// After restore / claim: write at most one startRecording for the issued utterance. + private func ensureClipboardStartCommandWritten() { + guard isClipboardCommandUtterance else { return } + let issued = ClipboardCommandResume.startIssuedUtteranceId() + let snapshot = FlowSessionBridge.readySnapshot() + let hostReason: ClipboardHostBusyReason? = { + switch snapshot?.reason { + case .recording: return .recording + case .processing: return .processing + default: return nil + } + }() + let action = ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: issued, + isFlowRecording: isFlowRecording, + currentUtteranceId: currentUtteranceId, + hostBusyUtteranceId: snapshot?.busyUtteranceId, + hostReason: hostReason, + hostReadyWithSession: snapshot?.sessionId != nil && FlowSessionBridge.isHostReady() + ) + + switch action { + case .none: + return + case .alreadyInFlight: + return + case .adoptBusy(let busyId, let reason): + currentUtteranceId = busyId + FlowSessionBridge.setPendingKeyboardUtteranceId(busyId) + ClipboardCommandResume.markStartIssued(busyId) + if reason == .recording { + isFlowRecording = true + noteClipboardHostRecordingConfirmed() + } + return + case .waitForHost: + recordWhenHostReady = true + startHostReadyWaitIfNeeded() + traceState("clipboard.ensureStart.waitHost") + return + case .writeStart(let utteranceId): + guard let sessionId = snapshot?.sessionId else { + recordWhenHostReady = true + startHostReadyWaitIfNeeded() + return + } + activeSessionId = sessionId + currentUtteranceId = utteranceId + FlowSessionBridge.setPendingKeyboardUtteranceId(utteranceId) + lastStoppedUtteranceId = nil + writeCommand(.startRecording) + isFlowRecording = true + clipboardAwaitingHostRecordConfirm = true + if clipboardPreparingStartedAt <= 0 { + clipboardPreparingStartedAt = Date().timeIntervalSince1970 + } + clipboardHostRecordConfirmedAt = nil + state.phase = .requestingPermissions + state.lastTranscript = ExtL10n.string("keyboard.placeholder.preparingRecording") + publishClipboardUIState() + scheduleClipboardPreparingWatchdog() + if let view = wakeLockView() { + ExtensionScreenWakeLock.acquire(from: view) + } + startUtteranceCountdown() + startFlowLevelWatchdog() + recomputeMicVoiceAvailability() + traceState( + "clipboard.ensureStart.written", + extra: "utterance=\(utteranceId.uuidString.prefix(8))" + ) + } + } + func pressBegan() { switch state.phase { case .idle, .denied, .error: @@ -713,6 +1193,13 @@ final class KeyboardFlowCoordinator { startFlowRecording() case .waitForHostReady(let recordWhenReady): detectAndStoreAppContext() + if isClipboardCommandUtterance { + // Clipboard never auto-records after warm-up (stable handoff). + deferClipboardUntilHostWarm(reason: "pressBegan.waitHost") + showClipboardHostWarmupHint() + traceState("pressBegan.clipboardDeferred", extra: "waitHost") + break + } recordWhenHostReady = recordWhenReady coldStartDebouncer.reset() startHostReadyWaitIfNeeded() @@ -722,6 +1209,15 @@ final class KeyboardFlowCoordinator { ) case .openHostColdStart: detectAndStoreAppContext() + if isClipboardCommandUtterance { + deferClipboardUntilHostWarm(reason: "pressBegan.coldStart") + if !isPendingFlowStart { + beginFlowStart(recordAfterHandoff: false) + } + showClipboardHostWarmupHint() + traceState("pressBegan.clipboardDeferred", extra: "coldStart") + break + } beginFlowStart(recordAfterHandoff: true) case .ignore: return @@ -749,6 +1245,7 @@ final class KeyboardFlowCoordinator { state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") startFlowResultWatchdog() recomputeMicVoiceAvailability() + publishClipboardUIState() } func beginFlowStart(recordAfterHandoff: Bool = false) { @@ -768,6 +1265,7 @@ final class KeyboardFlowCoordinator { flowStartDeadline = Date().timeIntervalSince1970 + FlowWatchdog.startTimeout state.lastTranscript = "" recomputeMicVoiceAvailability() + FlowSessionBridge.markPiPArmAttempt() OSGDiag.log( "beginFlowStart → openHostApp(startflow) recordAfterHandoff=\(recordAfterHandoff) " + "\(OSGDiag.memoryTag())", @@ -819,7 +1317,7 @@ final class KeyboardFlowCoordinator { state.level = 0 recomputeMicVoiceAvailability() } - endClipboardTaskSession() + resetClipboardUtteranceState() } // MARK: - Private @@ -835,11 +1333,7 @@ final class KeyboardFlowCoordinator { let mode: FlowUtteranceMode? = isClipboardCommandUtterance ? .clipboardCommand : nil let snapshot: String? = { guard isClipboardCommandUtterance, action == .startRecording else { return nil } - return clipboardTaskSession?.snapshot - }() - let previous: String? = { - guard isClipboardCommandUtterance, action == .startRecording else { return nil } - return clipboardTaskSession?.previousOutput + return clipboardFrozenSnapshot }() let command = FlowCommand( sessionId: activeSessionId, @@ -850,7 +1344,7 @@ final class KeyboardFlowCoordinator { fieldContext: action == .stopRecording ? fieldContextProvider() : nil, utteranceMode: mode, clipboardSnapshot: snapshot, - previousOutput: previous + previousOutput: nil ) FlowSessionBridge.writeCommand(command) debug( @@ -877,37 +1371,11 @@ final class KeyboardFlowCoordinator { stopFlowWatchdog() let wasClipboard = result.resolvedUtteranceMode == .clipboardCommand || isClipboardCommandUtterance - let replacePrevious: String? = { - guard wasClipboard, - let session = clipboardTaskSession, - let previous = session.lastInsertedText, - !previous.isEmpty else { return nil } - let fingerprint = fieldContextProvider()?.deliveryFingerprint - if let expected = session.fieldFingerprint, - let fingerprint, - expected != fingerprint { - return nil - } - return previous - }() + // Each clipboard round is independent — always append/insert, never replace. textInserter.handleFlowTranscript( - TranscriptionDelivery(text: text, polishWarning: result.warning), - replacePrevious: replacePrevious + TranscriptionDelivery(text: text, polishWarning: result.warning) ) - if wasClipboard { - let fingerprint = fieldContextProvider()?.deliveryFingerprint - if clipboardTaskSession == nil { - clipboardTaskSession = ClipboardCommandTaskSession( - snapshot: clipboardEligibility?.snapshot ?? "", - expiresAt: Date().timeIntervalSince1970 + ClipboardMaterialFilter.sessionDuration, - fieldFingerprint: fingerprint - ) - } - clipboardTaskSession?.noteSuccessfulInsert(text) - clipboardTaskSession?.fieldFingerprint = fingerprint - publishClipboardUIState() - } - isClipboardCommandUtterance = false + resetClipboardUtteranceState() FlowSessionBridge.writeAck( FlowAck( sessionId: result.sessionId, @@ -941,8 +1409,7 @@ final class KeyboardFlowCoordinator { ) isAwaitingFlowResult = false stopFlowWatchdog() - isClipboardCommandUtterance = false - publishClipboardUIState() + resetClipboardUtteranceState() FlowSessionBridge.writeAck( FlowAck( sessionId: result.sessionId, @@ -1179,9 +1646,22 @@ final class KeyboardFlowCoordinator { ) switch action { case .waitForHostReady(let recordWhenReady): + if isClipboardCommandUtterance { + deferClipboardUntilHostWarm(reason: "startFlowRecording.waitHost") + showClipboardHostWarmupHint() + return + } recordWhenHostReady = recordWhenReady startHostReadyWaitIfNeeded() case .openHostColdStart: + if isClipboardCommandUtterance { + deferClipboardUntilHostWarm(reason: "startFlowRecording.coldStart") + if !isPendingFlowStart { + beginFlowStart(recordAfterHandoff: false) + } + showClipboardHostWarmupHint() + return + } beginFlowStart(recordAfterHandoff: true) case .startRecording, .ignore: break @@ -1201,44 +1681,56 @@ final class KeyboardFlowCoordinator { hostStale: FlowSessionBridge.isHostStale(), withinReadyGrace: withinReadyGrace ) { - beginFlowStart(recordAfterHandoff: true) + if isClipboardCommandUtterance { + deferClipboardUntilHostWarm(reason: "startFlowRecording.missingSession.coldStart") + if !isPendingFlowStart { + beginFlowStart(recordAfterHandoff: false) + } + showClipboardHostWarmupHint() + } else { + beginFlowStart(recordAfterHandoff: true) + } + } else if isClipboardCommandUtterance { + deferClipboardUntilHostWarm(reason: "startFlowRecording.missingSession.wait") + showClipboardHostWarmupHint() } else { recordWhenHostReady = true startHostReadyWaitIfNeeded() } return } + // Clipboard: never send a second startRecording for the same round + // (paste-alert restore used to call pressBegan again → stuck「准备录音…」). + if isClipboardCommandUtterance { + if isFlowRecording { + scheduleClipboardPreparingWatchdog() + recoverClipboardPreparingIfHostMovedOn() + recomputeMicVoiceAvailability() + traceState("startFlowRecording.deduped", extra: "reason=alreadyInFlight") + return + } + ensureClipboardStartCommandWritten() + return + } + activeSessionId = sessionId currentUtteranceId = UUID() FlowSessionBridge.setPendingKeyboardUtteranceId(currentUtteranceId) lastStoppedUtteranceId = nil writeCommand(.startRecording) isFlowRecording = true - if isClipboardCommandUtterance { - clipboardAwaitingHostRecordConfirm = true - clipboardHostRecordConfirmedAt = nil - clipboardStopRequested = false - clipboardDeferredStopTask?.cancel() - clipboardDeferredStopTask = nil - state.phase = .requestingPermissions - state.lastTranscript = ExtL10n.string("keyboard.placeholder.preparingRecording") - } else { - clipboardAwaitingHostRecordConfirm = false - clipboardHostRecordConfirmedAt = nil - clipboardStopRequested = false - state.lastTranscript = "" - state.phase = .recording - } + clipboardAwaitingHostRecordConfirm = false + clipboardHostRecordConfirmedAt = nil + clipboardStopRequested = false + state.lastTranscript = "" + state.phase = .recording recomputeMicVoiceAvailability() if let view = wakeLockView() { ExtensionScreenWakeLock.acquire(from: view) } startUtteranceCountdown() startFlowLevelWatchdog() - traceState( - "startFlowRecording.started", - extra: isClipboardCommandUtterance ? "clipboardPreparing=1" : "clipboardPreparing=0" - ) + traceState("startFlowRecording.started", extra: "clipboardPreparing=0") } private func startUtteranceCountdown() { diff --git a/OSGKeyboardExt/Views/KeyboardRootView.swift b/OSGKeyboardExt/Views/KeyboardRootView.swift index 1d35e24..77b3628 100644 --- a/OSGKeyboardExt/Views/KeyboardRootView.swift +++ b/OSGKeyboardExt/Views/KeyboardRootView.swift @@ -132,7 +132,8 @@ public struct KeyboardRootView: View { transcript: state.lastTranscript, micVoiceAvailability: state.micVoiceAvailability, micDisabledHint: state.micDisabledHint, - clipboardCommandEligible: state.clipboardCommandEligible || state.clipboardCommandSessionActive, + clipboardCommandEligible: state.clipboardCommandEligible, + clipboardFailureHint: state.clipboardFailureHint, cursorDragHintActive: state.cursorDragActive, openSettings: state.openSettings ) @@ -177,18 +178,24 @@ public struct KeyboardRootView: View { return VStack(spacing: KeyboardLayoutMetrics.micToButtonGap) { HStack(spacing: 0) { cursorDragPad(enabled: cursorPadsEnabled) + .overlay { + clipboardSideHint( + ExtL10n.text("keyboard.clipboard.recordingLeft"), + visible: state.clipboardCommandRecording && !dragging + ) + } RecordButton( phase: buttonPhase, level: state.level, remainingSeconds: state.phase == .recording ? state.utteranceRemainingSeconds : nil, - isEnabled: !state.micDisabled, + isEnabled: micButtonEnabled, + isClipboardCommandRecording: state.clipboardCommandRecording, onToggle: state.tapMic, - onClipboardLongPressBegan: (state.clipboardCommandEligible || state.clipboardCommandSessionActive) + onClipboardLongPressBegan: (state.clipboardCommandEligible + || state.clipboardCommandUtteranceActive) + && micButtonEnabled ? state.beginClipboardCommand - : nil, - onClipboardLongPressEnded: (state.clipboardCommandEligible || state.clipboardCommandSessionActive) - ? state.endClipboardCommand : nil ) .frame(width: KeyboardLayoutMetrics.micSize, height: KeyboardLayoutMetrics.micSize) @@ -196,8 +203,15 @@ public struct KeyboardRootView: View { .opacity(dragging ? 0 : 1) cursorDragPad(enabled: cursorPadsEnabled) + .overlay { + clipboardSideHint( + ExtL10n.text("keyboard.clipboard.recordingRight"), + visible: state.clipboardCommandRecording && !dragging + ) + } } .frame(height: KeyboardLayoutMetrics.micSize) + .animation(.easeInOut(duration: 0.25), value: state.clipboardCommandRecording) GeometryReader { proxy in let widths = KeyboardChromeLayout.actionKeyWidths( @@ -240,6 +254,23 @@ public struct KeyboardRootView: View { .contentShape(Rectangle()) } + /// Side caption beside the mic during clipboard-command recording. + /// Vertically matches the mic disc (same upward offset); does not steal touches. + private func clipboardSideHint(_ text: Text, visible: Bool) -> some View { + text + // 22pt → ~18pt (−20%); softer than body so it doesn't compete with the mic. + .font(.system(size: 17.6, weight: .medium)) + .foregroundStyle(palette.textSecondary.opacity(0.42)) + .multilineTextAlignment(.center) + .lineLimit(3) + .minimumScaleFactor(0.7) + .padding(.horizontal, 2) + .offset(y: -KeyboardLayoutMetrics.micUpwardAdjustment) + .opacity(visible ? 1 : 0) + .allowsHitTesting(false) + .accessibilityHidden(!visible) + } + private func bottomDeleteButton(disabled: Bool) -> some View { RepeatingDeleteButton( disabled: disabled, @@ -287,6 +318,14 @@ public struct KeyboardRootView: View { } private var buttonPhase: RecordButton.Phase { + switch clipboardMicChrome { + case .preparingDisabled: + return .processing + case .recordingBlue: + return .recording + case .none: + break + } switch state.micVoiceAvailability { case .recording: return .recording @@ -298,6 +337,32 @@ public struct KeyboardRootView: View { return .idleReady } } + + /// Preparing clipboard capture: grey spinner, not tappable. + private var micButtonEnabled: Bool { + if state.micDisabled { return false } + if clipboardMicChrome == .preparingDisabled { return false } + return true + } + + private var clipboardMicChrome: ClipboardMicChrome { + let phase: ClipboardPreparingPhase = { + switch state.phase { + case .idle: return .idle + case .denied: return .denied + case .error: return .error + case .requestingPermissions: return .requestingPermissions + case .recording: return .recording + case .processing: return .processing + } + }() + return ClipboardPreparingPolicy.micChrome( + isClipboardUtterance: state.clipboardCommandUtteranceActive, + phase: phase, + awaitingHostConfirm: state.phase == .requestingPermissions + || (state.phase == .recording && !state.clipboardCommandRecording) + ) + } } // MARK: - State alias @@ -347,6 +412,7 @@ private struct TranscriptLine: View { let micVoiceAvailability: MicVoiceAvailability let micDisabledHint: String let clipboardCommandEligible: Bool + let clipboardFailureHint: String? let cursorDragHintActive: Bool let openSettings: () -> Void @@ -419,52 +485,60 @@ private struct TranscriptLine: View { @ViewBuilder private var idleHint: some View { - let isWarning: Bool = { - switch micVoiceAvailability { - case .unavailable(.hostNotReady), .unavailable(.preparingSession): - return false - case .unavailable: - return true - case .ready, .recording, .processing: - return false - } - }() - Group { - switch micVoiceAvailability { - case .ready: - if clipboardCommandEligible { - ExtL10n.text("keyboard.placeholder.idleClipboard") - } else { - ExtL10n.text("keyboard.placeholder.idle") - } - case .unavailable(.missingAPIKey): - Text(micDisabledHint) - case .unavailable(.hostNotReady): - if clipboardCommandEligible { - ExtL10n.text("keyboard.placeholder.idleClipboard") - } else { - ExtL10n.text("keyboard.placeholder.idle") - } - case .unavailable(.preparingSession): - if clipboardCommandEligible { - ExtL10n.text("keyboard.placeholder.idleClipboard") - } else { - ExtL10n.text("keyboard.placeholder.idle") - } - case .unavailable(.noFullAccess): - ExtL10n.text("keyboard.error.fullAccessRequired") - case .unavailable(.appGroupUnavailable): - ExtL10n.text("keyboard.error.appGroupCommunication") - case .unavailable(.onboardingIncomplete): - ExtL10n.text("keyboard.hint.finishSetupInApp") - case .recording, .processing: - EmptyView() + if let clipboardFailureHint, !clipboardFailureHint.isEmpty { + Text(clipboardFailureHint) + .font(TypeStyle.caption) + .foregroundStyle(palette.warning) + .lineLimit(1) + .truncationMode(.tail) + } else { + let isWarning: Bool = { + switch micVoiceAvailability { + case .unavailable(.hostNotReady), .unavailable(.preparingSession): + return false + case .unavailable: + return true + case .ready, .recording, .processing: + return false + } + }() + Group { + switch micVoiceAvailability { + case .ready: + if clipboardCommandEligible { + ExtL10n.text("keyboard.placeholder.idleClipboard") + } else { + ExtL10n.text("keyboard.placeholder.idle") + } + case .unavailable(.missingAPIKey): + Text(micDisabledHint) + case .unavailable(.hostNotReady): + if clipboardCommandEligible { + ExtL10n.text("keyboard.placeholder.idleClipboard") + } else { + ExtL10n.text("keyboard.placeholder.idle") + } + case .unavailable(.preparingSession): + if clipboardCommandEligible { + ExtL10n.text("keyboard.placeholder.idleClipboard") + } else { + ExtL10n.text("keyboard.placeholder.idle") + } + case .unavailable(.noFullAccess): + ExtL10n.text("keyboard.error.fullAccessRequired") + case .unavailable(.appGroupUnavailable): + ExtL10n.text("keyboard.error.appGroupCommunication") + case .unavailable(.onboardingIncomplete): + ExtL10n.text("keyboard.hint.finishSetupInApp") + case .recording, .processing: + EmptyView() + } } + .font(TypeStyle.caption) + .foregroundStyle(isWarning ? palette.warning : palette.textTertiary) + .lineLimit(1) + .truncationMode(.tail) } - .font(TypeStyle.caption) - .foregroundStyle(isWarning ? palette.warning : palette.textTertiary) - .lineLimit(1) - .truncationMode(.tail) } private func deniedMessage(for reason: KeyboardViewController.State.Phase.Reason) -> String { diff --git a/OSGKeyboardExt/en.lproj/Keyboard.strings b/OSGKeyboardExt/en.lproj/Keyboard.strings index f4ce920..7da0d6e 100644 --- a/OSGKeyboardExt/en.lproj/Keyboard.strings +++ b/OSGKeyboardExt/en.lproj/Keyboard.strings @@ -116,8 +116,21 @@ /* Keyboard (ext) */ "keyboard.placeholder.idle" = "Tap to talk"; "keyboard.placeholder.idleClipboard" = "Tap to talk, long-press for clipboard"; +"keyboard.clipboard.recordingLeft" = "Recording command"; +"keyboard.clipboard.recordingRight" = "Tap to finish"; "keyboard.placeholder.preparing" = "Preparing"; "keyboard.placeholder.preparingRecording" = "Preparing mic…"; +"keyboard.clipboard.reject.pasteDenied" = "Allow Paste to process the clipboard"; +"keyboard.clipboard.reject.empty" = "No text on the clipboard to process"; +"keyboard.clipboard.reject.phoneOrNumeric" = "Looks like a phone number — not started"; +"keyboard.clipboard.reject.emojiOrSymbolOnly" = "No usable text on the clipboard"; +"keyboard.clipboard.reject.verificationCode" = "Looks like a code — not started"; +"keyboard.clipboard.reject.tooShort" = "Clipboard text is too short"; +"keyboard.clipboard.reject.repetitiveSpam" = "Clipboard text isn’t usable"; +"keyboard.clipboard.reject.secureField" = "Clipboard commands aren’t available in password fields"; +"keyboard.clipboard.reject.noFullAccess" = "Full Access is required for clipboard commands"; +"keyboard.clipboard.reject.prepareFailed" = "Mic wasn’t ready in time — try again"; +"keyboard.clipboard.hint.hostStarting" = "Starting… return and long-press the mic again"; "keyboard.placeholder.processing" = "Processing"; "keyboard.placeholder.error" = "Polishing failed"; "keyboard.placeholder.localBadge" = "On-device"; diff --git a/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings b/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings index 328c403..ae95014 100644 --- a/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings +++ b/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings @@ -116,8 +116,21 @@ /* Keyboard (ext) */ "keyboard.placeholder.idle" = "点按说话"; "keyboard.placeholder.idleClipboard" = "点击说话，长按处理剪贴板"; +"keyboard.clipboard.recordingLeft" = "指令录制中"; +"keyboard.clipboard.recordingRight" = "点按结束处理"; "keyboard.placeholder.preparing" = "准备中…"; "keyboard.placeholder.preparingRecording" = "准备录音…"; +"keyboard.clipboard.reject.pasteDenied" = "需要允许粘贴才能处理剪贴板"; +"keyboard.clipboard.reject.empty" = "剪贴板里没有可处理的文字"; +"keyboard.clipboard.reject.phoneOrNumeric" = "看起来像号码，未开始处理"; +"keyboard.clipboard.reject.emojiOrSymbolOnly" = "没有可处理的文字内容"; +"keyboard.clipboard.reject.verificationCode" = "看起来像验证码，未开始处理"; +"keyboard.clipboard.reject.tooShort" = "内容太短，请复制更完整的文字"; +"keyboard.clipboard.reject.repetitiveSpam" = "内容无效，未开始处理"; +"keyboard.clipboard.reject.secureField" = "密码框中不能使用剪贴板指令"; +"keyboard.clipboard.reject.noFullAccess" = "需要开启完全访问才能处理剪贴板"; +"keyboard.clipboard.reject.prepareFailed" = "麦克风未能及时就绪，请再试一次"; +"keyboard.clipboard.hint.hostStarting" = "正在启动，返回后请再长按处理剪贴板"; "keyboard.placeholder.processing" = "处理中…"; "keyboard.placeholder.error" = "润色失败"; "keyboard.placeholder.localBadge" = "本地"; diff --git a/OSGKeyboardMac/MacTheme.swift b/OSGKeyboardMac/MacTheme.swift index 04c101d..e376d51 100644 --- a/OSGKeyboardMac/MacTheme.swift +++ b/OSGKeyboardMac/MacTheme.swift @@ -53,7 +53,8 @@ enum MacSystemPalette { ? Color.white.opacity(0.12) : Color.black.opacity(0.10), - recordRed: resolved(.systemRed, dark: dark) + recordRed: resolved(.systemRed, dark: dark), + recordBlue: resolved(.systemBlue, dark: dark) ) } diff --git a/OSGKeyboardShared/DesignSystem/RecordButton.swift b/OSGKeyboardShared/DesignSystem/RecordButton.swift index 8983e70..ee82193 100644 --- a/OSGKeyboardShared/DesignSystem/RecordButton.swift +++ b/OSGKeyboardShared/DesignSystem/RecordButton.swift @@ -23,10 +23,11 @@ public struct RecordButton: View { public let level: Double public let remainingSeconds: Int? public let isEnabled: Bool + /// When true (and `phase == .recording`), use blue clipboard-command chrome. + public let isClipboardCommandRecording: Bool public let onToggle: () -> Void - /// When non-nil, a 0.45s hold starts clipboard-command recording instead of toggle. + /// When non-nil, a 0.45s hold starts clipboard-command recording (tap again to stop). public let onClipboardLongPressBegan: (() -> Void)? - public let onClipboardLongPressEnded: (() -> Void)? @State private var breath = false @State private var longPressArmed = false @@ -36,17 +37,17 @@ public struct RecordButton: View { level: Double, remainingSeconds: Int? = nil, isEnabled: Bool = true, + isClipboardCommandRecording: Bool = false, onToggle: @escaping () -> Void, - onClipboardLongPressBegan: (() -> Void)? = nil, - onClipboardLongPressEnded: (() -> Void)? = nil + onClipboardLongPressBegan: (() -> Void)? = nil ) { self.phase = phase self.level = level self.remainingSeconds = remainingSeconds self.isEnabled = isEnabled + self.isClipboardCommandRecording = isClipboardCommandRecording self.onToggle = onToggle self.onClipboardLongPressBegan = onClipboardLongPressBegan - self.onClipboardLongPressEnded = onClipboardLongPressEnded } private var isUrgent: Bool { @@ -54,6 +55,17 @@ public struct RecordButton: View { return remainingSeconds <= 10 } + /// Active recording tint: blue for clipboard-command, red for dictation. + private var recordingTint: Color { + isClipboardCommandRecording ? palette.recordBlue : palette.recordRed + } + + private var waveformColor: Color { + isClipboardCommandRecording + ? Color(red: 0.78, green: 0.88, blue: 1.0) + : Color(red: 1.0, green: 0.78, blue: 0.78) + } + private enum Layout { static let disc: CGFloat = 95 static let outerRing: CGFloat = 106 @@ -64,16 +76,17 @@ public struct RecordButton: View { public var body: some View { ZStack { Circle() - .stroke(palette.recordRed.opacity(isUrgent ? 0.55 : 0.35), lineWidth: isUrgent ? 3 : 2) + .stroke(recordingTint.opacity(isUrgent ? 0.55 : 0.35), lineWidth: isUrgent ? 3 : 2) .frame(width: Layout.breathRing, height: Layout.breathRing) .scaleEffect(breath ? 1.18 : 0.95) .opacity(phase == .recording ? 1 : 0) .animation(Motion.breath, value: breath) + .animation(colorTransition, value: isClipboardCommandRecording) Circle() .fill( RadialGradient( - colors: [palette.recordRed.opacity(0.55), .clear], + colors: [recordingTint.opacity(0.55), .clear], center: .center, startRadius: 46, endRadius: 92 @@ -84,6 +97,7 @@ public struct RecordButton: View { .blur(radius: 18) .animation(Motion.soft, value: phase) .animation(Motion.soft, value: level) + .animation(colorTransition, value: isClipboardCommandRecording) Circle() .stroke(Color.white.opacity(isIdle ? 0.08 : 0.12), lineWidth: 0.5) @@ -114,7 +128,7 @@ public struct RecordButton: View { } WaveformView( level: level, - color: Color(red: 1.0, green: 0.78, blue: 0.78), + color: waveformColor, active: true ) .frame(width: 73, height: 32) @@ -137,6 +151,7 @@ public struct RecordButton: View { .frame(width: Layout.disc, height: Layout.disc) .animation(Motion.soft, value: phase) .animation(Motion.soft, value: remainingSeconds) + .animation(colorTransition, value: isClipboardCommandRecording) } .contentShape(Circle()) .modifier( @@ -146,8 +161,7 @@ public struct RecordButton: View { supportsClipboardLongPress: onClipboardLongPressBegan != nil, longPressArmed: $longPressArmed, onToggle: onToggle, - onClipboardLongPressBegan: onClipboardLongPressBegan, - onClipboardLongPressEnded: onClipboardLongPressEnded + onClipboardLongPressBegan: onClipboardLongPressBegan ) ) .onAppear { breath = (phase == .recording) } @@ -160,6 +174,11 @@ public struct RecordButton: View { .accessibilityLabel(Text(SharedL10n.string("keyboard.tapToTalkA11y"))) } + /// Red ↔ blue mode switch (~0.25s). + private var colorTransition: Animation { + .easeInOut(duration: 0.25) + } + private var isIdle: Bool { switch phase { case .idleReady, .idleUnavailable: @@ -179,8 +198,8 @@ public struct RecordButton: View { switch phase { case .recording: let colors: [Color] = isUrgent - ? [palette.recordRed, palette.recordRed.opacity(0.85)] - : [palette.recordRed.opacity(0.95), palette.recordRed.opacity(0.75)] + ? [recordingTint, recordingTint.opacity(0.85)] + : [recordingTint.opacity(0.95), recordingTint.opacity(0.75)] return LinearGradient(colors: colors, startPoint: .top, endPoint: .bottom) case .processing: return LinearGradient( @@ -213,45 +232,49 @@ private struct RecordButtonPressModifier: ViewModifier { @Binding var longPressArmed: Bool let onToggle: () -> Void let onClipboardLongPressBegan: (() -> Void)? - let onClipboardLongPressEnded: (() -> Void)? func body(content: Content) -> some View { - if supportsClipboardLongPress { - content.onLongPressGesture( - minimumDuration: ClipboardMaterialFilter.longPressDuration, - maximumDistance: 120, - pressing: { pressing in - if pressing { - longPressArmed = false - return - } - // Released. - if longPressArmed { - onClipboardLongPressEnded?() - longPressArmed = false - } else if phase != .processing, isEnabled || phase == .idleUnavailable { - // Short press → existing tap-toggle dictation. - guard phase != .recording else { - // If somehow recording without arming, end via toggle. - onToggle() - return - } - onToggle() - } - }, - perform: { - guard phase != .processing else { return } - guard isEnabled || phase == .idleUnavailable else { return } - longPressArmed = true - onClipboardLongPressBegan?() - } - ) - } else { + // While recording/processing, only tap-to-toggle — never treat finger-up + // from the starting long-press as stop (clipboard is explicitly tap-to-stop). + switch phase { + case .recording, .processing: content.onTapGesture { guard phase != .processing else { return } guard isEnabled || phase == .idleUnavailable else { return } onToggle() } + case .idleReady, .idleUnavailable, .error: + if supportsClipboardLongPress { + content.onLongPressGesture( + minimumDuration: ClipboardMaterialFilter.longPressDuration, + maximumDistance: 120, + pressing: { pressing in + if pressing { + longPressArmed = false + return + } + // Released before / without arming → short press = dictation toggle. + // Armed release is ignored here; stop happens on a later tap + // once phase becomes `.recording`. + if longPressArmed { + longPressArmed = false + return + } + guard isEnabled || phase == .idleUnavailable else { return } + onToggle() + }, + perform: { + guard isEnabled || phase == .idleUnavailable else { return } + longPressArmed = true + onClipboardLongPressBegan?() + } + ) + } else { + content.onTapGesture { + guard isEnabled || phase == .idleUnavailable else { return } + onToggle() + } + } } } } diff --git a/OSGKeyboardShared/DesignSystem/Theme.swift b/OSGKeyboardShared/DesignSystem/Theme.swift index 3dcd704..9930ac6 100644 --- a/OSGKeyboardShared/DesignSystem/Theme.swift +++ b/OSGKeyboardShared/DesignSystem/Theme.swift @@ -39,6 +39,8 @@ public struct ThemePalette: Sendable, Equatable { public let dividerStrong: Color public let recordRed: Color + /// Clipboard-command hold-to-talk recording (distinct from dictation red). + public let recordBlue: Color } public enum Palette { @@ -70,6 +72,7 @@ public enum Palette { // Recording state public static let recordRed = Color(red: 1.000, green: 0.231, blue: 0.188) // #FF3B30 + public static let recordBlue = Color(red: 0.000, green: 0.478, blue: 1.000) // #007AFF /// Canonical dark palette — preserves every legacy literal above so /// existing call sites that read `Palette.background` directly keep @@ -92,7 +95,8 @@ public enum Palette { textOnAccent: textOnAccent, divider: divider, dividerStrong: dividerStrong, - recordRed: recordRed + recordRed: recordRed, + recordBlue: recordBlue ) /// Light palette — warm gray backgrounds for daytime use. @@ -113,7 +117,8 @@ public enum Palette { textOnAccent: Color.white, divider: Color.black.opacity(0.06), dividerStrong: Color.black.opacity(0.10), - recordRed: Color(red: 1.000, green: 0.231, blue: 0.188) // #FF3B30 + recordRed: Color(red: 1.000, green: 0.231, blue: 0.188), // #FF3B30 + recordBlue: Color(red: 0.000, green: 0.478, blue: 1.000) // #007AFF ) } diff --git a/OSGKeyboardShared/Models/FlowHandoffPolicy.swift b/OSGKeyboardShared/Models/FlowHandoffPolicy.swift index ff3ce7b..821efd8 100644 --- a/OSGKeyboardShared/Models/FlowHandoffPolicy.swift +++ b/OSGKeyboardShared/Models/FlowHandoffPolicy.swift @@ -67,6 +67,53 @@ public enum FlowHandoffPolicy { ) } + /// Proactive keyboard-open PiP arm. Returns false when the host is already + /// ready, warming, busy, or a recent startflow is still in cooldown — + /// so Voice-tab / appear noise cannot re-jump to the app. + /// + /// `heartbeatStaleness`: seconds since last host heartbeat (`nil` = never). + /// A lingering `sessionActive` after force-quit must still allow arm once the + /// heartbeat is clearly gone — without treating brief switcher flaps as death. + public static let proactiveUnreachableArmGrace: TimeInterval = 8 + + public static func shouldProactivePiPArm( + hostReady: Bool, + snapshotReason: FlowReadySnapshot.Reason?, + sessionActive: Bool, + hostReachable: Bool, + hostStale: Bool, + withinReadyGrace: Bool, + inCooldown: Bool, + heartbeatStaleness: TimeInterval? = nil + ) -> Bool { + guard !hostReady, !inCooldown else { return false } + if let snapshotReason { + switch snapshotReason { + case .recording, .processing, .awaitingDelivery, .starting, + .waitingForAudioProof, .audioEngineNotLive: + return false + case .ready, .noSession, .permissionMissing, .appGroupUnavailable, + .hostLost, .error: + break + } + } + // Force-quit leaves sessionActive=true until the 60s zombie window. + // If heartbeat has been gone long enough, arm once (cooldown still applies). + if sessionActive, + !hostReachable, + !hostStale, + let staleness = heartbeatStaleness, + staleness >= proactiveUnreachableArmGrace { + return true + } + return shouldOpenHostColdStart( + sessionActive: sessionActive, + hostReachable: hostReachable, + hostStale: hostStale, + withinReadyGrace: withinReadyGrace + ) + } + /// Mic-press routing shared by the keyboard coordinator and unit tests. public static func micPressAction( availability: MicVoiceAvailability, diff --git a/OSGKeyboardShared/Models/TypingInputConfiguration.swift b/OSGKeyboardShared/Models/TypingInputConfiguration.swift index 3cff8e7..4d91b2c 100644 --- a/OSGKeyboardShared/Models/TypingInputConfiguration.swift +++ b/OSGKeyboardShared/Models/TypingInputConfiguration.swift @@ -22,6 +22,7 @@ public enum TypingInputSchema: String, CaseIterable, Identifiable, Codable, Send } } + /// Stable Rime schema `name:` (not UI copy). Keep Chinese so redeploy fingerprints stay stable. public var displayName: String { switch self { case .fullPinyin: return "全拼" @@ -29,6 +30,15 @@ public enum TypingInputSchema: String, CaseIterable, Identifiable, Codable, Send case .sogouDoublePinyin: return "搜狗双拼" } } + + /// Localizable UI label key (`AppL10n` / `SharedL10n`). + public var labelKey: String { + switch self { + case .fullPinyin: return "typing.schema.fullPinyin" + case .microsoftDoublePinyin: return "typing.schema.microsoftDoublePinyin" + case .sogouDoublePinyin: return "typing.schema.sogouDoublePinyin" + } + } } public enum PinyinFuzzyPair: String, CaseIterable, Identifiable, Codable, Sendable { diff --git a/OSGKeyboardShared/Services/ClipboardCommandEligibility.swift b/OSGKeyboardShared/Services/ClipboardCommandEligibility.swift index 251ea39..1ceb7af 100644 --- a/OSGKeyboardShared/Services/ClipboardCommandEligibility.swift +++ b/OSGKeyboardShared/Services/ClipboardCommandEligibility.swift @@ -1,96 +1,46 @@ // ClipboardCommandEligibility.swift // OSGKeyboard · Shared // -// Tracks clipboard open-window eligibility (30s from first sighting of a -// pasteboard change) for opportunity-read UI. Pure timing logic — no UIKit. +// User-visible failure reasons when long-press clipboard command cannot start. +// (30s eligibility window and continuous-rewrite sessions were removed.) import Foundation -public struct ClipboardCommandEligibility: Equatable, Sendable { - public var changeCount: Int - public var snapshot: String - public var startedAt: TimeInterval +/// Why a clipboard-command long-press did not start recording. +public enum ClipboardCommandFailure: Equatable, Sendable { + case pasteDenied + case secureField + case noFullAccess + /// Host never confirmed capture (double-start / mic not ready / timeout). + case prepareFailed + case material(ClipboardMaterialFilter.Rejection) - public init(changeCount: Int, snapshot: String, startedAt: TimeInterval = Date().timeIntervalSince1970) { - self.changeCount = changeCount - self.snapshot = snapshot - self.startedAt = startedAt - } - - public func isOpen(at now: TimeInterval = Date().timeIntervalSince1970) -> Bool { - now - startedAt <= ClipboardMaterialFilter.eligibilityDuration - } - - public func remaining(at now: TimeInterval = Date().timeIntervalSince1970) -> TimeInterval { - max(0, ClipboardMaterialFilter.eligibilityDuration - (now - startedAt)) - } -} - -public enum ClipboardCommandEligibilityTracker: Sendable { - /// Update eligibility from an opportunity-read sample. - /// - Parameters: - /// - changeCount: `UIPasteboard.general.changeCount` - /// - rawText: pasteboard string (may be nil) - /// - previous: last known eligibility - /// - now: clock - public static func refresh( - changeCount: Int, - rawText: String?, - previous: ClipboardCommandEligibility?, - now: TimeInterval = Date().timeIntervalSince1970 - ) -> ClipboardCommandEligibility? { - guard let rawText else { return nil } - - if let previous, previous.changeCount == changeCount { - return previous.isOpen(at: now) ? previous : nil - } - - switch ClipboardMaterialFilter.evaluate(rawText) { - case .eligible(let snapshot): - return ClipboardCommandEligibility( - changeCount: changeCount, - snapshot: snapshot, - startedAt: now - ) - case .rejected: - return nil + /// Localization key under the keyboard extension `Keyboard.strings` table. + public var localizationKey: String { + switch self { + case .pasteDenied: + return "keyboard.clipboard.reject.pasteDenied" + case .secureField: + return "keyboard.clipboard.reject.secureField" + case .noFullAccess: + return "keyboard.clipboard.reject.noFullAccess" + case .prepareFailed: + return "keyboard.clipboard.reject.prepareFailed" + case .material(let rejection): + switch rejection { + case .empty: + return "keyboard.clipboard.reject.empty" + case .phoneOrNumeric: + return "keyboard.clipboard.reject.phoneOrNumeric" + case .emojiOrSymbolOnly: + return "keyboard.clipboard.reject.emojiOrSymbolOnly" + case .verificationCode: + return "keyboard.clipboard.reject.verificationCode" + case .tooShort: + return "keyboard.clipboard.reject.tooShort" + case .repetitiveSpam: + return "keyboard.clipboard.reject.repetitiveSpam" + } } } } - -/// In-memory clipboard-command task session (plan §8 layer B). Owned by the keyboard. -public struct ClipboardCommandTaskSession: Equatable, Sendable { - public var snapshot: String - public var previousOutput: String? - public var lastInsertedText: String? - public var expiresAt: TimeInterval - public var fieldFingerprint: String? - - public init( - snapshot: String, - previousOutput: String? = nil, - lastInsertedText: String? = nil, - expiresAt: TimeInterval, - fieldFingerprint: String? = nil - ) { - self.snapshot = snapshot - self.previousOutput = previousOutput - self.lastInsertedText = lastInsertedText - self.expiresAt = expiresAt - self.fieldFingerprint = fieldFingerprint - } - - public func isActive(at now: TimeInterval = Date().timeIntervalSince1970) -> Bool { - now <= expiresAt - } - - public mutating func refreshExpiry(at now: TimeInterval = Date().timeIntervalSince1970) { - expiresAt = now + ClipboardMaterialFilter.sessionDuration - } - - public mutating func noteSuccessfulInsert(_ text: String, at now: TimeInterval = Date().timeIntervalSince1970) { - lastInsertedText = text - previousOutput = text - refreshExpiry(at: now) - } -} diff --git a/OSGKeyboardShared/Services/ClipboardCommandResume.swift b/OSGKeyboardShared/Services/ClipboardCommandResume.swift new file mode 100644 index 0000000..d01297f --- /dev/null +++ b/OSGKeyboardShared/Services/ClipboardCommandResume.swift @@ -0,0 +1,107 @@ +// ClipboardCommandResume.swift +// OSGKeyboard · Shared +// +// Sticky App Group flags so the system「允许粘贴」alert can dismiss / recreate +// the keyboard extension without losing "stay on voice + clipboard chrome". + +import Foundation + +public enum ClipboardCommandResume: Sendable { + private enum Key { + /// Prefer voice surface on the next keyboard presentation. + static let preferVoice = "clipboardCommand.preferVoice.v1" + /// Frozen snapshot captured before / during paste alert (optional). + static let snapshot = "clipboardCommand.pendingSnapshot.v1" + /// Wall time when prefer-voice was marked (drop stale flags). + static let markedAt = "clipboardCommand.preferVoiceAt.v1" + /// Utterance id already sent as startRecording — blocks duplicate starts. + static let startIssuedUtterance = "clipboardCommand.startIssuedUtterance.v1" + } + + /// How long a sticky prefer-voice / snapshot remains valid. + public static let stickyTTL: TimeInterval = 120 + /// Max time to wait in「准备录音…」for host confirm before failing closed. + public static let preparingTimeout: TimeInterval = 6 + + public static func markPreferVoice(defaults: UserDefaults? = nil) { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return } + store.set(true, forKey: Key.preferVoice) + store.set(Date().timeIntervalSince1970, forKey: Key.markedAt) + // Paste alert often jetsams the extension — flush before we block on + // UIPasteboard.string so a recreated process still sees prefer-voice. + store.synchronize() + } + + public static func storeSnapshot(_ text: String, defaults: UserDefaults? = nil) { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return } + let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return } + store.set(trimmed, forKey: Key.snapshot) + store.set(true, forKey: Key.preferVoice) + store.set(Date().timeIntervalSince1970, forKey: Key.markedAt) + store.synchronize() + } + + public static func markStartIssued(_ utteranceId: UUID, defaults: UserDefaults? = nil) { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return } + store.set(utteranceId.uuidString, forKey: Key.startIssuedUtterance) + store.set(true, forKey: Key.preferVoice) + store.set(Date().timeIntervalSince1970, forKey: Key.markedAt) + store.synchronize() + } + + public static func startIssuedUtteranceId(defaults: UserDefaults? = nil) -> UUID? { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return nil } + pruneIfStale(store: store) + guard let raw = store.string(forKey: Key.startIssuedUtterance) else { return nil } + return UUID(uuidString: raw) + } + + public static func hasStartIssued(defaults: UserDefaults? = nil) -> Bool { + startIssuedUtteranceId(defaults: defaults) != nil + } + + public static func clear(defaults: UserDefaults? = nil) { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return } + store.removeObject(forKey: Key.preferVoice) + store.removeObject(forKey: Key.snapshot) + store.removeObject(forKey: Key.markedAt) + store.removeObject(forKey: Key.startIssuedUtterance) + store.synchronize() + } + + public static func shouldPreferVoice(defaults: UserDefaults? = nil) -> Bool { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return false } + pruneIfStale(store: store) + return store.bool(forKey: Key.preferVoice) + } + + public static func pendingSnapshot(defaults: UserDefaults? = nil) -> String? { + guard let store = defaults ?? AppGroup.defaultsIfAvailable else { return nil } + pruneIfStale(store: store) + guard store.bool(forKey: Key.preferVoice) else { return nil } + return store.string(forKey: Key.snapshot) + } + + private static func pruneIfStale(store: UserDefaults) { + let markedAt = store.double(forKey: Key.markedAt) + guard markedAt > 0 else { + // Legacy / incomplete write — drop. + if store.object(forKey: Key.preferVoice) != nil + || store.object(forKey: Key.startIssuedUtterance) != nil { + store.removeObject(forKey: Key.preferVoice) + store.removeObject(forKey: Key.snapshot) + store.removeObject(forKey: Key.startIssuedUtterance) + store.synchronize() + } + return + } + if Date().timeIntervalSince1970 - markedAt > stickyTTL { + store.removeObject(forKey: Key.preferVoice) + store.removeObject(forKey: Key.snapshot) + store.removeObject(forKey: Key.markedAt) + store.removeObject(forKey: Key.startIssuedUtterance) + store.synchronize() + } + } +} diff --git a/OSGKeyboardShared/Services/ClipboardMaterialFilter.swift b/OSGKeyboardShared/Services/ClipboardMaterialFilter.swift index 43ec34f..4c75d68 100644 --- a/OSGKeyboardShared/Services/ClipboardMaterialFilter.swift +++ b/OSGKeyboardShared/Services/ClipboardMaterialFilter.swift @@ -10,12 +10,12 @@ public enum ClipboardMaterialFilter: Sendable { public static let minimumLength = 15 public static let maxSnapshotLength = 3_000 - public static let eligibilityDuration: TimeInterval = 30 - public static let sessionDuration: TimeInterval = 30 public static let longPressDuration: TimeInterval = 0.45 /// After the host confirms real capture, keep recording at least this long - /// before honoring finger-up (avoids near-silent cold-start tails). + /// before honoring an explicit stop tap (avoids near-silent cold-start tails). public static let minimumRecordingAfterHostConfirm: TimeInterval = 0.70 + /// How long a clipboard-command failure tip stays above the mic. + public static let failureHintDuration: TimeInterval = 2.5 public enum Rejection: String, Equatable, Sendable { case empty diff --git a/OSGKeyboardShared/Services/ClipboardPreparingPolicy.swift b/OSGKeyboardShared/Services/ClipboardPreparingPolicy.swift new file mode 100644 index 0000000..ce75df1 --- /dev/null +++ b/OSGKeyboardShared/Services/ClipboardPreparingPolicy.swift @@ -0,0 +1,192 @@ +// ClipboardPreparingPolicy.swift +// OSGKeyboard · Shared +// +// Pure decisions for clipboard「准备录音…」so paste-alert restore / double-start +// / host-failure recovery stay hermetic and regression-tested. + +import Foundation + +// MARK: - Restore after paste-alert / cold-start recreate + +public enum ClipboardRestoreAction: Equatable, Sendable { + /// Mid-flight claim exists — reattach preparing/recording, never pressBegan again. + case awaitExistingStart + /// Sticky voice + snapshot only (e.g. after cold-start). Force voice; do **not** auto-record. + case preferVoiceOnly + /// Already in a live clipboard phase — only refresh UI / recover. + case refreshOnly +} + +/// Whether a clipboard long-press may claim + start, or must warm the host first. +public enum ClipboardHostGateAction: Equatable, Sendable { + case startRecordingNow + case openHostColdStart + case waitForHost + case ignore +} + +/// Mic chrome while a clipboard round is live. +public enum ClipboardMicChrome: Equatable, Sendable { + /// Grey / spinner / not tappable — waiting for host capture confirm. + case preparingDisabled + /// Blue recording chrome + side captions. + case recordingBlue + /// Not a clipboard recording chrome state. + case none +} + +public enum ClipboardPreparingPolicy: Sendable { + + public static func restoreAction( + hasStartIssued: Bool, + phase: ClipboardPreparingPhase + ) -> ClipboardRestoreAction { + switch phase { + case .idle, .denied, .error: + // Cold-start return has snapshot/preferVoice but no startIssued → voice only. + return hasStartIssued ? .awaitExistingStart : .preferVoiceOnly + case .requestingPermissions, .recording, .processing: + return .refreshOnly + } + } + + /// Map the shared mic handoff decision onto clipboard (never auto-record after warm-up). + public static func hostGateAction( + micPressAction: FlowMicPressAction + ) -> ClipboardHostGateAction { + switch micPressAction { + case .startRecording: + return .startRecordingNow + case .openHostColdStart: + return .openHostColdStart + case .waitForHostReady: + // Clipboard does not set recordWhenHostReady — user long-presses again. + return .waitForHost + case .ignore: + return .ignore + } + } + + public static func micChrome( + isClipboardUtterance: Bool, + phase: ClipboardPreparingPhase, + awaitingHostConfirm: Bool + ) -> ClipboardMicChrome { + guard isClipboardUtterance else { return .none } + switch phase { + case .requestingPermissions: + return .preparingDisabled + case .recording: + return awaitingHostConfirm ? .preparingDisabled : .recordingBlue + case .idle, .denied, .error, .processing: + return .none + } + } + + // MARK: - Stop while preparing + + public static func stopWhilePreparing( + awaitingHostConfirm: Bool + ) -> ClipboardPreparingStopAction { + awaitingHostConfirm ? .abortPreparing : .requestStop + } + + // MARK: - Host moved on while preparing + + public static func recoverWhilePreparing( + awaitingHostConfirm: Bool, + currentUtteranceId: UUID?, + hostBusyUtteranceId: UUID?, + hostReason: ClipboardHostBusyReason?, + hasTerminalFailureForCurrent: Bool + ) -> ClipboardPreparingRecoverAction { + guard awaitingHostConfirm else { return .none } + + if hasTerminalFailureForCurrent { + return .abortForHostFailure + } + + guard let hostReason, let busyId = hostBusyUtteranceId else { + return .none + } + + switch hostReason { + case .recording: + if busyId == currentUtteranceId { + return .confirmRecording + } + return .adoptSibling(busyId) + case .processing: + if busyId == currentUtteranceId { + return .wait + } + return .adoptSibling(busyId) + } + } + + // MARK: - Ensure at most one startRecording + + public static func ensureStartAction( + issuedUtteranceId: UUID?, + isFlowRecording: Bool, + currentUtteranceId: UUID?, + hostBusyUtteranceId: UUID?, + hostReason: ClipboardHostBusyReason?, + hostReadyWithSession: Bool + ) -> ClipboardEnsureStartAction { + guard let issued = issuedUtteranceId else { return .none } + + if let busyId = hostBusyUtteranceId, let hostReason { + switch hostReason { + case .recording, .processing: + return .adoptBusy(busyId, hostReason) + } + } + + if isFlowRecording, currentUtteranceId == issued { + return .alreadyInFlight + } + + if hostReadyWithSession { + return .writeStart(issued) + } + + return .waitForHost + } +} + +/// Keyboard phase subset relevant to clipboard prepare/restore. +public enum ClipboardPreparingPhase: Equatable, Sendable { + case idle + case denied + case error + case requestingPermissions + case recording + case processing +} + +public enum ClipboardPreparingStopAction: Equatable, Sendable { + case abortPreparing + case requestStop +} + +public enum ClipboardHostBusyReason: Equatable, Sendable { + case recording + case processing +} + +public enum ClipboardPreparingRecoverAction: Equatable, Sendable { + case none + case wait + case confirmRecording + case adoptSibling(UUID) + case abortForHostFailure +} + +public enum ClipboardEnsureStartAction: Equatable, Sendable { + case none + case alreadyInFlight + case adoptBusy(UUID, ClipboardHostBusyReason) + case writeStart(UUID) + case waitForHost +} diff --git a/OSGKeyboardShared/Services/FlowSessionBridge.swift b/OSGKeyboardShared/Services/FlowSessionBridge.swift index 43cd552..00c7dbd 100644 --- a/OSGKeyboardShared/Services/FlowSessionBridge.swift +++ b/OSGKeyboardShared/Services/FlowSessionBridge.swift @@ -547,6 +547,20 @@ public enum FlowSessionBridge { setPendingHostBundleId(nil, defaults: defaults) } + /// True when a recent keyboard `startflow` arm should not be repeated. + public static func isPiPArmInCooldown(defaults: UserDefaults? = nil) -> Bool { + let store = resolvedDefaults(defaults) + let last = store.double(forKey: FlowSessionKeys.lastPiPArmAttemptAt) + guard last > 0 else { return false } + return Date().timeIntervalSince1970 - last < FlowSessionKeys.pipArmCooldown + } + + public static func markPiPArmAttempt(defaults: UserDefaults? = nil) { + let store = resolvedDefaults(defaults) + store.set(Date().timeIntervalSince1970, forKey: FlowSessionKeys.lastPiPArmAttemptAt) + flush(store) + } + // MARK: - Session validity (keyboard) /// True while the persistent PiP session contract is active. diff --git a/OSGKeyboardShared/Services/FlowSessionKeys.swift b/OSGKeyboardShared/Services/FlowSessionKeys.swift index 7a6b173..a59d91b 100644 --- a/OSGKeyboardShared/Services/FlowSessionKeys.swift +++ b/OSGKeyboardShared/Services/FlowSessionKeys.swift @@ -33,6 +33,10 @@ public enum FlowSessionKeys { public static let audioLevels = "flow.audioLevels" /// Bundle id of the app that opened `osgkeyboard://startflow` (scheme D). public static let pendingHostBundleId = "flow.pendingHostBundleId" + /// Wall-clock of the last keyboard→`startflow` PiP arm attempt (debounce re-jumps). + public static let lastPiPArmAttemptAt = "flow.lastPiPArmAttemptAt.v1" + /// Minimum gap between proactive / clipboard startflow jumps. + public static let pipArmCooldown: TimeInterval = 45 /// Wall-clock timestamp of the last utterance completion or session start. public static let lastActivityAt = "flow.lastActivityAt" /// One-shot token rotated by every host-process launch. State written by diff --git a/OSGKeyboardShared/Services/KeyboardOpenSurfacePolicy.swift b/OSGKeyboardShared/Services/KeyboardOpenSurfacePolicy.swift new file mode 100644 index 0000000..a767673 --- /dev/null +++ b/OSGKeyboardShared/Services/KeyboardOpenSurfacePolicy.swift @@ -0,0 +1,22 @@ +// KeyboardOpenSurfacePolicy.swift +// OSGKeyboard · Shared +// +// Pure open-surface decision used by the keyboard extension. Extracted so +// paste-alert sticky resume can be unit-tested without UIKit. + +import Foundation + +public enum KeyboardOpenSurfacePolicy: Sendable { + /// Surface to show on the first frame of a keyboard presentation. + public static func resolve( + locksTypingSurface: Bool, + clipboardCommandActive: Bool, + stickyPreferVoice: Bool, + preferred: KeyboardState.Surface + ) -> KeyboardState.Surface { + if locksTypingSurface || clipboardCommandActive || stickyPreferVoice { + return .voice + } + return preferred + } +} diff --git a/OSGKeyboardShared/Services/KeyboardState.swift b/OSGKeyboardShared/Services/KeyboardState.swift index 968e18b..8d96d31 100644 --- a/OSGKeyboardShared/Services/KeyboardState.swift +++ b/OSGKeyboardShared/Services/KeyboardState.swift @@ -130,10 +130,15 @@ public final class KeyboardState: ObservableObject { /// `true` while a cursor-drag pad is being pressed — drives the hint /// shown above the mic. @Published public var cursorDragActive: Bool = false - /// Opportunity-read: clipboard text is eligible for long-press command mode. + /// Idle affordance: pasteboard reports `hasStrings` (metadata only). @Published public var clipboardCommandEligible: Bool = false - /// Active clipboard-command task (continuous rewrite window). - @Published public var clipboardCommandSessionActive: Bool = false + /// True while a clipboard-command utterance is in flight (preparing or recording). + @Published public var clipboardCommandUtteranceActive: Bool = false + /// True only while a clipboard-command utterance is in `.recording` + /// (after host confirm) — drives blue mic chrome + side hints. + @Published public var clipboardCommandRecording: Bool = false + /// Transient tip after a failed clipboard long-press (auto-clears). + @Published public var clipboardFailureHint: String? = nil /// Whether translate-and-polish is armed for the current engine. public var isTranslationEffective: Bool { translationEnabled @@ -210,7 +215,6 @@ public final class KeyboardState: ObservableObject { public var endRecording: () -> Void = {} public var tapMic: () -> Void = {} public var beginClipboardCommand: () -> Void = {} - public var endClipboardCommand: () -> Void = {} public var refreshClipboardEligibility: () -> Void = {} public var openSettings: () -> Void = {} public var startFlowSession: () -> Void = {} diff --git a/OSGKeyboardTests/ClipboardCommandPromptComposerTests.swift b/OSGKeyboardTests/ClipboardCommandPromptComposerTests.swift index 7b3f6d8..cd0d68a 100644 --- a/OSGKeyboardTests/ClipboardCommandPromptComposerTests.swift +++ b/OSGKeyboardTests/ClipboardCommandPromptComposerTests.swift @@ -30,29 +30,22 @@ final class ClipboardCommandPromptComposerTests: XCTestCase { XCTAssertFalse(system.contains("不是向你提出的问题或命令")) } - func testEligibilityTrackerKeepsSameChangeCountWindow() { - let first = ClipboardCommandEligibilityTracker.refresh( - changeCount: 3, - rawText: "周末有空一起吃个饭吗？我想聊下项目进度。", - previous: nil, - now: 1_000 + func testFailureLocalizationKeysAreStable() { + XCTAssertEqual( + ClipboardCommandFailure.pasteDenied.localizationKey, + "keyboard.clipboard.reject.pasteDenied" ) - XCTAssertNotNil(first) - - let same = ClipboardCommandEligibilityTracker.refresh( - changeCount: 3, - rawText: "周末有空一起吃个饭吗？我想聊下项目进度。", - previous: first, - now: 1_010 + XCTAssertEqual( + ClipboardCommandFailure.material(.tooShort).localizationKey, + "keyboard.clipboard.reject.tooShort" ) - XCTAssertEqual(same?.startedAt, first?.startedAt) - - let expired = ClipboardCommandEligibilityTracker.refresh( - changeCount: 3, - rawText: "周末有空一起吃个饭吗？我想聊下项目进度。", - previous: first, - now: 1_040 + XCTAssertEqual( + ClipboardCommandFailure.secureField.localizationKey, + "keyboard.clipboard.reject.secureField" + ) + XCTAssertEqual( + ClipboardCommandFailure.prepareFailed.localizationKey, + "keyboard.clipboard.reject.prepareFailed" ) - XCTAssertNil(expired) } } diff --git a/OSGKeyboardTests/ClipboardCommandResumeTests.swift b/OSGKeyboardTests/ClipboardCommandResumeTests.swift new file mode 100644 index 0000000..572325d --- /dev/null +++ b/OSGKeyboardTests/ClipboardCommandResumeTests.swift @@ -0,0 +1,215 @@ +// ClipboardCommandResumeTests.swift +// OSGKeyboardTests + +import XCTest +@testable import OSGKeyboardShared + +final class ClipboardCommandResumeTests: XCTestCase { + private var defaults: UserDefaults! + private var suiteName: String! + + override func setUp() { + super.setUp() + suiteName = "ClipboardCommandResumeTests.\(UUID().uuidString)" + defaults = UserDefaults(suiteName: suiteName) + } + + override func tearDown() { + if let suiteName { + defaults?.removePersistentDomain(forName: suiteName) + } + defaults = nil + suiteName = nil + super.tearDown() + } + + func testMarkPreferVoiceSurvivesUntilCleared() { + XCTAssertFalse(ClipboardCommandResume.shouldPreferVoice(defaults: defaults)) + ClipboardCommandResume.markPreferVoice(defaults: defaults) + XCTAssertTrue(ClipboardCommandResume.shouldPreferVoice(defaults: defaults)) + ClipboardCommandResume.clear(defaults: defaults) + XCTAssertFalse(ClipboardCommandResume.shouldPreferVoice(defaults: defaults)) + } + + func testStoreSnapshotRoundTrip() { + ClipboardCommandResume.storeSnapshot("周末有空一起吃饭吗？", defaults: defaults) + XCTAssertTrue(ClipboardCommandResume.shouldPreferVoice(defaults: defaults)) + XCTAssertEqual( + ClipboardCommandResume.pendingSnapshot(defaults: defaults), + "周末有空一起吃饭吗？" + ) + } + + /// Simulates extension jetsam: writer process flushes, reader process is new. + func testStickySurvivesNewUserDefaultsInstanceAfterSynchronize() { + let text = "是AI语音输入法,更是好输入法。It is an AI voice input method." + ClipboardCommandResume.markPreferVoice(defaults: defaults) + ClipboardCommandResume.storeSnapshot(text, defaults: defaults) + + // New UserDefaults handle on the same suite ≈ new extension process. + let reopened = UserDefaults(suiteName: suiteName!) + XCTAssertNotNil(reopened) + guard let reopened else { return } + + XCTAssertTrue( + ClipboardCommandResume.shouldPreferVoice(defaults: reopened), + "Recreated process must still prefer voice after paste-alert jetsam" + ) + XCTAssertEqual( + ClipboardCommandResume.pendingSnapshot(defaults: reopened), + text + ) + + // And the open-surface policy must then force voice over default typing. + XCTAssertEqual( + KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: false, + stickyPreferVoice: ClipboardCommandResume.shouldPreferVoice(defaults: reopened), + preferred: .typing + ), + .voice + ) + } + + func testOpenSurfacePolicyForcesVoiceWhenStickyEvenIfDefaultTyping() { + let resolved = KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: false, + stickyPreferVoice: true, + preferred: .typing + ) + XCTAssertEqual(resolved, .voice) + } + + func testOpenSurfacePolicyHonorsDefaultTypingWhenNoSticky() { + let resolved = KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: false, + stickyPreferVoice: false, + preferred: .typing + ) + XCTAssertEqual(resolved, .typing) + } + + func testOpenSurfacePolicyClipboardActiveBeatsTypingDefault() { + let resolved = KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: true, + stickyPreferVoice: false, + preferred: .typing + ) + XCTAssertEqual(resolved, .voice) + } + + func testPasteAlertReopenDecisionMatrix() { + // Exact bug from device: default typing + sticky after Allow Paste. + XCTAssertEqual( + KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: false, + stickyPreferVoice: true, + preferred: .typing + ), + .voice, + "Allow Paste reopen must not land on default Chinese typing grid" + ) + + // Recording lock also forces voice. + XCTAssertEqual( + KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: true, + clipboardCommandActive: false, + stickyPreferVoice: false, + preferred: .typing + ), + .voice + ) + } + + func testMarkStartIssuedBlocksDuplicateRoundClaim() { + let id = UUID() + XCTAssertFalse(ClipboardCommandResume.hasStartIssued(defaults: defaults)) + ClipboardCommandResume.markStartIssued(id, defaults: defaults) + XCTAssertTrue(ClipboardCommandResume.hasStartIssued(defaults: defaults)) + XCTAssertEqual(ClipboardCommandResume.startIssuedUtteranceId(defaults: defaults), id) + XCTAssertTrue(ClipboardCommandResume.shouldPreferVoice(defaults: defaults)) + + let reopened = UserDefaults(suiteName: suiteName!) + XCTAssertEqual( + ClipboardCommandResume.startIssuedUtteranceId(defaults: reopened), + id, + "Recreated extension must see the already-issued start and not send another" + ) + + ClipboardCommandResume.clear(defaults: defaults) + XCTAssertFalse(ClipboardCommandResume.hasStartIssued(defaults: defaults)) + XCTAssertNil(ClipboardCommandResume.startIssuedUtteranceId(defaults: reopened)) + } + + func testClaimThenClearSurvivesTwentyAlternatingRounds() { + for round in 1...20 { + let id = UUID() + ClipboardCommandResume.storeSnapshot("材料-\(round)", defaults: defaults) + ClipboardCommandResume.markStartIssued(id, defaults: defaults) + + let reader = UserDefaults(suiteName: suiteName!) + XCTAssertEqual( + ClipboardCommandResume.startIssuedUtteranceId(defaults: reader), + id, + "round \(round) claim must survive new defaults handle" + ) + XCTAssertEqual( + KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: false, + stickyPreferVoice: ClipboardCommandResume.shouldPreferVoice(defaults: reader), + preferred: .typing + ), + .voice, + "round \(round) must stay on voice after paste-alert recreate" + ) + XCTAssertEqual( + ClipboardPreparingPolicy.restoreAction( + hasStartIssued: ClipboardCommandResume.hasStartIssued(defaults: reader), + phase: .idle + ), + .awaitExistingStart, + "round \(round) must not pressBegan again" + ) + + ClipboardCommandResume.clear(defaults: defaults) + XCTAssertFalse( + ClipboardCommandResume.hasStartIssued(defaults: reader), + "round \(round) clear must drop claim for next independent long-press" + ) + } + } + + func testPreparingTimeoutConstantIsPositive() { + XCTAssertGreaterThan(ClipboardCommandResume.preparingTimeout, 1) + XCTAssertLessThanOrEqual(ClipboardCommandResume.preparingTimeout, 15) + } + + func testColdStartStickyRestoreIsPreferVoiceOnlyWithoutClaim() { + ClipboardCommandResume.storeSnapshot("冻结材料", defaults: defaults) + XCTAssertFalse(ClipboardCommandResume.hasStartIssued(defaults: defaults)) + XCTAssertEqual( + ClipboardPreparingPolicy.restoreAction( + hasStartIssued: false, + phase: .idle + ), + .preferVoiceOnly + ) + XCTAssertEqual( + KeyboardOpenSurfacePolicy.resolve( + locksTypingSurface: false, + clipboardCommandActive: false, + stickyPreferVoice: ClipboardCommandResume.shouldPreferVoice(defaults: defaults), + preferred: .typing + ), + .voice, + "Default typing keyboard must still open voice after cold-start sticky" + ) + } +} diff --git a/OSGKeyboardTests/ClipboardMaterialFilterTests.swift b/OSGKeyboardTests/ClipboardMaterialFilterTests.swift index 43747c5..1726c7d 100644 --- a/OSGKeyboardTests/ClipboardMaterialFilterTests.swift +++ b/OSGKeyboardTests/ClipboardMaterialFilterTests.swift @@ -69,7 +69,6 @@ final class ClipboardMaterialFilterTests: XCTestCase { XCTAssertEqual(ClipboardMaterialFilter.maxSnapshotLength, 3_000) XCTAssertEqual(ClipboardMaterialFilter.longPressDuration, 0.45, accuracy: 0.001) XCTAssertEqual(ClipboardMaterialFilter.minimumRecordingAfterHostConfirm, 0.70, accuracy: 0.001) - XCTAssertEqual(ClipboardMaterialFilter.eligibilityDuration, 30, accuracy: 0.001) - XCTAssertEqual(ClipboardMaterialFilter.sessionDuration, 30, accuracy: 0.001) + XCTAssertEqual(ClipboardMaterialFilter.failureHintDuration, 2.5, accuracy: 0.001) } } diff --git a/OSGKeyboardTests/ClipboardPreparingPolicyTests.swift b/OSGKeyboardTests/ClipboardPreparingPolicyTests.swift new file mode 100644 index 0000000..ce97f86 --- /dev/null +++ b/OSGKeyboardTests/ClipboardPreparingPolicyTests.swift @@ -0,0 +1,350 @@ +// ClipboardPreparingPolicyTests.swift +// OSGKeyboardTests +// +// Decision-matrix coverage for clipboard prepare stuck / double-start bugs. + +import XCTest +@testable import OSGKeyboardShared + +final class ClipboardPreparingPolicyTests: XCTestCase { + + // MARK: - Restore (paste-alert recreate) + + func testRestoreAwaitExistingWhenStartAlreadyIssued() { + XCTAssertEqual( + ClipboardPreparingPolicy.restoreAction( + hasStartIssued: true, + phase: .idle + ), + .awaitExistingStart + ) + XCTAssertEqual( + ClipboardPreparingPolicy.restoreAction( + hasStartIssued: true, + phase: .error + ), + .awaitExistingStart + ) + } + + func testRestorePreferVoiceOnlyWhenNoClaim() { + XCTAssertEqual( + ClipboardPreparingPolicy.restoreAction( + hasStartIssued: false, + phase: .idle + ), + .preferVoiceOnly, + "Cold-start return must not auto-start recording" + ) + } + + func testRestoreRefreshOnlyWhenAlreadyLive() { + for phase: ClipboardPreparingPhase in [ + .requestingPermissions, .recording, .processing + ] { + XCTAssertEqual( + ClipboardPreparingPolicy.restoreAction( + hasStartIssued: true, + phase: phase + ), + .refreshOnly + ) + } + } + + /// Exact device bug: Allow Paste recreates keyboard while claim exists. + func testPasteAlertReopenMustNotPreferVoiceOnlyWhenClaimed() { + let action = ClipboardPreparingPolicy.restoreAction( + hasStartIssued: true, + phase: .idle + ) + XCTAssertNotEqual(action, .preferVoiceOnly) + XCTAssertEqual(action, .awaitExistingStart) + } + + func testHostGateNeverAutoRecordsAfterWarmup() { + XCTAssertEqual( + ClipboardPreparingPolicy.hostGateAction(micPressAction: .startRecording), + .startRecordingNow + ) + XCTAssertEqual( + ClipboardPreparingPolicy.hostGateAction(micPressAction: .openHostColdStart), + .openHostColdStart + ) + XCTAssertEqual( + ClipboardPreparingPolicy.hostGateAction( + micPressAction: .waitForHostReady(recordWhenReady: true) + ), + .waitForHost, + "Clipboard must ignore recordWhenReady and require a second long-press" + ) + XCTAssertEqual( + ClipboardPreparingPolicy.hostGateAction(micPressAction: .ignore), + .ignore + ) + } + + func testMicChromeGreyWhilePreparingBlueOnlyAfterConfirm() { + XCTAssertEqual( + ClipboardPreparingPolicy.micChrome( + isClipboardUtterance: true, + phase: .requestingPermissions, + awaitingHostConfirm: true + ), + .preparingDisabled + ) + XCTAssertEqual( + ClipboardPreparingPolicy.micChrome( + isClipboardUtterance: true, + phase: .recording, + awaitingHostConfirm: false + ), + .recordingBlue + ) + XCTAssertEqual( + ClipboardPreparingPolicy.micChrome( + isClipboardUtterance: false, + phase: .recording, + awaitingHostConfirm: false + ), + .none + ) + // Paste-acquire (idle + active flag) must not go blue. + XCTAssertEqual( + ClipboardPreparingPolicy.micChrome( + isClipboardUtterance: true, + phase: .idle, + awaitingHostConfirm: false + ), + .none + ) + } + + // MARK: - Stop while preparing + + func testTapDuringPreparingAbortsInsteadOfWaitingForever() { + XCTAssertEqual( + ClipboardPreparingPolicy.stopWhilePreparing(awaitingHostConfirm: true), + .abortPreparing + ) + XCTAssertEqual( + ClipboardPreparingPolicy.stopWhilePreparing(awaitingHostConfirm: false), + .requestStop + ) + } + + // MARK: - Recover while preparing + + func testRecoverConfirmsMatchingRecordingUtterance() { + let id = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: true, + currentUtteranceId: id, + hostBusyUtteranceId: id, + hostReason: .recording, + hasTerminalFailureForCurrent: false + ), + .confirmRecording + ) + } + + func testRecoverAdoptsSiblingOnDoubleStart() { + let ours = UUID() + let sibling = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: true, + currentUtteranceId: ours, + hostBusyUtteranceId: sibling, + hostReason: .recording, + hasTerminalFailureForCurrent: false + ), + .adoptSibling(sibling) + ) + XCTAssertEqual( + ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: true, + currentUtteranceId: ours, + hostBusyUtteranceId: sibling, + hostReason: .processing, + hasTerminalFailureForCurrent: false + ), + .adoptSibling(sibling) + ) + } + + func testRecoverAbortsOnHostTerminalFailure() { + let id = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: true, + currentUtteranceId: id, + hostBusyUtteranceId: nil, + hostReason: nil, + hasTerminalFailureForCurrent: true + ), + .abortForHostFailure + ) + } + + func testRecoverDoesNothingWhenNotAwaiting() { + XCTAssertEqual( + ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: false, + currentUtteranceId: UUID(), + hostBusyUtteranceId: UUID(), + hostReason: .recording, + hasTerminalFailureForCurrent: true + ), + .none + ) + } + + // MARK: - Ensure single startRecording + + func testEnsureStartSkipsWhenAlreadyInFlight() { + let id = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: id, + isFlowRecording: true, + currentUtteranceId: id, + hostBusyUtteranceId: nil, + hostReason: nil, + hostReadyWithSession: true + ), + .alreadyInFlight + ) + } + + func testEnsureStartAdoptsHostBusyInsteadOfSecondWrite() { + let issued = UUID() + let busy = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: issued, + isFlowRecording: false, + currentUtteranceId: issued, + hostBusyUtteranceId: busy, + hostReason: .recording, + hostReadyWithSession: true + ), + .adoptBusy(busy, .recording) + ) + } + + func testEnsureStartWritesOnceWhenHostReady() { + let id = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: id, + isFlowRecording: false, + currentUtteranceId: id, + hostBusyUtteranceId: nil, + hostReason: nil, + hostReadyWithSession: true + ), + .writeStart(id) + ) + } + + func testEnsureStartWaitsWhenHostNotReady() { + let id = UUID() + XCTAssertEqual( + ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: id, + isFlowRecording: false, + currentUtteranceId: id, + hostBusyUtteranceId: nil, + hostReason: nil, + hostReadyWithSession: false + ), + .waitForHost + ) + } + + /// Simulate paste-alert reopen loop: claim → restore → ensure must not + /// produce two writeStart for the same issued id across polls. + func testDoubleStartStressMatrixTwentyRounds() { + for round in 1...20 { + let claim = UUID() + // First long-press claims. + XCTAssertTrue(claim.uuidString.isEmpty == false) + + // Restore after Allow Paste / cold-start: + // - with claim → awaitExistingStart + // - without claim → preferVoiceOnly (no auto-record) + let restoreClaimed = ClipboardPreparingPolicy.restoreAction( + hasStartIssued: true, + phase: .idle + ) + XCTAssertEqual(restoreClaimed, .awaitExistingStart, "round \(round)") + let restoreWarm = ClipboardPreparingPolicy.restoreAction( + hasStartIssued: false, + phase: .idle + ) + XCTAssertEqual(restoreWarm, .preferVoiceOnly, "round \(round)") + + XCTAssertEqual( + ClipboardPreparingPolicy.hostGateAction(micPressAction: .openHostColdStart), + .openHostColdStart, + "round \(round)" + ) + + // First ensure writes once. + let first = ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: claim, + isFlowRecording: false, + currentUtteranceId: claim, + hostBusyUtteranceId: nil, + hostReason: nil, + hostReadyWithSession: true + ) + XCTAssertEqual(first, .writeStart(claim), "round \(round)") + + // Second ensure (same process or restore) must not write again. + let second = ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: claim, + isFlowRecording: true, + currentUtteranceId: claim, + hostBusyUtteranceId: nil, + hostReason: nil, + hostReadyWithSession: true + ) + XCTAssertEqual(second, .alreadyInFlight, "round \(round)") + + // Sibling double-start on host → adopt, never another write. + let sibling = UUID() + let adopt = ClipboardPreparingPolicy.ensureStartAction( + issuedUtteranceId: claim, + isFlowRecording: false, + currentUtteranceId: claim, + hostBusyUtteranceId: sibling, + hostReason: .recording, + hostReadyWithSession: true + ) + XCTAssertEqual(adopt, .adoptBusy(sibling, .recording), "round \(round)") + + // Mic-not-ready failure while preparing → abort. + XCTAssertEqual( + ClipboardPreparingPolicy.recoverWhilePreparing( + awaitingHostConfirm: true, + currentUtteranceId: claim, + hostBusyUtteranceId: nil, + hostReason: nil, + hasTerminalFailureForCurrent: true + ), + .abortForHostFailure, + "round \(round)" + ) + + // User tap while preparing cancels. + XCTAssertEqual( + ClipboardPreparingPolicy.stopWhilePreparing(awaitingHostConfirm: true), + .abortPreparing, + "round \(round)" + ) + } + } +} diff --git a/OSGKeyboardTests/FlowHandoffPolicyTests.swift b/OSGKeyboardTests/FlowHandoffPolicyTests.swift index 849aaa9..8f15229 100644 --- a/OSGKeyboardTests/FlowHandoffPolicyTests.swift +++ b/OSGKeyboardTests/FlowHandoffPolicyTests.swift @@ -330,4 +330,113 @@ final class FlowHandoffPolicyTests: XCTestCase { .silence ) } + + // MARK: - Proactive PiP arm (no re-jump) + + func testProactivePiPArmSkippedWhenHostReady() { + XCTAssertFalse( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: true, + snapshotReason: .ready, + sessionActive: true, + hostReachable: true, + hostStale: false, + withinReadyGrace: false, + inCooldown: false + ) + ) + } + + func testProactivePiPArmSkippedWhileStartingOrCooldown() { + XCTAssertFalse( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: false, + snapshotReason: .starting, + sessionActive: true, + hostReachable: true, + hostStale: false, + withinReadyGrace: false, + inCooldown: false + ), + "Warming session must not re-open startflow on Voice tab" + ) + XCTAssertFalse( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: false, + snapshotReason: nil, + sessionActive: false, + hostReachable: false, + hostStale: false, + withinReadyGrace: false, + inCooldown: true + ), + "Cooldown after a recent arm must block re-jump" + ) + } + + func testProactivePiPArmAllowsForceQuitZombieSessionAfterHeartbeatGrace() { + // sessionActive still true, but heartbeat gone > 8s → allow one arm. + XCTAssertTrue( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: false, + snapshotReason: .ready, + sessionActive: true, + hostReachable: false, + hostStale: false, + withinReadyGrace: false, + inCooldown: false, + heartbeatStaleness: FlowHandoffPolicy.proactiveUnreachableArmGrace + ) + ) + // Brief switcher flap (< grace) must NOT re-jump. + XCTAssertFalse( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: false, + snapshotReason: .ready, + sessionActive: true, + hostReachable: false, + hostStale: false, + withinReadyGrace: false, + inCooldown: false, + heartbeatStaleness: 3 + ) + ) + // Cooldown still wins after a recent arm. + XCTAssertFalse( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: false, + snapshotReason: .ready, + sessionActive: true, + hostReachable: false, + hostStale: false, + withinReadyGrace: false, + inCooldown: true, + heartbeatStaleness: 30 + ) + ) + } + + func testProactivePiPArmAllowedWhenHostTrulyDead() { + XCTAssertTrue( + FlowHandoffPolicy.shouldProactivePiPArm( + hostReady: false, + snapshotReason: .noSession, + sessionActive: false, + hostReachable: false, + hostStale: false, + withinReadyGrace: false, + inCooldown: false + ) + ) + } + + func testPiPArmCooldownRoundTrip() { + let suite = "PiPArmCooldown.\(UUID().uuidString)" + let defaults = UserDefaults(suiteName: suite)! + defer { defaults.removePersistentDomain(forName: suite) } + + XCTAssertFalse(FlowSessionBridge.isPiPArmInCooldown(defaults: defaults)) + FlowSessionBridge.markPiPArmAttempt(defaults: defaults) + XCTAssertTrue(FlowSessionBridge.isPiPArmInCooldown(defaults: defaults)) + } } diff --git a/Scripts/cold-start-stress.sh b/Scripts/cold-start-stress.sh new file mode 100755 index 0000000..d4db83c --- /dev/null +++ b/Scripts/cold-start-stress.sh @@ -0,0 +1,243 @@ +#!/usr/bin/env bash +# Cold-start stress on iOS Simulator. +# +# Usage: +# ./Scripts/cold-start-stress.sh [COUNT=100] [DEVICE_NAME=iPhone 17] +# +# Method: terminate → simctl launch (host cold start that auto-arms PiP). +# Keyboard URL wake (osgkeyboard://startflow) is optional once scheme is approved. +# +# Limitations (documented in summary): +# - Simulator VideoCall PiP is typically `unsupported` — not a device signal. +# - Keyboard extension process (KVC.init) is not spawned here. +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$ROOT" + +COUNT="${1:-100}" +DEVICE_NAME="${2:-iPhone 17}" +BUNDLE="com.osgkeyboard.ios" +OUT_DIR="${ROOT}/.tmp/cold-start-stress-$(date +%Y%m%d-%H%M%S)" +mkdir -p "$OUT_DIR" +REPORT="$OUT_DIR/report.jsonl" +SUMMARY="$OUT_DIR/summary.txt" +LOG_FILE="$OUT_DIR/unified.log" + +echo "==> Out: $OUT_DIR" +echo "==> Count: $COUNT Device: '$DEVICE_NAME'" + +# Exact device name match (avoid "iPhone 17" → "iPhone 17 Pro") +UDID="$(xcrun simctl list devices available | awk -F '[()]' -v n="$DEVICE_NAME" ' + { + line=$0 + # strip leading spaces + sub(/^[[:space:]]+/, "", line) + name=line + sub(/ \(.*/, "", name) + if (name == n && line ~ /Booted/) { print $2; exit } + } +')" +if [[ -z "$UDID" ]]; then + UDID="$(xcrun simctl list devices available | awk -F '[()]' -v n="$DEVICE_NAME" ' + { + line=$0 + sub(/^[[:space:]]+/, "", line) + name=line + sub(/ \(.*/, "", name) + if (name == n && line !~ /unavailable/) { print $2; exit } + } + ')" +fi +if [[ -z "$UDID" ]]; then + echo "error: exact device '$DEVICE_NAME' not found" >&2 + xcrun simctl list devices available | grep -i iphone | head -40 >&2 || true + exit 1 +fi +echo "==> UDID: $UDID" + +echo "==> Booting simulator" +xcrun simctl boot "$UDID" 2>/dev/null || true +xcrun simctl bootstatus "$UDID" -b + +# Prefer signed cold-start DerivedData build, else any existing Debug-iphonesimulator app +APP="" +if [[ -d "$ROOT/.derivedData-cold-start/Build/Products/Debug-iphonesimulator/OSGKeyboard.app" ]]; then + APP="$ROOT/.derivedData-cold-start/Build/Products/Debug-iphonesimulator/OSGKeyboard.app" +else + APP="$(ls -d "$HOME"/Library/Developer/Xcode/DerivedData/OSGKeyboard-*/Build/Products/Debug-iphonesimulator/OSGKeyboard.app 2>/dev/null | head -1 || true)" +fi +if [[ -z "$APP" || ! -d "$APP" ]]; then + echo "error: no built OSGKeyboard.app — build for simulator first" >&2 + exit 1 +fi +echo "==> Installing $APP" +xcrun simctl install "$UDID" "$APP" >/dev/null +xcrun simctl privacy "$UDID" grant microphone "$BUNDLE" 2>/dev/null || true + +seed_onboarding() { + echo "==> Seeding onboarding (App Group + Keychain path)" + xcrun simctl launch "$UDID" "$BUNDLE" >/dev/null || true + sleep 2 + xcrun simctl terminate "$UDID" "$BUNDLE" 2>/dev/null || true + sleep 0.5 + + local group_container prefs + group_container="$(xcrun simctl get_app_container "$UDID" "$BUNDLE" group.com.osgkeyboard.shared 2>/dev/null || true)" + if [[ -z "$group_container" || ! -d "$group_container" ]]; then + echo "warning: App Group container missing" >&2 + return 0 + fi + prefs="$group_container/Library/Preferences/group.com.osgkeyboard.shared.plist" + mkdir -p "$(dirname "$prefs")" + if [[ -f "$prefs" ]]; then + /usr/libexec/PlistBuddy -c "Set :config.hasCompletedOnboarding true" "$prefs" 2>/dev/null \ + || /usr/libexec/PlistBuddy -c "Add :config.hasCompletedOnboarding bool true" "$prefs" 2>/dev/null || true + else + /usr/libexec/PlistBuddy -c "Add :config.hasCompletedOnboarding bool true" "$prefs" 2>/dev/null || true + fi + echo "==> Seeded hasCompletedOnboarding=$(/usr/libexec/PlistBuddy -c 'Print :config.hasCompletedOnboarding' "$prefs" 2>/dev/null || echo missing)" + + # One launch so ProviderConfig mirrors App Group → Keychain + xcrun simctl launch "$UDID" "$BUNDLE" >/dev/null || true + sleep 2 + xcrun simctl terminate "$UDID" "$BUNDLE" 2>/dev/null || true + sleep 0.4 +} +seed_onboarding + +: >"$LOG_FILE" +xcrun simctl spawn "$UDID" log stream \ + --style compact \ + --level debug \ + --predicate 'process == "OSGKeyboard"' \ + >"$LOG_FILE" 2>&1 & +LOG_PID=$! +trap 'kill $LOG_PID 2>/dev/null || true' EXIT +sleep 1 + +logfile_window() { + local offset="$1" + local start=$((offset + 1)) + tail -c +"$start" "$LOG_FILE" 2>/dev/null || true +} + +# Returns: host|pip +# host: success|fail|unknown +# pip: success|unsupported|fail|permissions|onboarding|unknown +logfile_classify() { + local chunk="$1" + local host="unknown" pip="unknown" + + # Use here-string (not pipe) to avoid SIGPIPE under `set -o pipefail`. + if grep -Eq "OSGKeyboardApp\.init" <<<"$chunk"; then + host="success" + fi + if grep -Eq "MainAppRoot\.onAppear skip Flow.*onboarding incomplete" <<<"$chunk"; then + pip="onboarding" + echo "$host|$pip" + return + fi + if grep -Eq "activateOnForeground aborted reason=permissions|startSessionAsync\.blocked.*permissions" <<<"$chunk"; then + pip="permissions" + echo "$host|$pip" + return + fi + if grep -Eq "Flow session started \(PiP keep-alive\)|low-profile PiP active|startSessionAsync\.ready" <<<"$chunk"; then + pip="success" + elif grep -Eq "failure=unsupported|PiP keep-alive failed to start: unsupported" <<<"$chunk"; then + pip="unsupported" + elif grep -Eq "PiP keep-alive failed to start|PiP startAndWait failed|startSessionAsync\.failed.*pipUnavailable" <<<"$chunk"; then + pip="fail" + elif grep -Eq "activateOnForeground|startSession\.request.*autoPiP|startSessionAsync\.begin" <<<"$chunk"; then + pip="seen_no_result" + fi + + echo "$host|$pip" +} + +HOST_OK=0 +HOST_FAIL=0 +PIP_SUCCESS=0 +PIP_UNSUPPORTED=0 +PIP_FAIL=0 +PIP_PERMISSIONS=0 +PIP_ONBOARDING=0 +PIP_OTHER=0 + +echo "==> Running $COUNT cold starts (terminate → launch)" +for i in $(seq 1 "$COUNT"); do + xcrun simctl terminate "$UDID" "$BUNDLE" 2>/dev/null || true + sleep 0.35 + + OFFSET=$(wc -c <"$LOG_FILE" | tr -d ' ') + START_TS=$(date +%s) + + if ! xcrun simctl launch "$UDID" "$BUNDLE" >/dev/null 2>"$OUT_DIR/launch-$i.err"; then + printf '{"i":%d,"host":"fail","pip":"unknown","elapsed_s":0,"note":"launch_failed"}\n' "$i" >>"$REPORT" + printf "[%3d/%d] host=fail pip=unknown\n" "$i" "$COUNT" + HOST_FAIL=$((HOST_FAIL + 1)) + PIP_OTHER=$((PIP_OTHER + 1)) + continue + fi + + RESULT="unknown|unknown" + for _ in $(seq 1 50); do # ~15s + sleep 0.3 + CHUNK="$(logfile_window "$OFFSET")" + RESULT="$(logfile_classify "$CHUNK")" + PIP="${RESULT##*|}" + if [[ "$PIP" == "success" || "$PIP" == "unsupported" || "$PIP" == "fail" || "$PIP" == "permissions" || "$PIP" == "onboarding" ]]; then + break + fi + done + + ELAPSED=$(( $(date +%s) - START_TS )) + HOST="${RESULT%%|*}" + PIP="${RESULT##*|}" + + case "$HOST" in + success) HOST_OK=$((HOST_OK + 1)) ;; + *) HOST_FAIL=$((HOST_FAIL + 1)) ;; + esac + case "$PIP" in + success) PIP_SUCCESS=$((PIP_SUCCESS + 1)) ;; + unsupported) PIP_UNSUPPORTED=$((PIP_UNSUPPORTED + 1)) ;; + fail) PIP_FAIL=$((PIP_FAIL + 1)) ;; + permissions) PIP_PERMISSIONS=$((PIP_PERMISSIONS + 1)) ;; + onboarding) PIP_ONBOARDING=$((PIP_ONBOARDING + 1)) ;; + *) PIP_OTHER=$((PIP_OTHER + 1)) ;; + esac + + printf '{"i":%d,"host":"%s","pip":"%s","elapsed_s":%d}\n' "$i" "$HOST" "$PIP" "$ELAPSED" >>"$REPORT" + printf "[%3d/%d] host=%-8s pip=%-12s %2ds\n" "$i" "$COUNT" "$HOST" "$PIP" "$ELAPSED" +done + +{ + echo "Cold-start stress summary" + echo "device=$DEVICE_NAME udid=$UDID count=$COUNT" + echo "method=terminate + simctl launch (host cold start / autoPiP path)" + echo + echo "Host cold start (OSGKeyboardApp.init observed):" + echo " success=$HOST_OK fail=$HOST_FAIL" + echo " fail_rate=$(python3 -c "print(f'{$HOST_FAIL/$COUNT*100:.1f}%')")" + echo + echo "PiP keep-alive outcome:" + echo " success=$PIP_SUCCESS" + echo " unsupported=$PIP_UNSUPPORTED (expected on Simulator)" + echo " fail_other=$PIP_FAIL" + echo " permissions=$PIP_PERMISSIONS" + echo " onboarding=$PIP_ONBOARDING" + echo " unknown/other=$PIP_OTHER" + echo " non-unsupported fail_rate=$(python3 -c "print(f'{($PIP_FAIL+$PIP_PERMISSIONS+$PIP_ONBOARDING+$PIP_OTHER)/$COUNT*100:.1f}%')")" + echo + echo "Artifacts: $OUT_DIR" + echo "Caveats:" + echo " - Simulator PiP is typically 'unsupported'; device PiP is the release gate." + echo " - Keyboard extension KVC.init / openHostApp is not covered (needs physical keyboard enablement)." + echo " - Host wake path covered = cold launch of the process the keyboard would open." +} | tee "$SUMMARY" + +kill "$LOG_PID" 2>/dev/null || true +trap - EXIT +echo "==> Done" diff --git a/Scripts/compose_app_preview_zh.py b/Scripts/compose_app_preview_zh.py new file mode 100644 index 0000000..45eac6a --- /dev/null +++ b/Scripts/compose_app_preview_zh.py @@ -0,0 +1,312 @@ +#!/usr/bin/env python3 +"""Compose a Chinese App Store preview (iPhone 6.7\") for OSGKeyboard. + +Uses real Simulator / marketing UI stills + ffmpeg Ken Burns + ASS titles. +Output: docs/assets/app-preview/zh/OSGKeyboard-preview-6.7-zh.mp4 +""" + +from __future__ import annotations + +import subprocess +import sys +from pathlib import Path + +from PIL import Image, ImageDraw, ImageFont + +ROOT = Path(__file__).resolve().parents[1] +OUT_DIR = ROOT / "docs" / "assets" / "app-preview" / "zh" +WORK = ROOT / ".tmp" / "app-preview-zh" / "compose" +W, H = 1290, 2796 # App Store Connect 6.7" +FPS = 30 + +HOME_SRC = ROOT / ".tmp" / "app-preview-zh" / "shot-home-dark.png" +KB_IDLE = ROOT / "docs" / "assets" / "screenshots" / "zh" / "dark" / "iphone-keyboard_idle.png" +KB_REC = ROOT / "docs" / "assets" / "screenshots" / "zh" / "dark" / "iphone-keyboard_recording.png" +ICON = ROOT / "docs" / "assets" / "app-icon.png" + +# Prefer a clean Chinese UI font available on this machine. +FONT_CANDIDATES = [ + Path("/Users/rocky/Library/Fonts/OPPO Sans 4.0.ttf"), + Path("/System/Library/Fonts/STHeiti Medium.ttc"), + Path("/System/Library/Fonts/Supplemental/Songti.ttc"), +] + + +def pick_font(size: int) -> ImageFont.FreeTypeFont | ImageFont.ImageFont: + for path in FONT_CANDIDATES: + if not path.exists(): + continue + try: + return ImageFont.truetype(str(path), size=size, index=0) + except OSError: + continue + return ImageFont.load_default() + + +def cover(img: Image.Image) -> Image.Image: + """Scale to cover WxH and center-crop.""" + src = img.convert("RGB") + scale = max(W / src.width, H / src.height) + nw, nh = int(src.width * scale), int(src.height * scale) + resized = src.resize((nw, nh), Image.Resampling.LANCZOS) + left = (nw - W) // 2 + top = (nh - H) // 2 + return resized.crop((left, top, left + W, top + H)) + + +def patch_home_warning(src: Path) -> Image.Image: + """Hide Simulator-only Flow warning banner for marketing stills.""" + im = Image.open(src).convert("RGB") + # Banner sits under the three status dots (~y 520–580 @ 1320×2868). + sample = im.getpixel((im.width // 2, 500)) + draw = ImageDraw.Draw(im) + draw.rounded_rectangle((48, 518, im.width - 48, 586), radius=28, fill=sample) + return cover(im) + + +def title_card(lines: list[str], subtitle: str | None = None, with_icon: bool = False) -> Image.Image: + im = Image.new("RGB", (W, H), (12, 12, 14)) + # Soft brand glow + glow = Image.new("RGB", (W, H), (12, 12, 14)) + gdraw = ImageDraw.Draw(glow) + cx, cy = W // 2, int(H * 0.38) + for r, alpha in ((520, 28), (360, 40), (220, 55)): + color = (18 + alpha // 4, 40 + alpha // 3, 28 + alpha // 5) + gdraw.ellipse((cx - r, cy - r, cx + r, cy + r), fill=color) + im = Image.blend(im, glow, 0.55) + draw = ImageDraw.Draw(im) + + y = int(H * 0.34) + if with_icon and ICON.exists(): + icon = Image.open(ICON).convert("RGBA").resize((220, 220), Image.Resampling.LANCZOS) + # Rounded mask + mask = Image.new("L", (220, 220), 0) + ImageDraw.Draw(mask).rounded_rectangle((0, 0, 219, 219), radius=48, fill=255) + im.paste(icon, ((W - 220) // 2, y - 280), mask) + y = int(H * 0.42) + + title_font = pick_font(96 if len(lines) == 1 else 84) + sub_font = pick_font(44) + for line in lines: + bbox = draw.textbbox((0, 0), line, font=title_font) + tw = bbox[2] - bbox[0] + draw.text(((W - tw) // 2, y), line, fill=(245, 246, 248), font=title_font) + y += 120 + if subtitle: + bbox = draw.textbbox((0, 0), subtitle, font=sub_font) + tw = bbox[2] - bbox[0] + draw.text(((W - tw) // 2, y + 24), subtitle, fill=(140, 160, 145), font=sub_font) + return im + + +def overlay_caption(base: Image.Image, text: str) -> Image.Image: + """Bottom gradient + caption for ad-style supers.""" + im = base.copy().convert("RGBA") + shade = Image.new("RGBA", (W, H), (0, 0, 0, 0)) + d = ImageDraw.Draw(shade) + for i in range(420): + a = int(190 * (i / 419)) + y = H - 420 + i + d.line([(0, y), (W, y)], fill=(0, 0, 0, a)) + im = Image.alpha_composite(im, shade) + draw = ImageDraw.Draw(im) + font = pick_font(64) + bbox = draw.textbbox((0, 0), text, font=font) + tw = bbox[2] - bbox[0] + draw.text(((W - tw) // 2, H - 260), text, fill=(255, 255, 255, 255), font=font) + return im.convert("RGB") + + +def write_ass(path: Path) -> None: + # Kept for optional future hardsubs; primary captions are burned into stills. + path.write_text( + """[Script Info] +ScriptType: v4.00+ +PlayResX: 1290 +PlayResY: 2796 + +[V4+ Styles] +Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding +Style: Default,Heiti SC,64,&H00FFFFFF,&H000000FF,&H64000000,&H80000000,0,0,0,0,100,100,0,0,1,0,0,2,60,60,120,1 + +[Events] +Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text +""", + encoding="utf-8", + ) + + +def run(cmd: list[str]) -> None: + print("+", " ".join(cmd[:8]), "..." if len(cmd) > 8 else "") + subprocess.run(cmd, check=True) + + +def main() -> int: + if not HOME_SRC.exists(): + print(f"missing home still: {HOME_SRC}", file=sys.stderr) + return 1 + + WORK.mkdir(parents=True, exist_ok=True) + OUT_DIR.mkdir(parents=True, exist_ok=True) + + home = patch_home_warning(HOME_SRC) + kb_idle = cover(Image.open(KB_IDLE)) + kb_rec = cover(Image.open(KB_REC)) + + scenes = { + "01_open": title_card(["开口即文字"], with_icon=True), + "02_hook": overlay_caption(home, "说，就好了"), + "03_voice": overlay_caption(kb_rec, "说完就能用"), + "04_type": overlay_caption(kb_idle, "打字也在行"), + "05_privacy": overlay_caption(home, "默认不上传录音"), + "06_end": title_card(["OSGKeyboard"], "开口即文字", with_icon=True), + } + for name, img in scenes.items(): + img.save(WORK / f"{name}.png", optimize=True) + + # Durations (seconds) — total ~28s + timeline = [ + ("01_open", 2.2), + ("02_hook", 3.2), + ("03_voice", 7.0), + ("04_type", 6.0), + ("05_privacy", 5.0), + ("06_end", 4.0), + ] + + # Build concat list with zoompan Ken Burns per still. + parts: list[Path] = [] + for idx, (name, dur) in enumerate(timeline): + src = WORK / f"{name}.png" + part = WORK / f"part-{idx:02d}.mp4" + frames = max(1, int(dur * FPS)) + # Gentle zoom: alternate direction for rhythm + if idx % 2 == 0: + z = f"min(zoom+0.00045,1.08)" + x = "iw/2-(iw/zoom/2)" + y = "ih/2-(ih/zoom/2)" + else: + z = f"if(eq(on,1),1.08,max(zoom-0.00045,1.0))" + x = "iw/2-(iw/zoom/2)" + y = "ih/2-(ih/zoom/2)" + vf = ( + f"scale={W}:{H}:force_original_aspect_ratio=increase," + f"crop={W}:{H}," + f"zoompan=z='{z}':x='{x}':y='{y}':d={frames}:s={W}x{H}:fps={FPS}," + f"setsar=1,format=yuv420p" + ) + run( + [ + "ffmpeg", + "-y", + "-loop", + "1", + "-i", + str(src), + "-vf", + vf, + "-t", + f"{dur:.2f}", + "-c:v", + "libx264", + "-pix_fmt", + "yuv420p", + "-profile:v", + "high", + "-level", + "4.0", + "-crf", + "18", + "-r", + str(FPS), + str(part), + ] + ) + parts.append(part) + + concat_list = WORK / "concat.txt" + concat_list.write_text("".join(f"file '{p}'\n" for p in parts), encoding="utf-8") + + silent = WORK / "preview-silent.mp4" + run( + [ + "ffmpeg", + "-y", + "-f", + "concat", + "-safe", + "0", + "-i", + str(concat_list), + "-c:v", + "libx264", + "-pix_fmt", + "yuv420p", + "-movflags", + "+faststart", + "-r", + str(FPS), + str(silent), + ] + ) + + # Soft generated bed (no third-party music / no copyright risk). + final = OUT_DIR / "OSGKeyboard-preview-6.7-zh.mp4" + run( + [ + "ffmpeg", + "-y", + "-i", + str(silent), + "-f", + "lavfi", + "-i", + "sine=frequency=196:sample_rate=44100,volume=0.035,afade=t=in:st=0:d=1.2,afade=t=out:st=25:d=2.5", + "-f", + "lavfi", + "-i", + "sine=frequency=293.66:sample_rate=44100,volume=0.02,afade=t=in:st=0:d=1.5,afade=t=out:st=25:d=2.5", + "-filter_complex", + "[1:a][2:a]amix=inputs=2:duration=first:dropout_transition=2[a]", + "-map", + "0:v", + "-map", + "[a]", + "-c:v", + "copy", + "-c:a", + "aac", + "-b:a", + "128k", + "-shortest", + "-movflags", + "+faststart", + str(final), + ] + ) + + # Probe + probe = subprocess.check_output( + [ + "ffprobe", + "-v", + "error", + "-select_streams", + "v:0", + "-show_entries", + "stream=width,height,duration", + "-show_entries", + "format=duration,size", + "-of", + "default=noprint_wrappers=1", + str(final), + ], + text=True, + ) + print(probe) + print(f"Wrote {final}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/Scripts/device-pip-stress.sh b/Scripts/device-pip-stress.sh new file mode 100755 index 0000000..7746faf --- /dev/null +++ b/Scripts/device-pip-stress.sh @@ -0,0 +1,193 @@ +#!/usr/bin/env bash +# Physical-device PiP / host-wake stress using `devicectl --console` logs. +# +# Usage: +# ./Scripts/device-pip-stress.sh [UDID] [COUNT=50] +# +# Suites: +# cold — terminate-existing launch (force-quit / cold start) +# bgfg — open Safari (background) then relaunch OSG (foreground restore) +# hold — start PiP, wait 20s in session, relaunch to verify still recoverable +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +cd "$ROOT" + +UDID="${1:-00008130-001C249C0E52001C}" +COUNT="${2:-50}" +BUNDLE="com.osgkeyboard.ios" +SAFARI="com.apple.mobilesafari" +OUT_DIR="${ROOT}/.tmp/device-pip-stress-$(date +%Y%m%d-%H%M%S)" +mkdir -p "$OUT_DIR" +REPORT="$OUT_DIR/report.jsonl" +SUMMARY="$OUT_DIR/summary.txt" +APP="$ROOT/.derivedData-device-stress/Build/Products/Debug-iphoneos/OSGKeyboard.app" + +echo "==> Out: $OUT_DIR" +echo "==> Device: $UDID count/suite: $COUNT" + +if [[ ! -d "$APP" ]]; then + echo "error: missing $APP — build for device first" >&2 + exit 1 +fi + +echo "==> Ensuring install" +xcrun devicectl device install app --device "$UDID" "$APP" --timeout 180 >/dev/null + +# Capture one console launch (timeout seconds). Prints raw console to stdout file. +console_launch() { + local outfile="$1" + local secs="${2:-14}" + local extra_flags="${3:-}" # e.g. --terminate-existing + # shellcheck disable=SC2086 + timeout "$secs" xcrun devicectl device process launch \ + --device "$UDID" \ + --console \ + $extra_flags \ + "$BUNDLE" >"$outfile" 2>&1 || true +} + +classify_file() { + local f="$1" + local host="unknown" pip="unknown" mic="unknown" + + if grep -Eq "OSGKeyboardApp\.init|MainAppRoot\.onAppear|activateOnForeground" "$f"; then + host="success" + elif grep -Eq "Launched application with com\.osgkeyboard\.ios" "$f"; then + host="launch_only" + fi + + if grep -Eq "onboarding incomplete" "$f"; then + pip="onboarding" + elif grep -Eq "aborted reason=permissions|blocked.*permissions" "$f"; then + pip="permissions" + elif grep -Eq "Flow session started \(PiP keep-alive\)|low-profile PiP active|startSessionAsync\.ready" "$f"; then + pip="success" + elif grep -Eq "failure=unsupported|failed to start: unsupported" "$f"; then + pip="unsupported" + elif grep -Eq "PiP keep-alive failed to start|startSessionAsync\.failed.*pipUnavailable|startAndWait failed" "$f"; then + pip="fail" + elif grep -Eq "PiP start attempt failed" "$f"; then + # Retry path — only fail if we never saw success above + pip="retry_then_unknown" + elif grep -Eq "activateOnForeground|autoPiP|startSessionAsync\.begin" "$f"; then + pip="seen_no_result" + fi + + # Mic keep-alive contract: idle releases mic after PiP proves + if grep -Eq "mic released between utterances|released audio session and frame pump" "$f"; then + mic="released_ok" + elif grep -Eq "PiP audio session ready" "$f"; then + mic="armed" + else + mic="unknown" + fi + + # Promote retry_then_unknown if success markers appeared (grep order already handled) + if [[ "$pip" == "retry_then_unknown" ]]; then + if grep -Eq "low-profile PiP active|Flow session started \(PiP keep-alive\)" "$f"; then + pip="success" + else + pip="fail" + fi + fi + + echo "$host|$pip|$mic" +} + +run_suite() { + local suite="$1" + local i outfile result host pip mic start_ts elapsed flags + echo "==> Suite: $suite × $COUNT" + for i in $(seq 1 "$COUNT"); do + outfile="$OUT_DIR/${suite}-$i.console.log" + start_ts=$(date +%s) + flags="" + + case "$suite" in + cold) + flags="--terminate-existing" + console_launch "$outfile" 14 "$flags" + ;; + bgfg) + # Ensure app running with PiP first + console_launch "$OUT_DIR/${suite}-$i.prep.log" 12 "--terminate-existing" >/dev/null || true + # Background by opening Safari + xcrun devicectl device process launch --device "$UDID" "$SAFARI" >/dev/null 2>&1 || true + sleep 3 + # Resume OSG without terminate + console_launch "$outfile" 12 "" + ;; + hold) + console_launch "$OUT_DIR/${suite}-$i.prep.log" 12 "--terminate-existing" >/dev/null || true + # Leave session alive ~20s (PiP should keep host) + sleep 20 + # Background briefly then resume + xcrun devicectl device process launch --device "$UDID" "$SAFARI" >/dev/null 2>&1 || true + sleep 2 + console_launch "$outfile" 12 "" + ;; + esac + + elapsed=$(( $(date +%s) - start_ts )) + result="$(classify_file "$outfile")" + host="${result%%|*}" + rest="${result#*|}" + pip="${rest%%|*}" + mic="${rest##*|}" + + printf '{"suite":"%s","i":%d,"host":"%s","pip":"%s","mic":"%s","elapsed_s":%d}\n' \ + "$suite" "$i" "$host" "$pip" "$mic" "$elapsed" >>"$REPORT" + printf "[%s %3d/%d] host=%-11s pip=%-12s mic=%-12s %2ds\n" \ + "$suite" "$i" "$COUNT" "$host" "$pip" "$mic" "$elapsed" + done +} + +: >"$REPORT" +run_suite cold +run_suite bgfg +run_suite hold + +python3 - "$REPORT" "$SUMMARY" "$UDID" "$COUNT" "$OUT_DIR" <<'PY' +import json, collections, sys +from pathlib import Path +report, summary, udid, count, out = sys.argv[1:6] +rows = [json.loads(l) for l in Path(report).read_text().splitlines() if l.strip()] +by = collections.defaultdict(list) +for r in rows: + by[r["suite"]].append(r) + +lines = [ + "Device PiP / mic keep-alive stress summary", + f"device=Rocky 15 PM udid={udid} count_per_suite={count}", + f"total_rows={len(rows)}", + "", +] +for suite in ("cold", "bgfg", "hold"): + rs = by.get(suite, []) + n = max(len(rs), 1) + host_ok = sum(1 for r in rs if r["host"] in ("success", "launch_only")) + pip_c = collections.Counter(r["pip"] for r in rs) + mic_c = collections.Counter(r["mic"] for r in rs) + pip_ok = pip_c.get("success", 0) + lines += [ + f"[{suite}]", + f" host_ok={host_ok}/{len(rs)} ({host_ok/n*100:.1f}%)", + f" pip_success={pip_ok}/{len(rs)} ({pip_ok/n*100:.1f}%) breakdown={dict(pip_c)}", + f" mic={dict(mic_c)}", + "", + ] +lines += [ + f"Artifacts: {out}", + "Notes:", + " - cold uses --terminate-existing (force-quit recovery).", + " - bgfg backgrounds via Safari then resumes.", + " - hold keeps session ~20s then Safari background + resume.", + " - Keyboard-extension mic tap is not automated.", +] +text = "\n".join(lines) + "\n" +Path(summary).write_text(text) +print(text) +PY + +echo "==> Done" diff --git a/docs/assets/app-preview/zh/OSGKeyboard-preview-6.7-zh.mp4 b/docs/assets/app-preview/zh/OSGKeyboard-preview-6.7-zh.mp4 new file mode 100644 index 0000000..1af80ab Binary files /dev/null and b/docs/assets/app-preview/zh/OSGKeyboard-preview-6.7-zh.mp4 differ diff --git a/docs/osgkeyboardversion.html b/docs/osgkeyboardversion.html new file mode 100644 index 0000000..0b5a523 --- /dev/null +++ b/docs/osgkeyboardversion.html @@ -0,0 +1,457 @@ + + + + + + OSGKeyboard — What's New / 更新说明 + + + + + +
+
+

更新说明

+

本页展示当前版本及更早版本的更新内容

+
+
+

What's New

+

Notes for this version and earlier releases

+
+
+ +

暂无与当前版本匹配的更新说明。

+

No release notes match this app version.

+ + +
+
1.6.5
+ +
+
+

新功能

+ +
+

全新保活流程

+
    +
  • 键盘保活流程全新升级，大幅提升后台保活逻辑，大幅降低耗电量并提升稳定性
    • +
+
+ +
+

新增键盘输入功能

+
    +
  • 新增拼音与英文键盘：支持全拼、微软/搜狗双拼、词组候选、可选模糊音；拼音与英文均支持自定义词库
    • +
  • 键盘布局整体升级，语音 / 中文 / 英文外观统一；左侧 Logo 可跳转主 App，右侧改为输入方式切换与翻译入口
    • +
  • 新增中文候选词面板，候选池更大，长词和更多备选更容易找到；多字输入时可分词选择
    • +
  • 新增英文联想补全、自动纠错与候选撤销，输入更快更准
    • +
  • 新增键盘防误触和误输入纠正，整体体验更接近系统键盘
    • +
  • 新增按键音效与震动，以及轻 / 强震动设置
    • +
  • 新增长按删除加速，改长文更高效
    • +
  • 新增英文键盘智能大写：连续大写、句首自动大写
    • +
  • 新增键盘偏好设置，支持记住上次使用的语音或文字界面，再次打开无需反复切换
    • +
+
+ +
+

新增剪贴板润色模式

+
    +
  • 新增剪贴板处理模式：复制文本后长按麦克风进入文本处理模式
    • +
  • 在剪贴板处理模式下，可通过语音指令对剪贴板内容进行翻译、精简、回复等多种操作
    • +
+
+ +
+

润色模式整体升级

+
    +
  • 大幅升级趣味润色（含直男癌等），风格更鲜明，同时减少事实失真
    • +
  • 润色强度简化为轻度 / 重度：轻度优先保真，重度释放五种趣味风格的完整创意
    • +
  • 新增同音 / 近音纠错，语音识别后的错字修复更准
    • +
  • 长文本润色与翻译分段更清晰，列表结构更易读，翻译后保留段落格式
    • +
+
+ +
+

设置优化

+
    +
  • 设置页层级与卡片布局更清晰，转写与润色入口更好找
    • +
  • 简化首次设置：未完成语音配置时仍可正常打字
    • +
+
+
+ +
+

稳定性与可靠性

+
    +
  • 重构蓝牙与音频路由处理：切换蓝牙设备（耳机）后，录音恢复更可靠
    • +
  • 修复录音启动 / 取消竞态，减少识别完成后卡在润色阶段的问题
    • +
  • Apple 本地识别改用降噪接口：同房间旁人说话时，更倾向听清你的近讲声音
    • +
  • 为识别与润色增加明确超时与原文兜底，网络不稳时不再无限等待
    • +
  • 加强主 App 与键盘的结果确认与恢复，降低结果丢失或无法上屏的概率
    • +
  • 修复火山 ASR 连接问题；修复云端 ASR 配置验证误报
    • +
  • 重构键盘启动流程、降低内存峰值，减少闪退与被系统终止
    • +
  • 输入引擎按需加载，语音与文字切换更快、更省内存
    • +
  • 未使用麦克风时，不主动拉起主 App 占用内存
    • +
  • 移除冗余多步引导；优化会话未完成时的体验
    • +
  • 自定义词 / 本地识别相关能力打包更完整
    • +
+
+
+ +
+
+

New Features

+ +
+

Redesigned keep-alive flow

+
    +
  • Keyboard keep-alive is rebuilt for more reliable background continuity, lower power use, and better stability
    • +
+
+ +
+

Typing keyboards

+
    +
  • New Pinyin and English keyboards: full Pinyin, Microsoft/Sogou double Pinyin, phrase candidates, optional fuzzy Pinyin; both support custom dictionaries
    • +
  • Unified keyboard chrome across Voice / Chinese / English; left logo opens the main app; right side switches input mode and opens translation
    • +
  • New Chinese candidate panel with a larger pool for long phrases and alternatives; multi-character input supports segmented selection
    • +
  • English autocomplete, autocorrect, and candidate undo for faster, more accurate typing
    • +
  • Anti-mis-tap and mistype correction for a closer-to-system keyboard feel
    • +
  • New key sounds and haptics, with light / strong haptic options
    • +
  • Accelerating long-press delete for faster long-text edits
    • +
  • Smarter English capitalization: caps lock and sentence-start autocapitalization
    • +
  • New Keyboard Preferences: remember the last Voice or Typing surface so you don’t keep switching
    • +
+
+ +
+

Clipboard polish mode

+
    +
  • New clipboard processing mode: after copying text, long-press the mic to process that clipboard snapshot
    • +
  • In clipboard mode, speak instructions to translate, shorten, reply, and more
    • +
+
+ +
+

Polish upgrades

+
    +
  • Stronger fun polish styles (including Straight-Talk Rescue / 直男癌), with clearer voice and less factual drift
    • +
  • Intensity simplified to Light / Heavy: Light prioritizes fidelity; Heavy unlocks the full creativity of the five fun styles
    • +
  • New homophone / near-homophone correction after speech recognition
    • +
  • Clearer segmentation for long polish and translation; lists read better; translations keep paragraph structure
    • +
+
+ +
+

Settings

+
    +
  • Clearer Settings hierarchy and cards; transcription and polish are easier to find
    • +
  • Simpler first-run setup: you can type even before voice setup is finished
    • +
+
+
+ +
+

Stability & Reliability

+
    +
  • Rebuilt Bluetooth / audio-route handling so recording recovers more reliably after switching headsets
    • +
  • Fixed start / cancel races that could leave the UI stuck in polish after recognition finished
    • +
  • Apple on-device recognition uses voice-processing / noise reduction so your near-talk mic is preferred over other speakers in the room
    • +
  • Explicit timeouts and original-text fallback for recognition and polish — no endless waits on flaky networks
    • +
  • Stronger host ↔ keyboard result confirmation and recovery to reduce lost or uninsertable text
    • +
  • Fixed Volcengine ASR connection issues and false cloud ASR config-validation failures
    • +
  • Rebuilt keyboard launch path with lower peak memory to reduce crashes and jetsams
    • +
  • Input engines load on demand — faster Voice ↔ Typing switches and lower memory use
    • +
  • Does not wake the host app just to hold memory when the mic isn’t used
    • +
  • Removed redundant multi-step prompts; smoother experience when a session isn’t finished
    • +
  • More complete packaging for custom words and on-device recognition assets
    • +
+
+
+
+ + +
+
1.1.0
+ +
+
+

新功能

+
    +
  • 新增「润色风格包」：内置轻度清理、清晰结构、正式表达、日常聊天，并支持 iCloud 同步
    • +
  • 新增「趣味润色风格」：直男癌拯救器、装逼指南、大厂黑话、帝吧大神、小红书集美五种润色风格
    • +
  • 新增自定义润色风格功能
    • +
  • 新增「画中画」保活，并作为默认风格，支持在设置中切换回灵动岛；关掉画中画即可结束会话
    • +
  • 云端识别支持真流式（百炼 / 火山 / OpenAI），说话时文字会实时上屏
    • +
  • 新增历史记录可按天删除（只清当天）
    • +
  • 设置里对流式 ASR 供应商显示「流式识别」标签；OpenAI 云端识别默认走流式
    • +
+
+ +
+

体验改进

+
    +
  • 设置页重新设计，结构更清晰
    • +
  • 灵动岛保活默认改为 5 分钟（更省隐私与电量），支持多种时长设置
    • +
  • 更新 iPhone 历史、词库 Tab 图标，与 Mac / iPad 统一
    • +
  • 润色风格卡片与设置摘要更简洁，少装饰、少冗余状态字
    • +
  • 设置、风格等相关页面统一卡片留白与圆角
    • +
  • 更新权限与隐私说明
    • +
+
+ +
+

问题修复

+
    +
  • 完成新手引导后不再卡在最后一页
    • +
  • 删除重装后会重新出现欢迎引导
    • +
  • 松手后尾音更不容易丢字；识别过短时会整句再识别一次兜底
    • +
  • 按下录音时麦克风预热阶段的开头语音更不易被丢掉
    • +
  • 末段识别为空时，不再把前面已经识别好的字抹掉
    • +
  • 修复趣味润色在短句时的失真问题
    • +
+
+
+ +
+
+

New Features

+
    +
  • Polish style packs: Light Cleanup, Clear Structure, Formal, and Casual Chat, with iCloud sync
    • +
  • Fun polish styles: Straight-Talk Rescue, Flex Guide, Big-Tech Buzzwords, Tieba Legend, and Xiaohongshu Vibes
    • +
  • Custom polish styles
    • +
  • Picture-in-Picture keep-alive as the default, with Dynamic Island as an option in Settings; ending PiP ends the session
    • +
  • True streaming cloud ASR (Bailian / Volcengine / OpenAI) with live on-screen text while you speak
    • +
  • Delete history by day (clear only that day)
    • +
  • Settings shows a “Streaming ASR” badge for streaming providers; OpenAI cloud ASR defaults to streaming
    • +
+
+ +
+

Improvements

+
    +
  • Redesigned Settings with a clearer structure
    • +
  • Dynamic Island keep-alive defaults to 5 minutes (more privacy- and battery-friendly), with multiple duration options
    • +
  • Updated iPhone History and Dictionary tab icons to match Mac / iPad
    • +
  • Simpler polish-style cards and Settings summaries — less decoration and fewer status words
    • +
  • Consistent card padding and corner radius across Settings and style pages
    • +
  • Updated permission and privacy copy
    • +
+
+ +
+

Fixes

+
    +
  • Onboarding no longer sticks on the last page after completion
    • +
  • Welcome onboarding shows again after delete-and-reinstall
    • +
  • Less end-of-utterance truncation on release; very short captures retry the whole phrase
    • +
  • Speech at the start of mic warm-up is less likely to be dropped
    • +
  • An empty final segment no longer wipes earlier recognized text
    • +
  • Fixed fun-polish distortion on short sentences
    • +
+
+
+
+ + + + diff --git a/project.yml b/project.yml index 418c7fc..12b9120 100644 --- a/project.yml +++ b/project.yml @@ -52,7 +52,7 @@ settings: STRING_CATALOG_GENERATE_SYMBOLS: YES CLANG_CXX_LANGUAGE_STANDARD: c++17 MARKETING_VERSION: "1.6.5" - CURRENT_PROJECT_VERSION: "53" + CURRENT_PROJECT_VERSION: "55" # 签名配置来自 Signing.local.xcconfig（gitignored，不会被覆盖） # 项目级签名 xcconfig，适用于所有 target