diff --git a/CHANGELOG.md b/CHANGELOG.md index a50b583..62f66aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- **Volcengine ASR API Key auth**: settings can switch to the new-console single `X-Api-Key` mode while keeping legacy APP ID + Access Token as the default; SAUC resource is fixed to Doubao streaming 2.0 (`volc.seedasr.sauc.duration`). / **火山 ASR API Key 鉴权**：设置可切换到新控制台单字段 `X-Api-Key`，默认仍为旧版 APP ID + Access Token；SAUC 资源固定为豆包流式 2.0（`volc.seedasr.sauc.duration`）。 +- **Custom style mood emoji**: custom polish styles can opt in (default off) to allow emotion-matched emoji; the prompt overrides R5 and post-processing keeps them on screen. Paste-only prompts that declare emoji opt-in are detected automatically. / **自定义风格情绪 emoji**：自定义润色风格可单独开启（默认关）按情绪点缀 emoji；提示词覆盖 R5，后处理保留上屏。仅粘贴声明允许 emoji 的 prompt 也会自动识别。 + +### Fixed +- **PiP post-utterance yellow flash**: after a voice turn, the mic no longer briefly shows yellow「正在启动画中画」— hold ready once the session proved live, refresh host ready on ack, and avoid labeling an already-active PiP as `.starting`. / **PiP 句末黄色闪烁**：语音说完后麦克风不再短暂变黄并提示「正在启动画中画」——会话曾就绪后保持绿灯、ack 后立即刷新 host ready，且已激活的 PiP 不再标成 `.starting`。 +- **English Shift after Return**: sentence autocapitalization treats newline as a new-line boundary (system keyboard behavior), so Notes-style multi-line Return arms Shift again. / **回车后英文 Shift**：句首自动大写将换行视为新行边界（对齐系统键盘），备忘录等多行回车后会重新点亮 Shift。 +- **English Shift with stale proxy**: after our own insert/delete, autocap merges a local caret-prefix shadow when `documentContextBeforeInput` lags (e.g. period / Return in Notes). / **滞后前文下的英文 Shift**：自身插删后若 `documentContextBeforeInput` 滞后，自动大写会合并本地光标前文影子（如备忘录中的句号/回车）。 + +## [1.6.2] - 2026-08-06 + +### Changed +- **ASR capture voice processing**: iOS App Flow / preview capture switches from `.measurement` to `.voiceChat`, enables `AVAudioEngine` Voice Processing, and prefers a near-talk built-in mic pattern so competing talkers in the same room are suppressed more strongly (Control Center Voice Isolation remains available once VP is on). / **ASR 采音人声处理**：iOS App 的 Flow / 预览采音从 `.measurement` 改为 `.voiceChat`，打开 `AVAudioEngine` Voice Processing，并优先近讲内置麦指向，以更好压制同房间旁人说话（开启 VP 后控制中心仍可选人声隔离）。 + +### Fixed +- **Voice toolbar haptics**: delete / space / return on the voice keyboard follow Settings → General → Haptics (mic unchanged). / **语音底栏震动**：语音键盘的删除 / 空格 / 回车跟随设置 → 通用 → 震动（麦克风大圆钮不变）。 +- **Haptics after app switch**: re-prepare Taptic when the keyboard appears so feedback does not go cold after switching host apps. / **切 App 后震动消失**：键盘每次出现时重新预热触觉引擎，避免从 A 切到 B 后触感变冷。 +- **Typing switch after cold start**: a sticky App Group `hostHeavy` flag no longer silently blocks 中文/EN when the host died mid-warmup; the flag now expires and is cleared on Flow state reset / host relaunch. / **冷启动后无法切拼音/英文**：宿主中途被杀留下的 `hostHeavy` 不再静默挡住中文/EN；该标志会过期，并在 Flow 状态清理与宿主重启时清除。 +- **Chinese Shift uppercase**: with Shift / Caps Lock / Shift-hold on in Pinyin mode, letter keys insert Latin uppercase directly (system-keyboard style) instead of sending uppercase keycodes to Rime, which rejects them. / **拼音 Shift 大写**：拼音模式下开启 Shift / Caps Lock / 按住 Shift 时，字母键直接插入大写拉丁字母（对齐系统键盘），不再把大写键码送给会拒绝的 Rime。 + ## [1.6.1] - 2026-08-05 ### Added diff --git a/OSGKeyboard/AppIcon.icon/Assets/App Icon Template.svg b/OSGKeyboard/AppIcon.icon/Assets/App Icon Template.svg deleted file mode 100644 index afa301d..0000000 --- a/OSGKeyboard/AppIcon.icon/Assets/App Icon Template.svg +++ /dev/null @@ -1,22 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - diff --git a/OSGKeyboard/AppIcon.icon/Assets/Group 29.svg b/OSGKeyboard/AppIcon.icon/Assets/Group 29.svg new file mode 100644 index 0000000..ee97722 --- /dev/null +++ b/OSGKeyboard/AppIcon.icon/Assets/Group 29.svg @@ -0,0 +1,58 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/OSGKeyboard/AppIcon.icon/icon.json b/OSGKeyboard/AppIcon.icon/icon.json index 7a85758..a30ebb2 100644 --- a/OSGKeyboard/AppIcon.icon/icon.json +++ b/OSGKeyboard/AppIcon.icon/icon.json @@ -8,11 +8,8 @@ "hidden" : false, "layers" : [ { - "fill" : "none", - "hidden" : false, - "image-name" : "App Icon Template.svg", - "name" : "App Icon Template", - "opacity" : 1 + "image-name" : "Group 29.svg", + "name" : "Group 29" } ], "lighting" : "individual", diff --git a/OSGKeyboard/Services/FlowSessionManager.swift b/OSGKeyboard/Services/FlowSessionManager.swift index d0b6bbe..e10e44e 100644 --- a/OSGKeyboard/Services/FlowSessionManager.swift +++ b/OSGKeyboard/Services/FlowSessionManager.swift @@ -787,6 +787,11 @@ final class FlowSessionManager: ObservableObject { reason = .audioEngineNotLive } else if !usesPiPKeepAlive, !hasRecentAudio { reason = .waitingForAudioProof + } else if usesPiPKeepAlive, pipController.isPictureInPictureActive { + // PiP is already up — transient gates (!polling / interruption) + // are not a cold start. Prefer awaitingDelivery-adjacent idle over + // `.starting` so the keyboard never flashes「正在启动画中画」. + reason = .awaitingDelivery } else { reason = .starting } @@ -1331,6 +1336,10 @@ final class FlowSessionManager: ObservableObject { return } FlowSessionBridge.clearResult() + // Ack clears the terminal result; republish ready immediately so the + // keyboard does not linger on awaitingDelivery / starting between the + // 500 ms poll and the next heartbeat. + refreshHostReady() } private func hasUnacknowledgedTerminalResult() -> Bool { diff --git a/OSGKeyboard/Views/ASRSettingsCard.swift b/OSGKeyboard/Views/ASRSettingsCard.swift index 9a5fc1e..4a6c1b5 100644 --- a/OSGKeyboard/Views/ASRSettingsCard.swift +++ b/OSGKeyboard/Views/ASRSettingsCard.swift @@ -58,41 +58,62 @@ struct ASRSettingsCard: View { private var volcengineRows: some View { Group { - SettingsCredentialRow( - title: AppL10n.string("settings.asr.volcengine.appId"), - placeholder: "APP ID", - text: Binding( - get: { volcengineFields.appID }, - set: { updateVolcengine(appID: $0) } - ), - isSecret: true, - isMonospaced: true - ) + Toggle(isOn: volcengineAPIKeyModeBinding) { + VStack(alignment: .leading, spacing: Spacing.xxs) { + Text("settings.asr.volcengine.apiKeyMode.title") + .font(TypeStyle.body) + .foregroundStyle(palette.textPrimary) + Text("settings.asr.volcengine.apiKeyMode.subtitle") + .font(TypeStyle.caption2) + .foregroundStyle(palette.textTertiary) + .fixedSize(horizontal: false, vertical: true) + } + } + .tint(palette.accent) + .settingsListRow() + rowDivider - SettingsCredentialRow( - title: AppL10n.string("settings.asr.volcengine.accessToken"), - placeholder: "Access Token", - text: Binding( - get: { volcengineFields.accessToken }, - set: { updateVolcengine(accessToken: $0) } - ), - isSecret: true, - isMonospaced: true - ) - rowDivider - SettingsCredentialRow( - title: AppL10n.string("settings.asr.volcengine.resourceId"), - placeholder: CloudASRModelCatalog.defaultModel(for: "volcengine"), - text: Binding( - get: { volcengineFields.resourceID }, - set: { updateVolcengine(resourceID: $0) } - ), - isMonospaced: true, - defaultValue: CloudASRModelCatalog.defaultModel(for: "volcengine") - ) + + if volcengineFields.usesAPIKeyAuth { + SettingsCredentialRow( + title: AppL10n.string("settings.asr.volcengine.apiKey"), + placeholder: "API Key", + text: Binding( + get: { volcengineFields.apiKeyCredential }, + set: { updateVolcengine(apiKeyCredential: $0) } + ), + isSecret: true, + isMonospaced: true + ) + } else { + SettingsCredentialRow( + title: AppL10n.string("settings.asr.volcengine.appId"), + placeholder: "APP ID", + text: Binding( + get: { volcengineFields.appID }, + set: { updateVolcengine(appID: $0) } + ), + isSecret: true, + isMonospaced: true + ) + rowDivider + SettingsCredentialRow( + title: AppL10n.string("settings.asr.volcengine.accessToken"), + placeholder: "Access Token", + text: Binding( + get: { volcengineFields.accessToken }, + set: { updateVolcengine(accessToken: $0) } + ), + isSecret: true, + isMonospaced: true + ) + } + rowDivider SettingsProviderRow(title: AppL10n.string("settings.provider.note")) { - Text("settings.asr.volcengine.note") + Text(volcengineFields.usesAPIKeyAuth + ? "settings.asr.volcengine.note.apiKey" + : "settings.asr.volcengine.note.appToken") .font(TypeStyle.caption) .foregroundStyle(palette.textTertiary) .fixedSize(horizontal: false, vertical: true) @@ -105,22 +126,29 @@ struct ASRSettingsCard: View { } private var volcengineFields: VolcengineASRFields { - VolcengineASRFields.parse( - apiKey: config.asrApiKey, - resourceFallback: config.asrModel.isEmpty - ? CloudASRModelCatalog.defaultModel(for: "volcengine") - : config.asrModel + VolcengineASRFields.parse(apiKey: config.asrApiKey) + } + + private var volcengineAPIKeyModeBinding: Binding { + Binding( + get: { volcengineFields.usesAPIKeyAuth }, + set: { updateVolcengine(authMode: $0 ? .apiKey : .appToken) } ) } - private func updateVolcengine(appID: String? = nil, accessToken: String? = nil, resourceID: String? = nil) { + private func updateVolcengine( + authMode: VolcengineASRAuthMode? = nil, + appID: String? = nil, + accessToken: String? = nil, + apiKeyCredential: String? = nil + ) { var fields = volcengineFields + if let authMode { fields.authMode = authMode } if let appID { fields.appID = appID } if let accessToken { fields.accessToken = accessToken } - if let resourceID { - fields.resourceID = resourceID - config.asrModel = resourceID - } + if let apiKeyCredential { fields.apiKeyCredential = apiKeyCredential } + // Keep store model aligned with the locked SAUC 2.0 resource. + config.asrModel = VolcengineASRFields.fixedResourceID config.asrApiKey = fields.encodedAPIKey } diff --git a/OSGKeyboard/Views/PolishStylesView.swift b/OSGKeyboard/Views/PolishStylesView.swift index 5a49bd7..b012315 100644 --- a/OSGKeyboard/Views/PolishStylesView.swift +++ b/OSGKeyboard/Views/PolishStylesView.swift @@ -239,7 +239,8 @@ struct PolishStylesView: View { format: AppL10n.string("polishStyles.copyName"), pack.displayName(language: config.uiLanguage) ), - prompt: pack.prompt + prompt: pack.prompt, + allowsAddedEmoji: pack.allowsAddedEmoji ) showEditor = true } @@ -310,12 +311,14 @@ private struct PolishStyleEditorSheet: View { @Environment(\.themePalette) private var palette @State private var name: String @State private var prompt: String + @State private var allowsAddedEmoji: Bool init(pack: PolishStylePack?, onSave: @escaping (PolishStylePack) -> Void) { self.pack = pack self.onSave = onSave _name = State(initialValue: pack?.name ?? "") _prompt = State(initialValue: pack?.prompt ?? PolishStylePackCatalog.newUserPromptTemplate) + _allowsAddedEmoji = State(initialValue: pack?.allowsAddedEmoji ?? false) } var body: some View { @@ -324,10 +327,23 @@ private struct PolishStyleEditorSheet: View { Section("polishStyles.editor.name") { TextField("polishStyles.editor.namePlaceholder", text: $name) } + Section { + Toggle("polishStyles.editor.allowsAddedEmoji", isOn: $allowsAddedEmoji) + } footer: { + Text("polishStyles.editor.allowsAddedEmoji.hint") + } Section { TextEditor(text: $prompt) .font(.body.monospaced()) .frame(minHeight: 320) + .onChange(of: prompt) { _, newValue in + // Paste-only custom prompts that declare emoji opt-in + // should flip the toggle so post-processing keeps them. + if !allowsAddedEmoji, + PolishStylePack.promptDeclaresAddedEmojiOptIn(newValue) { + allowsAddedEmoji = true + } + } } header: { HStack { Text("polishStyles.editor.prompt") @@ -355,6 +371,8 @@ private struct PolishStyleEditorSheet: View { id: pack?.id ?? "user.\(UUID().uuidString.lowercased())", name: name, prompt: prompt, + allowsAddedEmoji: allowsAddedEmoji + || PolishStylePack.promptDeclaresAddedEmojiOptIn(prompt), kind: .user, createdAt: pack?.createdAt ?? Date() ) diff --git a/OSGKeyboard/en.lproj/Localizable.strings b/OSGKeyboard/en.lproj/Localizable.strings index 8400854..7c46919 100644 --- a/OSGKeyboard/en.lproj/Localizable.strings +++ b/OSGKeyboard/en.lproj/Localizable.strings @@ -137,8 +137,11 @@ "settings.provider.note" = "Note"; "settings.asr.volcengine.appId" = "APP ID"; "settings.asr.volcengine.accessToken" = "Access Token"; -"settings.asr.volcengine.resourceId" = "Resource ID"; -"settings.asr.volcengine.note" = "Secret Key is not required. Resource ID defaults to volc.seedasr.sauc.duration."; +"settings.asr.volcengine.apiKey" = "API Key"; +"settings.asr.volcengine.apiKeyMode.title" = "Use new API Key auth"; +"settings.asr.volcengine.apiKeyMode.subtitle" = "Turn on for the new Volcengine console. Keep off if you already use APP ID + Access Token."; +"settings.asr.volcengine.note.appToken" = "Legacy console: enter APP ID and Access Token. Secret Key is not required. Resource is fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration)."; +"settings.asr.volcengine.note.apiKey" = "New console: enter only the API Key. Resource is fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration)."; "provider.openai" = "OpenAI"; "provider.deepseek" = "DeepSeek"; "provider.qwen" = "Qwen (DashScope)"; @@ -404,6 +407,8 @@ "polishStyles.copyName" = "%@ Copy"; "polishStyles.editor.name" = "Name"; "polishStyles.editor.namePlaceholder" = "Style name"; +"polishStyles.editor.allowsAddedEmoji" = "Allow mood emojis"; +"polishStyles.editor.allowsAddedEmoji.hint" = "When on, polish may add a few emojis that match the draft’s emotion, and keeps them on screen. Off by default."; "polishStyles.editor.prompt" = "Complete prompt"; "polishStyles.editor.hint" = "Describe only the role, tone, style boundaries, and examples. The system appends the personal dictionary, safety contract, and output rules."; "polishStyles.error.title" = "Couldn’t save style"; diff --git a/OSGKeyboard/zh-Hans.lproj/Localizable.strings b/OSGKeyboard/zh-Hans.lproj/Localizable.strings index f7af6bb..66b6c23 100644 --- a/OSGKeyboard/zh-Hans.lproj/Localizable.strings +++ b/OSGKeyboard/zh-Hans.lproj/Localizable.strings @@ -137,8 +137,11 @@ "settings.provider.note" = "提示"; "settings.asr.volcengine.appId" = "APP ID"; "settings.asr.volcengine.accessToken" = "Access Token"; -"settings.asr.volcengine.resourceId" = "Resource ID"; -"settings.asr.volcengine.note" = "Secret Key 当前无需填写。Resource ID 默认使用 volc.seedasr.sauc.duration。"; +"settings.asr.volcengine.apiKey" = "API Key"; +"settings.asr.volcengine.apiKeyMode.title" = "使用新版 API Key 鉴权"; +"settings.asr.volcengine.apiKeyMode.subtitle" = "新控制台请打开此开关；已有 AppID + Token 可保持关闭。"; +"settings.asr.volcengine.note.appToken" = "旧版控制台：填写 APP ID 与 Access Token。Secret Key 无需填写。识别资源固定为豆包流式 2.0（volc.seedasr.sauc.duration）。"; +"settings.asr.volcengine.note.apiKey" = "新版控制台：只需填写 API Key。识别资源固定为豆包流式 2.0（volc.seedasr.sauc.duration）。"; "provider.openai" = "OpenAI"; "provider.deepseek" = "DeepSeek"; "provider.qwen" = "通义千问"; @@ -403,6 +406,8 @@ "polishStyles.copyName" = "%@副本"; "polishStyles.editor.name" = "名称"; "polishStyles.editor.namePlaceholder" = "风格名称"; +"polishStyles.editor.allowsAddedEmoji" = "允许按情绪添加 emoji"; +"polishStyles.editor.allowsAddedEmoji.hint" = "开启后，润色可按原文情绪点缀少量 emoji，并保留上屏。默认关闭。"; "polishStyles.editor.prompt" = "完整提示词"; "polishStyles.editor.hint" = "只需描述角色、语气、风格边界和示例。系统会自动追加个人词典、安全边界和输出契约。"; "polishStyles.error.title" = "无法保存风格"; diff --git a/OSGKeyboardExt/KeyboardViewController.swift b/OSGKeyboardExt/KeyboardViewController.swift index f694d29..802e12a 100644 --- a/OSGKeyboardExt/KeyboardViewController.swift +++ b/OSGKeyboardExt/KeyboardViewController.swift @@ -165,6 +165,9 @@ public final class KeyboardViewController: UIInputViewController { configSync.refreshConfigFromAppGroup() // Settings may have changed while the extension stayed alive. applyPreferredSurfaceOnOpen() + // Re-warm Taptic after host app switches: SwiftUI `onAppear` often + // skips when the extension process is reused, leaving generators cold. + KeyboardHapticFeedback.prepare() if state.surface == .typing { OSGDiag.log("KVC.viewWillAppear enterTypingMode", category: "boot") typingSession.enterTypingMode() diff --git a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift index e0c2a0c..a0f6dfd 100644 --- a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift +++ b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift @@ -41,6 +41,9 @@ final class KeyboardFlowCoordinator { /// stale sample never flashes the mic orange while the session is healthy. private var lastHostReadyAt: TimeInterval = 0 private static let hostReadyGrace: TimeInterval = 4 + /// Once the host has published ready for this session, hold green through + /// brief inter-utterance ready flaps instead of flashing preparingSession. + private var sessionProvenReady = false private var flowSessionMonitorTask: Task? private var isAwaitingFlowResult = false private var activeSessionId: UUID? @@ -168,9 +171,17 @@ final class KeyboardFlowCoordinator { // show red/white instead of a fake orange "starting" state. adoptHostBusyStateIfNeeded(snapshot: readySnapshot) - let hostReady = readySnapshot?.ready == true && FlowSessionBridge.isHostReady() + let hostReadyRaw = readySnapshot?.ready == true && FlowSessionBridge.isHostReady() + let sessionActive = FlowSessionBridge.isSessionActive() let now = Date().timeIntervalSince1970 - if hostReady { lastHostReadyAt = now } + if hostReadyRaw { + lastHostReadyAt = now + sessionProvenReady = true + } + if !sessionActive { + sessionProvenReady = false + lastHostReadyAt = 0 + } // Grace window: the host was ready very recently, so treat a momentary // stale heartbeat read as "still warming" rather than an outright // failure. `isSessionActive` is heartbeat-independent, so it stays true @@ -181,19 +192,30 @@ final class KeyboardFlowCoordinator { // it as preparingSession was the orange-stuck bug after cold start: // host utt.rec=1 → ready=false → keyboard forever "正在启动…". let hostBusy = FlowKeyboardHostWarming.isHostBusy(reason: readySnapshot?.reason) + // Hold green after the session already proved ready — PiP mic release / + // ack lag must not flash yellow「正在启动画中画». + let holdReady = FlowKeyboardHostWarming.shouldHoldReady( + hostReady: hostReadyRaw, + hostBusy: hostBusy, + sessionActive: sessionActive, + sessionProvenReady: sessionProvenReady, + isPendingFlowStart: isPendingFlowStart, + snapshotReason: readySnapshot?.reason + ) + let hostReady = hostReadyRaw || holdReady // PiP sessions publish `reason=.starting` while the small window is // coming up — treat that as warming so the mic stays orange (wait) // instead of jumping into another cold start. let hostWarming = FlowKeyboardHostWarming.isHostWarming( hostReady: hostReady, hostBusy: hostBusy, - sessionActive: FlowSessionBridge.isSessionActive(), + sessionActive: sessionActive, hostReachable: FlowSessionBridge.isHostReachable(), isPendingFlowStart: isPendingFlowStart, withinReadyGrace: withinReadyGrace, snapshotReason: readySnapshot?.reason ) - state.flowSessionActive = FlowSessionBridge.isSessionActive() + state.flowSessionActive = sessionActive state.debugPendingFlowStart = isPendingFlowStart state.debugFlowRecording = isFlowRecording state.debugAwaitingFlowResult = isAwaitingFlowResult @@ -626,6 +648,7 @@ final class KeyboardFlowCoordinator { "utterance=\(result.utteranceId.uuidString.prefix(8)) " + "commandSeq=\(result.commandSeq) warning=\(result.warning == nil ? 0 : 1)" ) + recomputeMicVoiceAvailability() return } if let result = matchingResult(), isTerminalFailure(result) { diff --git a/OSGKeyboardExt/Typing/TypingRootView.swift b/OSGKeyboardExt/Typing/TypingRootView.swift index fd52d33..4b03c9e 100644 --- a/OSGKeyboardExt/Typing/TypingRootView.swift +++ b/OSGKeyboardExt/Typing/TypingRootView.swift @@ -95,6 +95,8 @@ struct TypingRootView: View { if !hasContent { candidatePanelMounted = false } } .onAppear { + // Primary warm-up lives in KVC.viewWillAppear (covers app switch). + // Keep this for first mount / surface flip into typing. KeyboardHapticFeedback.prepare() } } @@ -489,8 +491,12 @@ struct TypingRootView: View { if !output.text.isEmpty { onInsert(output.text) } - // Proxy context is up to date after inserts/deletes — refresh Shift. - typing.syncAutocapitalization() + // Notes / some hosts lag `documentContextBeforeInput` — pass the edit we + // just applied so sentence Shift can arm after Return / "." immediately. + typing.syncAutocapitalization( + accountingForInsert: output.text, + deleteCount: output.deleteCount + ) } private func keyWeight(label: String, index: Int, rowIndex: Int) -> CGFloat { diff --git a/OSGKeyboardExt/Views/KeyboardRootView.swift b/OSGKeyboardExt/Views/KeyboardRootView.swift index 958e6a1..41d3e7e 100644 --- a/OSGKeyboardExt/Views/KeyboardRootView.swift +++ b/OSGKeyboardExt/Views/KeyboardRootView.swift @@ -234,14 +234,22 @@ public struct KeyboardRootView: View { } private func bottomDeleteButton(disabled: Bool) -> some View { - RepeatingDeleteButton(disabled: disabled) { + RepeatingDeleteButton( + disabled: disabled, + hapticIntensity: state.keyboardHapticIntensity + ) { state.deleteBackward() } .frame(height: KeyboardLayoutMetrics.bottomActionRowHeight) } private func bottomSpaceButton(disabled: Bool) -> some View { - RectangularToolbarButton(spaceStyle: true, label: "space", disabled: disabled) { + RectangularToolbarButton( + spaceStyle: true, + label: "space", + disabled: disabled, + hapticIntensity: state.keyboardHapticIntensity + ) { state.insertSpace() } .frame(height: KeyboardLayoutMetrics.bottomActionRowHeight) @@ -253,7 +261,8 @@ public struct KeyboardRootView: View { title: title, label: title, disabled: disabled, - isSend: state.returnKeyRole == .send + isSend: state.returnKeyRole == .send, + hapticIntensity: state.keyboardHapticIntensity ) { state.insertNewline() } diff --git a/OSGKeyboardExt/Views/KeyboardTopControls.swift b/OSGKeyboardExt/Views/KeyboardTopControls.swift index 74db5e9..3a4e715 100644 --- a/OSGKeyboardExt/Views/KeyboardTopControls.swift +++ b/OSGKeyboardExt/Views/KeyboardTopControls.swift @@ -154,7 +154,10 @@ struct KeyboardTopControls: View { return } onInsert(output.text) - typing.syncAutocapitalization() + typing.syncAutocapitalization( + accountingForInsert: output.text, + deleteCount: output.deleteCount + ) } private func accessibilityLabel(for tab: KeyboardInputTab) -> String { diff --git a/OSGKeyboardExt/Views/ToolbarActionButtons.swift b/OSGKeyboardExt/Views/ToolbarActionButtons.swift index 9570b1b..b701a13 100644 --- a/OSGKeyboardExt/Views/ToolbarActionButtons.swift +++ b/OSGKeyboardExt/Views/ToolbarActionButtons.swift @@ -94,7 +94,7 @@ struct RepeatingPressButton: View { var disabled: Bool = false /// Plays the system delete click on each fire (matches stock keyboard). var playsDeleteSound: Bool = true - /// Typing-grid haptic strength; `.off` skips haptics (voice toolbar default). + /// Settings → General → Haptics; `.off` skips impact feedback. var hapticIntensity: KeyboardHapticIntensity = .off let action: () -> Void @ViewBuilder let label: (_ isPressed: Bool) -> Label @@ -166,10 +166,16 @@ struct RepeatingDeleteButton: View { @Environment(\.colorScheme) private var colorScheme let disabled: Bool + /// Mirrors Settings → General → Haptics (same as typing delete). + var hapticIntensity: KeyboardHapticIntensity = .off let action: () -> Void var body: some View { - RepeatingPressButton(disabled: disabled, action: action) { isPressed in + RepeatingPressButton( + disabled: disabled, + hapticIntensity: hapticIntensity, + action: action + ) { isPressed in ToolbarKeySurface( isPressed: isPressed, cornerRadius: ToolbarButtonMetrics.cornerRadius, @@ -227,15 +233,24 @@ struct RectangularToolbarButton: View { let label: String let disabled: Bool let isSend: Bool + /// Settings → General → Haptics; space / return use `.action` role. + var hapticIntensity: KeyboardHapticIntensity = .off let action: () -> Void - init(systemName: String, label: String, disabled: Bool = false, action: @escaping () -> Void) { + init( + systemName: String, + label: String, + disabled: Bool = false, + hapticIntensity: KeyboardHapticIntensity = .off, + action: @escaping () -> Void + ) { self.systemName = systemName self.spaceStyle = false self.title = nil self.label = label self.disabled = disabled self.isSend = false + self.hapticIntensity = hapticIntensity self.action = action } @@ -244,6 +259,7 @@ struct RectangularToolbarButton: View { label: String, disabled: Bool = false, isSend: Bool = false, + hapticIntensity: KeyboardHapticIntensity = .off, action: @escaping () -> Void ) { self.systemName = nil @@ -251,17 +267,25 @@ struct RectangularToolbarButton: View { self.label = label self.disabled = disabled self.isSend = isSend + self.hapticIntensity = hapticIntensity self.action = action self.title = title } - init(spaceStyle: Bool, label: String, disabled: Bool = false, action: @escaping () -> Void) { + init( + spaceStyle: Bool, + label: String, + disabled: Bool = false, + hapticIntensity: KeyboardHapticIntensity = .off, + action: @escaping () -> Void + ) { self.systemName = nil self.spaceStyle = spaceStyle self.title = nil self.label = label self.disabled = disabled self.isSend = false + self.hapticIntensity = hapticIntensity self.action = action } @@ -299,13 +323,14 @@ struct RectangularToolbarButton: View { isSend ? .white : NativeKeyboardKeyColors.text(for: colorScheme) } - // 按下即响、按下即执行，与系统键盘保持一致（Button 默认松手才触发）。 + // 按下即响、即震、即执行，与系统键盘 / 打字面保持一致（Button 默认松手才触发）。 private var pressGesture: some Gesture { DragGesture(minimumDistance: 0) .onChanged { _ in guard !disabled, !isPressing else { return } isPressing = true KeyboardSoundFeedback.keyClick() + KeyboardHapticFeedback.play(role: .action, intensity: hapticIntensity) action() } .onEnded { _ in diff --git a/OSGKeyboardExtTests/EnglishTypingTests.swift b/OSGKeyboardExtTests/EnglishTypingTests.swift index a33f1a5..68a8e91 100644 --- a/OSGKeyboardExtTests/EnglishTypingTests.swift +++ b/OSGKeyboardExtTests/EnglishTypingTests.swift @@ -61,6 +61,15 @@ final class EnglishTypingTests: XCTestCase { XCTAssertTrue( TypingAutocapitalization.shouldCapitalize(precedingText: "Hello!\n", mode: .sentences) ) + // System keyboard (.sentences): Return starts a new line → capitalize. + // Notes-style document context after Return is typically "…\n". + XCTAssertTrue( + TypingAutocapitalization.shouldCapitalize(precedingText: "Hello\n", mode: .sentences), + "Return/newline must arm Shift (Notes multi-line)" + ) + XCTAssertTrue( + TypingAutocapitalization.shouldCapitalize(precedingText: "Hello\n\n", mode: .sentences) + ) XCTAssertFalse( TypingAutocapitalization.shouldCapitalize(precedingText: "Hello ", mode: .sentences) ) @@ -72,6 +81,55 @@ final class EnglishTypingTests: XCTestCase { ) } + @MainActor + func testEnglishShiftArmsAfterReturnLikeNotes() { + // Simulates Notes: proxy preceding text gains a trailing newline after Return. + var preceding = "Hello" + let typing = TypingSessionController() + typing.precedingTextProvider = { preceding } + typing.autocapitalizationModeProvider = { .sentences } + _ = typing.setLanguage(.english) + XCTAssertFalse(typing.shiftActive, "mid-word should not arm Shift") + + _ = typing.handleReturn() + preceding = "Hello\n" // host document after inserting newline + typing.syncAutocapitalization(accountingForInsert: "\n") + XCTAssertTrue( + typing.shiftActive, + "after Return, Shift must light for next line (system sentences behavior)" + ) + XCTAssertEqual(typing.keyRows.first?.first, "Q") + } + + @MainActor + func testEnglishShiftArmsWhenProxyLagsAfterReturn() { + // Notes often still reports pre-Return context right after insertText("\n"). + var preceding = "Hello" + let typing = TypingSessionController() + typing.precedingTextProvider = { preceding } + typing.autocapitalizationModeProvider = { .sentences } + _ = typing.setLanguage(.english) + + _ = typing.handleReturn() + // Proxy intentionally stale — still "Hello" without "\n". + typing.syncAutocapitalization(accountingForInsert: "\n") + XCTAssertTrue(typing.shiftActive) + XCTAssertEqual(typing.keyRows.first?.first, "Q") + } + + @MainActor + func testEnglishShiftArmsWhenProxyLagsAfterPeriod() { + var preceding = "Hello" + let typing = TypingSessionController() + typing.precedingTextProvider = { preceding } + typing.autocapitalizationModeProvider = { .sentences } + _ = typing.setLanguage(.english) + + _ = typing.handleKey(".") + typing.syncAutocapitalization(accountingForInsert: ".") + XCTAssertTrue(typing.shiftActive) + } + @MainActor func testEnglishIdleShowsNoCandidatesUntilLetterTyped() { let typing = TypingSessionController() diff --git a/OSGKeyboardExtTests/KeyboardSurfaceStateTests.swift b/OSGKeyboardExtTests/KeyboardSurfaceStateTests.swift index 4d51f52..52adc9b 100644 --- a/OSGKeyboardExtTests/KeyboardSurfaceStateTests.swift +++ b/OSGKeyboardExtTests/KeyboardSurfaceStateTests.swift @@ -154,4 +154,120 @@ final class KeyboardSurfaceStateTests: XCTestCase { XCTAssertFalse(typing.shiftHeld) XCTAssertFalse(typing.capsLock) } + + func testChineseShiftInsertsUppercaseLatinBypassingRime() { + let engine = TrackingStubRimeEngine() + let typing = TypingSessionController(engine: { engine }) + + _ = typing.handleKey("⇧") + XCTAssertTrue(typing.shiftActive) + + let output = typing.handleKey("N") + XCTAssertEqual(output, .insert("N")) + XCTAssertEqual(engine.processCharacterCallCount, 0) + XCTAssertFalse(typing.shiftActive, "one-shot Shift clears after Latin insert") + XCTAssertTrue(typing.composition.preedit.isEmpty) + } + + func testChineseShiftPreservesExistingComposition() { + let engine = TrackingStubRimeEngine() + let typing = TypingSessionController(engine: { engine }) + // Seed session composition via a lowercase letter (goes to Rime). + _ = typing.handleKey("n") + XCTAssertEqual(typing.composition.preedit, "n") + + _ = typing.handleKey("⇧") + let output = typing.handleKey("A") + XCTAssertEqual(output, .insert("A")) + XCTAssertEqual(engine.processCharacterCallCount, 1, "only the unshifted letter hits Rime") + XCTAssertEqual(typing.composition.preedit, "n", "Shift Latin must not clear preedit") + } + + func testChineseCapsLockKeepsInsertingUppercaseLatin() { + let engine = TrackingStubRimeEngine() + let typing = TypingSessionController(engine: { engine }) + + _ = typing.handleKey("⇧") + _ = typing.handleKey("⇧") // second tap → Caps Lock + XCTAssertTrue(typing.capsLock) + + XCTAssertEqual(typing.handleKey("O"), .insert("O")) + XCTAssertEqual(typing.handleKey("S"), .insert("S")) + XCTAssertEqual(engine.processCharacterCallCount, 0) + XCTAssertTrue(typing.capsLock) + } + + func testChineseUnshiftedLetterStillComposes() { + let engine = TrackingStubRimeEngine() + let typing = TypingSessionController(engine: { engine }) + + let output = typing.handleKey("n") + XCTAssertEqual(output, .none) + XCTAssertEqual(engine.processCharacterCallCount, 1) + XCTAssertEqual(engine.lastProcessedCharacter, "n") + XCTAssertEqual(typing.composition.preedit, "n") + } +} + +/// Stub that records `processCharacter` calls for Chinese Shift bypass tests. +@MainActor +private final class TrackingStubRimeEngine: RimeEngineBridging { + var composition: TypingComposition = .empty + var isReady: Bool = true + var schema: TypingInputSchema = .fullPinyin + private(set) var processCharacterCallCount = 0 + private(set) var lastProcessedCharacter: Character? + + func prepare() async throws {} + func teardown() { composition = .empty } + + func setLanguage(_ language: TypingInputLanguage) { + if language == .english { composition = .empty } + } + + @discardableResult + func setSchema(_ schema: TypingInputSchema) -> Bool { + self.schema = schema + return true + } + + func processCharacter(_ character: Character) -> String? { + processCharacterCallCount += 1 + lastProcessedCharacter = character + composition = TypingComposition( + preedit: String(character).lowercased(), + candidates: [TypingCandidate(text: "你")] + ) + return nil + } + + func processBackspace() -> String? { + composition = .empty + return nil + } + + func processSpace() -> String? { + composition = .empty + return " " + } + + func processReturn() -> String? { + composition = .empty + return "\n" + } + + func selectCandidate(at index: Int) -> String { + composition = .empty + return "" + } + + func flushPreedit() -> String { + let raw = composition.preedit + composition = .empty + return raw + } + + func clearComposition() { + composition = .empty + } } diff --git a/OSGKeyboardHostSupport/Services/CloudASR/VolcengineCloudASRClient.swift b/OSGKeyboardHostSupport/Services/CloudASR/VolcengineCloudASRClient.swift index e87e7c0..4c13243 100644 --- a/OSGKeyboardHostSupport/Services/CloudASR/VolcengineCloudASRClient.swift +++ b/OSGKeyboardHostSupport/Services/CloudASR/VolcengineCloudASRClient.swift @@ -29,19 +29,15 @@ struct VolcengineCloudASRClient: CloudASRTranscribing, CloudASRStreamingCapable onPartial: @escaping @Sendable (String) -> Void ) async throws -> any CloudASRStreamingSession { _ = locale - let credentials = try VolcengineCredentials.parse( - apiKey: apiKey, - fallbackResourceID: resolvedResourceID - ) + _ = resourceID // Product is locked to Doubao streaming 2.0 duration. + let credentials = VolcengineASRFields.parse(apiKey: apiKey) + guard credentials.hasUsableCredentials else { throw CloudASRError.noAPIKey } let url = try resolvedEndpointURL() let connectID = UUID().uuidString var request = URLRequest(url: url) request.timeoutInterval = 8 - request.setValue(credentials.appID, forHTTPHeaderField: "X-Api-App-Key") - request.setValue(credentials.accessToken, forHTTPHeaderField: "X-Api-Access-Key") - request.setValue(credentials.resourceID, forHTTPHeaderField: "X-Api-Resource-Id") - request.setValue(connectID, forHTTPHeaderField: "X-Api-Connect-Id") + credentials.applyWebSocketAuthHeaders(to: &request, connectID: connectID) let task = session.webSocketTask(with: request) task.resume() @@ -90,12 +86,6 @@ struct VolcengineCloudASRClient: CloudASRTranscribing, CloudASRStreamingCapable live.cancel() } - private var resolvedResourceID: String { - resourceID.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty - ? CloudASRModelCatalog.volcengineDefaultResourceID - : resourceID.trimmingCharacters(in: .whitespacesAndNewlines) - } - private func resolvedEndpointURL() throws -> URL { let raw = endpoint.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty ? CloudASRModelCatalog.volcengineEndpoint @@ -424,46 +414,6 @@ private final class VolcengineStreamingSession: CloudASRStreamingSession, @unche } } -private struct VolcengineCredentials { - let appID: String - let accessToken: String - let resourceID: String - - static func parse(apiKey: String, fallbackResourceID: String) throws -> VolcengineCredentials { - let trimmed = apiKey.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { throw CloudASRError.noAPIKey } - - if let data = trimmed.data(using: .utf8), - let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] { - let appID = string(json, keys: ["app_id", "appId", "appid"]) - let token = string(json, keys: ["access_token", "accessToken", "token"]) - let resourceID = string(json, keys: ["resource_id", "resourceId", "resource"]) - ?? fallbackResourceID - guard let appID, let token, !resourceID.isEmpty else { throw CloudASRError.noAPIKey } - return VolcengineCredentials(appID: appID, accessToken: token, resourceID: resourceID) - } - - let separators = CharacterSet(charactersIn: ":\n,") - let parts = trimmed - .components(separatedBy: separators) - .map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } - .filter { !$0.isEmpty } - guard parts.count >= 2 else { throw CloudASRError.noAPIKey } - let resourceID = parts.count >= 3 ? parts[2] : fallbackResourceID - return VolcengineCredentials(appID: parts[0], accessToken: parts[1], resourceID: resourceID) - } - - private static func string(_ json: [String: Any], keys: [String]) -> String? { - for key in keys { - if let value = json[key] as? String { - let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines) - if !trimmed.isEmpty { return trimmed } - } - } - return nil - } -} - enum VolcengineMessageType: UInt8 { case fullClientRequest = 0b0001 case audioOnlyRequest = 0b0010 diff --git a/OSGKeyboardHostSupport/Services/FlowAudioSessionCoordinator.swift b/OSGKeyboardHostSupport/Services/FlowAudioSessionCoordinator.swift index 7f15fa9..3982633 100644 --- a/OSGKeyboardHostSupport/Services/FlowAudioSessionCoordinator.swift +++ b/OSGKeyboardHostSupport/Services/FlowAudioSessionCoordinator.swift @@ -68,10 +68,13 @@ public final class FlowAudioSessionCoordinator: @unchecked Sendable { public func activateCapture() async throws -> FlowAudioSessionSnapshot { let activation: CaptureActivation = try await perform { if self.mode != .capture { + // `.voiceChat` turns on system speech DSP (noise suppression / + // AGC). `.measurement` feeds near-raw PCM and is weak against + // a competing talker in the same room. try self.session.setCategory( .playAndRecord, - mode: .measurement, - options: [.defaultToSpeaker, .allowBluetoothHFP, .mixWithOthers] + mode: FlowCaptureVoiceProcessing.captureMode, + options: FlowCaptureVoiceProcessing.captureOptions ) } if !self.active { @@ -95,6 +98,9 @@ public final class FlowAudioSessionCoordinator: @unchecked Sendable { category: "flow" ) } + } else { + // Built-in near-talk preference (cardioid when available). + FlowCaptureVoiceProcessing.preferNearTalkBuiltInMic(on: self.session) } self.mode = .capture return CaptureActivation( diff --git a/OSGKeyboardHostSupport/Services/FlowCaptureVoiceProcessing.swift b/OSGKeyboardHostSupport/Services/FlowCaptureVoiceProcessing.swift new file mode 100644 index 0000000..599aee8 --- /dev/null +++ b/OSGKeyboardHostSupport/Services/FlowCaptureVoiceProcessing.swift @@ -0,0 +1,108 @@ +// FlowCaptureVoiceProcessing.swift +// OSGKeyboard · Host Support +// +// Capture-side speech front-end for local ASR: Apple Voice Processing +// (noise suppression / AGC / AEC) plus near-talk built-in mic preference. +// `.measurement` delivers near-raw PCM and is a poor fit for competing +// talkers in the same room; `.voiceChat` + VP is the system path that +// also unlocks Control Center Mic Modes (incl. Voice Isolation). + +import AVFoundation +import Foundation +import OSGKeyboardShared + +public enum FlowCaptureVoiceProcessing { + + /// Speech-oriented session mode. Prefer over `.measurement` for ASR. + public static let captureMode: AVAudioSession.Mode = .voiceChat + + public static let captureOptions: AVAudioSession.CategoryOptions = [ + .defaultToSpeaker, .allowBluetoothHFP, .mixWithOthers + ] + + /// Enable Apple Voice Processing on a stopped engine. Format may change + /// afterward — callers must re-read `inputNode` formats before `installTap`. + @discardableResult + public static func enableVoiceProcessing(on engine: AVAudioEngine) -> Bool { + let input = engine.inputNode + if input.isVoiceProcessingEnabled { return true } + do { + try input.setVoiceProcessingEnabled(true) + OSGDiag.log("voiceProcessing enabled", category: "flow") + logActiveMicrophoneMode() + return true + } catch { + OSGDiag.log( + "voiceProcessing enable failed: \(error.localizedDescription)", + category: "flow" + ) + return false + } + } + + /// Prefer a near-field built-in data source (front/lower + cardioid when + /// available). No-op while Bluetooth HFP is preferred or active — keep + /// the existing headset route. + public static func preferNearTalkBuiltInMic(on session: AVAudioSession) { + if session.currentRoute.inputs.first?.portType == .bluetoothHFP { return } + if session.preferredInput?.portType == .bluetoothHFP { return } + + guard let builtIn = session.availableInputs?.first(where: { + $0.portType == .builtInMic + }) else { return } + + let sources = builtIn.dataSources ?? [] + let preferred = + sources.first(where: { $0.orientation == .front }) + ?? sources.first(where: { $0.location == .lower }) + ?? sources.first + + if let preferred { + if preferred.supportedPolarPatterns?.contains(.cardioid) == true { + do { + try preferred.setPreferredPolarPattern(.cardioid) + } catch { + OSGDiag.log( + "cardioid polar pattern failed: \(error.localizedDescription)", + category: "flow" + ) + } + } + do { + try builtIn.setPreferredDataSource(preferred) + } catch { + OSGDiag.log( + "preferred data source failed: \(error.localizedDescription)", + category: "flow" + ) + } + } + + do { + try session.setPreferredInput(builtIn) + } catch { + OSGDiag.log( + "preferred built-in mic failed: \(error.localizedDescription)", + category: "flow" + ) + } + } + + public static func logActiveMicrophoneMode() { + let preferred = micModeLabel(AVCaptureDevice.preferredMicrophoneMode) + let active = micModeLabel(AVCaptureDevice.activeMicrophoneMode) + OSGDiag.log( + "micMode preferred=\(preferred) active=\(active)", + category: "flow" + ) + } + + private static func micModeLabel(_ mode: AVCaptureDevice.MicrophoneMode) -> String { + switch mode { + case .standard: return "standard" + case .wideSpectrum: return "wideSpectrum" + case .voiceIsolation: return "voiceIsolation" + @unknown default: return "unknown(\(mode.rawValue))" + } + } +} diff --git a/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift b/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift index e58d154..38540a6 100644 --- a/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift +++ b/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift @@ -604,13 +604,25 @@ public final class FlowContinuousCapture { FlowAudioEngineHandle(audioEngine) ) + // Voice Processing must be enabled while the engine is stopped, and + // it can change the RemoteIO format — enable first, then sync rates. var candidateEngine = AVAudioEngine() + var voiceProcessingOn = FlowCaptureVoiceProcessing.enableVoiceProcessing( + on: candidateEngine + ) var inputNode = candidateEngine.inputNode var hardwareFormat = inputNode.inputFormat(forBus: 0) var outputFormat = inputNode.outputFormat(forBus: 0) - for _ in 0..<3 where abs(hardwareFormat.sampleRate - sessionSnapshot.sampleRate) >= 1 { + var resolvedSession = sessionSnapshot + for _ in 0..<3 where abs(hardwareFormat.sampleRate - resolvedSession.sampleRate) >= 1 { try? await Task.sleep(nanoseconds: 50_000_000) + // Re-enter capture so the session rate tracks VP / route shifts. + resolvedSession = (try? await FlowAudioSessionCoordinator.shared.activateCapture()) + ?? resolvedSession candidateEngine = AVAudioEngine() + voiceProcessingOn = FlowCaptureVoiceProcessing.enableVoiceProcessing( + on: candidateEngine + ) inputNode = candidateEngine.inputNode hardwareFormat = inputNode.inputFormat(forBus: 0) outputFormat = inputNode.outputFormat(forBus: 0) @@ -619,8 +631,9 @@ public final class FlowContinuousCapture { "audioSession.active", "hwRate=\(Int(hardwareFormat.sampleRate)) hwChannels=\(hardwareFormat.channelCount) " + "outputRate=\(Int(outputFormat.sampleRate)) " - + "sessionRate=\(Int(sessionSnapshot.sampleRate)) " - + "route=\(sessionSnapshot.inputPortType)" + + "sessionRate=\(Int(resolvedSession.sampleRate)) " + + "route=\(resolvedSession.inputPortType) " + + "voiceProcessing=\(voiceProcessingOn ? 1 : 0)" ) guard hardwareFormat.sampleRate > 0, hardwareFormat.channelCount > 0 else { FlowTrace.warn( @@ -632,8 +645,8 @@ public final class FlowContinuousCapture { channels: Int(hardwareFormat.channelCount) ) } - guard sessionSnapshot.sampleRate <= 0 - || abs(hardwareFormat.sampleRate - sessionSnapshot.sampleRate) < 1 else { + guard resolvedSession.sampleRate <= 0 + || abs(hardwareFormat.sampleRate - resolvedSession.sampleRate) < 1 else { throw StartError.invalidHardwareFormat( sampleRate: hardwareFormat.sampleRate, channels: Int(hardwareFormat.channelCount) @@ -703,7 +716,7 @@ public final class FlowContinuousCapture { throw StartError.engineStartFailed(error.localizedDescription) } audioEngine = candidateEngine - activeRouteSnapshot = sessionSnapshot + activeRouteSnapshot = resolvedSession engineActivationCount += 1 lastActivationAt = Date() FlowTrace.capture("engine.started", "running=\(audioEngine.isRunning ? 1 : 0)") diff --git a/OSGKeyboardHostSupport/Services/LiveDictationController.swift b/OSGKeyboardHostSupport/Services/LiveDictationController.swift index f14bc83..5c6e144 100644 --- a/OSGKeyboardHostSupport/Services/LiveDictationController.swift +++ b/OSGKeyboardHostSupport/Services/LiveDictationController.swift @@ -177,30 +177,27 @@ public final class LiveDictationController: ObservableObject { // 3. Audio session — only configure once per process. // - // Category is `.record` (not `.playAndRecord`) because the - // preview never plays back audio — it just records from the - // mic and hands the buffers to `SpeechAnalyzer`. On the - // iOS Simulator, `.playAndRecord` requires the - // `AURemoteIO` Audio Unit's *output* side to also be - // enabled, but the simulator's "speaker" reports a 0 Hz - // hardware format, so `AURemoteIO::enable` fails with - // `kAudioUnitErr_FormatNotSupported` (-10851) and any - // subsequent `installTap` traps with "Failed to create tap - // due to format mismatch". `.record` skips the output - // side entirely, so the simulator can record. - // - // The real keyboard extension (`OSGKeyboardExt`) keeps - // `.playAndRecord` because it runs on a real device where - // the output side has a real hardware format, and may want - // to play click sounds / haptic feedback. Only the preview - // needs the simulator-friendly category. + // On device we use `.playAndRecord` + `.voiceChat` so Apple Voice + // Processing can suppress competing talkers. The iOS Simulator's + // speaker reports a 0 Hz output format, so `.playAndRecord` fails + // with `kAudioUnitErr_FormatNotSupported` (-10851); keep `.record` + // there so preview still works under CoreSimulator. if !didConfigureAudioSession { do { let session = AVAudioSession.sharedInstance() - try session.setCategory(.record, - mode: .measurement, - options: []) + #if targetEnvironment(simulator) + try session.setCategory(.record, mode: .default, options: []) + #else + try session.setCategory( + .playAndRecord, + mode: FlowCaptureVoiceProcessing.captureMode, + options: FlowCaptureVoiceProcessing.captureOptions + ) + #endif try session.setActive(true, options: .notifyOthersOnDeactivation) + #if !targetEnvironment(simulator) + FlowCaptureVoiceProcessing.preferNearTalkBuiltInMic(on: session) + #endif didConfigureAudioSession = true } catch { debug("audio session failed: \(error.localizedDescription)") @@ -214,6 +211,8 @@ public final class LiveDictationController: ObservableObject { // The route may have changed while the preview was closed. A fresh // engine created after session activation avoids a stale input node. audioEngine = AVAudioEngine() + // Enable VP before reading hardware format — RemoteIO rate can shift. + _ = FlowCaptureVoiceProcessing.enableVoiceProcessing(on: audioEngine) // 4. Spin up the engine + ASR. phase = .recording diff --git a/OSGKeyboardMac/MacPolishStylesView.swift b/OSGKeyboardMac/MacPolishStylesView.swift index 891d919..64bbba0 100644 --- a/OSGKeyboardMac/MacPolishStylesView.swift +++ b/OSGKeyboardMac/MacPolishStylesView.swift @@ -146,7 +146,8 @@ struct MacPolishStylesView: View { duplicate: { editingPack = PolishStylePack( name: "\(pack.displayName(language: lang)) \(MacL10n.string("mac.styles.copy", language: lang))", - prompt: pack.prompt + prompt: pack.prompt, + allowsAddedEmoji: pack.allowsAddedEmoji ) showEditor = true }, @@ -347,6 +348,7 @@ private struct MacPolishStyleEditor: View { @Environment(\.themePalette) private var palette @State private var name: String @State private var prompt: String + @State private var allowsAddedEmoji: Bool init( pack: PolishStylePack?, @@ -358,6 +360,7 @@ private struct MacPolishStyleEditor: View { self.onSave = onSave _name = State(initialValue: pack?.name ?? "") _prompt = State(initialValue: pack?.prompt ?? PolishStylePackCatalog.newUserPromptTemplate) + _allowsAddedEmoji = State(initialValue: pack?.allowsAddedEmoji ?? false) } var body: some View { @@ -366,6 +369,13 @@ private struct MacPolishStyleEditor: View { .font(TypeStyle.title2) TextField(MacL10n.string("mac.styles.name", language: language), text: $name) .textFieldStyle(.roundedBorder) + Toggle( + MacL10n.string("mac.styles.allowsAddedEmoji", language: language), + isOn: $allowsAddedEmoji + ) + Text(MacL10n.string("mac.styles.allowsAddedEmoji.hint", language: language)) + .font(TypeStyle.caption2) + .foregroundStyle(palette.textTertiary) HStack { Text(MacL10n.string("mac.styles.prompt", language: language)) .font(MacSettingsType.sectionTitle) @@ -380,13 +390,19 @@ private struct MacPolishStyleEditor: View { } TextEditor(text: $prompt) .font(.body.monospaced()) - .frame(minHeight: 360) + .frame(minHeight: 320) .padding(4) .background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.medium)) .overlay( RoundedRectangle(cornerRadius: Radius.medium) .stroke(palette.divider, lineWidth: 1) ) + .onChange(of: prompt) { _, newValue in + if !allowsAddedEmoji, + PolishStylePack.promptDeclaresAddedEmojiOptIn(newValue) { + allowsAddedEmoji = true + } + } Text(MacL10n.string("mac.styles.hint", language: language)) .font(TypeStyle.caption2) .foregroundStyle(palette.textTertiary) @@ -399,6 +415,8 @@ private struct MacPolishStyleEditor: View { id: pack?.id ?? "user.\(UUID().uuidString.lowercased())", name: name, prompt: prompt, + allowsAddedEmoji: allowsAddedEmoji + || PolishStylePack.promptDeclaresAddedEmojiOptIn(prompt), kind: .user, createdAt: pack?.createdAt ?? Date() ) @@ -414,7 +432,7 @@ private struct MacPolishStyleEditor: View { } } .padding(Spacing.xl) - .frame(width: 680, height: 590) + .frame(width: 680, height: 640) .background(palette.background) } } diff --git a/OSGKeyboardMac/MacSettingsView.swift b/OSGKeyboardMac/MacSettingsView.swift index 701293d..3f6678a 100644 --- a/OSGKeyboardMac/MacSettingsView.swift +++ b/OSGKeyboardMac/MacSettingsView.swift @@ -188,34 +188,52 @@ struct MacSettingsView: View { @ViewBuilder private var volcengineAsrRows: some View { - MacCredentialField( - title: MacL10n.string("mac.settings.volcengineAppId", language: lang), - placeholder: "APP ID", - text: Binding( - get: { macVolcengineFields.appID }, - set: { updateMacVolcengine(appID: $0) } - ), - isSecret: true + MacProviderThinkingRow( + title: MacL10n.string("mac.settings.volcengineApiKeyMode", language: lang), + subtitle: MacL10n.string("mac.settings.volcengineApiKeyModeSubtitle", language: lang), + isOn: Binding( + get: { macVolcengineFields.usesAPIKeyAuth }, + set: { updateMacVolcengine(authMode: $0 ? .apiKey : .appToken) } + ) ) - MacCredentialField( - title: MacL10n.string("mac.settings.volcengineAccessToken", language: lang), - placeholder: "Access Token", - text: Binding( - get: { macVolcengineFields.accessToken }, - set: { updateMacVolcengine(accessToken: $0) } - ), - isSecret: true + if macVolcengineFields.usesAPIKeyAuth { + MacCredentialField( + title: MacL10n.string("mac.settings.volcengineApiKey", language: lang), + placeholder: "API Key", + text: Binding( + get: { macVolcengineFields.apiKeyCredential }, + set: { updateMacVolcengine(apiKeyCredential: $0) } + ), + isSecret: true + ) + } else { + MacCredentialField( + title: MacL10n.string("mac.settings.volcengineAppId", language: lang), + placeholder: "APP ID", + text: Binding( + get: { macVolcengineFields.appID }, + set: { updateMacVolcengine(appID: $0) } + ), + isSecret: true + ) + MacCredentialField( + title: MacL10n.string("mac.settings.volcengineAccessToken", language: lang), + placeholder: "Access Token", + text: Binding( + get: { macVolcengineFields.accessToken }, + set: { updateMacVolcengine(accessToken: $0) } + ), + isSecret: true + ) + } + MacProviderNoteRow( + text: MacL10n.string( + macVolcengineFields.usesAPIKeyAuth + ? "mac.settings.volcengineNoteApiKey" + : "mac.settings.volcengineNoteAppToken", + language: lang + ) ) - MacCredentialField( - title: MacL10n.string("mac.settings.volcengineResourceId", language: lang), - placeholder: CloudASRModelCatalog.defaultModel(for: "volcengine"), - text: Binding( - get: { macVolcengineFields.resourceID }, - set: { updateMacVolcengine(resourceID: $0) } - ), - defaultValue: CloudASRModelCatalog.defaultModel(for: "volcengine") - ) - MacProviderNoteRow(text: MacL10n.string("mac.settings.volcengineNote", language: lang)) } @ViewBuilder @@ -407,22 +425,21 @@ struct MacSettingsView: View { } private var macVolcengineFields: VolcengineASRFields { - VolcengineASRFields.parse( - apiKey: viewModel.config.asrApiKey, - resourceFallback: viewModel.config.asrModel.isEmpty - ? CloudASRModelCatalog.defaultModel(for: "volcengine") - : viewModel.config.asrModel - ) + VolcengineASRFields.parse(apiKey: viewModel.config.asrApiKey) } - private func updateMacVolcengine(appID: String? = nil, accessToken: String? = nil, resourceID: String? = nil) { + private func updateMacVolcengine( + authMode: VolcengineASRAuthMode? = nil, + appID: String? = nil, + accessToken: String? = nil, + apiKeyCredential: String? = nil + ) { var fields = macVolcengineFields + if let authMode { fields.authMode = authMode } if let appID { fields.appID = appID } if let accessToken { fields.accessToken = accessToken } - if let resourceID { - fields.resourceID = resourceID - viewModel.config.asrModel = resourceID - } + if let apiKeyCredential { fields.apiKeyCredential = apiKeyCredential } + viewModel.config.asrModel = VolcengineASRFields.fixedResourceID viewModel.config.asrApiKey = fields.encodedAPIKey } diff --git a/OSGKeyboardShared/Models/AppGroupConfiguration.swift b/OSGKeyboardShared/Models/AppGroupConfiguration.swift index c8be66f..6353446 100644 --- a/OSGKeyboardShared/Models/AppGroupConfiguration.swift +++ b/OSGKeyboardShared/Models/AppGroupConfiguration.swift @@ -140,7 +140,13 @@ public struct AppGroupConfiguration: Sendable, Equatable { public var isCloudASRKeyMissing: Bool { guard engineMode == "cloud" else { return false } - return asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + let key = asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines) + guard !key.isEmpty else { return true } + // Volcengine may store auth_mode JSON before credentials are filled. + if asrProviderId == "volcengine" { + return !VolcengineASRFields.parse(apiKey: key).hasUsableCredentials + } + return false } public var isPolishKeyMissing: Bool { diff --git a/OSGKeyboardShared/Models/LLMProvider.swift b/OSGKeyboardShared/Models/LLMProvider.swift index c015850..4a72bcc 100644 --- a/OSGKeyboardShared/Models/LLMProvider.swift +++ b/OSGKeyboardShared/Models/LLMProvider.swift @@ -191,7 +191,7 @@ public struct LLMProvider: Identifiable, Codable, Hashable, Sendable { defaultBaseURL: "wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async", defaultModel: "volc.seedasr.sauc.duration", apiKeyURL: URL(string: "https://console.volcengine.com/speech"), - blurb: "流式大模型 ASR · API Key 填 appId:accessToken[:resourceId]", + blurb: "流式大模型 ASR · 旧版 AppID+Token / 新版 API Key", isUserSelectable: false ), .init( diff --git a/OSGKeyboardShared/Models/PolishStylePack.swift b/OSGKeyboardShared/Models/PolishStylePack.swift index 815d7b1..bdf1e2e 100644 --- a/OSGKeyboardShared/Models/PolishStylePack.swift +++ b/OSGKeyboardShared/Models/PolishStylePack.swift @@ -15,6 +15,9 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable { public let id: String public var name: String public var prompt: String + /// When true, polish may keep model-added emoji and the prompt overrides R5. + /// Defaults off so existing / builtin styles stay emoji-strict. + public var allowsAddedEmoji: Bool public let kind: Kind public let createdAt: Date public var updatedAt: Date @@ -23,6 +26,7 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable { id: String = "user.\(UUID().uuidString.lowercased())", name: String, prompt: String, + allowsAddedEmoji: Bool = false, kind: Kind = .user, createdAt: Date = Date(), updatedAt: Date? = nil @@ -30,6 +34,7 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable { self.id = id self.name = name self.prompt = prompt + self.allowsAddedEmoji = allowsAddedEmoji self.kind = kind self.createdAt = createdAt self.updatedAt = updatedAt ?? createdAt @@ -39,6 +44,46 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable { guard kind == .builtin else { return name } return SharedL10n.string("polishStyle.\(id.dropFirst("builtin.".count))", language: language) } + + /// Effective emoji policy for polish: explicit toggle, or a custom prompt that + /// clearly opts in (so paste-only custom styles still keep model-added emoji). + public var effectiveAllowsAddedEmoji: Bool { + if allowsAddedEmoji { return true } + guard kind == .user else { return false } + return Self.promptDeclaresAddedEmojiOptIn(prompt) + } + + /// Heuristic for custom prompts that declare “add mood emoji” themselves. + public static func promptDeclaresAddedEmojiOptIn(_ prompt: String) -> Bool { + let markers = [ + "允许新增 emoji", + "允许新增emoji", + "按情绪点缀", + "按原文情绪", + "Emoji 覆盖", + "outranks global R5", + "may add emojis", + "allow mood emoji", + "allowsAddedEmoji", + ] + return markers.contains { prompt.localizedCaseInsensitiveContains($0) } + } + + private enum CodingKeys: String, CodingKey { + case id, name, prompt, allowsAddedEmoji, kind, createdAt, updatedAt + } + + public init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + id = try container.decode(String.self, forKey: .id) + name = try container.decode(String.self, forKey: .name) + prompt = try container.decode(String.self, forKey: .prompt) + // Older synced packs omit the key — stay emoji-strict. + allowsAddedEmoji = try container.decodeIfPresent(Bool.self, forKey: .allowsAddedEmoji) ?? false + kind = try container.decode(Kind.self, forKey: .kind) + createdAt = try container.decode(Date.self, forKey: .createdAt) + updatedAt = try container.decode(Date.self, forKey: .updatedAt) + } } public enum PolishStyleLimits { @@ -92,6 +137,7 @@ public struct PolishStyleCatalog: Codable, Equatable, Sendable { var updated = pack updated.name = name updated.prompt = prompt + updated.allowsAddedEmoji = pack.allowsAddedEmoji updated.updatedAt = date entries[index] = updated } else { @@ -101,6 +147,7 @@ public struct PolishStyleCatalog: Codable, Equatable, Sendable { var created = pack created.name = name created.prompt = prompt + created.allowsAddedEmoji = pack.allowsAddedEmoji created.updatedAt = date entries.append(created) } diff --git a/OSGKeyboardShared/Models/ProviderConfig.swift b/OSGKeyboardShared/Models/ProviderConfig.swift index 892e153..4616734 100644 --- a/OSGKeyboardShared/Models/ProviderConfig.swift +++ b/OSGKeyboardShared/Models/ProviderConfig.swift @@ -313,7 +313,14 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable { public var isASRConfigured: Bool { guard !isLocalEngine else { return true } - return !asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + let key = asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines) + let hasKey: Bool = { + if asrProviderId == "volcengine" { + return VolcengineASRFields.parse(apiKey: key).hasUsableCredentials + } + return !key.isEmpty + }() + return hasKey && (!asrBaseURL.isEmpty || CloudASRModelCatalog.strategy(for: asrProviderId) != .prompt) } diff --git a/OSGKeyboardShared/Models/VolcengineASRFields.swift b/OSGKeyboardShared/Models/VolcengineASRFields.swift index 97b786a..cd50048 100644 --- a/OSGKeyboardShared/Models/VolcengineASRFields.swift +++ b/OSGKeyboardShared/Models/VolcengineASRFields.swift @@ -2,65 +2,155 @@ // OSGKeyboard · Shared // // Parse / encode Volcengine SAUC credentials stored in the ASR API key field. +// Supports legacy AppID+AccessToken (old console) and single API Key (new console). import Foundation +/// Volcengine speech console auth style for SAUC streaming ASR. +public enum VolcengineASRAuthMode: String, Sendable, Equatable { + /// Old console: `X-Api-App-Key` + `X-Api-Access-Key`. + case appToken = "app_token" + /// New console: single `X-Api-Key`. + case apiKey = "api_key" +} + public struct VolcengineASRFields: Sendable, Equatable { + public var authMode: VolcengineASRAuthMode public var appID: String public var accessToken: String - public var resourceID: String + /// New-console API Key (`X-Api-Key`). Kept alongside app-token fields so the + /// settings toggle can switch modes without wiping the other credential set. + public var apiKeyCredential: String + + /// Locked product: Doubao streaming ASR 2.0 · duration billing. + public static let fixedResourceID = CloudASRModelCatalog.volcengineDefaultResourceID public init( + authMode: VolcengineASRAuthMode = .appToken, appID: String = "", accessToken: String = "", - resourceID: String = CloudASRModelCatalog.defaultModel(for: "volcengine") + apiKeyCredential: String = "" ) { + self.authMode = authMode self.appID = appID self.accessToken = accessToken - self.resourceID = resourceID + self.apiKeyCredential = apiKeyCredential + } + + /// Always `volc.seedasr.sauc.duration` — not user-editable. + public var resourceID: String { Self.fixedResourceID } + + public var usesAPIKeyAuth: Bool { authMode == .apiKey } + + public var hasUsableCredentials: Bool { + switch authMode { + case .appToken: + return !appID.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + && !accessToken.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + case .apiKey: + return !apiKeyCredential.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + } } public var encodedAPIKey: String { - let object = [ - "app_id": appID, - "access_token": accessToken, - "resource_id": resourceID, + var object: [String: String] = [ + "auth_mode": authMode.rawValue, + "resource_id": Self.fixedResourceID, ] + // Persist both credential sets so toggling auth mode is non-destructive. + if !appID.isEmpty { object["app_id"] = appID } + if !accessToken.isEmpty { object["access_token"] = accessToken } + if !apiKeyCredential.isEmpty { object["api_key"] = apiKeyCredential } + guard let data = try? JSONSerialization.data(withJSONObject: object), let string = String(data: data, encoding: .utf8) else { - return [appID, accessToken, resourceID].joined(separator: ":") + switch authMode { + case .appToken: + return [appID, accessToken, Self.fixedResourceID].joined(separator: ":") + case .apiKey: + return apiKeyCredential + } } return string } - public static func parse(apiKey: String, resourceFallback: String) -> VolcengineASRFields { + /// Apply SAUC WebSocket handshake headers for the active auth mode. + public func applyWebSocketAuthHeaders(to request: inout URLRequest, connectID: String) { + request.setValue(Self.fixedResourceID, forHTTPHeaderField: "X-Api-Resource-Id") + request.setValue(connectID, forHTTPHeaderField: "X-Api-Connect-Id") + switch authMode { + case .apiKey: + request.setValue( + apiKeyCredential.trimmingCharacters(in: .whitespacesAndNewlines), + forHTTPHeaderField: "X-Api-Key" + ) + case .appToken: + request.setValue( + appID.trimmingCharacters(in: .whitespacesAndNewlines), + forHTTPHeaderField: "X-Api-App-Key" + ) + request.setValue( + accessToken.trimmingCharacters(in: .whitespacesAndNewlines), + forHTTPHeaderField: "X-Api-Access-Key" + ) + } + } + + /// - Parameter resourceFallback: Ignored; resource is always `fixedResourceID`. + /// Kept so call sites stay source-compatible. + public static func parse(apiKey: String, resourceFallback: String = "") -> VolcengineASRFields { + _ = resourceFallback let trimmed = apiKey.trimmingCharacters(in: .whitespacesAndNewlines) - var fields = VolcengineASRFields( - appID: "", - accessToken: "", - resourceID: resourceFallback.isEmpty - ? CloudASRModelCatalog.defaultModel(for: "volcengine") - : resourceFallback - ) + var fields = VolcengineASRFields() + + guard !trimmed.isEmpty else { return fields } if let data = trimmed.data(using: .utf8), let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] { fields.appID = string(json, keys: ["app_id", "appId", "appid"]) ?? "" fields.accessToken = string(json, keys: ["access_token", "accessToken", "token"]) ?? "" - fields.resourceID = string(json, keys: ["resource_id", "resourceId", "resource"]) ?? fields.resourceID + fields.apiKeyCredential = string(json, keys: ["api_key", "apiKey"]) ?? "" + fields.authMode = resolveAuthMode( + raw: string(json, keys: ["auth_mode", "authMode"]), + hasAPIKey: !fields.apiKeyCredential.isEmpty, + hasAppToken: !fields.appID.isEmpty && !fields.accessToken.isEmpty + ) return fields } + // Legacy colon form is always old-console app-token auth. let parts = trimmed .components(separatedBy: CharacterSet(charactersIn: ":\n,")) .map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } .filter { !$0.isEmpty } if parts.indices.contains(0) { fields.appID = parts[0] } if parts.indices.contains(1) { fields.accessToken = parts[1] } - if parts.indices.contains(2) { fields.resourceID = parts[2] } + fields.authMode = .appToken return fields } + private static func resolveAuthMode( + raw: String?, + hasAPIKey: Bool, + hasAppToken: Bool + ) -> VolcengineASRAuthMode { + if let raw { + let normalized = raw.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + if normalized == VolcengineASRAuthMode.apiKey.rawValue || normalized == "apikey" { + return .apiKey + } + if normalized == VolcengineASRAuthMode.appToken.rawValue + || normalized == "apptoken" + || normalized == "app_id_token" { + return .appToken + } + } + // Legacy JSON without auth_mode: prefer app-token when present. + if hasAppToken { return .appToken } + if hasAPIKey { return .apiKey } + return .appToken + } + private static func string(_ json: [String: Any], keys: [String]) -> String? { for key in keys { if let value = json[key] as? String { diff --git a/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift b/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift index 5cb9a51..d7f09a9 100644 --- a/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift +++ b/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift @@ -14,8 +14,39 @@ public enum FlowKeyboardHostWarming { reason == .recording || reason == .processing || reason == .awaitingDelivery } + /// Keep the mic green after the session has already proven ready. + /// + /// Inter-utterance PiP flaps (mic release, ack lag, brief `reason=.starting`) + /// used to flash yellow「正在启动画中画」even though Picture in Picture was + /// already running. Hold ready through those windows; real cold starts still + /// go through `isHostWarming` while `sessionProvenReady` is false. + public static func shouldHoldReady( + hostReady: Bool, + hostBusy: Bool, + sessionActive: Bool, + sessionProvenReady: Bool, + isPendingFlowStart: Bool, + snapshotReason: FlowReadySnapshot.Reason? + ) -> Bool { + guard !hostReady, + sessionProvenReady, + sessionActive, + !isPendingFlowStart else { + return false + } + // After insert the host may still publish awaitingDelivery until it + // consumes the ack — that is not a PiP restart. + if hostBusy { + return snapshotReason == .awaitingDelivery + } + return true + } + /// Session lives but ready contract is not fresh — keep mic orange (wait) /// instead of launching another cold start. + /// + /// `withinReadyGrace` is intentionally unused for warming: a recent ready + /// must hold green via `shouldHoldReady`, not flash preparingSession. public static func isHostWarming( hostReady: Bool, hostBusy: Bool, @@ -25,13 +56,13 @@ public enum FlowKeyboardHostWarming { withinReadyGrace: Bool, snapshotReason: FlowReadySnapshot.Reason? ) -> Bool { - !hostReady + _ = withinReadyGrace + return !hostReady && !hostBusy && sessionActive && ( hostReachable || isPendingFlowStart - || withinReadyGrace || snapshotReason == .starting ) } diff --git a/OSGKeyboardShared/Services/FlowSessionBridge.swift b/OSGKeyboardShared/Services/FlowSessionBridge.swift index 0230440..5b11c07 100644 --- a/OSGKeyboardShared/Services/FlowSessionBridge.swift +++ b/OSGKeyboardShared/Services/FlowSessionBridge.swift @@ -671,6 +671,8 @@ public enum FlowSessionBridge { store.removeObject(forKey: FlowSessionKeys.audioLevels) store.removeObject(forKey: FlowSessionKeys.lastActivityAt) clearHostReady(defaults: store, notify: false) + // Previous generation may have died mid Rime/CLM/ASR with hostHeavy=1. + clearHostHeavy(defaults: store) flush(store) } @@ -701,13 +703,47 @@ public enum FlowSessionBridge { /// avoid stacking typing-engine RSS on top. public static func setHostHeavy(_ heavy: Bool, defaults: UserDefaults? = nil) { let store = resolvedDefaults(defaults) - store.set(heavy, forKey: FlowSessionKeys.hostHeavy) + if heavy { + store.set(true, forKey: FlowSessionKeys.hostHeavy) + store.set(Date().timeIntervalSince1970, forKey: FlowSessionKeys.hostHeavyAt) + } else { + clearHostHeavy(defaults: store) + } flush(store) OSGDiag.log("hostHeavy=\(heavy ? 1 : 0) \(OSGDiag.memoryTag())", category: "flow") } + /// True only while the host recently marked itself busy. A sticky `true` + /// left by a dead host (no `setHostHeavy(false)`) expires after + /// `hostHeavyMaxAge` so typing 中文/EN is not silently blocked forever. public static func isHostHeavy(defaults: UserDefaults? = nil) -> Bool { - resolvedDefaults(defaults).bool(forKey: FlowSessionKeys.hostHeavy) + let store = resolvedDefaults(defaults) + guard store.bool(forKey: FlowSessionKeys.hostHeavy) else { return false } + let markedAt = store.double(forKey: FlowSessionKeys.hostHeavyAt) + // Legacy writes had the bool but no timestamp — treat as stale so a + // pre-fix sticky flag cannot brick typing after upgrade. + guard markedAt > 0 else { + clearHostHeavy(defaults: store) + flush(store) + OSGDiag.log("hostHeavy stale missingAt — cleared \(OSGDiag.memoryTag())", category: "flow") + return false + } + let age = Date().timeIntervalSince1970 - markedAt + guard age >= 0, age <= FlowSessionKeys.hostHeavyMaxAge else { + clearHostHeavy(defaults: store) + flush(store) + OSGDiag.log( + "hostHeavy stale age=\(Int(age))s — cleared \(OSGDiag.memoryTag())", + category: "flow" + ) + return false + } + return true + } + + private static func clearHostHeavy(defaults: UserDefaults) { + defaults.set(false, forKey: FlowSessionKeys.hostHeavy) + defaults.removeObject(forKey: FlowSessionKeys.hostHeavyAt) } /// True when the host has published a fresh ready contract (stricter than heartbeat alone). @@ -948,6 +984,7 @@ public enum FlowSessionBridge { store.removeObject(forKey: FlowSessionKeys.pendingHostBundleId) store.removeObject(forKey: FlowSessionKeys.lastActivityAt) clearHostReady(defaults: store, notify: false) + clearHostHeavy(defaults: store) flush(store) } diff --git a/OSGKeyboardShared/Services/FlowSessionKeys.swift b/OSGKeyboardShared/Services/FlowSessionKeys.swift index 2e2bba7..7a6b173 100644 --- a/OSGKeyboardShared/Services/FlowSessionKeys.swift +++ b/OSGKeyboardShared/Services/FlowSessionKeys.swift @@ -43,6 +43,15 @@ public enum FlowSessionKeys { /// Host is mid heavy work (Rime/CLM/ASR). Extension should stay on voice /// and skip typing engine prepare until this clears. public static let hostHeavy = "flow.hostHeavy.v1" + /// Wall-clock timestamp paired with `hostHeavy` (seconds since 1970). + /// Lets the keyboard ignore a sticky flag left behind when the host died + /// mid-warmup without ever clearing App Group state. + public static let hostHeavyAt = "flow.hostHeavyAt.v1" + + /// `hostHeavy` older than this is treated as stale (host likely jetsammed + /// or force-quit before `setHostHeavy(false)`). Rime/CLM/ASR bursts are + /// expected well under this window. + public static let hostHeavyMaxAge: TimeInterval = 120 /// Heartbeat older than this → host is not actively reachable for recording. public static let heartbeatStaleInterval: TimeInterval = 3 diff --git a/OSGKeyboardShared/Services/PolishPromptComposer.swift b/OSGKeyboardShared/Services/PolishPromptComposer.swift index 7705689..4420f0a 100644 --- a/OSGKeyboardShared/Services/PolishPromptComposer.swift +++ b/OSGKeyboardShared/Services/PolishPromptComposer.swift @@ -211,6 +211,10 @@ public enum PolishPromptComposer { dictionaryBlock, useChineseGuidance: useChineseGuidance ) + let emojiOverride = emojiOverrideSection( + allowsAddedEmoji: style.effectiveAllowsAddedEmoji, + useChineseGuidance: useChineseGuidance + ) if usesHeavyFunPipeline { let formatting = useChineseGuidance ? chineseFunFormattingPrompt @@ -226,6 +230,7 @@ public enum PolishPromptComposer { \(personality) \(outputInstruction) + \(emojiOverride) """ } @@ -262,6 +267,7 @@ public enum PolishPromptComposer { fieldHints: context.fieldHints, useChineseGuidance: true ))用户消息即为待处理的转写文本。只输出处理后的文本。 + \(emojiOverride) """ } @@ -284,6 +290,7 @@ public enum PolishPromptComposer { fieldHints: context.fieldHints, useChineseGuidance: false ))The user message is the transcript to process. Output the processed text only. + \(emojiOverride) """ } @@ -346,6 +353,8 @@ public enum PolishPromptComposer { useChineseGuidance: Bool ) -> String { let body = PolishStylePackCatalog.runtimePersonality(for: style) + // Emoji override is appended at the end of compose(), not here — + // it must outrank core R5 and later context premises. if style.kind == .user { return useChineseGuidance ? """ @@ -370,6 +379,30 @@ public enum PolishPromptComposer { """ } + /// When a style opts in, override global R5 so the model may add emoji. + /// Placed last in the composed prompt so it wins over core R5. + private static func emojiOverrideSection( + allowsAddedEmoji: Bool, + useChineseGuidance: Bool + ) -> String { + guard allowsAddedEmoji else { return "" } + return useChineseGuidance + ? """ + + # Emoji 覆盖（本风格开启 · 最终优先级） + 本风格允许新增 emoji，优先级高于全局 R5「不新增 emoji」以及上文任何「不要加 emoji」表述。 + 仅按原文已表达的情绪点缀 0–2 个贴合语气的 emoji；中性安排、正式通知与极短确认词不加。 + 原文已有 emoji 时只整理文字，不替换、不堆叠。禁止无关装饰与 emoji 墙。 + """ + : """ + + # Emoji override (enabled for this style · final priority) + This style may add emojis and outranks global R5 (“add no emojis”) and any earlier “do not add emojis” guidance. + Add 0–2 tone-matching emojis only when the draft already expresses emotion; skip neutral schedules, formal notices, and ultra-short acks. + If the draft already has emojis, keep them and do not replace or stack. No decorative spam or emoji walls. + """ + } + /// Neutralize envelope-breaking tags inside user-controlled transcript text. internal static func sanitizeEnvelopeContent(_ text: String) -> String { let maxCharacters = 16_000 diff --git a/OSGKeyboardShared/Services/PolishingService.swift b/OSGKeyboardShared/Services/PolishingService.swift index 759b21f..fb82bc0 100644 --- a/OSGKeyboardShared/Services/PolishingService.swift +++ b/OSGKeyboardShared/Services/PolishingService.swift @@ -285,7 +285,15 @@ public actor PolishingService { // One prompt, one model request. Deterministic validation may reject a // result locally, but it never starts a second polish request. - let firstCandidate = TranscriptPostProcessor.process(original: trimmed, llmOutput: first) + let activeStyle = PolishStylePackCatalog.resolve( + id: store.activePolishStyleId, + userCatalog: store.polishStyleCatalog + ) + let firstCandidate = TranscriptPostProcessor.process( + original: trimmed, + llmOutput: first, + allowsAddedEmoji: activeStyle.effectiveAllowsAddedEmoji + ) let firstViolations = PolishOutputValidator.validate( input: trimmed, output: firstCandidate, diff --git a/OSGKeyboardShared/Services/TranscriptPostProcessor.swift b/OSGKeyboardShared/Services/TranscriptPostProcessor.swift index e8768bc..7f8aecb 100644 --- a/OSGKeyboardShared/Services/TranscriptPostProcessor.swift +++ b/OSGKeyboardShared/Services/TranscriptPostProcessor.swift @@ -169,9 +169,17 @@ public enum TranscriptPostProcessor: Sendable { // MARK: - Post-LLM pipeline /// Apply deterministic cleanup and quality gate to LLM output. - public static func process(original: String, llmOutput: String) -> String { + public static func process( + original: String, + llmOutput: String, + allowsAddedEmoji: Bool = false + ) -> String { let trimmedOriginal = original.trimmingCharacters(in: .whitespacesAndNewlines) - let decision = qualityGate(original: trimmedOriginal, candidate: llmOutput) + let decision = qualityGate( + original: trimmedOriginal, + candidate: llmOutput, + allowsAddedEmoji: allowsAddedEmoji + ) switch decision { case .accept(let text): return text @@ -191,7 +199,11 @@ public enum TranscriptPostProcessor: Sendable { /// back when the model returned genuinely unusable output (empty, or /// pure explanation), and even then we prefer a cleaned candidate /// over the raw transcript. - public static func qualityGate(original: String, candidate: String) -> GateDecision { + public static func qualityGate( + original: String, + candidate: String, + allowsAddedEmoji: Bool = false + ) -> GateDecision { var text = candidate.trimmingCharacters(in: .whitespacesAndNewlines) if text.isEmpty { @@ -201,7 +213,9 @@ public enum TranscriptPostProcessor: Sendable { text = stripExplanatoryPrefix(from: text) text = stripPauseMarkers(from: text) text = unwrapSurroundingQuotes(text) - text = stripAddedEmojis(original: original, output: text) + if !allowsAddedEmoji { + text = stripAddedEmojis(original: original, output: text) + } text = repairMidSentenceLineBreaks(text) text = normalizeWhitespaceAndPunctuation(text) text = normalizeNumberedLists(text) diff --git a/OSGKeyboardShared/Typing/TypingAutocapitalization.swift b/OSGKeyboardShared/Typing/TypingAutocapitalization.swift index 932b6d4..5763135 100644 --- a/OSGKeyboardShared/Typing/TypingAutocapitalization.swift +++ b/OSGKeyboardShared/Typing/TypingAutocapitalization.swift @@ -42,20 +42,22 @@ public enum TypingAutocapitalization: Sendable { private static func needsSentenceCapitalization(_ preceding: String?) -> Bool { guard let preceding, !preceding.isEmpty else { return true } - // Walk backward past trailing whitespace / newlines; capitalize when - // the field is empty or the previous visible character ends a sentence. + // Walk backward from the caret. Trailing spaces are ignored; a newline + // itself starts a new line (system .sentences behavior, e.g. Notes + // after Return). Otherwise capitalize only after a sentence terminator. var index = preceding.endIndex - var sawContent = false while index > preceding.startIndex { index = preceding.index(before: index) let character = preceding[index] - if character.isWhitespace || character.isNewline { + if character.isNewline { + return true + } + if character.isWhitespace { continue } - sawContent = true return isSentenceTerminator(character) } - return !sawContent + return true } private static func isSentenceTerminator(_ character: Character) -> Bool { diff --git a/OSGKeyboardShared/Typing/TypingSessionController.swift b/OSGKeyboardShared/Typing/TypingSessionController.swift index b405304..b7e270d 100644 --- a/OSGKeyboardShared/Typing/TypingSessionController.swift +++ b/OSGKeyboardShared/Typing/TypingSessionController.swift @@ -68,6 +68,10 @@ public final class TypingSessionController: ObservableObject { private var shiftPrimedByUser = false /// True if any key was typed while the current Shift hold was active. private var typedWhileShiftHeld = false + /// Local caret-prefix mirror so autocap survives stale `documentContextBeforeInput` + /// (common in Notes). Capped; reseeds from the proxy when it looks fresh. + private var precedingShadow = "" + private static let precedingShadowLimit = 400 public init( engine: (@MainActor () -> RimeEngineBridging)? = nil, @@ -287,6 +291,14 @@ public final class TypingSessionController: ObservableObject { return handleEnglishCharacter(ch) } + // Chinese + Shift: insert Latin directly (iOS-style mix-in), leave Rime + // composition untouched. Rime's alphabet is lowercase-only, so uppercase + // keycodes would otherwise be rejected with no output. + if isShiftEnabled, ch.isLetter { + clearOneShotShiftIfNeeded() + return .insert(String(ch)) + } + // Chinese letters → compose let committed = engine.processCharacter(ch) ?? "" composition = engine.composition @@ -490,17 +502,94 @@ public final class TypingSessionController: ObservableObject { /// Arms Shift for sentence / word starts using the host field traits. /// Manual one-shot, hold, and Caps Lock always win over autocapitalization. - public func syncAutocapitalization() { + /// - Parameters: + /// - insert: Text just written through the document proxy (may not be + /// reflected in `documentContextBeforeInput` yet). + /// - deleteCount: Characters just deleted before `insert` (replace path). + public func syncAutocapitalization( + accountingForInsert insert: String = "", + deleteCount: Int = 0 + ) { guard language == .english, page == .letters else { return } guard !capsLock, !shiftHeld, !shiftPrimedByUser else { return } let mode = autocapitalizationModeProvider?() ?? .sentences - let preceding = precedingTextProvider?() + let preceding = resolvedPrecedingText( + accountingForInsert: insert, + deleteCount: deleteCount + ) shiftActive = TypingAutocapitalization.shouldCapitalize( precedingText: preceding, mode: mode ) } + /// Prefer a fresh proxy; when the host lags (Notes), merge our just-applied edit. + private func resolvedPrecedingText( + accountingForInsert insert: String, + deleteCount: Int + ) -> String? { + // Host-driven refresh (appear / textDidChange): reseed from proxy. + if deleteCount == 0, insert.isEmpty { + if let proxy = precedingTextProvider?() { + return storePrecedingShadow(proxy) + } + return precedingShadow.isEmpty ? nil : precedingShadow + } + + let proxy = precedingTextProvider?() + + if deleteCount > 0, !insert.isEmpty { + if let proxy { + if proxy.hasSuffix(insert) { + return storePrecedingShadow(proxy) + } + if proxy.count >= deleteCount { + return storePrecedingShadow(String(proxy.dropLast(deleteCount)) + insert) + } + } + trimPrecedingShadow(by: deleteCount) + return storePrecedingShadow(precedingShadow + insert) + } + + if deleteCount > 0 { + if let proxy, proxy.count + deleteCount == precedingShadow.count + || (precedingShadow.count >= deleteCount + && String(precedingShadow.dropLast(deleteCount)) == proxy) { + return storePrecedingShadow(proxy) + } + if precedingShadow.count >= deleteCount { + return storePrecedingShadow(String(precedingShadow.dropLast(deleteCount))) + } + if let proxy { return storePrecedingShadow(proxy) } + precedingShadow = "" + return "" + } + + // insert only + if let proxy { + if proxy.hasSuffix(insert) { + return storePrecedingShadow(proxy) + } + return storePrecedingShadow(proxy + insert) + } + return storePrecedingShadow(precedingShadow + insert) + } + + @discardableResult + private func storePrecedingShadow(_ value: String) -> String { + precedingShadow = String(value.suffix(Self.precedingShadowLimit)) + return precedingShadow + } + + private func trimPrecedingShadow(by count: Int) { + guard count > 0 else { return } + if precedingShadow.count >= count { + precedingShadow.removeLast(count) + } else { + precedingShadow = "" + } + } + // MARK: - Shift /// Tap cycle: off → one-shot → Caps Lock → off (iOS-style second tap). @@ -536,6 +625,7 @@ public final class TypingSessionController: ObservableObject { shiftHeld = false shiftPrimedByUser = false typedWhileShiftHeld = false + precedingShadow = "" } private func clearEnglishWordState(keepPrevious: Bool) { diff --git a/OSGKeyboardShared/en.lproj/Shared.strings b/OSGKeyboardShared/en.lproj/Shared.strings index deec117..23529aa 100644 --- a/OSGKeyboardShared/en.lproj/Shared.strings +++ b/OSGKeyboardShared/en.lproj/Shared.strings @@ -158,6 +158,8 @@ "mac.styles.error" = "Couldn’t Save Style"; "mac.styles.validation" = "Check the name, prompt length, and the 8-style limit."; "mac.styles.name" = "Style name"; +"mac.styles.allowsAddedEmoji" = "Allow mood emojis"; +"mac.styles.allowsAddedEmoji.hint" = "When on, polish may add a few emojis that match the draft’s emotion, and keeps them on screen. Off by default."; "mac.styles.prompt" = "Complete prompt"; "mac.styles.hint" = "Use {{DICTIONARY}} to place the personal dictionary. System rules are appended automatically."; "mac.section.settings" = "Settings"; @@ -262,8 +264,11 @@ "mac.settings.translationOff" = "Don't translate"; "mac.settings.volcengineAppId" = "APP ID"; "mac.settings.volcengineAccessToken" = "Access Token"; -"mac.settings.volcengineResourceId" = "Resource ID"; -"mac.settings.volcengineNote" = "Secret Key is not required. Resource ID defaults to volc.seedasr.sauc.duration."; +"mac.settings.volcengineApiKey" = "API Key"; +"mac.settings.volcengineApiKeyMode" = "New API Key auth"; +"mac.settings.volcengineApiKeyModeSubtitle" = "On for the new console; off keeps APP ID + Access Token."; +"mac.settings.volcengineNoteAppToken" = "Legacy console: APP ID + Access Token. Secret Key not required. Resource fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration)."; +"mac.settings.volcengineNoteApiKey" = "New console: API Key only. Resource fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration)."; "mac.settings.recognition" = "RECOGNITION METHOD"; "mac.settings.cloudEngine" = "Cloud Engine & AI Refinement"; "mac.settings.cloudEngineDesc" = "Premium transcription via your provider's API, plus AI grammar and style polishing."; diff --git a/OSGKeyboardShared/zh-Hans.lproj/Shared.strings b/OSGKeyboardShared/zh-Hans.lproj/Shared.strings index 5946f5e..0bb2fe5 100644 --- a/OSGKeyboardShared/zh-Hans.lproj/Shared.strings +++ b/OSGKeyboardShared/zh-Hans.lproj/Shared.strings @@ -157,6 +157,8 @@ "mac.styles.error" = "无法保存风格"; "mac.styles.validation" = "请检查名称、提示词长度及 8 个风格的数量上限。"; "mac.styles.name" = "风格名称"; +"mac.styles.allowsAddedEmoji" = "允许按情绪添加 emoji"; +"mac.styles.allowsAddedEmoji.hint" = "开启后，润色可按原文情绪点缀少量 emoji，并保留上屏。默认关闭。"; "mac.styles.prompt" = "完整提示词"; "mac.styles.hint" = "使用 {{DICTIONARY}} 指定个人词典位置；系统规则会自动追加。"; "mac.section.settings" = "设置"; @@ -261,8 +263,11 @@ "mac.settings.translationOff" = "不翻译"; "mac.settings.volcengineAppId" = "APP ID"; "mac.settings.volcengineAccessToken" = "Access Token"; -"mac.settings.volcengineResourceId" = "Resource ID"; -"mac.settings.volcengineNote" = "Secret Key 当前无需填写。Resource ID 默认使用 volc.seedasr.sauc.duration。"; +"mac.settings.volcengineApiKey" = "API Key"; +"mac.settings.volcengineApiKeyMode" = "使用新版 API Key 鉴权"; +"mac.settings.volcengineApiKeyModeSubtitle" = "新控制台请打开；已有 AppID + Token 可保持关闭。"; +"mac.settings.volcengineNoteAppToken" = "旧版控制台：填写 APP ID 与 Access Token。Secret Key 无需填写。识别资源固定为豆包流式 2.0（volc.seedasr.sauc.duration）。"; +"mac.settings.volcengineNoteApiKey" = "新版控制台：只需填写 API Key。识别资源固定为豆包流式 2.0（volc.seedasr.sauc.duration）。"; "mac.settings.recognition" = "识别方式"; "mac.settings.cloudEngine" = "云端引擎与 AI 润色"; "mac.settings.cloudEngineDesc" = "通过服务商 API 进行高质量转写，并自动润色语法与风格。"; diff --git a/OSGKeyboardTests/CloudASRTests.swift b/OSGKeyboardTests/CloudASRTests.swift index 707a04b..73199e1 100644 --- a/OSGKeyboardTests/CloudASRTests.swift +++ b/OSGKeyboardTests/CloudASRTests.swift @@ -130,7 +130,10 @@ final class CloudASRTests: XCTestCase { let fields = VolcengineASRFields.parse(apiKey: json, resourceFallback: "") XCTAssertEqual(fields.appID, "app-1") XCTAssertEqual(fields.accessToken, "tok-2") - XCTAssertEqual(fields.resourceID, "res-3") + XCTAssertEqual(fields.authMode, .appToken) + // Custom resource IDs are ignored; product is locked to SAUC 2.0 duration. + XCTAssertEqual(fields.resourceID, VolcengineASRFields.fixedResourceID) + XCTAssertTrue(fields.hasUsableCredentials) } func testVolcengineASRFieldsColonParsing() { @@ -140,8 +143,70 @@ final class CloudASRTests: XCTestCase { ) XCTAssertEqual(fields.appID, "app-1") XCTAssertEqual(fields.accessToken, "tok-2") - XCTAssertEqual(fields.resourceID, "res-3") + XCTAssertEqual(fields.authMode, .appToken) + XCTAssertEqual(fields.resourceID, VolcengineASRFields.fixedResourceID) XCTAssertTrue(fields.encodedAPIKey.contains("app-1")) + XCTAssertTrue(fields.encodedAPIKey.contains("auth_mode")) + } + + func testVolcengineASRFieldsAPIKeyModeParsing() { + let json = #"{"auth_mode":"api_key","api_key":"vk-new-console"}"# + let fields = VolcengineASRFields.parse(apiKey: json) + XCTAssertEqual(fields.authMode, .apiKey) + XCTAssertEqual(fields.apiKeyCredential, "vk-new-console") + XCTAssertTrue(fields.hasUsableCredentials) + XCTAssertEqual(fields.resourceID, VolcengineASRFields.fixedResourceID) + + var request = URLRequest(url: URL(string: "wss://example.invalid")!) + fields.applyWebSocketAuthHeaders(to: &request, connectID: "conn-1") + XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-Key"), "vk-new-console") + XCTAssertNil(request.value(forHTTPHeaderField: "X-Api-App-Key")) + XCTAssertNil(request.value(forHTTPHeaderField: "X-Api-Access-Key")) + XCTAssertEqual( + request.value(forHTTPHeaderField: "X-Api-Resource-Id"), + VolcengineASRFields.fixedResourceID + ) + XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-Connect-Id"), "conn-1") + } + + func testVolcengineASRFieldsAppTokenHeaders() { + let fields = VolcengineASRFields( + authMode: .appToken, + appID: "app-1", + accessToken: "tok-2" + ) + var request = URLRequest(url: URL(string: "wss://example.invalid")!) + fields.applyWebSocketAuthHeaders(to: &request, connectID: "conn-2") + XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-App-Key"), "app-1") + XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-Access-Key"), "tok-2") + XCTAssertNil(request.value(forHTTPHeaderField: "X-Api-Key")) + XCTAssertEqual( + request.value(forHTTPHeaderField: "X-Api-Resource-Id"), + VolcengineASRFields.fixedResourceID + ) + } + + func testVolcengineASRFieldsTogglePreservesBothCredentialSets() { + var fields = VolcengineASRFields( + authMode: .appToken, + appID: "app-1", + accessToken: "tok-2", + apiKeyCredential: "vk-keep" + ) + fields.authMode = .apiKey + let encoded = fields.encodedAPIKey + let parsed = VolcengineASRFields.parse(apiKey: encoded) + XCTAssertEqual(parsed.authMode, .apiKey) + XCTAssertEqual(parsed.apiKeyCredential, "vk-keep") + XCTAssertEqual(parsed.appID, "app-1") + XCTAssertEqual(parsed.accessToken, "tok-2") + } + + func testVolcengineASRFieldsEmptyAPIKeyModeIsNotUsable() { + let json = #"{"auth_mode":"api_key"}"# + let fields = VolcengineASRFields.parse(apiKey: json) + XCTAssertEqual(fields.authMode, .apiKey) + XCTAssertFalse(fields.hasUsableCredentials) } func testPersonalDictionaryASRHotwordsDedupesTerms() { diff --git a/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift b/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift index f410e60..d498f6b 100644 --- a/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift +++ b/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift @@ -42,6 +42,90 @@ final class FlowKeyboardPoliciesTests: XCTestCase { XCTAssertTrue(warming) } + func testRecentReadyGraceDoesNotForcePreparingSession() { + // withinReadyGrace used to OR into warming and flash yellow after each + // utterance; sticky ready must hold green instead. + let warming = FlowKeyboardHostWarming.isHostWarming( + hostReady: false, + hostBusy: false, + sessionActive: true, + hostReachable: false, + isPendingFlowStart: false, + withinReadyGrace: true, + snapshotReason: .starting + ) + // Still warming via reason=.starting when not yet proven ready on the + // keyboard side; grace alone must not be the trigger. + XCTAssertTrue(warming) + + XCTAssertTrue( + FlowKeyboardHostWarming.shouldHoldReady( + hostReady: false, + hostBusy: false, + sessionActive: true, + sessionProvenReady: true, + isPendingFlowStart: false, + snapshotReason: .starting + ), + "proven-ready session must hold green through starting flaps" + ) + XCTAssertTrue( + FlowKeyboardHostWarming.shouldHoldReady( + hostReady: false, + hostBusy: true, + sessionActive: true, + sessionProvenReady: true, + isPendingFlowStart: false, + snapshotReason: .awaitingDelivery + ), + "ack lag awaitingDelivery must not drop to preparingSession" + ) + XCTAssertFalse( + FlowKeyboardHostWarming.shouldHoldReady( + hostReady: false, + hostBusy: true, + sessionActive: true, + sessionProvenReady: true, + isPendingFlowStart: false, + snapshotReason: .recording + ), + "live recording must not be masked as sticky ready" + ) + XCTAssertFalse( + FlowKeyboardHostWarming.shouldHoldReady( + hostReady: false, + hostBusy: false, + sessionActive: true, + sessionProvenReady: false, + isPendingFlowStart: true, + snapshotReason: .starting + ), + "cold start pending must still show preparing" + ) + } + + func testHeldReadySuppressesWarming() { + let hold = FlowKeyboardHostWarming.shouldHoldReady( + hostReady: false, + hostBusy: false, + sessionActive: true, + sessionProvenReady: true, + isPendingFlowStart: false, + snapshotReason: .starting + ) + XCTAssertTrue(hold) + let warming = FlowKeyboardHostWarming.isHostWarming( + hostReady: true, // effective ready after hold + hostBusy: false, + sessionActive: true, + hostReachable: true, + isPendingFlowStart: false, + withinReadyGrace: true, + snapshotReason: .starting + ) + XCTAssertFalse(warming) + } + // MARK: - Adopt busy func testReAdoptsRecordingAfterExtensionProcessLoss() { diff --git a/OSGKeyboardTests/FlowReliabilityTests.swift b/OSGKeyboardTests/FlowReliabilityTests.swift index 047fa74..abb08de 100644 --- a/OSGKeyboardTests/FlowReliabilityTests.swift +++ b/OSGKeyboardTests/FlowReliabilityTests.swift @@ -220,4 +220,14 @@ final class FlowReliabilityTests: XCTestCase { ) ) } + + func testCaptureVoiceProcessingUsesSpeechOrientedSessionMode() { + XCTAssertEqual(FlowCaptureVoiceProcessing.captureMode, .voiceChat) + XCTAssertTrue( + FlowCaptureVoiceProcessing.captureOptions.contains(.allowBluetoothHFP) + ) + XCTAssertTrue( + FlowCaptureVoiceProcessing.captureOptions.contains(.defaultToSpeaker) + ) + } } diff --git a/OSGKeyboardTests/FlowSessionBridgeTests.swift b/OSGKeyboardTests/FlowSessionBridgeTests.swift index 00853b4..1f22a8e 100644 --- a/OSGKeyboardTests/FlowSessionBridgeTests.swift +++ b/OSGKeyboardTests/FlowSessionBridgeTests.swift @@ -428,6 +428,41 @@ final class FlowSessionBridgeTests: XCTestCase { ) } + func testHostHeavyClearsOnFlowStateReset() { + let defaults = makeDefaults() + FlowSessionBridge.setHostHeavy(true, defaults: defaults) + XCTAssertTrue(FlowSessionBridge.isHostHeavy(defaults: defaults)) + + FlowSessionBridge.clearFlowState(defaults: defaults) + XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults)) + + FlowSessionBridge.setHostHeavy(true, defaults: defaults) + FlowSessionBridge.setPendingHostBundleId("com.example.host", defaults: defaults) + FlowSessionBridge.clearFlowStateOnHostLaunch(defaults: defaults) + XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults)) + XCTAssertEqual( + FlowSessionBridge.pendingHostBundleId(defaults: defaults), + "com.example.host" + ) + } + + func testStaleHostHeavyDoesNotBlockTyping() { + let defaults = makeDefaults() + // Legacy sticky bool with no timestamp — must not brick 中文/EN. + defaults.set(true, forKey: FlowSessionKeys.hostHeavy) + XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults)) + XCTAssertFalse(defaults.bool(forKey: FlowSessionKeys.hostHeavy)) + + FlowSessionBridge.setHostHeavy(true, defaults: defaults) + XCTAssertTrue(FlowSessionBridge.isHostHeavy(defaults: defaults)) + + // Expired timestamp → treat as clear so cold keyboard can switch. + let expired = Date().timeIntervalSince1970 - FlowSessionKeys.hostHeavyMaxAge - 1 + defaults.set(expired, forKey: FlowSessionKeys.hostHeavyAt) + XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults)) + XCTAssertFalse(defaults.bool(forKey: FlowSessionKeys.hostHeavy)) + } + func testRotateHostGenerationReturnsPreviousToken() { let defaults = makeDefaults() XCTAssertNil(FlowSessionBridge.rotateHostGeneration(defaults: defaults)) diff --git a/OSGKeyboardTests/IntelligentPolishTests.swift b/OSGKeyboardTests/IntelligentPolishTests.swift index f9fc110..d72a7ff 100644 --- a/OSGKeyboardTests/IntelligentPolishTests.swift +++ b/OSGKeyboardTests/IntelligentPolishTests.swift @@ -340,6 +340,54 @@ final class IntelligentPolishTests: XCTestCase { XCTAssertTrue(result.contains("完成")) } + func testPolishServiceKeepsAddedEmojiWhenStyleAllows() async throws { + store.setEngineMode("local") + var catalog = PolishStyleCatalog() + let pack = PolishStylePack( + id: "user.emoji", + name: "Emoji", + prompt: "保持口语，可按情绪加 emoji。", + allowsAddedEmoji: true + ) + try catalog.upsert(pack) + store.setPolishStyleCatalog(catalog) + store.setActivePolishStyleId(pack.id) + + let emojiClient = FixedResponseLLMClient(response: "今天太开心了，终于搞定了😆") + let service = PolishingService(store: store, client: emojiClient) + let result = try await service.polish( + "今天太开心了终于搞定了", + context: PolishContext() + ) + XCTAssertTrue(result.contains("😆"), "Allowed-emoji styles must keep model-added emoji. Got: \(result)") + XCTAssertTrue(result.contains("开心")) + } + + func testPolishServiceKeepsAddedEmojiWhenPromptOptsInWithoutToggle() async throws { + store.setEngineMode("local") + var catalog = PolishStyleCatalog() + let pack = PolishStylePack( + id: "user.paste-emoji", + name: "PasteEmoji", + prompt: "本风格允许新增 emoji。按情绪点缀合适表情。", + allowsAddedEmoji: false + ) + try catalog.upsert(pack) + store.setPolishStyleCatalog(catalog) + store.setActivePolishStyleId(pack.id) + + let emojiClient = FixedResponseLLMClient(response: "辛苦你了，真的谢谢🙏") + let service = PolishingService(store: store, client: emojiClient) + let result = try await service.polish( + "辛苦你了真的谢谢", + context: PolishContext() + ) + XCTAssertTrue( + result.contains("🙏"), + "Prompt opt-in must keep emoji even when toggle is off. Got: \(result)" + ) + } + func testPolishServiceFallsBackWhenOutputEmpty() async throws { store.setEngineMode("local") let emptyClient = FixedResponseLLMClient(response: " ") @@ -410,6 +458,30 @@ final class IntelligentPolishTests: XCTestCase { XCTAssertEqual(result, "好的") } + func testQualityGateKeepsAddedEmojiWhenAllowed() { + let decision = TranscriptPostProcessor.qualityGate( + original: "今天太开心了", + candidate: "今天太开心了😆", + allowsAddedEmoji: true + ) + guard case .accept(let text) = decision else { + return XCTFail("Expected accept") + } + XCTAssertTrue(text.contains("😆")) + } + + func testQualityGateStripsAddedEmojiByDefault() { + let decision = TranscriptPostProcessor.qualityGate( + original: "今天太开心了", + candidate: "今天太开心了😆", + allowsAddedEmoji: false + ) + guard case .accept(let text) = decision else { + return XCTFail("Expected accept") + } + XCTAssertFalse(text.contains("😆")) + } + func testQualityGateStripsResidualPauseMarkers() { let result = TranscriptPostProcessor.process( original: "第一段 ⟨0.8s⟩ 第二段", diff --git a/OSGKeyboardTests/PolishStylePackTests.swift b/OSGKeyboardTests/PolishStylePackTests.swift index 56dca0d..2d3ee90 100644 --- a/OSGKeyboardTests/PolishStylePackTests.swift +++ b/OSGKeyboardTests/PolishStylePackTests.swift @@ -206,6 +206,90 @@ final class PolishStylePackTests: XCTestCase { XCTAssertTrue(prompt.contains("词典命中优先于同音猜测")) XCTAssertTrue(prompt.contains("风格接入（纠错之后）")) XCTAssertFalse(prompt.contains("原始内容")) + XCTAssertFalse(prompt.contains("Emoji 覆盖")) + } + + func testComposerInjectsEmojiOverrideWhenStyleAllows() { + let style = PolishStylePack( + id: "user.emoji", + name: "Emoji", + prompt: "ROLE", + allowsAddedEmoji: true + ) + let prompt = PolishPromptComposer.compose( + text: "今天太开心了", + style: style, + context: PolishContext(), + dictionaryBlock: "", + useChineseGuidance: true + ) + XCTAssertTrue(prompt.contains("Emoji 覆盖（本风格开启 · 最终优先级）")) + XCTAssertTrue(prompt.contains("优先级高于全局 R5")) + // Override must appear after core R5 so it wins. + let r5 = prompt.range(of: "R5 不新增 emoji") + let override = prompt.range(of: "Emoji 覆盖（本风格开启 · 最终优先级）") + XCTAssertNotNil(r5) + XCTAssertNotNil(override) + XCTAssertLessThan(r5!.lowerBound, override!.lowerBound) + } + + func testPromptOptInEnablesEmojiWithoutToggle() async throws { + let style = PolishStylePack( + id: "user.paste", + name: "Paste", + prompt: """ + # 最高优先级覆盖 + 本风格允许新增 emoji。当与全局「不新增 emoji」规则冲突时，以本风格为准。 + """, + allowsAddedEmoji: false + ) + XCTAssertTrue(style.effectiveAllowsAddedEmoji) + + let composed = PolishPromptComposer.compose( + text: "今天太开心了", + style: style, + context: PolishContext(), + dictionaryBlock: "", + useChineseGuidance: true + ) + XCTAssertTrue(composed.contains("Emoji 覆盖（本风格开启 · 最终优先级）")) + } + + func testLegacyPackDecodeDefaultsAllowsAddedEmojiToFalse() throws { + let encoder = JSONEncoder() + encoder.dateEncodingStrategy = .iso8601 + // Encode without the new key by decoding a minimal legacy payload. + let legacyJSON = """ + { + "id": "user.legacy", + "name": "Legacy", + "prompt": "ROLE", + "kind": "user", + "createdAt": 0, + "updatedAt": 0 + } + """ + let decoder = JSONDecoder() + decoder.dateDecodingStrategy = .secondsSince1970 + let pack = try decoder.decode(PolishStylePack.self, from: Data(legacyJSON.utf8)) + XCTAssertFalse(pack.allowsAddedEmoji) + } + + func testUpsertPreservesAllowsAddedEmoji() throws { + var catalog = PolishStyleCatalog() + let pack = PolishStylePack( + name: "Emoji", + prompt: "ROLE", + allowsAddedEmoji: true + ) + try catalog.upsert(pack) + XCTAssertEqual(catalog.entries.first?.allowsAddedEmoji, true) + + var updated = pack + updated.prompt = "ROLE 2" + try catalog.upsert(updated) + XCTAssertEqual(catalog.entries.first?.allowsAddedEmoji, true) + XCTAssertEqual(catalog.entries.first?.prompt, "ROLE 2") } func testComposerKeepsHomophoneRepairWhenDictionaryPresent() { diff --git a/docs/keyboard-memory-budget.md b/docs/keyboard-memory-budget.md index 122a677..a943ed4 100644 --- a/docs/keyboard-memory-budget.md +++ b/docs/keyboard-memory-budget.md @@ -13,7 +13,10 @@ wiring; jetsam behavior is device-only. - **Do not** stack CLM + Rime + ASR in the same second after onboarding. - ASR warmup runs on **first mic press** (`beginUtterance`), gated by `HostMemoryBudget` (~260 MB RSS). `hostHeavy` is set only while heavy work - actually runs, then cleared. + actually runs, then cleared. A sticky `hostHeavy` (host died mid-work) + expires after `hostHeavyMaxAge` (~120 s) and is cleared on + `clearFlowState` / host-launch reconciliation so typing 中文/EN is not + permanently blocked. ## Console checklist diff --git a/project.yml b/project.yml index 057edc7..52850b0 100644 --- a/project.yml +++ b/project.yml @@ -51,8 +51,8 @@ settings: ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES STRING_CATALOG_GENERATE_SYMBOLS: YES CLANG_CXX_LANGUAGE_STANDARD: c++17 - MARKETING_VERSION: "1.6.1" - CURRENT_PROJECT_VERSION: "45" + MARKETING_VERSION: "1.6.2" + CURRENT_PROJECT_VERSION: "47" # 签名配置来自 Signing.local.xcconfig（gitignored，不会被覆盖） # 项目级签名 xcconfig，适用于所有 target