From 631617ee53863253ca80e4c9798857b801d5a27c Mon Sep 17 00:00:00 2001 From: Rocky <72559939+hkgood@users.noreply.github.com> Date: Thu, 13 Aug 2026 15:47:32 +0800 Subject: [PATCH] feat(keyboard): add clipboard AI skills, hint keywords, and voice session fixes Idle chips show entities with category icons; a fresh copy surfaces Reply/Summarize/Translate; abort/cancel/empty-tap no longer leave the mic stuck. --- AGENTS.md | 2 +- CHANGELOG.md | 22 ++ .../Services/AIHintRefreshService.swift | 5 +- OSGKeyboard/Services/FlowSessionManager.swift | 180 +++++++++---- .../Views/ClipboardHistoryDemoView.swift | 2 +- .../Views/Components/MinimalTabBar.swift | 96 +++---- .../Views/Components/TabBarVisibility.swift | 16 +- OSGKeyboard/Views/HomeView.swift | 8 +- OSGKeyboard/Views/MainTabView.swift | 2 +- OSGKeyboard/en.lproj/Localizable.strings | 2 +- OSGKeyboard/zh-Hans.lproj/Localizable.strings | 2 +- OSGKeyboardExt/KeyboardViewController.swift | 9 + .../Services/AIKeyboardCoordinator.swift | 48 +++- .../Services/KeyboardFlowCoordinator.swift | 170 +++++++++---- OSGKeyboardExt/Views/AIKeyboardView.swift | 110 ++++++-- OSGKeyboardExt/Views/KeyboardRootView.swift | 7 +- .../Views/KeyboardTopControls.swift | 13 +- .../Views/ToolbarActionButtons.swift | 42 +++- OSGKeyboardExt/en.lproj/Keyboard.strings | 3 + OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings | 3 + OSGKeyboardExtTests/KeyboardStateTests.swift | 5 + .../Services/FlowContinuousCapture.swift | 6 + OSGKeyboardShared/Models/AIHintModels.swift | 50 +++- .../Models/AIHintVisualKind.swift | 58 +++++ OSGKeyboardShared/Models/AISessionState.swift | 2 +- .../Models/TranslationLanguage.swift | 39 +++ .../Services/AIClipboardPrompt.swift | 5 + .../Services/AIClipboardSkill.swift | 129 ++++++++++ .../Services/AIHintKeywordCompressor.swift | 92 +++---- .../Services/AIHintKeywordExtractor.swift | 162 ++++++++++++ .../Services/AIHintLocalCatalog.swift | 18 +- OSGKeyboardShared/Services/AIHintPool.swift | 35 ++- .../Services/FlowKeyboardPolicies.swift | 70 ++++++ .../Services/FlowSessionKeys.swift | 2 + .../Services/KeyboardState.swift | 9 +- .../Utilities/FlowUtterancePCMStore.swift | 5 + OSGKeyboardTests/AIClipboardPromptTests.swift | 9 + .../AIHintKeywordExtractorTests.swift | 238 ++++++++++++++++++ OSGKeyboardTests/AIHintPoolTests.swift | 20 +- .../FlowKeyboardPoliciesTests.swift | 209 +++++++++++++++ .../FlowUtterancePCMStoreTests.swift | 8 + README.en.md | 2 +- README.md | 2 +- docs/APPSTORE_METADATA.md | 35 +-- docs/index.html | 8 +- docs/llms.txt | 2 +- docs/osgkeyboardversion.html | 6 +- project.yml | 4 +- 48 files changed, 1627 insertions(+), 345 deletions(-) create mode 100644 OSGKeyboardShared/Models/AIHintVisualKind.swift create mode 100644 OSGKeyboardShared/Services/AIClipboardSkill.swift create mode 100644 OSGKeyboardShared/Services/AIHintKeywordExtractor.swift create mode 100644 OSGKeyboardTests/AIHintKeywordExtractorTests.swift diff --git a/AGENTS.md b/AGENTS.md index 293dc00..b9db30d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ writing commit messages that will ship to users. ### Version format -The current source-of-truth version is **1.7.0 (build 65)**. Releases use stable SemVer: +The current source-of-truth version is **1.7.5 (build 67)**. Releases use stable SemVer: | Field | File | Rule | |-------|------|------| diff --git a/CHANGELOG.md b/CHANGELOG.md index 8212011..36062f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed +- **AI idle hint keywords**: chips show the entity (from feed `metadata.title` / city / holiday name) instead of a long sentence; category prefixes like「全网热点：」are dropped, and LLM compression is only a last resort. / **AI 空闲建议关键词**：芯片展示实体（来自 feed 的 `metadata.title` / 城市 / 节日名）而不再是长句；去掉「全网热点：」一类前缀，LLM 压缩仅作兜底。 +- **AI idle hint chrome**: each rotating suggestion sits in a Liquid Glass capsule with a category SF Symbol (calendar, weather, news, stocks, trending, search). / **AI 空闲建议样式**：轮播建议放入 Liquid Glass 胶囊，左侧为类型 SF Symbol（日历、天气、新闻、股票、热搜、搜索）。 +- **Clipboard AI skills**: within 30 seconds of a copy, AI idle shows compact circular Reply / Summarize / Translate buttons; translate follows the keyboard language setting, or Chinese ↔ English when unset. The skill list is catalog-based so more actions can be added later. / **剪贴板 AI 技能**：复制后约 30 秒内，空闲态改为小圆形「回复 / 总结 / 翻译」按钮；翻译跟随键盘目标语言，未设置时中英互译。技能来自目录，便于日后扩展。 +- **Clipboard Translate label**: the Translate chip shows the live direction — `中译英` / `To JP` when a target is set, `中↔英` / `CN↔EN` when unset, and `简↔繁` when Chinese UI targets 简体 or 繁體. / **剪贴板翻译文案**：翻译按钮按当前设置显示方向——已选目标为「中译英」/「To JP」，未设置为「中↔英」/「CN↔EN」，中文界面目标为简繁时为「简↔繁」。 + +- **AI idle capsule size**: rotating suggestion chips keep a 44 pt tap height with 16 pt side padding. / **AI 空闲胶囊尺寸**：轮播建议芯片保持 44 pt 点击高度，左右各 16 pt 内边距。 +- **Undo / translation chrome**: voice mic-row undo and translation controls are 52 pt circular Liquid Glass buttons. / **撤销与翻译按钮**：语音麦克风行的撤销、翻译改为 52 pt 圆形 Liquid Glass 按钮。 +- **Clipboard skill circles**: Reply / Summarize / Translate idle buttons match the same 52 pt circle. / **剪贴板技能圆钮**：回复 / 总结 / 翻译空闲按钮与语音侧键同为 52 pt 圆。 +- **Home tab selection**: the dock stays Liquid Glass at its original height; the selected tab is a wider green fill capsule with a 5 pt inset, not a second glass chip. Dock items are 24 pt icons with Home / Styles / Settings labels. / **首页 Tab 选中态**：dock 仍是原高度 Liquid Glass；选中项改为更宽的绿色填充胶囊，距栏边 5 pt，不再套第二层玻璃。dock 为 24 pt 图标加「首页 / 风格 / 设置」文字。 +- **Home library card titles**: History and Personal dictionary headers use a 16 pt icon and 13 pt label. / **首页资料卡标题**：历史与个性词库标题改为 16 pt 图标、13 pt 文字。 + +### Fixed +- **Clipboard skills jumped to a sentence chip**: copying no longer swaps the three skill buttons for a leftover carousel card such as “translate the clipboard”. Clipboard sentence cards stay out of the idle rotation. / **剪贴板技能跳成句子芯片**：复制后不再把三个技能按钮换成「把剪贴板翻译成…」这类旧轮播卡片；剪贴板句子不再进入空闲轮播。 +- **Clipboard skill status XML**: tapping Reply / Summarize / Translate no longer flashes the internal `` envelope above the mic. / **剪贴板技能状态 XML**：点回复 / 总结 / 翻译不再把内部 `` 信封闪现在麦克风上方。 +- **Idle keyboard re-adopted recognition**: aborting a session now remembers that utterance id, so a lagging host `processing` snapshot cannot reopen the voice keyboard in 「识别中」. / **空闲打开误进识别中**：中止会话会记住该句 id，滞后的主机 processing 快照不会在下次打开语音键盘时显示「识别中」。 +- **Voice cancel chrome until result**: Cancel (X) stays up through ASR/polish and through abort wait after leaving AI Agent, so the voice mic no longer looks ready while the host is still busy. / **语音取消直到出结果**：X 一直保留到识别/润色结束，以及离开 AI Agent 后的中断收尾；语音麦克风不再在宿主仍忙时显示为可用。 +- **Empty double-tap skip**: a mic press shorter than 300 ms with near-silence (or no samples) is discarded before ASR, so accidental double taps no longer wait on a no-speech error. / **空连点跳过**：按下短于 300ms 且接近静音（或尚无样本）的录音在进入识别前丢弃，避免误触后长时间等待「没听清」。 +- **AI hint left the mic stuck**: prefilled AI questions now drop the host processing gate after the answer or error; consuming an ack for the live utterance also heals a leaked gate. Reopening the keyboard only aborts when App Group already acked that utterance and the result is gone — a live LLM/ASR wait with no result yet is left running. Cancel during generate no longer delivers the answer afterwards. / **AI 建议卡住麦克风**：预填 AI 问题在出答案或失败后会关掉宿主 processing 闸门；若键盘已 ack 当前句而闸门仍开着，宿主消费 ack 时一并放闸。再次打开键盘仅在 App Group 已 ack 且结果已空时才中断残留闸门；LLM/ASR 仍在跑、尚无结果时继续等待。生成中取消后不再把答案写回来。 + +## [1.7.5] - 2026-08-13 + ### Added - **AI idle hint carousel**: AI mode empty state rotates one-line suggestions (local evergreen + optional remote hot topics); tap sends the card prompt to the LLM without the mic. Coexists with the clipboard suggestion strip in the top bar. / **AI 空闲建议轮播**：AI 模式空状态轮播单行建议（内置常驻 + 可选远程热点）；点按即跳过麦克风把卡片 prompt 发给模型。与顶栏剪贴板建议条并存。 - **Hint feed refresh**: main app silently fetches `https://key.osglab.com/hints` about every 12 hours, compresses titles with the user’s polish LLM, and writes App Group ready packs for the keyboard. / **建议源刷新**：主 App 约每 12 小时静默拉取 `https://key.osglab.com/hints`，用用户润色 LLM 压缩标题，写入 App Group 供键盘只读。 diff --git a/OSGKeyboard/Services/AIHintRefreshService.swift b/OSGKeyboard/Services/AIHintRefreshService.swift index 44eff84..a3830a1 100644 --- a/OSGKeyboard/Services/AIHintRefreshService.swift +++ b/OSGKeyboard/Services/AIHintRefreshService.swift @@ -1,8 +1,9 @@ // AIHintRefreshService.swift // OSGKeyboard · Main App // -// Silent 12h refresh: fetch remote packs, compress titles with polish LLM, -// merge local evergreen cards, write App Group ready packs for the keyboard. + // Silent 12h refresh: fetch remote packs, extract keywords (LLM only + // for leftovers), merge local evergreen cards, write App Group ready + // packs for the keyboard. import Foundation import OSGKeyboardShared diff --git a/OSGKeyboard/Services/FlowSessionManager.swift b/OSGKeyboard/Services/FlowSessionManager.swift index 6f4d624..8a0eca9 100644 --- a/OSGKeyboard/Services/FlowSessionManager.swift +++ b/OSGKeyboard/Services/FlowSessionManager.swift @@ -1069,8 +1069,20 @@ final class FlowSessionManager: ObservableObject { FlowSessionBridge.clearResult() // Ack clears the terminal result; republish ready immediately so the // keyboard does not linger on awaitingDelivery / starting between the - // 500 ms poll and the next heartbeat. - refreshHostReady() + // 500 ms poll and the next heartbeat. If processing is still true for + // this same utterance, the writer forgot to drop the gate — heal here. + if FlowHostAckGatePolicy.shouldDropProcessingGate( + ackUtteranceId: ack.utteranceId, + currentUtteranceId: currentUtteranceId, + isUtteranceProcessing: isUtteranceProcessing + ) { + completeFinalizeCleanup( + sessionId: ack.sessionId, + utteranceId: ack.utteranceId + ) + } else { + refreshHostReady() + } } private func hasUnacknowledgedTerminalResult() -> Bool { @@ -1280,6 +1292,10 @@ final class FlowSessionManager: ObservableObject { case .cancelPrimeAudio: cancelAudioPrime(command) case .endAIConversation: + if currentUtteranceMode == .aiQuestion, + (isUtteranceProcessing || isUtteranceRecording) { + abortUtterance() + } if let conversationID = command.aiConversationID { Task { await aiConversations.removeConversation(conversationID) } } @@ -1371,25 +1387,30 @@ final class FlowSessionManager: ObservableObject { utteranceId: UUID, commandSeq: Int64 ) async { - guard let conversationID else { - guard claimTerminal(utteranceId: utteranceId) else { return } - storeFinalizedError( - AppL10n.string("flow.error.aiQuestionFailed"), - kind: .generic, + // Hint-card / prefilled questions never go through `finalizeUtterance`, + // so this path must drop the processing gate itself. Leaving it set + // keeps `reason=processing` forever and the keyboard mic unclickable. + defer { + completeFinalizeCleanup( sessionId: sessionId, - utteranceId: utteranceId, - commandSeq: commandSeq + utteranceId: utteranceId ) + } + guard let conversationID else { + claimAndStoreTerminal(utteranceId: utteranceId) { + storeFinalizedError( + AppL10n.string("flow.error.aiQuestionFailed"), + kind: .generic, + sessionId: sessionId, + utteranceId: utteranceId, + commandSeq: commandSeq + ) + } return } - storeRawCandidate( - question, - sessionId: sessionId, - utteranceId: utteranceId, - commandSeq: commandSeq - ) - + // Do not publish the prompt as a "transcript" — clipboard skills + // send an XML envelope that must never appear above the mic. let pipelineStore = AppGroupStore() do { let service = try AIQuestionService.configured( @@ -1413,6 +1434,8 @@ final class FlowSessionManager: ObservableObject { ) } } + // Abort during the LLM await must not commit or deliver. + guard canStoreTerminal(for: utteranceId) else { return } publishStreamingAIAnswerIfNeeded( answer, sessionId: sessionId, @@ -1421,30 +1444,32 @@ final class FlowSessionManager: ObservableObject { aiConversationID: conversationID, force: true ) - guard claimTerminal(utteranceId: utteranceId) else { return } await service.commitSuccessfulTurn( question: question, answer: answer, conversationID: conversationID ) - storeFinalizedResult( - answer, - warning: nil, - sessionId: sessionId, - utteranceId: utteranceId, - commandSeq: commandSeq, - aiConversationID: conversationID - ) + claimAndStoreTerminal(utteranceId: utteranceId) { + storeFinalizedResult( + answer, + warning: nil, + sessionId: sessionId, + utteranceId: utteranceId, + commandSeq: commandSeq, + aiConversationID: conversationID + ) + } } catch { - guard claimTerminal(utteranceId: utteranceId) else { return } - storeFinalizedError( - AppL10n.string("flow.error.aiQuestionFailed"), - kind: .generic, - sessionId: sessionId, - utteranceId: utteranceId, - commandSeq: commandSeq, - aiConversationID: conversationID - ) + claimAndStoreTerminal(utteranceId: utteranceId) { + storeFinalizedError( + AppL10n.string("flow.error.aiQuestionFailed"), + kind: .generic, + sessionId: sessionId, + utteranceId: utteranceId, + commandSeq: commandSeq, + aiConversationID: conversationID + ) + } } } @@ -1993,6 +2018,26 @@ final class FlowSessionManager: ObservableObject { private func endUtterance() { guard isUtteranceRecording else { return } + let duration = utteranceRecordingStartedAt.map { Date().timeIntervalSince($0) } ?? 0 + if currentUtteranceMode != .editLastInput, + duration < FlowEmptyTapSkipPolicy.maxDurationSeconds { + let samples = capture.utterancePCMSnapshot() + let peak = FlowEmptyTapSkipPolicy.peakAbs(samples) + if FlowEmptyTapSkipPolicy.shouldSkip( + durationSeconds: duration, + sampleCount: samples.count, + peakAmplitude: peak + ) { + FlowTrace.pipeline( + "utterance.skipEmptyTap", + "duration=\(String(format: "%.3f", duration))s " + + "samples=\(samples.count) peak=\(String(format: "%.4f", peak))" + ) + abortUtterance(kind: .discardedEmpty, message: "") + return + } + } + // Close the mic gate first, then mark processing before dropping the // recording flag so the poll loop cannot start a second utterance. isUtteranceRecording = false @@ -2040,12 +2085,15 @@ final class FlowSessionManager: ObservableObject { debug("utterance stopped, draining tail") } - private func abortUtterance() { + private func abortUtterance( + kind: FlowSessionKeys.TranscriptionErrorKind = .recognitionInterrupted, + message: String? = nil + ) { let utteranceId = currentUtteranceId if claimTerminal(utteranceId: utteranceId) { storeCurrentError( - AppL10n.string("flow.error.recognitionInterrupted"), - kind: .recognitionInterrupted, + message ?? AppL10n.string("flow.error.recognitionInterrupted"), + kind: kind, status: .aborted ) } @@ -2125,6 +2173,24 @@ final class FlowSessionManager: ObservableObject { return true } + /// After an `await`, refuse to write if this utterance was aborted or replaced. + private func canStoreTerminal(for utteranceId: UUID?) -> Bool { + guard let utteranceId else { return false } + return FlowTerminalStorePolicy.canStore( + currentUtteranceId: currentUtteranceId, + finishedUtteranceId: utteranceId, + alreadyTerminal: terminalUtteranceIds.contains(utteranceId) + ) + } + + /// Claim + write with no `await` between them, so abort cannot sneak a + /// second payload in after `claimTerminal`. + private func claimAndStoreTerminal(utteranceId: UUID?, store: () -> Void) { + guard canStoreTerminal(for: utteranceId) else { return } + guard claimTerminal(utteranceId: utteranceId) else { return } + store() + } + private func clearPendingInstructionState() { pendingEditSourceText = nil pendingSourceHistoryEntryID = nil @@ -2358,6 +2424,8 @@ final class FlowSessionManager: ObservableObject { ) } } + // Abort during the LLM await must not commit or deliver. + guard canStoreTerminal(for: finalizeUtteranceId) else { return } // Flush the final draft when throttle skipped the last characters. publishStreamingAIAnswerIfNeeded( answer, @@ -2367,30 +2435,32 @@ final class FlowSessionManager: ObservableObject { aiConversationID: aiConversationID, force: true ) - guard claimTerminal(utteranceId: finalizeUtteranceId) else { return } await service.commitSuccessfulTurn( question: question, answer: answer, conversationID: aiConversationID ) - storeFinalizedResult( - answer, - warning: chunkNote, - sessionId: finalizeSessionId, - utteranceId: finalizeUtteranceId, - commandSeq: finalizeCommandSeq, - aiConversationID: aiConversationID - ) + claimAndStoreTerminal(utteranceId: finalizeUtteranceId) { + storeFinalizedResult( + answer, + warning: chunkNote, + sessionId: finalizeSessionId, + utteranceId: finalizeUtteranceId, + commandSeq: finalizeCommandSeq, + aiConversationID: aiConversationID + ) + } } catch { - guard claimTerminal(utteranceId: finalizeUtteranceId) else { return } - storeFinalizedError( - AppL10n.string("flow.error.aiQuestionFailed"), - kind: .generic, - sessionId: finalizeSessionId, - utteranceId: finalizeUtteranceId, - commandSeq: finalizeCommandSeq, - aiConversationID: aiConversationID - ) + claimAndStoreTerminal(utteranceId: finalizeUtteranceId) { + storeFinalizedError( + AppL10n.string("flow.error.aiQuestionFailed"), + kind: .generic, + sessionId: finalizeSessionId, + utteranceId: finalizeUtteranceId, + commandSeq: finalizeCommandSeq, + aiConversationID: aiConversationID + ) + } } return } diff --git a/OSGKeyboard/Views/ClipboardHistoryDemoView.swift b/OSGKeyboard/Views/ClipboardHistoryDemoView.swift index 1fa6156..4740bb3 100644 --- a/OSGKeyboard/Views/ClipboardHistoryDemoView.swift +++ b/OSGKeyboard/Views/ClipboardHistoryDemoView.swift @@ -13,7 +13,7 @@ import OSGKeyboardShared struct ClipboardHistoryDemoView: View { private enum Layout { static let micSize: CGFloat = 121 - static let undoSize: CGFloat = 44 + static let undoSize: CGFloat = 52 static let micToButtonGap: CGFloat = 8 static let actionClusterTopGap: CGFloat = Spacing.xl static let micUpwardAdjustment: CGFloat = diff --git a/OSGKeyboard/Views/Components/MinimalTabBar.swift b/OSGKeyboard/Views/Components/MinimalTabBar.swift index 105ba13..91ce86e 100644 --- a/OSGKeyboard/Views/Components/MinimalTabBar.swift +++ b/OSGKeyboard/Views/Components/MinimalTabBar.swift @@ -1,9 +1,9 @@ // MinimalTabBar.swift // OSGKeyboard · Main App // -// Bottom tab bar — three icons, no labels. -// Capsule uses iOS 26 Liquid Glass (.regular.interactive) so content -// behind the dock refracts through on scroll. +// Bottom tab bar — icon + label for Home / Styles / Settings. +// The dock capsule is iOS 26 Liquid Glass; the selected tab is a green +// fill inside that capsule (Photos-style), not a second glass layer. // History + dictionary live as Home cards (not dock tabs). import SwiftUI @@ -53,66 +53,66 @@ enum AppTab: Int, CaseIterable { struct MinimalTabBar: View { @Environment(\.themePalette) private var palette: ThemePalette @Environment(\.colorScheme) private var colorScheme - @Namespace private var selectionGlassNamespace + @Namespace private var selectionNamespace @Binding var selection: AppTab var body: some View { - GlassEffectContainer(spacing: 0) { - HStack(spacing: 0) { - ForEach(AppTab.allCases, id: \.rawValue) { tab in - Button { - withAnimation(Motion.soft) { - selection = tab - } - } label: { + HStack(spacing: 0) { + ForEach(AppTab.allCases, id: \.rawValue) { tab in + Button { + withAnimation(Motion.soft) { + selection = tab + } + } label: { + VStack(spacing: 2) { Group { if let sfSymbol = tab.sfSymbol { Image(systemName: sfSymbol) - .font(.system(size: 20, weight: .regular)) + .font(.system(size: TabBarDockMetrics.iconSize, weight: .regular)) } else { - MaterialIcon(name: tab.icon, size: 24) + MaterialIcon(name: tab.icon, size: TabBarDockMetrics.iconSize) } } - .foregroundStyle(tabIconColor(for: tab)) - .frame(maxWidth: .infinity) - .frame(height: 48) - .background { - if selection == tab { - // Capsule, not circle: a circle would inscribe to the - // smaller edge and leave the tab slot looking empty. - Color.clear - .frame(width: 52, height: 44) - .glassEffect( - .regular - .tint(palette.accent.opacity(0.18)) - .interactive(), - in: .capsule - ) - .glassEffectID( - "main-tab-selection", - in: selectionGlassNamespace - ) - .glassEffectTransition(.matchedGeometry) - .matchedGeometryEffect( - id: "main-tab-selection", - in: selectionGlassNamespace - ) - } - } - .contentShape(Rectangle()) + Text(tab.accessibilityKey) + .font(TypeStyle.caption2) + .lineLimit(1) + .minimumScaleFactor(0.8) } - .buttonStyle(.plain) - .accessibilityLabel(tab.accessibilityKey) - .accessibilityAddTraits(selection == tab ? .isSelected : []) + .foregroundStyle(tabIconColor(for: tab)) + .frame(maxWidth: .infinity) + .frame(height: TabBarDockMetrics.itemHeight) + .background { + if selection == tab { + // Stretch into the dock padding so top/bottom + // leftover matches the side leftover (~5 pt). + Capsule() + .fill(palette.accentMuted) + .frame(maxWidth: .infinity, maxHeight: .infinity) + .padding(.horizontal, TabBarDockMetrics.selectionInset) + .padding( + .vertical, + TabBarDockMetrics.selectionInset + - TabBarDockMetrics.dockInsetVertical + ) + .matchedGeometryEffect( + id: "main-tab-selection", + in: selectionNamespace + ) + } + } + .contentShape(Rectangle()) } + .buttonStyle(.plain) + .accessibilityLabel(tab.accessibilityKey) + .accessibilityAddTraits(selection == tab ? .isSelected : []) } - .padding(.horizontal, Spacing.md) - .padding(.vertical, Spacing.sm) - .glassEffect(.regular.interactive(), in: .capsule) } + .padding(.horizontal, TabBarDockMetrics.dockInsetHorizontal) + .padding(.vertical, TabBarDockMetrics.dockInsetVertical) + .glassEffect(.regular.interactive(), in: .capsule) .frame(maxWidth: 280) .frame(maxWidth: .infinity, alignment: .center) - .padding(.bottom, Spacing.xs) + .padding(.bottom, TabBarDockMetrics.bottomPadding) } private func tabIconColor(for tab: AppTab) -> Color { diff --git a/OSGKeyboard/Views/Components/TabBarVisibility.swift b/OSGKeyboard/Views/Components/TabBarVisibility.swift index 3deab01..4b3cb16 100644 --- a/OSGKeyboard/Views/Components/TabBarVisibility.swift +++ b/OSGKeyboard/Views/Components/TabBarVisibility.swift @@ -52,8 +52,20 @@ extension View { } enum TabBarDockMetrics { - /// Clearance above the floating dock (icon row + vertical padding + home indicator). - static let scrollClearance: CGFloat = 100 + static let itemHeight: CGFloat = 52 + static let iconSize: CGFloat = 24 + /// Horizontal glass pad is 0: the selected capsule's `selectionInset` + /// is the only side gap. `Spacing.md` (16) plus that 5 pt was ~3–4× + /// the 5 pt top/bottom gap. + static let dockInsetHorizontal: CGFloat = 0 + static let dockInsetVertical: CGFloat = Spacing.sm + /// Gap between the selected fill and the glass dock / neighbouring tabs. + static let selectionInset: CGFloat = 5 + static let bottomPadding: CGFloat = Spacing.xs + /// Clearance above the floating dock (bar + home-indicator slack). + static var scrollClearance: CGFloat { + itemHeight + dockInsetVertical * 2 + bottomPadding + 20 + } } private struct TabBarScrollBottomPaddingModifier: ViewModifier { diff --git a/OSGKeyboard/Views/HomeView.swift b/OSGKeyboard/Views/HomeView.swift index 3e37448..9040fe7 100644 --- a/OSGKeyboard/Views/HomeView.swift +++ b/OSGKeyboard/Views/HomeView.swift @@ -205,8 +205,8 @@ struct HomeView: View { // MARK: - History / dictionary cards - /// Two independent cards; each header mirrors the stats tiles (accent icon - /// + small uppercase label) and the body grows with its rows. + /// Two independent cards; each header is an accent icon + uppercase label + /// and the body grows with its rows. private var homeLibrarySection: some View { VStack(spacing: Spacing.md) { historyCard @@ -290,11 +290,11 @@ struct HomeView: View { VStack(alignment: .leading, spacing: Spacing.xs) { HStack(spacing: Spacing.xs) { Image(systemName: systemImage) - .font(.system(size: 13, weight: .semibold)) + .font(.system(size: 16, weight: .semibold)) .foregroundStyle(palette.accent) .symbolRenderingMode(.hierarchical) Text(titleKey) - .font(TypeStyle.caption2) + .font(.system(size: 13, weight: .semibold)) .tracking(0.6) .textCase(.uppercase) .foregroundStyle(palette.textTertiary) diff --git a/OSGKeyboard/Views/MainTabView.swift b/OSGKeyboard/Views/MainTabView.swift index 262c4af..13a0981 100644 --- a/OSGKeyboard/Views/MainTabView.swift +++ b/OSGKeyboard/Views/MainTabView.swift @@ -44,7 +44,7 @@ struct MainTabView: View { if !isTabBarHidden { // Match floating dock + home-indicator clearance so // page footers / scroll ends sit above MinimalTabBar. - Color.clear.frame(height: 100) + Color.clear.frame(height: TabBarDockMetrics.scrollClearance) } } .onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in diff --git a/OSGKeyboard/en.lproj/Localizable.strings b/OSGKeyboard/en.lproj/Localizable.strings index 7e964c5..df445f2 100644 --- a/OSGKeyboard/en.lproj/Localizable.strings +++ b/OSGKeyboard/en.lproj/Localizable.strings @@ -423,7 +423,7 @@ "home.engine.attentionBanner" = "Local ASR model is not ready yet. Download it and wait for warm-up before dictating."; /* Tabs */ -"tab.keyboard" = "Keyboard"; +"tab.keyboard" = "Home"; "tab.history" = "History"; "tab.dictionary" = "Dictionary"; "tab.styles" = "Styles"; diff --git a/OSGKeyboard/zh-Hans.lproj/Localizable.strings b/OSGKeyboard/zh-Hans.lproj/Localizable.strings index 097240f..f41e7da 100644 --- a/OSGKeyboard/zh-Hans.lproj/Localizable.strings +++ b/OSGKeyboard/zh-Hans.lproj/Localizable.strings @@ -422,7 +422,7 @@ "home.engine.attentionBanner" = "本地语音识别模型尚未就绪，请先下载并等待加载完成后再使用语音输入。"; /* Tabs */ -"tab.keyboard" = "键盘"; +"tab.keyboard" = "首页"; "tab.history" = "历史"; "tab.dictionary" = "词库"; "tab.styles" = "风格"; diff --git a/OSGKeyboardExt/KeyboardViewController.swift b/OSGKeyboardExt/KeyboardViewController.swift index f1c697c..42f608b 100644 --- a/OSGKeyboardExt/KeyboardViewController.swift +++ b/OSGKeyboardExt/KeyboardViewController.swift @@ -446,6 +446,9 @@ public final class KeyboardViewController: UIInputViewController { flowCoordinator.onAIRecognitionStarted = { [weak self] utteranceID in self?.aiKeyboardCoordinator.recognitionStarted(utteranceID) } + flowCoordinator.onAIGeneratingStarted = { [weak self] utteranceID in + self?.aiKeyboardCoordinator.generatingStarted(utteranceID) + } flowCoordinator.onAITranscript = { [weak self] transcript, utteranceID, status in self?.aiKeyboardCoordinator.receiveTranscript( transcript, @@ -465,6 +468,9 @@ public final class KeyboardViewController: UIInputViewController { flowCoordinator.onAIFailure = { [weak self] message, utteranceID in self?.aiKeyboardCoordinator.fail(message, utteranceID: utteranceID) } + flowCoordinator.onAICancelled = { [weak self] in + self?.state.aiSession.cancelCurrentWork() + } _ = textInserter.recoverPendingEditTransactionIfNeeded() cursorDrag = CursorDragController( @@ -511,6 +517,9 @@ public final class KeyboardViewController: UIInputViewController { state.submitAIHint = { [weak self] card in self?.aiKeyboardCoordinator.submitHintCard(card) } + state.submitAIClipboardSkill = { [weak self] skill in + self?.aiKeyboardCoordinator.submitClipboardSkill(skill) + } state.openSettings = { [weak self] in self?.openHostApp() } state.openInputMethodSetup = { [weak self] in self?.openHostApp(path: "deployrime") } state.openClipboardSettings = { [weak self] in diff --git a/OSGKeyboardExt/Services/AIKeyboardCoordinator.swift b/OSGKeyboardExt/Services/AIKeyboardCoordinator.swift index d4a25c9..ddaa662 100644 --- a/OSGKeyboardExt/Services/AIKeyboardCoordinator.swift +++ b/OSGKeyboardExt/Services/AIKeyboardCoordinator.swift @@ -66,20 +66,44 @@ final class AIKeyboardCoordinator { } } + /// Tap a clipboard skill chip: same fail-closed material path as hint cards. + func submitClipboardSkill(_ skill: AIClipboardSkill) { + guard canAcceptIdleSubmit else { return } + enterIfNeeded() + let instruction = AIClipboardSkillCatalog.instruction( + for: skill, + locale: AIHintLocaleResolver.packLocale(), + translationTargetLocaleId: state.translationTargetLocaleId + ) + let resolution = AIClipboardPrompt.resolve( + instruction: instruction, + material: ClipboardHistoryStore.shared.newestAIHintEligibleEntry()?.text + ) + submitResolvedPrompt(resolution) + } + /// Tap an idle hint card: resolve its material, skip the mic, ask the host. func submitHintCard(_ card: AIHintCard) { - switch state.aiSession.phase { - case .inactive, .idle, .failed: - break - case .preparing, .listening, .recognizing, .generating, - .ready, .awaitingSend, .inserted, .sent: - return - } + guard canAcceptIdleSubmit else { return } enterIfNeeded() let resolution = AIHintPool.resolvePrompt( for: card, clipboardText: ClipboardHistoryStore.shared.newestAIHintEligibleEntry()?.text ) + submitResolvedPrompt(resolution) + } + + private var canAcceptIdleSubmit: Bool { + switch state.aiSession.phase { + case .inactive, .idle, .failed: + return true + case .preparing, .listening, .recognizing, .generating, + .ready, .awaitingSend, .inserted, .sent: + return false + } + } + + private func submitResolvedPrompt(_ resolution: AIClipboardPrompt.Resolution) { guard case .ready(let prompt) = resolution else { // The clipboard window closed between rendering and this tap. state.aiSession.fail( @@ -132,11 +156,21 @@ final class AIKeyboardCoordinator { state.aiSession.beginRecognizing(utteranceID: utteranceID) } + func generatingStarted(_ utteranceID: UUID) { + state.aiSession.beginGenerating(question: "", utteranceID: utteranceID) + } + func receiveTranscript( _ transcript: String, utteranceID: UUID, status: FlowResult.Status ) { + if AIClipboardPrompt.isInternalPrompt(transcript) { + if status == .rawReady { + state.aiSession.beginGenerating(question: "", utteranceID: utteranceID) + } + return + } state.aiSession.updateTranscript(transcript, utteranceID: utteranceID) if status == .rawReady { state.aiSession.beginGenerating( diff --git a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift index 746b4d1..261305f 100644 --- a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift +++ b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift @@ -61,10 +61,12 @@ final class KeyboardFlowCoordinator { var onAIUtterancePrepared: (UUID) -> Void = { _ in } var onAIRecordingStarted: (UUID) -> Void = { _ in } var onAIRecognitionStarted: (UUID) -> Void = { _ in } + var onAIGeneratingStarted: (UUID) -> Void = { _ in } var onAITranscript: (String, UUID, FlowResult.Status) -> Void = { _, _, _ in } var onAIStreamingAnswer: (String, UUID) -> Void = { _, _ in } var onAIResult: (FlowResult) -> Void = { _ in } var onAIFailure: (String, UUID?) -> Void = { _, _ in } + var onAICancelled: () -> Void = {} /// Utterance whose final result we already inserted (or failed). Prevents /// `adoptHostBusyStateIfNeeded` from re-entering `.processing` after a /// stale App Group snapshot still says `reason=processing`. @@ -414,6 +416,29 @@ final class KeyboardFlowCoordinator { isFlowRecording = false stopUtteranceCountdown() ExtensionScreenWakeLock.release() + if FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing( + busyUtteranceId: busyId, + latestResult: FlowSessionBridge.latestResult(), + latestAck: FlowSessionBridge.latestAck() + ) { + // Already acked, result gone, host forgot to drop the gate. + // Abort unsticks the host. Do not await — claimTerminal already + // ran, so abort will not write a new result. Remember the id + // so this same refresh cannot re-adopt before the host poll. + writeCommand(.abort) + lastConsumedUtteranceId = busyId + lastStoppedUtteranceId = busyId + isAwaitingFlowResult = false + currentUtteranceId = nil + state.phase = .idle + state.lastTranscript = "" + traceState( + "adoptHostBusy.staleProcessingReleased", + extra: "utterance=\(busyId.uuidString.prefix(8))" + ) + return + } + // Missing result with no matching ack is live ASR/LLM — wait. state.phase = .processing if state.lastTranscript.isEmpty { state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") @@ -557,6 +582,12 @@ final class KeyboardFlowCoordinator { case .requestingPermissions: break case .idle, .denied, .error: + // A second tap while still waiting for the host must cancel the + // pending start instead of stacking another utterance. + if currentUtteranceRequest != nil { + cancelCurrentDictation() + return + } _ = startUtterance(.dictation) case .processing: break @@ -655,34 +686,8 @@ final class KeyboardFlowCoordinator { func cancelAIRecording() { guard currentUtteranceRequest?.isAIQuestion == true else { return } - recordWhenHostReady = false - recordAfterHandoff = false - isPendingFlowStart = false - flowStartDeadline = 0 - coldStartDebouncer.reset() - stopHostReadyWait() - stopFlowWatchdog() - stopUtteranceCountdown() - ExtensionScreenWakeLock.release() - state.level = 0 - state.phase = .idle - state.lastTranscript = "" - - guard let utteranceID = currentUtteranceId, - isFlowRecording || isAwaitingFlowResult else { - clearUnissuedUtterance() - isFlowRecording = false - isAwaitingFlowResult = false - recomputeMicVoiceAvailability() - return - } - - cancelledAIUtteranceIDs.insert(utteranceID) - writeCommand(.abort) - isFlowRecording = false - isAwaitingFlowResult = true - startFlowResultWatchdog() - recomputeMicVoiceAvailability() + prepareLocalCancel() + beginAwaitingAbort(tracking: &cancelledAIUtteranceIDs) } func endAIConversation(_ conversationID: UUID) { @@ -708,11 +713,27 @@ final class KeyboardFlowCoordinator { } func cancelCurrentDictation() { - guard currentUtteranceRequest?.isEdit != true, - currentUtteranceRequest?.isAIQuestion != true else { + if currentUtteranceRequest?.isAIQuestion == true { + cancelAIRecording() return } + guard currentUtteranceRequest?.isEdit != true else { return } + prepareLocalCancel() + let hadIssuedTransport = currentUtteranceId != nil + && (isFlowRecording || isAwaitingFlowResult) + beginAwaitingAbort(tracking: &cancelledDictationUtteranceIDs) + traceState( + "dictation.cancelled", + extra: hadIssuedTransport + ? "utterance=\(currentUtteranceId?.uuidString.prefix(8) ?? "none")" + : "transport=localOnly" + ) + } + + /// Stop local wait/prime timers before an abort. Does not change phase; + /// `beginAwaitingAbort` keeps `.processing` until the host acks. + private func prepareLocalCancel() { recordWhenHostReady = false recordAfterHandoff = false isPendingFlowStart = false @@ -723,29 +744,37 @@ final class KeyboardFlowCoordinator { stopUtteranceCountdown() ExtensionScreenWakeLock.release() state.level = 0 - state.phase = .idle state.lastTranscript = "" + } + /// Keep the cancel chrome (X + white mic) until the host finishes abort. + private func beginAwaitingAbort(tracking cancelledIDs: inout Set) { guard let utteranceID = currentUtteranceId, isFlowRecording || isAwaitingFlowResult else { - clearUnissuedUtterance() - isFlowRecording = false - isAwaitingFlowResult = false - recomputeMicVoiceAvailability() - traceState("dictation.cancelled", extra: "transport=localOnly") + finishLocalCancel() return } - - cancelledDictationUtteranceIDs.insert(utteranceID) + if cancelledIDs.contains(utteranceID), isAwaitingFlowResult { + state.phase = .processing + recomputeMicVoiceAvailability() + return + } + cancelledIDs.insert(utteranceID) writeCommand(.abort) isFlowRecording = false isAwaitingFlowResult = true + state.phase = .processing startFlowResultWatchdog() recomputeMicVoiceAvailability() - traceState( - "dictation.cancelled", - extra: "utterance=\(utteranceID.uuidString.prefix(8))" - ) + } + + private func finishLocalCancel() { + clearUnissuedUtterance() + isFlowRecording = false + isAwaitingFlowResult = false + state.phase = .idle + state.lastTranscript = "" + recomputeMicVoiceAvailability() } func abortEditRecording() { @@ -1085,9 +1114,11 @@ final class KeyboardFlowCoordinator { if isFlowRecording { writeCommand(.abort) ExtensionScreenWakeLock.release() + // Remember the aborted id so the next keyboard open cannot + // re-adopt a lagging host snapshot as 「识别中」. + lastStoppedUtteranceId = currentUtteranceId } currentUtteranceId = nil - lastStoppedUtteranceId = nil isFlowRecording = false isPendingFlowStart = false recordAfterHandoff = false @@ -1174,6 +1205,10 @@ final class KeyboardFlowCoordinator { consumeCancelledDictationResultIfNeeded(result) { return } + if let result = matchingResult(), + consumeDiscardedEmptyResultIfNeeded(result) { + return + } if let result = matchingResult(), consumeCancelledEditResultIfNeeded(result) { return @@ -1311,6 +1346,42 @@ final class KeyboardFlowCoordinator { return true } + private func consumeDiscardedEmptyResultIfNeeded(_ result: FlowResult) -> Bool { + guard result.errorKind == .discardedEmpty, + result.status == .aborted || isTerminalFailure(result) else { + return false + } + FlowSessionBridge.writeAck( + FlowAck( + sessionId: result.sessionId, + utteranceId: result.utteranceId, + commandSeq: result.commandSeq, + hostGeneration: result.hostGeneration, + revision: result.revision, + deliveryOutcome: .rejected + ) + ) + lastConsumedUtteranceId = result.utteranceId + lastStoppedUtteranceId = nil + stopUtteranceCountdown() + stopFlowWatchdog() + ExtensionScreenWakeLock.release() + let wasAI = currentUtteranceRequest?.isAIQuestion == true + resetEditTransportState() + state.level = 0 + state.phase = .idle + state.lastTranscript = "" + recomputeMicVoiceAvailability() + if wasAI { + onAICancelled() + } + traceState( + "utterance.discardedEmptyTap", + extra: "utterance=\(result.utteranceId.uuidString.prefix(8))" + ) + return true + } + private func consumeCancelledEditResultIfNeeded(_ result: FlowResult) -> Bool { guard cancelledEditUtteranceIDs.contains(result.utteranceId), result.status == .final || isTerminalFailure(result) else { @@ -1754,10 +1825,10 @@ final class KeyboardFlowCoordinator { writeSubmitAIQuestion(question) isFlowRecording = false isAwaitingFlowResult = true - state.lastTranscript = question + state.lastTranscript = "" state.phase = .processing if let currentUtteranceId { - onAIRecognitionStarted(currentUtteranceId) + onAIGeneratingStarted(currentUtteranceId) } startFlowResultWatchdog() recomputeMicVoiceAvailability() @@ -1935,6 +2006,11 @@ final class KeyboardFlowCoordinator { if (result.status == .partial || result.status == .rawReady), let partial = result.text, !partial.isEmpty { + if result.resolvedUtteranceMode == .aiQuestion, + AIClipboardPrompt.isInternalPrompt(partial) { + onAIGeneratingStarted(result.utteranceId) + return + } state.lastTranscript = partial if result.resolvedUtteranceMode == .aiQuestion { onAITranscript(partial, result.utteranceId, result.status) @@ -1982,6 +2058,10 @@ final class KeyboardFlowCoordinator { self.consumeCancelledDictationResultIfNeeded(result) { return } + if let result = self.matchingResult(), + self.consumeDiscardedEmptyResultIfNeeded(result) { + return + } if let result = self.matchingResult(), self.consumeCancelledEditResultIfNeeded(result) { return diff --git a/OSGKeyboardExt/Views/AIKeyboardView.swift b/OSGKeyboardExt/Views/AIKeyboardView.swift index 180cfd8..fb43f32 100644 --- a/OSGKeyboardExt/Views/AIKeyboardView.swift +++ b/OSGKeyboardExt/Views/AIKeyboardView.swift @@ -15,6 +15,7 @@ struct AIKeyboardView: View { static let actionButtonMaxWidth: CGFloat = 150 static let statusHeight: CGFloat = 20 static let carouselInterval: TimeInterval = 4 + static let skillButtonSize: CGFloat = 52 } @Environment(\.colorScheme) private var colorScheme @@ -61,6 +62,7 @@ struct AIKeyboardView: View { // Reduce Motion stops the rotation, not the data: a card whose // clipboard window has closed must still leave the carousel. reloadHintPool(resetBag: false) + guard !showsClipboardSkills else { return } if reduceMotion, let hint = currentHint, poolCards.contains(where: { $0.id == hint.id }) { return @@ -127,8 +129,13 @@ struct AIKeyboardView: View { private var answerArea: some View { ZStack(alignment: .bottom) { if showsPlaceholder { - hintCarousel - .frame(maxWidth: .infinity, maxHeight: .infinity) + if showsClipboardSkills { + clipboardSkillRow + .frame(maxWidth: .infinity, maxHeight: .infinity) + } else { + hintCarousel + .frame(maxWidth: .infinity, maxHeight: .infinity) + } } else { ScrollViewReader { proxy in ScrollView(.vertical) { @@ -173,30 +180,82 @@ struct AIKeyboardView: View { guard let hint = currentHint else { return } state.submitAIHint(hint) } label: { - Text(currentHint?.displayText ?? ExtL10n.string("keyboard.ai.placeholder")) - .font(TypeStyle.body) - .foregroundStyle(palette.textTertiary) - .multilineTextAlignment(.center) - .lineLimit(1) - .truncationMode(.tail) - .padding(.horizontal, Spacing.md) - .opacity(hintOpacity) - .frame(maxWidth: .infinity, maxHeight: .infinity) - .contentShape(Rectangle()) + HStack(spacing: 6) { + if let hint = currentHint { + Image(systemName: hint.visualKind.systemImage) + .font(.system(size: 14, weight: .semibold)) + .foregroundStyle(palette.textPrimary.opacity(0.55)) + } + Text(currentHint.map(\.resolvedDisplayText) ?? ExtL10n.string("keyboard.ai.placeholder")) + .font(TypeStyle.bodyEmph) + .foregroundStyle(palette.textPrimary) + .lineLimit(1) + .truncationMode(.tail) + } + .padding(.horizontal, 16) + .frame(height: 44) + .opacity(hintOpacity) + .glassEffect(.regular.interactive(), in: Capsule()) } .buttonStyle(.plain) + .fixedSize() // A busy session already owns the surface; the status line explains a // missing LLM. Both keep the hint from being a tap with no outcome. .disabled(currentHint == nil || !state.aiServiceAvailable || state.aiSession.isBusy) .accessibilityLabel( Text( currentHint.map { - "\(ExtL10n.string("keyboard.ai.hintA11yPrefix"))\($0.displayText)" + "\(ExtL10n.string("keyboard.ai.hintA11yPrefix"))\($0.resolvedDisplayText)" } ?? ExtL10n.string("keyboard.ai.placeholder") ) ) } + private var clipboardSkillRow: some View { + HStack(spacing: Spacing.lg) { + ForEach(AIClipboardSkillCatalog.visible()) { skill in + Button { + state.submitAIClipboardSkill(skill) + } label: { + VStack(spacing: 6) { + Image(systemName: skill.systemImage) + .font(.system(size: 18, weight: .semibold)) + .foregroundStyle(palette.textPrimary.opacity(0.85)) + .frame(width: Layout.skillButtonSize, height: Layout.skillButtonSize) + .glassEffect(.regular.interactive(), in: Circle()) + Text(clipboardSkillTitle(skill)) + .font(TypeStyle.caption2) + .foregroundStyle(palette.textSecondary) + } + } + .buttonStyle(.plain) + .disabled(!state.aiServiceAvailable || state.aiSession.isBusy) + .accessibilityLabel(Text(clipboardSkillTitle(skill))) + } + } + .frame(maxWidth: .infinity, maxHeight: .infinity) + } + + /// Translate follows the keyboard target; Reply / Summarize stay static. + private func clipboardSkillTitle(_ skill: AIClipboardSkill) -> String { + if skill.id == AIClipboardSkillCatalog.translateID { + return AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: state.translationTargetLocaleId, + uiLanguage: AppGroupStore().uiLanguage + ) + } + return ExtL10n.string(skill.titleKey) + } + + /// Copy-then-30s window: skill chips replace the rotating hint. + private var showsClipboardSkills: Bool { + guard showsPlaceholder, state.clipboardHistoryEnabled else { return false } + return AIHintPool.isClipboardSkillWindowActive( + clipboardHistoryEnabled: true, + newestClipboard: clipboardHistory.newestEntry + ) + } + /// No draft/answer yet — show the centered hint carousel instead of a scroll body. private var showsPlaceholder: Bool { let hasDraft = !(state.aiSession.draftAnswerText?.isEmpty ?? true) @@ -367,16 +426,20 @@ struct AIKeyboardView: View { ? ExtL10n.string("keyboard.ai.listening") : state.aiSession.transcript case .recognizing: - return state.aiSession.transcript.isEmpty - ? ExtL10n.string("keyboard.ai.recognizing") - : state.aiSession.transcript + if state.aiSession.transcript.isEmpty + || AIClipboardPrompt.isInternalPrompt(state.aiSession.transcript) { + return ExtL10n.string("keyboard.ai.recognizing") + } + return state.aiSession.transcript case .generating: if let draft = state.aiSession.draftAnswerText, !draft.isEmpty { return ExtL10n.string("keyboard.ai.generating") } - return state.aiSession.transcript.isEmpty - ? ExtL10n.string("keyboard.ai.thinking") - : state.aiSession.transcript + if state.aiSession.transcript.isEmpty + || AIClipboardPrompt.isInternalPrompt(state.aiSession.transcript) { + return ExtL10n.string("keyboard.ai.thinking") + } + return state.aiSession.transcript case .failed: return ExtL10n.string("keyboard.ai.error.requestFailed") } @@ -391,19 +454,18 @@ struct AIKeyboardView: View { // MARK: - Carousel /// Rebuild the pool and show a card right away, without a fade. + /// Skip advancing the chip while clipboard skills own the surface, so a + /// leftover clipboard sentence cannot replace the three buttons. private func resetCarousel() { reloadHintPool(resetBag: true) + guard !showsClipboardSkills else { return } showNextHint(animated: false) } private func reloadHintPool(resetBag: Bool) { let locale = AIHintLocaleResolver.packLocale() let pack = AIHintStore.resolvedPack(locale: locale) - poolCards = AIHintPool.activeCards( - pack: pack, - clipboardHistoryEnabled: state.clipboardHistoryEnabled, - newestClipboard: clipboardHistory.newestEntry - ) + poolCards = AIHintPool.activeCards(pack: pack) if resetBag { carouselBag.reset() } diff --git a/OSGKeyboardExt/Views/KeyboardRootView.swift b/OSGKeyboardExt/Views/KeyboardRootView.swift index 8aee3c5..3c59241 100644 --- a/OSGKeyboardExt/Views/KeyboardRootView.swift +++ b/OSGKeyboardExt/Views/KeyboardRootView.swift @@ -22,8 +22,8 @@ import OSGKeyboardShared private enum KeyboardLayoutMetrics { static let micSize: CGFloat = 121 static let micToButtonGap: CGFloat = 8 - /// Square undo key beside the mic (outer edge, aligned with delete). - static let undoButtonSize: CGFloat = 44 + /// Circular undo / translation keys beside the mic (outer edge). + static let undoButtonSize: CGFloat = 52 static let bottomActionRowHeight: CGFloat = KeyboardChromeLayout.actionKeyHeight static let bottomActionSpacing: CGFloat = KeyboardChromeLayout.actionKeySpacing /// Gap between the top control row and the transcript / hint line. @@ -419,7 +419,7 @@ public struct KeyboardRootView: View { .frame(height: KeyboardLayoutMetrics.bottomActionRowHeight) } - /// Square undo key on the outer drag pad — same chrome / haptic / click + /// Circular undo key on the outer drag pad — same chrome / haptic / click /// as space & return. Vertically matches the mic disc. private func undoButton(disabled: Bool, visible: Bool) -> some View { RectangularToolbarButton( @@ -427,6 +427,7 @@ public struct KeyboardRootView: View { label: ExtL10n.string("keyboard.undoA11y"), disabled: disabled, usesLiquidGlass: true, + usesCircleGlass: true, hapticIntensity: state.keyboardHapticIntensity ) { state.undoLastInsertion() diff --git a/OSGKeyboardExt/Views/KeyboardTopControls.swift b/OSGKeyboardExt/Views/KeyboardTopControls.swift index 18524c9..697e749 100644 --- a/OSGKeyboardExt/Views/KeyboardTopControls.swift +++ b/OSGKeyboardExt/Views/KeyboardTopControls.swift @@ -280,25 +280,18 @@ struct KeyboardTranslationMenuButton: View, Equatable { } } } label: { - // Match the adjacent undo key: 44×44 rounded Liquid Glass control. ZStack { Color.clear Image(systemName: isEnabled ? "character.bubble.fill" : "character.bubble") - .font(.system(size: 14, weight: .semibold)) + .font(.system(size: 17, weight: .semibold)) .foregroundStyle( isEnabled ? palette.accent : palette.textSecondary ) } - .contentShape(Rectangle()) - .glassEffect( - .regular.interactive(), - in: RoundedRectangle( - cornerRadius: KeyboardChromeLayout.actionKeyCornerRadius, - style: .continuous - ) - ) + .contentShape(Circle()) + .glassEffect(.regular.interactive(), in: Circle()) } .menuStyle(.button) .accessibilityLabel(Text(SharedL10n.string("keyboard.translation.a11y"))) diff --git a/OSGKeyboardExt/Views/ToolbarActionButtons.swift b/OSGKeyboardExt/Views/ToolbarActionButtons.swift index 1d5768a..f72d15a 100644 --- a/OSGKeyboardExt/Views/ToolbarActionButtons.swift +++ b/OSGKeyboardExt/Views/ToolbarActionButtons.swift @@ -10,6 +10,8 @@ import OSGKeyboardShared private enum ToolbarButtonMetrics { static let iconSize: CGFloat = 14 + /// Slightly larger glyph so a 52 pt circular glass key does not look sparse. + static let circleIconSize: CGFloat = 17 static let titleSize: CGFloat = 16 static let cornerRadius: CGFloat = KeyboardChromeLayout.actionKeyCornerRadius static let spaceBarCapsuleWidth: CGFloat = 31 @@ -201,6 +203,7 @@ struct RectangularToolbarButton: View { let disabled: Bool let isSend: Bool let usesLiquidGlass: Bool + let usesCircleGlass: Bool /// Settings → General → Haptics; space / return use `.action` role. var hapticIntensity: KeyboardHapticIntensity = .off let action: () -> Void @@ -210,6 +213,7 @@ struct RectangularToolbarButton: View { label: String, disabled: Bool = false, usesLiquidGlass: Bool = false, + usesCircleGlass: Bool = false, hapticIntensity: KeyboardHapticIntensity = .off, action: @escaping () -> Void ) { @@ -220,6 +224,7 @@ struct RectangularToolbarButton: View { self.disabled = disabled self.isSend = false self.usesLiquidGlass = usesLiquidGlass + self.usesCircleGlass = usesCircleGlass self.hapticIntensity = hapticIntensity self.action = action } @@ -239,6 +244,7 @@ struct RectangularToolbarButton: View { self.disabled = disabled self.isSend = isSend self.usesLiquidGlass = usesLiquidGlass + self.usesCircleGlass = false self.hapticIntensity = hapticIntensity self.action = action self.title = title @@ -259,6 +265,7 @@ struct RectangularToolbarButton: View { self.disabled = disabled self.isSend = false self.usesLiquidGlass = usesLiquidGlass + self.usesCircleGlass = false self.hapticIntensity = hapticIntensity self.action = action } @@ -267,7 +274,7 @@ struct RectangularToolbarButton: View { var body: some View { buttonSurface - .contentShape(Rectangle()) + .contentShape(usesCircleGlass ? AnyShape(Circle()) : AnyShape(Rectangle())) .gesture(pressGesture) .opacity(disabled ? 0.38 : 1) .allowsHitTesting(!disabled) @@ -282,13 +289,7 @@ struct RectangularToolbarButton: View { Color.clear buttonContent } - .glassEffect( - .regular.interactive(), - in: RoundedRectangle( - cornerRadius: ToolbarButtonMetrics.cornerRadius, - style: .continuous - ) - ) + .modifier(ToolbarLiquidGlass(isCircle: usesCircleGlass)) // The custom press gesture fires on touch-down; mirror that state // visually while Liquid Glass supplies its native light response. .scaleEffect(isPressing ? 0.97 : 1) @@ -312,7 +313,12 @@ struct RectangularToolbarButton: View { .frame(width: ToolbarButtonMetrics.spaceBarCapsuleWidth, height: 3) } else if let systemName { Image(systemName: systemName) - .font(.system(size: ToolbarButtonMetrics.iconSize, weight: .semibold)) + .font(.system( + size: usesCircleGlass + ? ToolbarButtonMetrics.circleIconSize + : ToolbarButtonMetrics.iconSize, + weight: .semibold + )) .foregroundStyle(buttonForeground) } else if let title { Text(title) @@ -340,3 +346,21 @@ struct RectangularToolbarButton: View { } } } + +private struct ToolbarLiquidGlass: ViewModifier { + let isCircle: Bool + + func body(content: Content) -> some View { + if isCircle { + content.glassEffect(.regular.interactive(), in: Circle()) + } else { + content.glassEffect( + .regular.interactive(), + in: RoundedRectangle( + cornerRadius: ToolbarButtonMetrics.cornerRadius, + style: .continuous + ) + ) + } + } +} diff --git a/OSGKeyboardExt/en.lproj/Keyboard.strings b/OSGKeyboardExt/en.lproj/Keyboard.strings index 7a1689e..39367d3 100644 --- a/OSGKeyboardExt/en.lproj/Keyboard.strings +++ b/OSGKeyboardExt/en.lproj/Keyboard.strings @@ -294,3 +294,6 @@ "keyboard.ai.error.requestTimeout" = "AI response timed out. Try again"; "keyboard.ai.error.requestFailed" = "AI response failed. Try again"; "keyboard.ai.error.clipboardUnavailable" = "This clipboard suggestion expired. Copy the text again"; +"keyboard.ai.skill.reply" = "Reply"; +"keyboard.ai.skill.summarize" = "Summarize"; +"keyboard.ai.skill.translate" = "Translate"; diff --git a/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings b/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings index 6ff39a2..b16546a 100644 --- a/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings +++ b/OSGKeyboardExt/zh-Hans.lproj/Keyboard.strings @@ -294,3 +294,6 @@ "keyboard.ai.error.requestTimeout" = "AI 回答超时，请重试"; "keyboard.ai.error.requestFailed" = "AI 回答失败，请重试"; "keyboard.ai.error.clipboardUnavailable" = "剪贴板建议已过期，请重新复制文本"; +"keyboard.ai.skill.reply" = "回复"; +"keyboard.ai.skill.summarize" = "总结"; +"keyboard.ai.skill.translate" = "翻译"; diff --git a/OSGKeyboardExtTests/KeyboardStateTests.swift b/OSGKeyboardExtTests/KeyboardStateTests.swift index ccc49ff..5fd8ff4 100644 --- a/OSGKeyboardExtTests/KeyboardStateTests.swift +++ b/OSGKeyboardExtTests/KeyboardStateTests.swift @@ -110,6 +110,11 @@ final class KeyboardStateTests: XCTestCase { KeyboardState.Phase.ErrorKind.fromFlowTranscription(flowError), .hostTranscriptionFailed("asr failed") ) + let discarded = FlowTranscriptionError(message: "", kind: .discardedEmpty) + XCTAssertEqual( + KeyboardState.Phase.ErrorKind.fromFlowTranscription(discarded), + .noSpeechDetected + ) } func testInputModeIsPolishOnly() { diff --git a/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift b/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift index 2176add..70faf78 100644 --- a/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift +++ b/OSGKeyboardHostSupport/Services/FlowContinuousCapture.swift @@ -1126,6 +1126,12 @@ public final class FlowContinuousCapture { return report } + /// Copy of utterance PCM so the host can decide empty-tap skip + /// without consuming the buffer used by the normal drain path. + public func utterancePCMSnapshot() -> [Float] { + utterancePCMStore.snapshot() + } + /// Returns the utterance PCM accumulated during the last recording cycle. public func consumeUtteranceSamples() -> [Float] { utterancePCMStore.consume() diff --git a/OSGKeyboardShared/Models/AIHintModels.swift b/OSGKeyboardShared/Models/AIHintModels.swift index bec5040..c5b4c69 100644 --- a/OSGKeyboardShared/Models/AIHintModels.swift +++ b/OSGKeyboardShared/Models/AIHintModels.swift @@ -1,11 +1,39 @@ // AIHintModels.swift // OSGKeyboard · Shared // -// Hint cards for the AI-mode idle carousel. Remote packs use `text`; the -// host compresses that into `displayText` before writing the ready pack. +// Hint cards for the AI-mode idle carousel. Remote packs use `text` plus +// optional `metadata`; the host extracts a keyword `displayText` for the chip. import Foundation +public struct AIHintMetadata: Codable, Equatable, Sendable { + public var title: String? + public var city: String? + public var tempC: Double? + public var soul: String? + public var name: String? + public var date: String? + public var day: String? + + public init( + title: String? = nil, + city: String? = nil, + tempC: Double? = nil, + soul: String? = nil, + name: String? = nil, + date: String? = nil, + day: String? = nil + ) { + self.title = title + self.city = city + self.tempC = tempC + self.soul = soul + self.name = name + self.date = date + self.day = day + } +} + public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { public let id: String /// One-line carousel label (after host keyword pass, or local catalog). @@ -17,6 +45,7 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { public var source: String public var locale: String public var conditions: [String] + public var metadata: AIHintMetadata? public init( id: String, @@ -26,7 +55,8 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { priority: Int = 50, source: String = "local", locale: String = "zh", - conditions: [String] = [] + conditions: [String] = [], + metadata: AIHintMetadata? = nil ) { self.id = id self.displayText = displayText @@ -36,14 +66,24 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { self.source = source self.locale = locale self.conditions = conditions + self.metadata = metadata } public var requiresClipboard30s: Bool { conditions.contains("clipboard_30s") || category == "clipboard" } + /// Keyword shown in the idle chip; prefers feed metadata over raw `text`. + public var resolvedDisplayText: String { + AIHintKeywordExtractor.displayText(for: self) + } + + public var visualKind: AIHintVisualKind { + AIHintVisualKind.resolve(self) + } + enum CodingKeys: String, CodingKey { - case id, displayText, text, prompt, category, priority, source, locale, conditions + case id, displayText, text, prompt, category, priority, source, locale, conditions, metadata } public init(from decoder: Decoder) throws { @@ -55,6 +95,7 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { source = try container.decodeIfPresent(String.self, forKey: .source) ?? "remote" locale = try container.decodeIfPresent(String.self, forKey: .locale) ?? "zh" conditions = try container.decodeIfPresent([String].self, forKey: .conditions) ?? [] + metadata = try container.decodeIfPresent(AIHintMetadata.self, forKey: .metadata) if let display = try container.decodeIfPresent(String.self, forKey: .displayText), !display.isEmpty { displayText = display @@ -73,6 +114,7 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { try container.encode(source, forKey: .source) try container.encode(locale, forKey: .locale) try container.encode(conditions, forKey: .conditions) + try container.encodeIfPresent(metadata, forKey: .metadata) } } diff --git a/OSGKeyboardShared/Models/AIHintVisualKind.swift b/OSGKeyboardShared/Models/AIHintVisualKind.swift new file mode 100644 index 0000000..398132f --- /dev/null +++ b/OSGKeyboardShared/Models/AIHintVisualKind.swift @@ -0,0 +1,58 @@ +// AIHintVisualKind.swift +// OSGKeyboard · Shared +// +// Maps hint-feed category/source onto the idle-chip SF Symbol. + +import Foundation + +public enum AIHintVisualKind: String, Equatable, Sendable { + case calendar + case weather + case news + case stocks + case trending + case search + + public var systemImage: String { + switch self { + case .calendar: return "calendar" + case .weather: return "cloud.sun.fill" + case .news: return "newspaper.fill" + case .stocks: return "chart.line.uptrend.xyaxis" + case .trending: return "flame.fill" + case .search: return "magnifyingglass" + } + } + + public static func resolve(_ card: AIHintCard) -> Self { + let category = card.category.lowercased() + let source = card.source.lowercased() + let id = card.id.lowercased() + + if category == "weather" || source.contains("meteo") { + return .weather + } + if category == "economy" || id.contains("stock") { + return .stocks + } + if category == "society" + || source.contains("open-hot") + || source.contains("tophub-open") { + return .trending + } + if category == "holiday" + || category == "history" + || source.contains("holiday") + || card.conditions.contains("date") { + return .calendar + } + if category == "daily" { + if card.metadata?.soul != nil || id.contains("soul") { + return .search + } + return .news + } + // Quotes, encyclopedia, how-to, and unknown capability cards. + return .search + } +} diff --git a/OSGKeyboardShared/Models/AISessionState.swift b/OSGKeyboardShared/Models/AISessionState.swift index 54ae5b1..64d1407 100644 --- a/OSGKeyboardShared/Models/AISessionState.swift +++ b/OSGKeyboardShared/Models/AISessionState.swift @@ -161,7 +161,7 @@ public struct AISessionState: Equatable, Sendable { /// replace the previous committed `answer` until `receiveAnswer`. public mutating func receivePartialAnswer(_ text: String, utteranceID: UUID) { guard isActive, activeUtteranceID == utteranceID else { return } - if phase == .recognizing { + if phase == .recognizing || phase == .preparing { phase = .generating } guard phase == .generating else { return } diff --git a/OSGKeyboardShared/Models/TranslationLanguage.swift b/OSGKeyboardShared/Models/TranslationLanguage.swift index b1541fe..a25249c 100644 --- a/OSGKeyboardShared/Models/TranslationLanguage.swift +++ b/OSGKeyboardShared/Models/TranslationLanguage.swift @@ -25,6 +25,45 @@ public struct TranslationLanguage: Identifiable, Hashable, Sendable { self.promptLanguageName = promptLanguageName self.nativeName = nativeName } + + /// One-character Chinese token for compact chips such as「中译英」. + public var chineseShort: String { + switch id { + case "en": return "英" + case "zh-Hans": return "中" + case "zh-Hant": return "繁" + case "ja": return "日" + case "ko": return "韩" + case "fr": return "法" + case "de": return "德" + case "es": return "西" + case "ru": return "俄" + case "pt": return "葡" + default: return nativeName + } + } + + /// Country-style English code for compact chips such as「To JP」. + public var englishShort: String { + switch id { + case "en": return "EN" + case "zh-Hans": return "CN" + case "zh-Hant": return "TW" + case "ja": return "JP" + case "ko": return "KR" + case "fr": return "FR" + case "de": return "DE" + case "es": return "ES" + case "ru": return "RU" + case "pt": return "PT" + default: return id.uppercased() + } + } + + /// True when this target is a Chinese script (简体 or 繁體). + public var isChineseScript: Bool { + id == "zh-Hans" || id == "zh-Hant" + } } public enum TranslationLanguageCatalog { diff --git a/OSGKeyboardShared/Services/AIClipboardPrompt.swift b/OSGKeyboardShared/Services/AIClipboardPrompt.swift index d9565f1..a614e4e 100644 --- a/OSGKeyboardShared/Services/AIClipboardPrompt.swift +++ b/OSGKeyboardShared/Services/AIClipboardPrompt.swift @@ -48,6 +48,11 @@ public enum AIClipboardPrompt: Sendable { return resolve(instruction: question, material: material) } + /// True when `text` is the clipboard-AI XML envelope, not user-visible copy. + public static func isInternalPrompt(_ text: String) -> Bool { + text.contains(" String { trimmed(prompt.replacingOccurrences(of: materialPlaceholder, with: "")) diff --git a/OSGKeyboardShared/Services/AIClipboardSkill.swift b/OSGKeyboardShared/Services/AIClipboardSkill.swift new file mode 100644 index 0000000..f7d0b56 --- /dev/null +++ b/OSGKeyboardShared/Services/AIClipboardSkill.swift @@ -0,0 +1,129 @@ +// AIClipboardSkill.swift +// OSGKeyboard · Shared +// +// Built-in clipboard actions for AI idle. The catalog is an ordered list so +// Settings can later persist a subset or permutation without changing the view. + +import Foundation + +public struct AIClipboardSkill: Identifiable, Equatable, Sendable { + public let id: String + public let systemImage: String + /// Keyboard.strings key for the short button title. + public let titleKey: String + + public init(id: String, systemImage: String, titleKey: String) { + self.id = id + self.systemImage = systemImage + self.titleKey = titleKey + } +} + +public enum AIClipboardSkillCatalog: Sendable { + public static let replyID = "reply" + public static let summarizeID = "summarize" + public static let translateID = "translate" + + /// Default set, in display order. Future skills append here. + public static let builtIn: [AIClipboardSkill] = [ + AIClipboardSkill( + id: replyID, + systemImage: "arrowshape.turn.up.left.fill", + titleKey: "keyboard.ai.skill.reply" + ), + AIClipboardSkill( + id: summarizeID, + systemImage: "doc.text.magnifyingglass", + titleKey: "keyboard.ai.skill.summarize" + ), + AIClipboardSkill( + id: translateID, + systemImage: "character.bubble.fill", + titleKey: "keyboard.ai.skill.translate" + ), + ] + + /// `enabledIDs` is the future Settings hook: `nil` keeps the built-in list. + public static func visible(enabledIDs: [String]? = nil) -> [AIClipboardSkill] { + guard let enabledIDs, !enabledIDs.isEmpty else { return builtIn } + let byID = Dictionary(uniqueKeysWithValues: builtIn.map { ($0.id, $0) }) + return enabledIDs.compactMap { byID[$0] } + } + + public static func instruction( + for skill: AIClipboardSkill, + locale: String, + translationTargetLocaleId: String + ) -> String { + instruction( + skillID: skill.id, + locale: locale, + translationTargetLocaleId: translationTargetLocaleId + ) + } + + /// Compact Translate-chip label. Unset target → 中英互译; Chinese UI + /// targeting 简/繁 → 简繁互转 (avoids「中译中」); otherwise 中译× / To XX. + public static func translateButtonTitle( + translationTargetLocaleId: String, + uiLanguage: AppUILanguage + ) -> String { + let isChineseUI = uiLanguage.resolvedLanguageCode() == "zh-Hans" + if TranslationLanguageCatalog.isOff(translationTargetLocaleId) { + return isChineseUI ? "中↔英" : "CN↔EN" + } + let target = TranslationLanguageCatalog.resolve(translationTargetLocaleId) + if isChineseUI, target.isChineseScript { + return "简↔繁" + } + if isChineseUI { + return "中译\(target.chineseShort)" + } + return "To \(target.englishShort)" + } + + public static func instruction( + skillID: String, + locale: String, + translationTargetLocaleId: String + ) -> String { + let zh = locale == "zh" + switch skillID { + case replyID: + return zh + ? "请根据剪贴板内容起草一段礼貌、简洁的回复，语气自然，可直接发送。" + : "Draft a concise, polite reply the user can send, based on the clipboard text." + case summarizeID: + return zh + ? "请概括剪贴板内容的核心意思，保留关键事实与结论，不要改写成可发送的短消息。" + : "Summarize the clipboard text: keep the key facts and conclusions; do not rewrite it as a sendable short message." + case translateID: + return translateInstruction( + locale: locale, + translationTargetLocaleId: translationTargetLocaleId + ) + default: + return zh + ? "请根据剪贴板内容完成用户选择的操作。" + : "Complete the selected action using the clipboard text." + } + } + + /// Uses the keyboard translation target when set; otherwise Chinese ↔ English. + private static func translateInstruction( + locale: String, + translationTargetLocaleId: String + ) -> String { + let zh = locale == "zh" + if !TranslationLanguageCatalog.isOff(translationTargetLocaleId) { + let language = TranslationLanguageCatalog.resolve(translationTargetLocaleId) + let name = language.promptLanguageName + return zh + ? "请将剪贴板内容翻译成\(name)，保留原意与语气。" + : "Translate the clipboard text into \(name), preserving meaning and tone." + } + return zh + ? "请将剪贴板内容在中文与英文之间互译：若原文主要是中文则译成自然英文，若主要是英文则译成自然中文。保留原意与语气。" + : "Translate the clipboard between Chinese and English: if it is primarily Chinese, produce natural English; if primarily English, produce natural Chinese. Preserve meaning and tone." + } +} diff --git a/OSGKeyboardShared/Services/AIHintKeywordCompressor.swift b/OSGKeyboardShared/Services/AIHintKeywordCompressor.swift index 4574564..b333090 100644 --- a/OSGKeyboardShared/Services/AIHintKeywordCompressor.swift +++ b/OSGKeyboardShared/Services/AIHintKeywordCompressor.swift @@ -1,8 +1,8 @@ // AIHintKeywordCompressor.swift // OSGKeyboard · Shared // -// Uses the user's polish LLM to compress remote hint titles into one-line -// display labels. Failure leaves the previous ready pack untouched (caller). +// Optional LLM pass after deterministic keyword extraction. Failure leaves +// the extracted labels in place (hard truncate only as a last resort). import Foundation @@ -19,8 +19,15 @@ public struct AIHintKeywordCompressor: Sendable { cards: [AIHintCard], locale: String ) async -> [AIHintCard] { - let candidates = cards.filter { shouldCompress($0) } - guard !candidates.isEmpty else { return cards } + let prepared = cards.map { card -> AIHintCard in + var copy = card + copy.displayText = AIHintKeywordExtractor.displayText(for: card) + return copy + } + let candidates = zip(cards, prepared).compactMap { original, extracted -> AIHintCard? in + shouldCompress(extracted) ? original : nil + } + guard !candidates.isEmpty else { return prepared } do { let client = try resolveClient() @@ -28,6 +35,7 @@ public struct AIHintKeywordCompressor: Sendable { [ "id": $0.id, "text": $0.displayText, + "title": $0.metadata?.title ?? "", "category": $0.category, "source": $0.source, ] @@ -48,32 +56,27 @@ public struct AIHintKeywordCompressor: Sendable { return result } let mapping = Self.parseDisplayMap(from: raw) - guard !mapping.isEmpty else { return cards } - return cards.map { card in + guard !mapping.isEmpty else { return prepared } + return prepared.map { card in guard let display = mapping[card.id], !display.isEmpty else { return card } var copy = card - copy.displayText = Self.sanitizeDisplay(display, locale: locale) + copy.displayText = AIHintKeywordExtractor.finalize(display, locale: locale) return copy } } catch { #if DEBUG print("⚠️ [AIHintKeywordCompressor] failed: \(error)") #endif - return cards.map { card in - var copy = card - copy.displayText = Self.fallbackTruncate(card.displayText, locale: locale) - return copy - } + return prepared } } private func shouldCompress(_ card: AIHintCard) -> Bool { if isHistoricalToday(card) { return false } - if card.locale == "zh" || card.displayText.contains(where: { $0.isCJKUnifiedIdeograph }) { - return card.displayText.count > 12 || card.displayText.contains("…") - || card.displayText.contains("全网热点") - } - return card.displayText.count > 28 + if card.requiresClipboard30s { return false } + let limit = AIHintKeywordExtractor.characterLimit(locale: card.locale) + return card.displayText.count > limit + || card.displayText.contains("…") } private func isHistoricalToday(_ card: AIHintCard) -> Bool { @@ -103,34 +106,30 @@ public struct AIHintKeywordCompressor: Sendable { private static func systemPrompt(locale: String) -> String { if locale == "zh" { return """ - 你是输入法 AI 空闲轮播的文案压缩器。 - 输入是 JSON 数组，每项含 id/text/category/source。 + 你是输入法 AI 空闲轮播的关键词提取器。 + 输入是 JSON 数组，每项含 id/text/title/category/source。 输出 JSON 数组，每项仅 {"id","displayText"}。 硬性规则： - - displayText 必须单行，不要省略号结尾 - - 中文约 5–12 字 - - 按意图选句式，禁止统一加「聊聊」前缀： - · 讨论类热点 →「聊聊+实体」 - · 剪贴板动作 →「帮我回复剪贴板」「把剪贴板译成英文」等 - · 天气查询 →「上海天气怎么样」 - · 早报/行情 →「看今日早报」「今天大盘如何」 - · 生成类 →「来句今日金句」「讲个有趣概念」 - · 节日 →「中秋节怎么过」 + - displayText 必须是实体/关键词，不要写成问句或动作句 + - 禁止「聊聊」「看看」「帮我」等动词前缀 + - 单行，不要省略号结尾 + - 中文约 4–10 字；优先用 title 字段 - 丢弃「历史上的今天」类条目（不要输出它们的 id） - 不要改写 prompt；不要 Markdown；只输出 JSON """ } return """ - You compress AI keyboard idle hint titles. - Input: JSON array of {id,text,category,source}. + You extract keywords for AI keyboard idle hint chips. + Input: JSON array of {id,text,title,category,source}. Output: JSON array of {"id","displayText"} only. Rules: - - displayText must be one line, no trailing ellipsis - - English: ≤28 characters, NO "Chat"/"Chat about" prefix - - Match intent (action / query / discuss) with a short natural label - - Drop "On this day" / historical-today style items (omit their ids) + - displayText is the entity/keyword, not a question or action sentence + - No "Chat"/"Chat about" prefix + - One line, no trailing ellipsis + - English: ≤22 characters; prefer the title field + - Drop "On this day" / historical-today items (omit their ids) - Do not change prompts; JSON only, no Markdown """ } @@ -163,31 +162,10 @@ public struct AIHintKeywordCompressor: Sendable { } public static func sanitizeDisplay(_ text: String, locale: String) -> String { - var value = text - .replacingOccurrences(of: "\n", with: " ") - .trimmingCharacters(in: .whitespacesAndNewlines) - while value.hasSuffix("…") || value.hasSuffix("...") { - if value.hasSuffix("...") { - value = String(value.dropLast(3)) - } else { - value = String(value.dropLast()) - } - value = value.trimmingCharacters(in: .whitespacesAndNewlines) - } - return fallbackTruncate(value, locale: locale) + AIHintKeywordExtractor.finalize(text, locale: locale) } public static func fallbackTruncate(_ text: String, locale: String) -> String { - let limit = locale == "zh" ? 12 : 28 - guard text.count > limit else { return text } - return String(text.prefix(limit)) - } -} - -private extension Character { - var isCJKUnifiedIdeograph: Bool { - unicodeScalars.contains { scalar in - (0x4E00...0x9FFF).contains(scalar.value) - } + AIHintKeywordExtractor.finalize(text, locale: locale) } } diff --git a/OSGKeyboardShared/Services/AIHintKeywordExtractor.swift b/OSGKeyboardShared/Services/AIHintKeywordExtractor.swift new file mode 100644 index 0000000..5918d91 --- /dev/null +++ b/OSGKeyboardShared/Services/AIHintKeywordExtractor.swift @@ -0,0 +1,162 @@ +// AIHintKeywordExtractor.swift +// OSGKeyboard · Shared +// +// Deterministic idle-chip labels: the icon carries type, the text is the +// entity. LLM compression is only a last resort for leftovers. + +import Foundation + +public enum AIHintKeywordExtractor: Sendable { + public static func characterLimit(locale: String) -> Int { + locale == "zh" ? 10 : 22 + } + + public static func displayText(for card: AIHintCard) -> String { + let locale = card.locale == "zh" ? "zh" : "en" + let kind = AIHintVisualKind.resolve(card) + let raw = keyword(for: card, kind: kind, locale: locale) + return finalize(raw, locale: locale) + } + + /// Strip prefixes, pick a fitting chunk, then enforce the character cap. + public static func finalize(_ text: String, locale: String) -> String { + var value = stripPrefixes(text, locale: locale) + .replacingOccurrences(of: "\n", with: " ") + .trimmingCharacters(in: .whitespacesAndNewlines) + value = stripTrailingPunctuation(value) + let limit = characterLimit(locale: locale) + if value.count <= limit { return value } + if let clause = firstFittingClause(value, limit: limit) { return clause } + if let latin = leadingLatinPhrase(value, limit: limit) { return latin } + if let chunk = lastFittingChunk(value, limit: limit) { return chunk } + return String(value.prefix(limit)) + } + + // MARK: - Keyword by kind + + private static func keyword( + for card: AIHintCard, + kind: AIHintVisualKind, + locale: String + ) -> String { + switch kind { + case .trending: + return firstNonEmpty(card.metadata?.title, card.displayText) + case .weather: + if let city = trimmed(card.metadata?.city) { + if let temp = card.metadata?.tempC { + return "\(city) \(Int(temp.rounded()))°" + } + return city + } + return card.displayText + case .news: + return locale == "zh" ? "今日早报" : "Today's briefing" + case .stocks: + return locale == "zh" ? "今日大盘" : "Markets" + case .calendar: + if locale != "zh", let name = trimmed(card.metadata?.name) { + return name + } + return card.displayText + case .search: + if card.metadata?.soul != nil { + return locale == "zh" ? "今日金句" : "A quote" + } + return card.displayText + } + } + + // MARK: - Prefixes + + public static func stripPrefixes(_ text: String, locale: String) -> String { + var value = text.trimmingCharacters(in: .whitespacesAndNewlines) + let prefixes = locale == "zh" ? zhPrefixes : enPrefixes + var changed = true + while changed { + changed = false + for prefix in prefixes where value.hasPrefix(prefix) { + value = String(value.dropFirst(prefix.count)) + .trimmingCharacters(in: .whitespacesAndNewlines) + changed = true + break + } + } + return value + } + + private static let zhPrefixes = [ + "全网热点：", "全网热点:", "临近节日：", "临近节日:", + "历史上的今天：", "历史上的今天:", "今日一句：", "今日一句:", + "查百科：", "查百科:", "聊聊", "看看", + ] + + private static let enPrefixes = [ + "Trending: ", "Upcoming: ", "On this day: ", + "Chat about ", "Chat ", "Weather in ", + ] + + // MARK: - Chunks + + private static func firstFittingClause(_ text: String, limit: Int) -> String? { + let separators = CharacterSet(charactersIn: "，。；;,.!?？！") + let parts = text.components(separatedBy: separators) + .map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } + .filter { !$0.isEmpty } + guard let first = parts.first, first.count <= limit, first.count >= 2 else { + return nil + } + return first + } + + private static func lastFittingChunk(_ text: String, limit: Int) -> String? { + let parts = text.split { $0 == " " || $0 == "：" || $0 == ":" } + .map(String.init) + .filter { !$0.isEmpty } + guard let last = parts.last else { return nil } + let cleaned = stripTrailingPunctuation(last) + guard (4...limit).contains(cleaned.count) else { return nil } + return cleaned + } + + private static func leadingLatinPhrase(_ text: String, limit: Int) -> String? { + var scalars: [Unicode.Scalar] = [] + for scalar in text.unicodeScalars { + let isLatin = (0x41...0x5A).contains(scalar.value) + || (0x61...0x7A).contains(scalar.value) + || (0x30...0x39).contains(scalar.value) + || scalar == "." || scalar == "-" + let isSpace = scalar == " " + if isLatin || (isSpace && !scalars.isEmpty) { + scalars.append(scalar) + } else if !scalars.isEmpty { + break + } + } + var phrase = String(String.UnicodeScalarView(scalars)) + .trimmingCharacters(in: .whitespaces) + guard phrase.count >= 2 else { return nil } + if phrase.count <= limit { return phrase } + while phrase.count > limit { + guard let lastSpace = phrase.lastIndex(of: " "), lastSpace > phrase.startIndex else { + return String(phrase.prefix(limit)) + } + phrase = String(phrase[.. String { + text.trimmingCharacters(in: CharacterSet(charactersIn: "…。.!?？！、,")) + } + + private static func firstNonEmpty(_ values: String?...) -> String { + values.compactMap(trimmed).first ?? "" + } + + private static func trimmed(_ value: String?) -> String? { + guard let value else { return nil } + let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines) + return trimmed.isEmpty ? nil : trimmed + } +} diff --git a/OSGKeyboardShared/Services/AIHintLocalCatalog.swift b/OSGKeyboardShared/Services/AIHintLocalCatalog.swift index e89707d..81b7739 100644 --- a/OSGKeyboardShared/Services/AIHintLocalCatalog.swift +++ b/OSGKeyboardShared/Services/AIHintLocalCatalog.swift @@ -44,7 +44,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-zh-encyclopedia", - displayText: "讲个有趣概念", + displayText: "有趣概念", prompt: "用通俗易懂的中文解释一个有趣但常见的概念，并给一个生活里的例子（4-6 句）。", category: "capability", priority: 40, @@ -53,7 +53,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-zh-stocks", - displayText: "今天大盘如何", + displayText: "今日大盘", prompt: "请用非专业口吻概括今天 A 股/港股/美股中至少一个市场的整体表现、" + "可能驱动因素，并提醒这并非投资建议（4-6 句）。", category: "economy", @@ -63,7 +63,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-zh-daily-brief", - displayText: "看今日早报", + displayText: "今日早报", prompt: "请用中文写一份简洁的「今日早报」：国内外各 2–3 条要点、一条财经/科技、" + "一条轻松话题；每条一句话，总计不超过 12 句。不确定处请标明。", category: "daily", @@ -73,7 +73,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-zh-quote", - displayText: "来句今日金句", + displayText: "今日金句", prompt: "请给一句适合今天分享的中文金句，并附上一两句简短解释。", category: "capability", priority: 38, @@ -82,7 +82,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-zh-howto", - displayText: "给我一个小技巧", + displayText: "生活技巧", prompt: "分享一个实用的生活或工作效率小技巧，用中文说清步骤与适用场景（4-6 句）。", category: "capability", priority: 36, @@ -124,7 +124,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-en-encyclopedia", - displayText: "Explain a concept", + displayText: "A concept", prompt: "Explain an interesting everyday concept in plain English with one real-life example (4-6 sentences).", category: "capability", priority: 40, @@ -133,7 +133,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-en-stocks", - displayText: "Market pulse", + displayText: "Markets", prompt: "Summarize today's broad market mood (US or global) in plain English, " + "note possible drivers, and add this is not financial advice (4-6 sentences).", category: "economy", @@ -153,7 +153,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-en-quote", - displayText: "Share a quote", + displayText: "A quote", prompt: "Share one short quote worth sending today, plus one or two sentences of context.", category: "capability", priority: 38, @@ -162,7 +162,7 @@ public enum AIHintLocalCatalog: Sendable { ), AIHintCard( id: "local-en-howto", - displayText: "Give a tip", + displayText: "A tip", prompt: "Share one practical life or productivity tip in English, with steps and when it helps (4-6 sentences).", category: "capability", priority: 36, diff --git a/OSGKeyboardShared/Services/AIHintPool.swift b/OSGKeyboardShared/Services/AIHintPool.swift index 200530d..690104d 100644 --- a/OSGKeyboardShared/Services/AIHintPool.swift +++ b/OSGKeyboardShared/Services/AIHintPool.swift @@ -1,34 +1,21 @@ // AIHintPool.swift // OSGKeyboard · Shared // -// Builds the idle carousel pool: 100% clipboard cards while eligible, -// otherwise a shuffled mix of non-clipboard local + remote cards. +// Builds the idle carousel pool from local + remote cards. Clipboard +// sentence cards are excluded: the 30s copy window shows skill chips. import Foundation public enum AIHintPool: Sendable { public static func activeCards( - pack: AIHintPack, - clipboardHistoryEnabled: Bool, - newestClipboard: ClipboardHistoryEntry?, - now: Date = Date() + pack: AIHintPack ) -> [AIHintCard] { - let clipboardEligible = clipboardHistoryEnabled - && newestClipboard.map { ClipboardHistoryPolicy.isEligibleForAIHint($0, now: now) } == true - - let clipboardCards = pack.cards.filter(\.requiresClipboard30s) let regularCards = pack.cards.filter { !$0.requiresClipboard30s } .filter { !isHistoricalToday($0) } - // Within 30s: only clipboard-related sentences. - if clipboardEligible { - let pool = clipboardCards.isEmpty - ? AIHintLocalCatalog.cards(locale: pack.locale).filter(\.requiresClipboard30s) - : clipboardCards - return pool.sorted { $0.priority > $1.priority } - } - - // Otherwise: drop clipboard-conditioned cards entirely. + // Clipboard-conditioned sentences no longer rotate in the carousel; + // the 30s window shows skill chips instead. Keep evergreen content + // loaded so leaving the window does not flash a leftover clipboard card. var merged = regularCards let localRegular = AIHintLocalCatalog.cards(locale: pack.locale) .filter { !$0.requiresClipboard30s } @@ -38,6 +25,16 @@ public enum AIHintPool: Sendable { return merged.sorted { $0.priority > $1.priority } } + /// Copy-then-30s window where clipboard skill chips replace the carousel. + public static func isClipboardSkillWindowActive( + clipboardHistoryEnabled: Bool, + newestClipboard: ClipboardHistoryEntry?, + now: Date = Date() + ) -> Bool { + clipboardHistoryEnabled + && newestClipboard.map { ClipboardHistoryPolicy.isEligibleForAIHint($0, now: now) } == true + } + /// Prompt for a tapped card. Clipboard cards fail closed so an expired /// window can never send an instruction without its material. public static func resolvePrompt( diff --git a/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift b/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift index b1c060a..50a4a93 100644 --- a/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift +++ b/OSGKeyboardShared/Services/FlowKeyboardPolicies.swift @@ -110,11 +110,81 @@ public enum FlowKeyboardAdoptBusyPolicy { guard !isAwaitingFlowResult else { return .none } guard let busyId = snapshot.busyUtteranceId else { return .none } guard busyId != lastConsumedUtteranceId else { return .none } + guard busyId != lastStoppedUtteranceId else { return .none } return .adoptProcessing(sessionId: sessionId, utteranceId: busyId) default: return .none } } + + /// Host still advertises `processing` for an utterance the keyboard already + /// acked, and the result mailbox is empty. That is a leaked gate — not a + /// live ASR/LLM wait (those have no ack yet). + public static func isStaleDeliveredProcessing( + busyUtteranceId: UUID, + latestResult: FlowResult?, + latestAck: FlowAck? + ) -> Bool { + guard let ack = latestAck, ack.utteranceId == busyUtteranceId else { + return false + } + if latestResult?.utteranceId == busyUtteranceId { + return false + } + return true + } +} + +// MARK: - Terminal store after await + +public enum FlowTerminalStorePolicy { + /// After an `await`, only the still-current, not-yet-terminal utterance + /// may write a final/error payload. Abort during LLM must not deliver. + public static func canStore( + currentUtteranceId: UUID?, + finishedUtteranceId: UUID, + alreadyTerminal: Bool + ) -> Bool { + currentUtteranceId == finishedUtteranceId && !alreadyTerminal + } +} + +// MARK: - Ack must drop a leaked processing gate + +public enum FlowHostAckGatePolicy { + /// The keyboard acked this live utterance; the processing flag must not + /// outlive that ack (hint-card used to leak `reason=processing`). + public static func shouldDropProcessingGate( + ackUtteranceId: UUID, + currentUtteranceId: UUID?, + isUtteranceProcessing: Bool + ) -> Bool { + isUtteranceProcessing && currentUtteranceId == ackUtteranceId + } +} + +// MARK: - Empty double-tap skip + +/// Drop a take before ASR when the press was too short to be speech. +public enum FlowEmptyTapSkipPolicy { + public static let maxDurationSeconds: TimeInterval = 0.3 + public static let silencePeakThreshold: Float = + FlowCaptureTailDrainPolicy.flowDefault.silenceRMSThreshold + + public static func peakAbs(_ samples: [Float]) -> Float { + samples.reduce(Float(0)) { max($0, abs($1)) } + } + + /// `sampleCount == 0` or a missing peak counts as silence. + public static func shouldSkip( + durationSeconds: TimeInterval, + sampleCount: Int, + peakAmplitude: Float? + ) -> Bool { + guard durationSeconds < maxDurationSeconds else { return false } + if sampleCount <= 0 { return true } + return (peakAmplitude ?? 0) < silencePeakThreshold + } } // MARK: - Result matching diff --git a/OSGKeyboardShared/Services/FlowSessionKeys.swift b/OSGKeyboardShared/Services/FlowSessionKeys.swift index 5aa0a0d..adeac48 100644 --- a/OSGKeyboardShared/Services/FlowSessionKeys.swift +++ b/OSGKeyboardShared/Services/FlowSessionKeys.swift @@ -139,5 +139,7 @@ public enum FlowSessionKeys { case audioUnavailable case asrFailed case generic + /// Short silent tap discarded before ASR — keyboard returns to idle quietly. + case discardedEmpty } } diff --git a/OSGKeyboardShared/Services/KeyboardState.swift b/OSGKeyboardShared/Services/KeyboardState.swift index 126595b..ee49223 100644 --- a/OSGKeyboardShared/Services/KeyboardState.swift +++ b/OSGKeyboardShared/Services/KeyboardState.swift @@ -239,6 +239,8 @@ public final class KeyboardState: ObservableObject { public var sendAIAnswer: () -> Void = {} /// Sends a tapped idle hint card as the AI question (skip microphone). public var submitAIHint: (AIHintCard) -> Void = { _ in } + /// Sends a clipboard skill (reply / summarize / translate / future). + public var submitAIClipboardSkill: (AIClipboardSkill) -> Void = { _ in } public var openSettings: () -> Void = {} /// Opens the host app straight to input-resource deployment. Used by the /// typing surface when Rime resources have not been deployed yet. @@ -303,8 +305,9 @@ public final class KeyboardState: ObservableObject { surface == .ai && aiSession.isBusy } - /// Normal dictation can be discarded from initial microphone startup - /// through ASR / polish processing. Edit mode owns its separate close flow. + /// Normal dictation can be discarded from microphone startup through + /// ASR / polish, including the abort-wait after Cancel until the host + /// acks (coordinator keeps `phase == .processing` for that window). public var canCancelVoiceInput: Bool { guard !editSession.isActive else { return false } switch phase { @@ -360,6 +363,8 @@ extension KeyboardState.Phase.ErrorKind { return .hostAudioUnavailable case .asrFailed, .generic: return .hostTranscriptionFailed(error.message) + case .discardedEmpty: + return .noSpeechDetected } } } \ No newline at end of file diff --git a/OSGKeyboardShared/Utilities/FlowUtterancePCMStore.swift b/OSGKeyboardShared/Utilities/FlowUtterancePCMStore.swift index fa42122..8b7229d 100644 --- a/OSGKeyboardShared/Utilities/FlowUtterancePCMStore.swift +++ b/OSGKeyboardShared/Utilities/FlowUtterancePCMStore.swift @@ -36,6 +36,11 @@ public final class FlowUtterancePCMStore: @unchecked Sendable { lock.withLock { samples.count } } + /// Copy of accumulated samples without clearing the store. + public func snapshot() -> [Float] { + lock.withLock { samples } + } + /// Returns accumulated samples and clears the store. public func consume() -> [Float] { lock.withLock { diff --git a/OSGKeyboardTests/AIClipboardPromptTests.swift b/OSGKeyboardTests/AIClipboardPromptTests.swift index 567c96d..d177d4e 100644 --- a/OSGKeyboardTests/AIClipboardPromptTests.swift +++ b/OSGKeyboardTests/AIClipboardPromptTests.swift @@ -5,6 +5,15 @@ import XCTest @testable import OSGKeyboardShared final class AIClipboardPromptTests: XCTestCase { + func testInternalClipboardEnvelopeIsDetected() { + let prompt = AIClipboardPrompt.compose( + instruction: "请翻译剪贴板", + material: "hello" + ) + XCTAssertTrue(AIClipboardPrompt.isInternalPrompt(prompt)) + XCTAssertFalse(AIClipboardPrompt.isInternalPrompt("请翻译剪贴板")) + } + func testComposeKeepsInstructionAndMaterialInSeparateBlocks() { let prompt = AIClipboardPrompt.compose( instruction: "请翻译剪贴板", diff --git a/OSGKeyboardTests/AIHintKeywordExtractorTests.swift b/OSGKeyboardTests/AIHintKeywordExtractorTests.swift new file mode 100644 index 0000000..032a7da --- /dev/null +++ b/OSGKeyboardTests/AIHintKeywordExtractorTests.swift @@ -0,0 +1,238 @@ +// AIHintKeywordExtractorTests.swift +// OSGKeyboardTests + +import XCTest +@testable import OSGKeyboardShared + +final class AIHintKeywordExtractorTests: XCTestCase { + func testTrendingPrefersMetadataTitleOverHotPrefix() throws { + let json = """ + {"id":"hot-1","text":"全网热点：朱镕基同志逝世","prompt":"请概括","category":"society","source":"tophub-open-hot","locale":"zh","metadata":{"title":"朱镕基同志逝世"}} + """ + let card = try JSONDecoder().decode(AIHintCard.self, from: Data(json.utf8)) + XCTAssertEqual(card.visualKind, .trending) + XCTAssertEqual(card.visualKind.systemImage, "flame.fill") + XCTAssertEqual(card.resolvedDisplayText, "朱镕基同志逝世") + } + + func testWeatherUsesCityAndTemperature() { + let card = AIHintCard( + id: "weather-zh-上海", + displayText: "上海天气速览", + prompt: "请说明天气", + category: "weather", + source: "open-meteo", + locale: "zh", + metadata: AIHintMetadata(city: "上海", tempC: 25.6) + ) + XCTAssertEqual(card.visualKind, .weather) + XCTAssertEqual(card.resolvedDisplayText, "上海 26°") + } + + func testHolidayStripsPrefixAndUsesChineseName() { + let card = AIHintCard( + id: "holiday-next-cn", + displayText: "临近节日：中秋节", + prompt: "介绍中秋", + category: "holiday", + source: "nager-holidays", + locale: "zh", + conditions: ["date"], + metadata: AIHintMetadata(name: "Mid-Autumn Festival") + ) + XCTAssertEqual(card.visualKind, .calendar) + XCTAssertEqual(card.resolvedDisplayText, "中秋节") + } + + func testEnglishHolidayUsesMetadataName() { + let card = AIHintCard( + id: "holiday-next-us", + displayText: "Upcoming: Labour Day", + prompt: "Explain Labour Day", + category: "holiday", + source: "nager-holidays", + locale: "en", + metadata: AIHintMetadata(name: "Labour Day") + ) + XCTAssertEqual(card.resolvedDisplayText, "Labour Day") + } + + func testLongOrgTitleFallsBackToLastChunk() { + let text = "中共中央 全国人大常委会 国务院 全国政协讣告 朱镕基同志逝世" + XCTAssertEqual( + AIHintKeywordExtractor.finalize(text, locale: "zh"), + "朱镕基同志逝世" + ) + } + + func testMixedTitlePrefersLeadingLatin() { + XCTAssertEqual( + AIHintKeywordExtractor.finalize( + "DeepSeek Pro 正式版已经发布，如何评价该模型？", + locale: "zh" + ), + "DeepSeek" + ) + } + + func testDailyBriefIsNewsAndSoulQuoteIsSearch() { + let brief = AIHintCard( + id: "tophub-daily-brief-1", + displayText: "看看今日早报", + prompt: "写早报", + category: "daily", + source: "tophub-daily", + locale: "zh" + ) + XCTAssertEqual(brief.visualKind, .news) + XCTAssertEqual(brief.resolvedDisplayText, "今日早报") + + let soul = AIHintCard( + id: "tophub-daily-soul-1", + displayText: "今日一句：展开聊聊", + prompt: "解释这句话", + category: "daily", + source: "tophub-daily", + locale: "zh", + metadata: AIHintMetadata(soul: "为了防止我这个月又乱花钱") + ) + XCTAssertEqual(soul.visualKind, .search) + XCTAssertEqual(soul.resolvedDisplayText, "今日金句") + } + + func testCapabilityMapsToSearch() { + let card = AIHintLocalCatalog.cards(locale: "zh") + .first { $0.id == "local-zh-encyclopedia" }! + XCTAssertEqual(card.visualKind, .search) + XCTAssertEqual(card.resolvedDisplayText, "有趣概念") + } + + func testStocksMapsToChartIcon() { + let card = AIHintLocalCatalog.cards(locale: "zh") + .first { $0.id == "local-zh-stocks" }! + XCTAssertEqual(card.visualKind, .stocks) + XCTAssertEqual(card.resolvedDisplayText, "今日大盘") + } +} + +final class AIClipboardSkillTests: XCTestCase { + func testVisibleDefaultsAreReplySummarizeTranslate() { + XCTAssertEqual( + AIClipboardSkillCatalog.visible().map(\.id), + ["reply", "summarize", "translate"] + ) + } + + func testVisibleRespectsEnabledIDsForFutureSettings() { + XCTAssertEqual( + AIClipboardSkillCatalog.visible(enabledIDs: ["translate", "reply"]).map(\.id), + ["translate", "reply"] + ) + XCTAssertEqual( + AIClipboardSkillCatalog.visible(enabledIDs: ["unknown"]).map(\.id), + [] + ) + } + + func testTranslateFollowsUserTargetLanguage() { + let ja = AIClipboardSkillCatalog.instruction( + skillID: AIClipboardSkillCatalog.translateID, + locale: "zh", + translationTargetLocaleId: "ja" + ) + XCTAssertTrue(ja.contains("Japanese")) + XCTAssertFalse(ja.contains("互译")) + } + + func testTranslateFallsBackToChineseEnglishWhenUnset() { + let zh = AIClipboardSkillCatalog.instruction( + skillID: AIClipboardSkillCatalog.translateID, + locale: "zh", + translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId + ) + XCTAssertTrue(zh.contains("中文与英文")) + let en = AIClipboardSkillCatalog.instruction( + skillID: AIClipboardSkillCatalog.translateID, + locale: "en", + translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId + ) + XCTAssertTrue(en.lowercased().contains("chinese")) + XCTAssertTrue(en.lowercased().contains("english")) + } + + func testTranslateButtonTitleUsesDirectionPairsAndTargetShortNames() { + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId, + uiLanguage: .chinese + ), + "中↔英" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId, + uiLanguage: .english + ), + "CN↔EN" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "en", + uiLanguage: .chinese + ), + "中译英" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "de", + uiLanguage: .chinese + ), + "中译德" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "ja", + uiLanguage: .english + ), + "To JP" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "zh-Hans", + uiLanguage: .english + ), + "To CN" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "zh-Hant", + uiLanguage: .english + ), + "To TW" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "zh-Hans", + uiLanguage: .chinese + ), + "简↔繁" + ) + XCTAssertEqual( + AIClipboardSkillCatalog.translateButtonTitle( + translationTargetLocaleId: "zh-Hant", + uiLanguage: .chinese + ), + "简↔繁" + ) + } + + func testSummarizeAsksForOverviewNotShortening() { + let prompt = AIClipboardSkillCatalog.instruction( + skillID: AIClipboardSkillCatalog.summarizeID, + locale: "zh", + translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId + ) + XCTAssertTrue(prompt.contains("概括")) + XCTAssertTrue(prompt.contains("不要改写成可发送的短消息")) + } +} diff --git a/OSGKeyboardTests/AIHintPoolTests.swift b/OSGKeyboardTests/AIHintPoolTests.swift index 7ab0b1e..2793426 100644 --- a/OSGKeyboardTests/AIHintPoolTests.swift +++ b/OSGKeyboardTests/AIHintPoolTests.swift @@ -12,19 +12,21 @@ final class AIHintPoolTests: XCTestCase { XCTAssertEqual(AIHintLocaleResolver.packLocale(preferredLanguages: ["en-US"]), "en") } - func testClipboardWindowForcesOnlyClipboardCards() { + func testClipboardWindowNeverPutsClipboardCardsInCarousel() { let pack = AIHintPack( locale: "zh", cards: AIHintLocalCatalog.cards(locale: "zh") ) let recent = ClipboardHistoryEntry(text: "hello", createdAt: Date()) - let cards = AIHintPool.activeCards( - pack: pack, - clipboardHistoryEnabled: true, - newestClipboard: recent + XCTAssertTrue( + AIHintPool.isClipboardSkillWindowActive( + clipboardHistoryEnabled: true, + newestClipboard: recent + ) ) + let cards = AIHintPool.activeCards(pack: pack) XCTAssertFalse(cards.isEmpty) - XCTAssertTrue(cards.allSatisfy(\.requiresClipboard30s)) + XCTAssertTrue(cards.allSatisfy { !$0.requiresClipboard30s }) } func testClipboardDisabledDropsClipboardCards() { @@ -32,11 +34,7 @@ final class AIHintPoolTests: XCTestCase { locale: "zh", cards: AIHintLocalCatalog.cards(locale: "zh") ) - let cards = AIHintPool.activeCards( - pack: pack, - clipboardHistoryEnabled: false, - newestClipboard: ClipboardHistoryEntry(text: "hello") - ) + let cards = AIHintPool.activeCards(pack: pack) XCTAssertFalse(cards.isEmpty) XCTAssertTrue(cards.allSatisfy { !$0.requiresClipboard30s }) } diff --git a/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift b/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift index d498f6b..4a7d06a 100644 --- a/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift +++ b/OSGKeyboardTests/FlowKeyboardPoliciesTests.swift @@ -188,6 +188,28 @@ final class FlowKeyboardPoliciesTests: XCTestCase { ), .none ) + + let processing = FlowReadySnapshot( + sessionId: sessionId, + ready: false, + reason: .processing, + engineMode: "cloud", + localeId: "zh-Hans", + busyUtteranceId: utteranceId, + hostGeneration: "gen-1" + ) + XCTAssertEqual( + FlowKeyboardAdoptBusyPolicy.decide( + snapshot: processing, + currentHostGeneration: "gen-1", + isFlowRecording: false, + isAwaitingFlowResult: false, + lastConsumedUtteranceId: nil, + lastStoppedUtteranceId: utteranceId + ), + .none, + "an aborted utterance must not come back as 识别中 on the next open" + ) } func testIgnoresDeadHostGenerationSnapshot() { @@ -234,6 +256,138 @@ final class FlowKeyboardPoliciesTests: XCTestCase { ) } + func testStaleDeliveredProcessingRequiresMatchingAckWithoutResult() { + let busyId = UUID() + let sessionId = UUID() + XCTAssertFalse( + FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing( + busyUtteranceId: busyId, + latestResult: nil, + latestAck: nil + ), + "live ASR/LLM has no result and no ack yet — must wait, not abort" + ) + + let otherAck = FlowAck( + sessionId: sessionId, + utteranceId: UUID(), + commandSeq: 1 + ) + XCTAssertFalse( + FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing( + busyUtteranceId: busyId, + latestResult: nil, + latestAck: otherAck + ), + "ack for a previous utterance must not abort the live one" + ) + + let matchingAck = FlowAck( + sessionId: sessionId, + utteranceId: busyId, + commandSeq: 1 + ) + XCTAssertTrue( + FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing( + busyUtteranceId: busyId, + latestResult: nil, + latestAck: matchingAck + ), + "acked + empty mailbox + still processing is a leaked gate" + ) + + let liveResult = FlowResult( + sessionId: sessionId, + utteranceId: busyId, + commandSeq: 1, + status: .streaming, + text: "draft" + ) + XCTAssertFalse( + FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing( + busyUtteranceId: busyId, + latestResult: liveResult, + latestAck: matchingAck + ), + "result still sitting for this utterance is waitable, not stale" + ) + XCTAssertFalse( + FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing( + busyUtteranceId: UUID(), + latestResult: liveResult, + latestAck: matchingAck + ) + ) + } + + func testTerminalStorePolicyRejectsAbortAndReplacement() { + let live = UUID() + XCTAssertTrue( + FlowTerminalStorePolicy.canStore( + currentUtteranceId: live, + finishedUtteranceId: live, + alreadyTerminal: false + ) + ) + XCTAssertFalse( + FlowTerminalStorePolicy.canStore( + currentUtteranceId: nil, + finishedUtteranceId: live, + alreadyTerminal: false + ), + "abort cleared currentUtteranceId — do not deliver" + ) + XCTAssertFalse( + FlowTerminalStorePolicy.canStore( + currentUtteranceId: UUID(), + finishedUtteranceId: live, + alreadyTerminal: false + ), + "a newer utterance owns the gate" + ) + XCTAssertFalse( + FlowTerminalStorePolicy.canStore( + currentUtteranceId: live, + finishedUtteranceId: live, + alreadyTerminal: true + ), + "already terminal — abort already claimed" + ) + } + + func testAckGateDropsProcessingOnlyForTheLiveUtterance() { + let live = UUID() + XCTAssertTrue( + FlowHostAckGatePolicy.shouldDropProcessingGate( + ackUtteranceId: live, + currentUtteranceId: live, + isUtteranceProcessing: true + ) + ) + XCTAssertFalse( + FlowHostAckGatePolicy.shouldDropProcessingGate( + ackUtteranceId: live, + currentUtteranceId: live, + isUtteranceProcessing: false + ) + ) + XCTAssertFalse( + FlowHostAckGatePolicy.shouldDropProcessingGate( + ackUtteranceId: live, + currentUtteranceId: UUID(), + isUtteranceProcessing: true + ), + "must not clobber a newer utterance" + ) + XCTAssertFalse( + FlowHostAckGatePolicy.shouldDropProcessingGate( + ackUtteranceId: live, + currentUtteranceId: nil, + isUtteranceProcessing: true + ) + ) + } + // MARK: - Result matcher func testMatchingResultRequiresAlignedSessionAndUtterance() { @@ -371,3 +525,58 @@ final class FlowKeyboardPoliciesTests: XCTestCase { ) } } + +// MARK: - Empty tap skip + +extension FlowKeyboardPoliciesTests { + func testEmptyTapSkipRequiresShortDuration() { + XCTAssertTrue( + FlowEmptyTapSkipPolicy.shouldSkip( + durationSeconds: 0.2, + sampleCount: 0, + peakAmplitude: nil + ) + ) + XCTAssertFalse( + FlowEmptyTapSkipPolicy.shouldSkip( + durationSeconds: 0.35, + sampleCount: 0, + peakAmplitude: nil + ) + ) + } + + func testEmptyTapSkipTreatsMissingSamplesAsSilence() { + XCTAssertTrue( + FlowEmptyTapSkipPolicy.shouldSkip( + durationSeconds: 0.1, + sampleCount: 16, + peakAmplitude: nil + ) + ) + } + + func testEmptyTapSkipKeepsShortSpeech() { + XCTAssertFalse( + FlowEmptyTapSkipPolicy.shouldSkip( + durationSeconds: 0.2, + sampleCount: 3_200, + peakAmplitude: 0.2 + ) + ) + } + + func testEmptyTapSkipDropsShortSilence() { + XCTAssertTrue( + FlowEmptyTapSkipPolicy.shouldSkip( + durationSeconds: 0.2, + sampleCount: 3_200, + peakAmplitude: 0.001 + ) + ) + XCTAssertEqual( + FlowEmptyTapSkipPolicy.peakAbs([0.001, -0.004, 0.002]), + 0.004 + ) + } +} diff --git a/OSGKeyboardTests/FlowUtterancePCMStoreTests.swift b/OSGKeyboardTests/FlowUtterancePCMStoreTests.swift index e4f4af6..6c6d131 100644 --- a/OSGKeyboardTests/FlowUtterancePCMStoreTests.swift +++ b/OSGKeyboardTests/FlowUtterancePCMStoreTests.swift @@ -20,4 +20,12 @@ final class FlowUtterancePCMStoreTests: XCTestCase { store.append([1, 2, 3, 4, 5]) XCTAssertEqual(store.consume(), [2, 3, 4, 5]) } + + func testSnapshotDoesNotClear() { + let store = FlowUtterancePCMStore(maxSampleCount: 100) + store.append([0.1, -0.2]) + XCTAssertEqual(store.snapshot(), [0.1, -0.2]) + XCTAssertEqual(store.sampleCount, 2) + XCTAssertEqual(store.consume(), [0.1, -0.2]) + } } diff --git a/README.en.md b/README.en.md index a38d133..786a7c1 100644 --- a/README.en.md +++ b/README.en.md @@ -7,7 +7,7 @@ Voice input for iPhone, iPad, and Mac. Speak in any app — polished text lands ![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple) ![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple) ![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift) -![Version](https://img.shields.io/badge/version-1.7.0-3aa05a) +![Version](https://img.shields.io/badge/version-1.7.5-3aa05a) ![License](https://img.shields.io/badge/license-Source%20Available-blue) [Website](https://hkgood.github.io/OSGKeyboard/) · [中文 README](./README.md) · [Privacy Policy](https://hkgood.github.io/OSGKeyboard/privacy/) diff --git a/README.md b/README.md index 488184b..fa2c5e0 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ ![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple) ![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple) ![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift) -![Version](https://img.shields.io/badge/version-1.7.0-3aa05a) +![Version](https://img.shields.io/badge/version-1.7.5-3aa05a) ![License](https://img.shields.io/badge/license-Source%20Available-blue) [官网](https://hkgood.github.io/OSGKeyboard/) · [English](./README.en.md) · [隐私政策](https://hkgood.github.io/OSGKeyboard/privacy/) diff --git a/docs/APPSTORE_METADATA.md b/docs/APPSTORE_METADATA.md index e05cf3c..38a818b 100644 --- a/docs/APPSTORE_METADATA.md +++ b/docs/APPSTORE_METADATA.md @@ -1,4 +1,4 @@ -# App Store Connect — OSGKeyboard 1.7.0 (build 65) +# App Store Connect — OSGKeyboard 1.7.5 (build 66) > Current metadata baseline for the iOS/iPadOS App Store build. Version and build > numbers come from `project.yml`. The repository also contains a separate @@ -11,7 +11,7 @@ | App name | `OSGKeyboard` | ≤ 30 characters | | Subtitle | `Voice input, everywhere` | ≤ 30 characters | | Bundle ID | `com.osgkeyboard.ios` | iOS host target | -| Version / build | `1.7.0` / `65` | `MARKETING_VERSION` / `CURRENT_PROJECT_VERSION` | +| Version / build | `1.7.5` / `66` | `MARKETING_VERSION` / `CURRENT_PROJECT_VERSION` | | Minimum system | iOS/iPadOS 26 | iPhone and iPad | | Primary locale | `en-US` | Simplified Chinese is also bundled | | Primary category | Utilities | | @@ -101,22 +101,25 @@ Voice input anywhere, with on-device recognition by default. Add your own AI key keyboard,voice,dictation,speech,transcribe,AI,pinyin,Chinese,English,polish,typing,productivity ``` -## What's new in 1.7.0 +## What's new in 1.7.5 ```text NEW -• AI keyboard mode turns spoken questions into reviewable answers, with - explicit Insert and Send actions. -• Provider-supported web search is available for AI questions, with a - no-search fallback if the provider rejects the request. -• Long-press the microphone to describe an edit to the last verified - OSGKeyboard insertion. +• Optional clipboard history (latest 15 plain-text items) with a top-bar + panel and suggestion strip; AI idle hints can use a fresh copy for ~30s. +• AI mode empty state rotates one-line suggestions; tap sends the card + prompt without using the mic. +• Spoken clipboard requests in AI mode attach stored clipboard text when + you name the clipboard out loud. +• Settings → AI Agent adds Response length (Short / Medium / Detailed). +• AI answers stream into the keyboard as the model writes. CHANGED -• Polish and AI mode now use only the API key you configure; the built-in - DeepSeek fallback has been removed. -• iPad voice and typing surfaces include the system globe key, and the - typing layout adds iPad-specific sizing and editing controls. +• History and Personal dictionary move from the tab bar onto Home preview + cards; engine status scrolls with the page. +• Translation control sits beside the mic (mirrored with Undo); the former + top-bar slot is the Clipboard button. +• Undo covers clipboard pastes as well as dictation, AI answers, and edits. ``` ## App Review information @@ -198,10 +201,10 @@ standard HTTPS. Re-evaluate this answer if non-exempt cryptography is added. ## Submission checklist -- [ ] Confirm `project.yml` still reads version 1.7.0 / build 65 -- [ ] Generate the project with `./Scripts/generate-xcodeproj.sh` +- [ ] Confirm `project.yml` still reads version 1.7.5 / build 66 +- [ ] Open the existing Xcode project (do not regenerate unless needed) - [ ] Run the release build and test suites on macOS with Xcode 26 - [ ] Replace screenshots with captures from the submitted build - [ ] Verify the privacy answers against the submitted provider features - [ ] Confirm the tip product remains optional and unlocks no feature -- [ ] Upload, select build 65, add review notes, and submit +- [ ] Upload, select build 66, add review notes, and submit diff --git a/docs/index.html b/docs/index.html index 81d87da..47120c0 100644 --- a/docs/index.html +++ b/docs/index.html @@ -68,7 +68,7 @@ "image": "https://hkgood.github.io/OSGKeyboard/assets/app-icon.png", "screenshot": "https://hkgood.github.io/OSGKeyboard/assets/hero-devices.png", "description": "Source-available voice input for iPhone, iPad, and Mac. On-device speech recognition by default, BYOK AI polish, personal dictionary, and Mac global hotkey dictation.", - "softwareVersion": "1.7.0", + "softwareVersion": "1.7.5", "isAccessibleForFree": true, "offers": { "@type": "Offer", @@ -1280,7 +1280,7 @@