feat(keyboard): add clipboard AI skills, hint keywords, and voice session fixes

Idle chips show entities with category icons; a fresh copy surfaces Reply/Summarize/Translate; abort/cancel/empty-tap no longer leave the mic stuck.
This commit is contained in:
Rocky
2026-08-13 15:47:32 +08:00
parent 9f308fadd2
commit 631617ee53
48 changed files with 1627 additions and 345 deletions
+1 -1
View File
@@ -14,7 +14,7 @@ writing commit messages that will ship to users.
### Version format ### Version format
The current source-of-truth version is **1.7.0 (build 65)**. Releases use stable SemVer: The current source-of-truth version is **1.7.5 (build 67)**. Releases use stable SemVer:
| Field | File | Rule | | Field | File | Rule |
|-------|------|------| |-------|------|------|
+22
View File
@@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Changed
- **AI idle hint keywords**: chips show the entity (from feed `metadata.title` / city / holiday name) instead of a long sentence; category prefixes like「全网热点:」are dropped, and LLM compression is only a last resort. / **AI 空闲建议关键词**:芯片展示实体(来自 feed 的 `metadata.title` / 城市 / 节日名)而不再是长句;去掉「全网热点:」一类前缀,LLM 压缩仅作兜底。
- **AI idle hint chrome**: each rotating suggestion sits in a Liquid Glass capsule with a category SF Symbol (calendar, weather, news, stocks, trending, search). / **AI 空闲建议样式**:轮播建议放入 Liquid Glass 胶囊,左侧为类型 SF Symbol(日历、天气、新闻、股票、热搜、搜索)。
- **Clipboard AI skills**: within 30 seconds of a copy, AI idle shows compact circular Reply / Summarize / Translate buttons; translate follows the keyboard language setting, or Chinese ↔ English when unset. The skill list is catalog-based so more actions can be added later. / **剪贴板 AI 技能**:复制后约 30 秒内,空闲态改为小圆形「回复 / 总结 / 翻译」按钮;翻译跟随键盘目标语言,未设置时中英互译。技能来自目录,便于日后扩展。
- **Clipboard Translate label**: the Translate chip shows the live direction — `中译英` / `To JP` when a target is set, `中↔英` / `CN↔EN` when unset, and `简↔繁` when Chinese UI targets 简体 or 繁體. / **剪贴板翻译文案**:翻译按钮按当前设置显示方向——已选目标为「中译英」/「To JP」,未设置为「中↔英」/「CN↔EN」,中文界面目标为简繁时为「简↔繁」。
- **AI idle capsule size**: rotating suggestion chips keep a 44 pt tap height with 16 pt side padding. / **AI 空闲胶囊尺寸**:轮播建议芯片保持 44 pt 点击高度,左右各 16 pt 内边距。
- **Undo / translation chrome**: voice mic-row undo and translation controls are 52 pt circular Liquid Glass buttons. / **撤销与翻译按钮**:语音麦克风行的撤销、翻译改为 52 pt 圆形 Liquid Glass 按钮。
- **Clipboard skill circles**: Reply / Summarize / Translate idle buttons match the same 52 pt circle. / **剪贴板技能圆钮**:回复 / 总结 / 翻译空闲按钮与语音侧键同为 52 pt 圆。
- **Home tab selection**: the dock stays Liquid Glass at its original height; the selected tab is a wider green fill capsule with a 5 pt inset, not a second glass chip. Dock items are 24 pt icons with Home / Styles / Settings labels. / **首页 Tab 选中态**dock 仍是原高度 Liquid Glass;选中项改为更宽的绿色填充胶囊,距栏边 5 pt,不再套第二层玻璃。dock 为 24 pt 图标加「首页 / 风格 / 设置」文字。
- **Home library card titles**: History and Personal dictionary headers use a 16 pt icon and 13 pt label. / **首页资料卡标题**:历史与个性词库标题改为 16 pt 图标、13 pt 文字。
### Fixed
- **Clipboard skills jumped to a sentence chip**: copying no longer swaps the three skill buttons for a leftover carousel card such as “translate the clipboard”. Clipboard sentence cards stay out of the idle rotation. / **剪贴板技能跳成句子芯片**:复制后不再把三个技能按钮换成「把剪贴板翻译成…」这类旧轮播卡片;剪贴板句子不再进入空闲轮播。
- **Clipboard skill status XML**: tapping Reply / Summarize / Translate no longer flashes the internal `<clipboard_request>` envelope above the mic. / **剪贴板技能状态 XML**:点回复 / 总结 / 翻译不再把内部 `<clipboard_request>` 信封闪现在麦克风上方。
- **Idle keyboard re-adopted recognition**: aborting a session now remembers that utterance id, so a lagging host `processing` snapshot cannot reopen the voice keyboard in 「识别中」. / **空闲打开误进识别中**:中止会话会记住该句 id,滞后的主机 processing 快照不会在下次打开语音键盘时显示「识别中」。
- **Voice cancel chrome until result**: Cancel (X) stays up through ASR/polish and through abort wait after leaving AI Agent, so the voice mic no longer looks ready while the host is still busy. / **语音取消直到出结果**:X 一直保留到识别/润色结束,以及离开 AI Agent 后的中断收尾;语音麦克风不再在宿主仍忙时显示为可用。
- **Empty double-tap skip**: a mic press shorter than 300 ms with near-silence (or no samples) is discarded before ASR, so accidental double taps no longer wait on a no-speech error. / **空连点跳过**:按下短于 300ms 且接近静音(或尚无样本)的录音在进入识别前丢弃,避免误触后长时间等待「没听清」。
- **AI hint left the mic stuck**: prefilled AI questions now drop the host processing gate after the answer or error; consuming an ack for the live utterance also heals a leaked gate. Reopening the keyboard only aborts when App Group already acked that utterance and the result is gone — a live LLM/ASR wait with no result yet is left running. Cancel during generate no longer delivers the answer afterwards. / **AI 建议卡住麦克风**:预填 AI 问题在出答案或失败后会关掉宿主 processing 闸门;若键盘已 ack 当前句而闸门仍开着,宿主消费 ack 时一并放闸。再次打开键盘仅在 App Group 已 ack 且结果已空时才中断残留闸门;LLM/ASR 仍在跑、尚无结果时继续等待。生成中取消后不再把答案写回来。
## [1.7.5] - 2026-08-13
### Added ### Added
- **AI idle hint carousel**: AI mode empty state rotates one-line suggestions (local evergreen + optional remote hot topics); tap sends the card prompt to the LLM without the mic. Coexists with the clipboard suggestion strip in the top bar. / **AI 空闲建议轮播**:AI 模式空状态轮播单行建议(内置常驻 + 可选远程热点);点按即跳过麦克风把卡片 prompt 发给模型。与顶栏剪贴板建议条并存。 - **AI idle hint carousel**: AI mode empty state rotates one-line suggestions (local evergreen + optional remote hot topics); tap sends the card prompt to the LLM without the mic. Coexists with the clipboard suggestion strip in the top bar. / **AI 空闲建议轮播**:AI 模式空状态轮播单行建议(内置常驻 + 可选远程热点);点按即跳过麦克风把卡片 prompt 发给模型。与顶栏剪贴板建议条并存。
- **Hint feed refresh**: main app silently fetches `https://key.osglab.com/hints` about every 12 hours, compresses titles with the users polish LLM, and writes App Group ready packs for the keyboard. / **建议源刷新**:主 App 约每 12 小时静默拉取 `https://key.osglab.com/hints`,用用户润色 LLM 压缩标题,写入 App Group 供键盘只读。 - **Hint feed refresh**: main app silently fetches `https://key.osglab.com/hints` about every 12 hours, compresses titles with the users polish LLM, and writes App Group ready packs for the keyboard. / **建议源刷新**:主 App 约每 12 小时静默拉取 `https://key.osglab.com/hints`,用用户润色 LLM 压缩标题,写入 App Group 供键盘只读。
@@ -1,8 +1,9 @@
// AIHintRefreshService.swift // AIHintRefreshService.swift
// OSGKeyboard · Main App // OSGKeyboard · Main App
// //
// Silent 12h refresh: fetch remote packs, compress titles with polish LLM, // Silent 12h refresh: fetch remote packs, extract keywords (LLM only
// merge local evergreen cards, write App Group ready packs for the keyboard. // for leftovers), merge local evergreen cards, write App Group ready
// packs for the keyboard.
import Foundation import Foundation
import OSGKeyboardShared import OSGKeyboardShared
+86 -16
View File
@@ -1069,9 +1069,21 @@ final class FlowSessionManager: ObservableObject {
FlowSessionBridge.clearResult() FlowSessionBridge.clearResult()
// Ack clears the terminal result; republish ready immediately so the // Ack clears the terminal result; republish ready immediately so the
// keyboard does not linger on awaitingDelivery / starting between the // keyboard does not linger on awaitingDelivery / starting between the
// 500 ms poll and the next heartbeat. // 500 ms poll and the next heartbeat. If processing is still true for
// this same utterance, the writer forgot to drop the gate heal here.
if FlowHostAckGatePolicy.shouldDropProcessingGate(
ackUtteranceId: ack.utteranceId,
currentUtteranceId: currentUtteranceId,
isUtteranceProcessing: isUtteranceProcessing
) {
completeFinalizeCleanup(
sessionId: ack.sessionId,
utteranceId: ack.utteranceId
)
} else {
refreshHostReady() refreshHostReady()
} }
}
private func hasUnacknowledgedTerminalResult() -> Bool { private func hasUnacknowledgedTerminalResult() -> Bool {
guard let result = FlowSessionBridge.latestResult(), guard let result = FlowSessionBridge.latestResult(),
@@ -1280,6 +1292,10 @@ final class FlowSessionManager: ObservableObject {
case .cancelPrimeAudio: case .cancelPrimeAudio:
cancelAudioPrime(command) cancelAudioPrime(command)
case .endAIConversation: case .endAIConversation:
if currentUtteranceMode == .aiQuestion,
(isUtteranceProcessing || isUtteranceRecording) {
abortUtterance()
}
if let conversationID = command.aiConversationID { if let conversationID = command.aiConversationID {
Task { await aiConversations.removeConversation(conversationID) } Task { await aiConversations.removeConversation(conversationID) }
} }
@@ -1371,8 +1387,17 @@ final class FlowSessionManager: ObservableObject {
utteranceId: UUID, utteranceId: UUID,
commandSeq: Int64 commandSeq: Int64
) async { ) async {
// Hint-card / prefilled questions never go through `finalizeUtterance`,
// so this path must drop the processing gate itself. Leaving it set
// keeps `reason=processing` forever and the keyboard mic unclickable.
defer {
completeFinalizeCleanup(
sessionId: sessionId,
utteranceId: utteranceId
)
}
guard let conversationID else { guard let conversationID else {
guard claimTerminal(utteranceId: utteranceId) else { return } claimAndStoreTerminal(utteranceId: utteranceId) {
storeFinalizedError( storeFinalizedError(
AppL10n.string("flow.error.aiQuestionFailed"), AppL10n.string("flow.error.aiQuestionFailed"),
kind: .generic, kind: .generic,
@@ -1380,16 +1405,12 @@ final class FlowSessionManager: ObservableObject {
utteranceId: utteranceId, utteranceId: utteranceId,
commandSeq: commandSeq commandSeq: commandSeq
) )
}
return return
} }
storeRawCandidate( // Do not publish the prompt as a "transcript" clipboard skills
question, // send an XML envelope that must never appear above the mic.
sessionId: sessionId,
utteranceId: utteranceId,
commandSeq: commandSeq
)
let pipelineStore = AppGroupStore() let pipelineStore = AppGroupStore()
do { do {
let service = try AIQuestionService.configured( let service = try AIQuestionService.configured(
@@ -1413,6 +1434,8 @@ final class FlowSessionManager: ObservableObject {
) )
} }
} }
// Abort during the LLM await must not commit or deliver.
guard canStoreTerminal(for: utteranceId) else { return }
publishStreamingAIAnswerIfNeeded( publishStreamingAIAnswerIfNeeded(
answer, answer,
sessionId: sessionId, sessionId: sessionId,
@@ -1421,12 +1444,12 @@ final class FlowSessionManager: ObservableObject {
aiConversationID: conversationID, aiConversationID: conversationID,
force: true force: true
) )
guard claimTerminal(utteranceId: utteranceId) else { return }
await service.commitSuccessfulTurn( await service.commitSuccessfulTurn(
question: question, question: question,
answer: answer, answer: answer,
conversationID: conversationID conversationID: conversationID
) )
claimAndStoreTerminal(utteranceId: utteranceId) {
storeFinalizedResult( storeFinalizedResult(
answer, answer,
warning: nil, warning: nil,
@@ -1435,8 +1458,9 @@ final class FlowSessionManager: ObservableObject {
commandSeq: commandSeq, commandSeq: commandSeq,
aiConversationID: conversationID aiConversationID: conversationID
) )
}
} catch { } catch {
guard claimTerminal(utteranceId: utteranceId) else { return } claimAndStoreTerminal(utteranceId: utteranceId) {
storeFinalizedError( storeFinalizedError(
AppL10n.string("flow.error.aiQuestionFailed"), AppL10n.string("flow.error.aiQuestionFailed"),
kind: .generic, kind: .generic,
@@ -1447,6 +1471,7 @@ final class FlowSessionManager: ObservableObject {
) )
} }
} }
}
private func beginAudioPrime(_ command: FlowCommand) { private func beginAudioPrime(_ command: FlowCommand) {
guard startingUtteranceId == nil, guard startingUtteranceId == nil,
@@ -1993,6 +2018,26 @@ final class FlowSessionManager: ObservableObject {
private func endUtterance() { private func endUtterance() {
guard isUtteranceRecording else { return } guard isUtteranceRecording else { return }
let duration = utteranceRecordingStartedAt.map { Date().timeIntervalSince($0) } ?? 0
if currentUtteranceMode != .editLastInput,
duration < FlowEmptyTapSkipPolicy.maxDurationSeconds {
let samples = capture.utterancePCMSnapshot()
let peak = FlowEmptyTapSkipPolicy.peakAbs(samples)
if FlowEmptyTapSkipPolicy.shouldSkip(
durationSeconds: duration,
sampleCount: samples.count,
peakAmplitude: peak
) {
FlowTrace.pipeline(
"utterance.skipEmptyTap",
"duration=\(String(format: "%.3f", duration))s "
+ "samples=\(samples.count) peak=\(String(format: "%.4f", peak))"
)
abortUtterance(kind: .discardedEmpty, message: "")
return
}
}
// Close the mic gate first, then mark processing before dropping the // Close the mic gate first, then mark processing before dropping the
// recording flag so the poll loop cannot start a second utterance. // recording flag so the poll loop cannot start a second utterance.
isUtteranceRecording = false isUtteranceRecording = false
@@ -2040,12 +2085,15 @@ final class FlowSessionManager: ObservableObject {
debug("utterance stopped, draining tail") debug("utterance stopped, draining tail")
} }
private func abortUtterance() { private func abortUtterance(
kind: FlowSessionKeys.TranscriptionErrorKind = .recognitionInterrupted,
message: String? = nil
) {
let utteranceId = currentUtteranceId let utteranceId = currentUtteranceId
if claimTerminal(utteranceId: utteranceId) { if claimTerminal(utteranceId: utteranceId) {
storeCurrentError( storeCurrentError(
AppL10n.string("flow.error.recognitionInterrupted"), message ?? AppL10n.string("flow.error.recognitionInterrupted"),
kind: .recognitionInterrupted, kind: kind,
status: .aborted status: .aborted
) )
} }
@@ -2125,6 +2173,24 @@ final class FlowSessionManager: ObservableObject {
return true return true
} }
/// After an `await`, refuse to write if this utterance was aborted or replaced.
private func canStoreTerminal(for utteranceId: UUID?) -> Bool {
guard let utteranceId else { return false }
return FlowTerminalStorePolicy.canStore(
currentUtteranceId: currentUtteranceId,
finishedUtteranceId: utteranceId,
alreadyTerminal: terminalUtteranceIds.contains(utteranceId)
)
}
/// Claim + write with no `await` between them, so abort cannot sneak a
/// second payload in after `claimTerminal`.
private func claimAndStoreTerminal(utteranceId: UUID?, store: () -> Void) {
guard canStoreTerminal(for: utteranceId) else { return }
guard claimTerminal(utteranceId: utteranceId) else { return }
store()
}
private func clearPendingInstructionState() { private func clearPendingInstructionState() {
pendingEditSourceText = nil pendingEditSourceText = nil
pendingSourceHistoryEntryID = nil pendingSourceHistoryEntryID = nil
@@ -2358,6 +2424,8 @@ final class FlowSessionManager: ObservableObject {
) )
} }
} }
// Abort during the LLM await must not commit or deliver.
guard canStoreTerminal(for: finalizeUtteranceId) else { return }
// Flush the final draft when throttle skipped the last characters. // Flush the final draft when throttle skipped the last characters.
publishStreamingAIAnswerIfNeeded( publishStreamingAIAnswerIfNeeded(
answer, answer,
@@ -2367,12 +2435,12 @@ final class FlowSessionManager: ObservableObject {
aiConversationID: aiConversationID, aiConversationID: aiConversationID,
force: true force: true
) )
guard claimTerminal(utteranceId: finalizeUtteranceId) else { return }
await service.commitSuccessfulTurn( await service.commitSuccessfulTurn(
question: question, question: question,
answer: answer, answer: answer,
conversationID: aiConversationID conversationID: aiConversationID
) )
claimAndStoreTerminal(utteranceId: finalizeUtteranceId) {
storeFinalizedResult( storeFinalizedResult(
answer, answer,
warning: chunkNote, warning: chunkNote,
@@ -2381,8 +2449,9 @@ final class FlowSessionManager: ObservableObject {
commandSeq: finalizeCommandSeq, commandSeq: finalizeCommandSeq,
aiConversationID: aiConversationID aiConversationID: aiConversationID
) )
}
} catch { } catch {
guard claimTerminal(utteranceId: finalizeUtteranceId) else { return } claimAndStoreTerminal(utteranceId: finalizeUtteranceId) {
storeFinalizedError( storeFinalizedError(
AppL10n.string("flow.error.aiQuestionFailed"), AppL10n.string("flow.error.aiQuestionFailed"),
kind: .generic, kind: .generic,
@@ -2392,6 +2461,7 @@ final class FlowSessionManager: ObservableObject {
aiConversationID: aiConversationID aiConversationID: aiConversationID
) )
} }
}
return return
} }
@@ -13,7 +13,7 @@ import OSGKeyboardShared
struct ClipboardHistoryDemoView: View { struct ClipboardHistoryDemoView: View {
private enum Layout { private enum Layout {
static let micSize: CGFloat = 121 static let micSize: CGFloat = 121
static let undoSize: CGFloat = 44 static let undoSize: CGFloat = 52
static let micToButtonGap: CGFloat = 8 static let micToButtonGap: CGFloat = 8
static let actionClusterTopGap: CGFloat = Spacing.xl static let actionClusterTopGap: CGFloat = Spacing.xl
static let micUpwardAdjustment: CGFloat = static let micUpwardAdjustment: CGFloat =
@@ -1,9 +1,9 @@
// MinimalTabBar.swift // MinimalTabBar.swift
// OSGKeyboard · Main App // OSGKeyboard · Main App
// //
// Bottom tab bar three icons, no labels. // Bottom tab bar icon + label for Home / Styles / Settings.
// Capsule uses iOS 26 Liquid Glass (.regular.interactive) so content // The dock capsule is iOS 26 Liquid Glass; the selected tab is a green
// behind the dock refracts through on scroll. // fill inside that capsule (Photos-style), not a second glass layer.
// History + dictionary live as Home cards (not dock tabs). // History + dictionary live as Home cards (not dock tabs).
import SwiftUI import SwiftUI
@@ -53,11 +53,10 @@ enum AppTab: Int, CaseIterable {
struct MinimalTabBar: View { struct MinimalTabBar: View {
@Environment(\.themePalette) private var palette: ThemePalette @Environment(\.themePalette) private var palette: ThemePalette
@Environment(\.colorScheme) private var colorScheme @Environment(\.colorScheme) private var colorScheme
@Namespace private var selectionGlassNamespace @Namespace private var selectionNamespace
@Binding var selection: AppTab @Binding var selection: AppTab
var body: some View { var body: some View {
GlassEffectContainer(spacing: 0) {
HStack(spacing: 0) { HStack(spacing: 0) {
ForEach(AppTab.allCases, id: \.rawValue) { tab in ForEach(AppTab.allCases, id: \.rawValue) { tab in
Button { Button {
@@ -65,37 +64,39 @@ struct MinimalTabBar: View {
selection = tab selection = tab
} }
} label: { } label: {
VStack(spacing: 2) {
Group { Group {
if let sfSymbol = tab.sfSymbol { if let sfSymbol = tab.sfSymbol {
Image(systemName: sfSymbol) Image(systemName: sfSymbol)
.font(.system(size: 20, weight: .regular)) .font(.system(size: TabBarDockMetrics.iconSize, weight: .regular))
} else { } else {
MaterialIcon(name: tab.icon, size: 24) MaterialIcon(name: tab.icon, size: TabBarDockMetrics.iconSize)
} }
} }
Text(tab.accessibilityKey)
.font(TypeStyle.caption2)
.lineLimit(1)
.minimumScaleFactor(0.8)
}
.foregroundStyle(tabIconColor(for: tab)) .foregroundStyle(tabIconColor(for: tab))
.frame(maxWidth: .infinity) .frame(maxWidth: .infinity)
.frame(height: 48) .frame(height: TabBarDockMetrics.itemHeight)
.background { .background {
if selection == tab { if selection == tab {
// Capsule, not circle: a circle would inscribe to the // Stretch into the dock padding so top/bottom
// smaller edge and leave the tab slot looking empty. // leftover matches the side leftover (~5 pt).
Color.clear Capsule()
.frame(width: 52, height: 44) .fill(palette.accentMuted)
.glassEffect( .frame(maxWidth: .infinity, maxHeight: .infinity)
.regular .padding(.horizontal, TabBarDockMetrics.selectionInset)
.tint(palette.accent.opacity(0.18)) .padding(
.interactive(), .vertical,
in: .capsule TabBarDockMetrics.selectionInset
- TabBarDockMetrics.dockInsetVertical
) )
.glassEffectID(
"main-tab-selection",
in: selectionGlassNamespace
)
.glassEffectTransition(.matchedGeometry)
.matchedGeometryEffect( .matchedGeometryEffect(
id: "main-tab-selection", id: "main-tab-selection",
in: selectionGlassNamespace in: selectionNamespace
) )
} }
} }
@@ -106,13 +107,12 @@ struct MinimalTabBar: View {
.accessibilityAddTraits(selection == tab ? .isSelected : []) .accessibilityAddTraits(selection == tab ? .isSelected : [])
} }
} }
.padding(.horizontal, Spacing.md) .padding(.horizontal, TabBarDockMetrics.dockInsetHorizontal)
.padding(.vertical, Spacing.sm) .padding(.vertical, TabBarDockMetrics.dockInsetVertical)
.glassEffect(.regular.interactive(), in: .capsule) .glassEffect(.regular.interactive(), in: .capsule)
}
.frame(maxWidth: 280) .frame(maxWidth: 280)
.frame(maxWidth: .infinity, alignment: .center) .frame(maxWidth: .infinity, alignment: .center)
.padding(.bottom, Spacing.xs) .padding(.bottom, TabBarDockMetrics.bottomPadding)
} }
private func tabIconColor(for tab: AppTab) -> Color { private func tabIconColor(for tab: AppTab) -> Color {
@@ -52,8 +52,20 @@ extension View {
} }
enum TabBarDockMetrics { enum TabBarDockMetrics {
/// Clearance above the floating dock (icon row + vertical padding + home indicator). static let itemHeight: CGFloat = 52
static let scrollClearance: CGFloat = 100 static let iconSize: CGFloat = 24
/// Horizontal glass pad is 0: the selected capsule's `selectionInset`
/// is the only side gap. `Spacing.md` (16) plus that 5 pt was ~34×
/// the 5 pt top/bottom gap.
static let dockInsetHorizontal: CGFloat = 0
static let dockInsetVertical: CGFloat = Spacing.sm
/// Gap between the selected fill and the glass dock / neighbouring tabs.
static let selectionInset: CGFloat = 5
static let bottomPadding: CGFloat = Spacing.xs
/// Clearance above the floating dock (bar + home-indicator slack).
static var scrollClearance: CGFloat {
itemHeight + dockInsetVertical * 2 + bottomPadding + 20
}
} }
private struct TabBarScrollBottomPaddingModifier: ViewModifier { private struct TabBarScrollBottomPaddingModifier: ViewModifier {
+4 -4
View File
@@ -205,8 +205,8 @@ struct HomeView: View {
// MARK: - History / dictionary cards // MARK: - History / dictionary cards
/// Two independent cards; each header mirrors the stats tiles (accent icon /// Two independent cards; each header is an accent icon + uppercase label
/// + small uppercase label) and the body grows with its rows. /// and the body grows with its rows.
private var homeLibrarySection: some View { private var homeLibrarySection: some View {
VStack(spacing: Spacing.md) { VStack(spacing: Spacing.md) {
historyCard historyCard
@@ -290,11 +290,11 @@ struct HomeView: View {
VStack(alignment: .leading, spacing: Spacing.xs) { VStack(alignment: .leading, spacing: Spacing.xs) {
HStack(spacing: Spacing.xs) { HStack(spacing: Spacing.xs) {
Image(systemName: systemImage) Image(systemName: systemImage)
.font(.system(size: 13, weight: .semibold)) .font(.system(size: 16, weight: .semibold))
.foregroundStyle(palette.accent) .foregroundStyle(palette.accent)
.symbolRenderingMode(.hierarchical) .symbolRenderingMode(.hierarchical)
Text(titleKey) Text(titleKey)
.font(TypeStyle.caption2) .font(.system(size: 13, weight: .semibold))
.tracking(0.6) .tracking(0.6)
.textCase(.uppercase) .textCase(.uppercase)
.foregroundStyle(palette.textTertiary) .foregroundStyle(palette.textTertiary)
+1 -1
View File
@@ -44,7 +44,7 @@ struct MainTabView: View {
if !isTabBarHidden { if !isTabBarHidden {
// Match floating dock + home-indicator clearance so // Match floating dock + home-indicator clearance so
// page footers / scroll ends sit above MinimalTabBar. // page footers / scroll ends sit above MinimalTabBar.
Color.clear.frame(height: 100) Color.clear.frame(height: TabBarDockMetrics.scrollClearance)
} }
} }
.onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in .onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in
+1 -1
View File
@@ -423,7 +423,7 @@
"home.engine.attentionBanner" = "Local ASR model is not ready yet. Download it and wait for warm-up before dictating."; "home.engine.attentionBanner" = "Local ASR model is not ready yet. Download it and wait for warm-up before dictating.";
/* Tabs */ /* Tabs */
"tab.keyboard" = "Keyboard"; "tab.keyboard" = "Home";
"tab.history" = "History"; "tab.history" = "History";
"tab.dictionary" = "Dictionary"; "tab.dictionary" = "Dictionary";
"tab.styles" = "Styles"; "tab.styles" = "Styles";
@@ -422,7 +422,7 @@
"home.engine.attentionBanner" = "本地语音识别模型尚未就绪,请先下载并等待加载完成后再使用语音输入。"; "home.engine.attentionBanner" = "本地语音识别模型尚未就绪,请先下载并等待加载完成后再使用语音输入。";
/* Tabs */ /* Tabs */
"tab.keyboard" = "键盘"; "tab.keyboard" = "首页";
"tab.history" = "历史"; "tab.history" = "历史";
"tab.dictionary" = "词库"; "tab.dictionary" = "词库";
"tab.styles" = "风格"; "tab.styles" = "风格";
@@ -446,6 +446,9 @@ public final class KeyboardViewController: UIInputViewController {
flowCoordinator.onAIRecognitionStarted = { [weak self] utteranceID in flowCoordinator.onAIRecognitionStarted = { [weak self] utteranceID in
self?.aiKeyboardCoordinator.recognitionStarted(utteranceID) self?.aiKeyboardCoordinator.recognitionStarted(utteranceID)
} }
flowCoordinator.onAIGeneratingStarted = { [weak self] utteranceID in
self?.aiKeyboardCoordinator.generatingStarted(utteranceID)
}
flowCoordinator.onAITranscript = { [weak self] transcript, utteranceID, status in flowCoordinator.onAITranscript = { [weak self] transcript, utteranceID, status in
self?.aiKeyboardCoordinator.receiveTranscript( self?.aiKeyboardCoordinator.receiveTranscript(
transcript, transcript,
@@ -465,6 +468,9 @@ public final class KeyboardViewController: UIInputViewController {
flowCoordinator.onAIFailure = { [weak self] message, utteranceID in flowCoordinator.onAIFailure = { [weak self] message, utteranceID in
self?.aiKeyboardCoordinator.fail(message, utteranceID: utteranceID) self?.aiKeyboardCoordinator.fail(message, utteranceID: utteranceID)
} }
flowCoordinator.onAICancelled = { [weak self] in
self?.state.aiSession.cancelCurrentWork()
}
_ = textInserter.recoverPendingEditTransactionIfNeeded() _ = textInserter.recoverPendingEditTransactionIfNeeded()
cursorDrag = CursorDragController( cursorDrag = CursorDragController(
@@ -511,6 +517,9 @@ public final class KeyboardViewController: UIInputViewController {
state.submitAIHint = { [weak self] card in state.submitAIHint = { [weak self] card in
self?.aiKeyboardCoordinator.submitHintCard(card) self?.aiKeyboardCoordinator.submitHintCard(card)
} }
state.submitAIClipboardSkill = { [weak self] skill in
self?.aiKeyboardCoordinator.submitClipboardSkill(skill)
}
state.openSettings = { [weak self] in self?.openHostApp() } state.openSettings = { [weak self] in self?.openHostApp() }
state.openInputMethodSetup = { [weak self] in self?.openHostApp(path: "deployrime") } state.openInputMethodSetup = { [weak self] in self?.openHostApp(path: "deployrime") }
state.openClipboardSettings = { [weak self] in state.openClipboardSettings = { [weak self] in
@@ -66,20 +66,44 @@ final class AIKeyboardCoordinator {
} }
} }
/// Tap a clipboard skill chip: same fail-closed material path as hint cards.
func submitClipboardSkill(_ skill: AIClipboardSkill) {
guard canAcceptIdleSubmit else { return }
enterIfNeeded()
let instruction = AIClipboardSkillCatalog.instruction(
for: skill,
locale: AIHintLocaleResolver.packLocale(),
translationTargetLocaleId: state.translationTargetLocaleId
)
let resolution = AIClipboardPrompt.resolve(
instruction: instruction,
material: ClipboardHistoryStore.shared.newestAIHintEligibleEntry()?.text
)
submitResolvedPrompt(resolution)
}
/// Tap an idle hint card: resolve its material, skip the mic, ask the host. /// Tap an idle hint card: resolve its material, skip the mic, ask the host.
func submitHintCard(_ card: AIHintCard) { func submitHintCard(_ card: AIHintCard) {
switch state.aiSession.phase { guard canAcceptIdleSubmit else { return }
case .inactive, .idle, .failed:
break
case .preparing, .listening, .recognizing, .generating,
.ready, .awaitingSend, .inserted, .sent:
return
}
enterIfNeeded() enterIfNeeded()
let resolution = AIHintPool.resolvePrompt( let resolution = AIHintPool.resolvePrompt(
for: card, for: card,
clipboardText: ClipboardHistoryStore.shared.newestAIHintEligibleEntry()?.text clipboardText: ClipboardHistoryStore.shared.newestAIHintEligibleEntry()?.text
) )
submitResolvedPrompt(resolution)
}
private var canAcceptIdleSubmit: Bool {
switch state.aiSession.phase {
case .inactive, .idle, .failed:
return true
case .preparing, .listening, .recognizing, .generating,
.ready, .awaitingSend, .inserted, .sent:
return false
}
}
private func submitResolvedPrompt(_ resolution: AIClipboardPrompt.Resolution) {
guard case .ready(let prompt) = resolution else { guard case .ready(let prompt) = resolution else {
// The clipboard window closed between rendering and this tap. // The clipboard window closed between rendering and this tap.
state.aiSession.fail( state.aiSession.fail(
@@ -132,11 +156,21 @@ final class AIKeyboardCoordinator {
state.aiSession.beginRecognizing(utteranceID: utteranceID) state.aiSession.beginRecognizing(utteranceID: utteranceID)
} }
func generatingStarted(_ utteranceID: UUID) {
state.aiSession.beginGenerating(question: "", utteranceID: utteranceID)
}
func receiveTranscript( func receiveTranscript(
_ transcript: String, _ transcript: String,
utteranceID: UUID, utteranceID: UUID,
status: FlowResult.Status status: FlowResult.Status
) { ) {
if AIClipboardPrompt.isInternalPrompt(transcript) {
if status == .rawReady {
state.aiSession.beginGenerating(question: "", utteranceID: utteranceID)
}
return
}
state.aiSession.updateTranscript(transcript, utteranceID: utteranceID) state.aiSession.updateTranscript(transcript, utteranceID: utteranceID)
if status == .rawReady { if status == .rawReady {
state.aiSession.beginGenerating( state.aiSession.beginGenerating(
@@ -61,10 +61,12 @@ final class KeyboardFlowCoordinator {
var onAIUtterancePrepared: (UUID) -> Void = { _ in } var onAIUtterancePrepared: (UUID) -> Void = { _ in }
var onAIRecordingStarted: (UUID) -> Void = { _ in } var onAIRecordingStarted: (UUID) -> Void = { _ in }
var onAIRecognitionStarted: (UUID) -> Void = { _ in } var onAIRecognitionStarted: (UUID) -> Void = { _ in }
var onAIGeneratingStarted: (UUID) -> Void = { _ in }
var onAITranscript: (String, UUID, FlowResult.Status) -> Void = { _, _, _ in } var onAITranscript: (String, UUID, FlowResult.Status) -> Void = { _, _, _ in }
var onAIStreamingAnswer: (String, UUID) -> Void = { _, _ in } var onAIStreamingAnswer: (String, UUID) -> Void = { _, _ in }
var onAIResult: (FlowResult) -> Void = { _ in } var onAIResult: (FlowResult) -> Void = { _ in }
var onAIFailure: (String, UUID?) -> Void = { _, _ in } var onAIFailure: (String, UUID?) -> Void = { _, _ in }
var onAICancelled: () -> Void = {}
/// Utterance whose final result we already inserted (or failed). Prevents /// Utterance whose final result we already inserted (or failed). Prevents
/// `adoptHostBusyStateIfNeeded` from re-entering `.processing` after a /// `adoptHostBusyStateIfNeeded` from re-entering `.processing` after a
/// stale App Group snapshot still says `reason=processing`. /// stale App Group snapshot still says `reason=processing`.
@@ -414,6 +416,29 @@ final class KeyboardFlowCoordinator {
isFlowRecording = false isFlowRecording = false
stopUtteranceCountdown() stopUtteranceCountdown()
ExtensionScreenWakeLock.release() ExtensionScreenWakeLock.release()
if FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing(
busyUtteranceId: busyId,
latestResult: FlowSessionBridge.latestResult(),
latestAck: FlowSessionBridge.latestAck()
) {
// Already acked, result gone, host forgot to drop the gate.
// Abort unsticks the host. Do not await claimTerminal already
// ran, so abort will not write a new result. Remember the id
// so this same refresh cannot re-adopt before the host poll.
writeCommand(.abort)
lastConsumedUtteranceId = busyId
lastStoppedUtteranceId = busyId
isAwaitingFlowResult = false
currentUtteranceId = nil
state.phase = .idle
state.lastTranscript = ""
traceState(
"adoptHostBusy.staleProcessingReleased",
extra: "utterance=\(busyId.uuidString.prefix(8))"
)
return
}
// Missing result with no matching ack is live ASR/LLM wait.
state.phase = .processing state.phase = .processing
if state.lastTranscript.isEmpty { if state.lastTranscript.isEmpty {
state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing")
@@ -557,6 +582,12 @@ final class KeyboardFlowCoordinator {
case .requestingPermissions: case .requestingPermissions:
break break
case .idle, .denied, .error: case .idle, .denied, .error:
// A second tap while still waiting for the host must cancel the
// pending start instead of stacking another utterance.
if currentUtteranceRequest != nil {
cancelCurrentDictation()
return
}
_ = startUtterance(.dictation) _ = startUtterance(.dictation)
case .processing: case .processing:
break break
@@ -655,34 +686,8 @@ final class KeyboardFlowCoordinator {
func cancelAIRecording() { func cancelAIRecording() {
guard currentUtteranceRequest?.isAIQuestion == true else { return } guard currentUtteranceRequest?.isAIQuestion == true else { return }
recordWhenHostReady = false prepareLocalCancel()
recordAfterHandoff = false beginAwaitingAbort(tracking: &cancelledAIUtteranceIDs)
isPendingFlowStart = false
flowStartDeadline = 0
coldStartDebouncer.reset()
stopHostReadyWait()
stopFlowWatchdog()
stopUtteranceCountdown()
ExtensionScreenWakeLock.release()
state.level = 0
state.phase = .idle
state.lastTranscript = ""
guard let utteranceID = currentUtteranceId,
isFlowRecording || isAwaitingFlowResult else {
clearUnissuedUtterance()
isFlowRecording = false
isAwaitingFlowResult = false
recomputeMicVoiceAvailability()
return
}
cancelledAIUtteranceIDs.insert(utteranceID)
writeCommand(.abort)
isFlowRecording = false
isAwaitingFlowResult = true
startFlowResultWatchdog()
recomputeMicVoiceAvailability()
} }
func endAIConversation(_ conversationID: UUID) { func endAIConversation(_ conversationID: UUID) {
@@ -708,11 +713,27 @@ final class KeyboardFlowCoordinator {
} }
func cancelCurrentDictation() { func cancelCurrentDictation() {
guard currentUtteranceRequest?.isEdit != true, if currentUtteranceRequest?.isAIQuestion == true {
currentUtteranceRequest?.isAIQuestion != true else { cancelAIRecording()
return return
} }
guard currentUtteranceRequest?.isEdit != true else { return }
prepareLocalCancel()
let hadIssuedTransport = currentUtteranceId != nil
&& (isFlowRecording || isAwaitingFlowResult)
beginAwaitingAbort(tracking: &cancelledDictationUtteranceIDs)
traceState(
"dictation.cancelled",
extra: hadIssuedTransport
? "utterance=\(currentUtteranceId?.uuidString.prefix(8) ?? "none")"
: "transport=localOnly"
)
}
/// Stop local wait/prime timers before an abort. Does not change phase;
/// `beginAwaitingAbort` keeps `.processing` until the host acks.
private func prepareLocalCancel() {
recordWhenHostReady = false recordWhenHostReady = false
recordAfterHandoff = false recordAfterHandoff = false
isPendingFlowStart = false isPendingFlowStart = false
@@ -723,29 +744,37 @@ final class KeyboardFlowCoordinator {
stopUtteranceCountdown() stopUtteranceCountdown()
ExtensionScreenWakeLock.release() ExtensionScreenWakeLock.release()
state.level = 0 state.level = 0
state.phase = .idle
state.lastTranscript = "" state.lastTranscript = ""
guard let utteranceID = currentUtteranceId,
isFlowRecording || isAwaitingFlowResult else {
clearUnissuedUtterance()
isFlowRecording = false
isAwaitingFlowResult = false
recomputeMicVoiceAvailability()
traceState("dictation.cancelled", extra: "transport=localOnly")
return
} }
cancelledDictationUtteranceIDs.insert(utteranceID) /// Keep the cancel chrome (X + white mic) until the host finishes abort.
private func beginAwaitingAbort(tracking cancelledIDs: inout Set<UUID>) {
guard let utteranceID = currentUtteranceId,
isFlowRecording || isAwaitingFlowResult else {
finishLocalCancel()
return
}
if cancelledIDs.contains(utteranceID), isAwaitingFlowResult {
state.phase = .processing
recomputeMicVoiceAvailability()
return
}
cancelledIDs.insert(utteranceID)
writeCommand(.abort) writeCommand(.abort)
isFlowRecording = false isFlowRecording = false
isAwaitingFlowResult = true isAwaitingFlowResult = true
state.phase = .processing
startFlowResultWatchdog() startFlowResultWatchdog()
recomputeMicVoiceAvailability() recomputeMicVoiceAvailability()
traceState( }
"dictation.cancelled",
extra: "utterance=\(utteranceID.uuidString.prefix(8))" private func finishLocalCancel() {
) clearUnissuedUtterance()
isFlowRecording = false
isAwaitingFlowResult = false
state.phase = .idle
state.lastTranscript = ""
recomputeMicVoiceAvailability()
} }
func abortEditRecording() { func abortEditRecording() {
@@ -1085,9 +1114,11 @@ final class KeyboardFlowCoordinator {
if isFlowRecording { if isFlowRecording {
writeCommand(.abort) writeCommand(.abort)
ExtensionScreenWakeLock.release() ExtensionScreenWakeLock.release()
// Remember the aborted id so the next keyboard open cannot
// re-adopt a lagging host snapshot as .
lastStoppedUtteranceId = currentUtteranceId
} }
currentUtteranceId = nil currentUtteranceId = nil
lastStoppedUtteranceId = nil
isFlowRecording = false isFlowRecording = false
isPendingFlowStart = false isPendingFlowStart = false
recordAfterHandoff = false recordAfterHandoff = false
@@ -1174,6 +1205,10 @@ final class KeyboardFlowCoordinator {
consumeCancelledDictationResultIfNeeded(result) { consumeCancelledDictationResultIfNeeded(result) {
return return
} }
if let result = matchingResult(),
consumeDiscardedEmptyResultIfNeeded(result) {
return
}
if let result = matchingResult(), if let result = matchingResult(),
consumeCancelledEditResultIfNeeded(result) { consumeCancelledEditResultIfNeeded(result) {
return return
@@ -1311,6 +1346,42 @@ final class KeyboardFlowCoordinator {
return true return true
} }
private func consumeDiscardedEmptyResultIfNeeded(_ result: FlowResult) -> Bool {
guard result.errorKind == .discardedEmpty,
result.status == .aborted || isTerminalFailure(result) else {
return false
}
FlowSessionBridge.writeAck(
FlowAck(
sessionId: result.sessionId,
utteranceId: result.utteranceId,
commandSeq: result.commandSeq,
hostGeneration: result.hostGeneration,
revision: result.revision,
deliveryOutcome: .rejected
)
)
lastConsumedUtteranceId = result.utteranceId
lastStoppedUtteranceId = nil
stopUtteranceCountdown()
stopFlowWatchdog()
ExtensionScreenWakeLock.release()
let wasAI = currentUtteranceRequest?.isAIQuestion == true
resetEditTransportState()
state.level = 0
state.phase = .idle
state.lastTranscript = ""
recomputeMicVoiceAvailability()
if wasAI {
onAICancelled()
}
traceState(
"utterance.discardedEmptyTap",
extra: "utterance=\(result.utteranceId.uuidString.prefix(8))"
)
return true
}
private func consumeCancelledEditResultIfNeeded(_ result: FlowResult) -> Bool { private func consumeCancelledEditResultIfNeeded(_ result: FlowResult) -> Bool {
guard cancelledEditUtteranceIDs.contains(result.utteranceId), guard cancelledEditUtteranceIDs.contains(result.utteranceId),
result.status == .final || isTerminalFailure(result) else { result.status == .final || isTerminalFailure(result) else {
@@ -1754,10 +1825,10 @@ final class KeyboardFlowCoordinator {
writeSubmitAIQuestion(question) writeSubmitAIQuestion(question)
isFlowRecording = false isFlowRecording = false
isAwaitingFlowResult = true isAwaitingFlowResult = true
state.lastTranscript = question state.lastTranscript = ""
state.phase = .processing state.phase = .processing
if let currentUtteranceId { if let currentUtteranceId {
onAIRecognitionStarted(currentUtteranceId) onAIGeneratingStarted(currentUtteranceId)
} }
startFlowResultWatchdog() startFlowResultWatchdog()
recomputeMicVoiceAvailability() recomputeMicVoiceAvailability()
@@ -1935,6 +2006,11 @@ final class KeyboardFlowCoordinator {
if (result.status == .partial || result.status == .rawReady), if (result.status == .partial || result.status == .rawReady),
let partial = result.text, let partial = result.text,
!partial.isEmpty { !partial.isEmpty {
if result.resolvedUtteranceMode == .aiQuestion,
AIClipboardPrompt.isInternalPrompt(partial) {
onAIGeneratingStarted(result.utteranceId)
return
}
state.lastTranscript = partial state.lastTranscript = partial
if result.resolvedUtteranceMode == .aiQuestion { if result.resolvedUtteranceMode == .aiQuestion {
onAITranscript(partial, result.utteranceId, result.status) onAITranscript(partial, result.utteranceId, result.status)
@@ -1982,6 +2058,10 @@ final class KeyboardFlowCoordinator {
self.consumeCancelledDictationResultIfNeeded(result) { self.consumeCancelledDictationResultIfNeeded(result) {
return return
} }
if let result = self.matchingResult(),
self.consumeDiscardedEmptyResultIfNeeded(result) {
return
}
if let result = self.matchingResult(), if let result = self.matchingResult(),
self.consumeCancelledEditResultIfNeeded(result) { self.consumeCancelledEditResultIfNeeded(result) {
return return
+81 -19
View File
@@ -15,6 +15,7 @@ struct AIKeyboardView: View {
static let actionButtonMaxWidth: CGFloat = 150 static let actionButtonMaxWidth: CGFloat = 150
static let statusHeight: CGFloat = 20 static let statusHeight: CGFloat = 20
static let carouselInterval: TimeInterval = 4 static let carouselInterval: TimeInterval = 4
static let skillButtonSize: CGFloat = 52
} }
@Environment(\.colorScheme) private var colorScheme @Environment(\.colorScheme) private var colorScheme
@@ -61,6 +62,7 @@ struct AIKeyboardView: View {
// Reduce Motion stops the rotation, not the data: a card whose // Reduce Motion stops the rotation, not the data: a card whose
// clipboard window has closed must still leave the carousel. // clipboard window has closed must still leave the carousel.
reloadHintPool(resetBag: false) reloadHintPool(resetBag: false)
guard !showsClipboardSkills else { return }
if reduceMotion, let hint = currentHint, if reduceMotion, let hint = currentHint,
poolCards.contains(where: { $0.id == hint.id }) { poolCards.contains(where: { $0.id == hint.id }) {
return return
@@ -127,8 +129,13 @@ struct AIKeyboardView: View {
private var answerArea: some View { private var answerArea: some View {
ZStack(alignment: .bottom) { ZStack(alignment: .bottom) {
if showsPlaceholder { if showsPlaceholder {
if showsClipboardSkills {
clipboardSkillRow
.frame(maxWidth: .infinity, maxHeight: .infinity)
} else {
hintCarousel hintCarousel
.frame(maxWidth: .infinity, maxHeight: .infinity) .frame(maxWidth: .infinity, maxHeight: .infinity)
}
} else { } else {
ScrollViewReader { proxy in ScrollViewReader { proxy in
ScrollView(.vertical) { ScrollView(.vertical) {
@@ -173,30 +180,82 @@ struct AIKeyboardView: View {
guard let hint = currentHint else { return } guard let hint = currentHint else { return }
state.submitAIHint(hint) state.submitAIHint(hint)
} label: { } label: {
Text(currentHint?.displayText ?? ExtL10n.string("keyboard.ai.placeholder")) HStack(spacing: 6) {
.font(TypeStyle.body) if let hint = currentHint {
.foregroundStyle(palette.textTertiary) Image(systemName: hint.visualKind.systemImage)
.multilineTextAlignment(.center) .font(.system(size: 14, weight: .semibold))
.foregroundStyle(palette.textPrimary.opacity(0.55))
}
Text(currentHint.map(\.resolvedDisplayText) ?? ExtL10n.string("keyboard.ai.placeholder"))
.font(TypeStyle.bodyEmph)
.foregroundStyle(palette.textPrimary)
.lineLimit(1) .lineLimit(1)
.truncationMode(.tail) .truncationMode(.tail)
.padding(.horizontal, Spacing.md) }
.padding(.horizontal, 16)
.frame(height: 44)
.opacity(hintOpacity) .opacity(hintOpacity)
.frame(maxWidth: .infinity, maxHeight: .infinity) .glassEffect(.regular.interactive(), in: Capsule())
.contentShape(Rectangle())
} }
.buttonStyle(.plain) .buttonStyle(.plain)
.fixedSize()
// A busy session already owns the surface; the status line explains a // A busy session already owns the surface; the status line explains a
// missing LLM. Both keep the hint from being a tap with no outcome. // missing LLM. Both keep the hint from being a tap with no outcome.
.disabled(currentHint == nil || !state.aiServiceAvailable || state.aiSession.isBusy) .disabled(currentHint == nil || !state.aiServiceAvailable || state.aiSession.isBusy)
.accessibilityLabel( .accessibilityLabel(
Text( Text(
currentHint.map { currentHint.map {
"\(ExtL10n.string("keyboard.ai.hintA11yPrefix"))\($0.displayText)" "\(ExtL10n.string("keyboard.ai.hintA11yPrefix"))\($0.resolvedDisplayText)"
} ?? ExtL10n.string("keyboard.ai.placeholder") } ?? ExtL10n.string("keyboard.ai.placeholder")
) )
) )
} }
private var clipboardSkillRow: some View {
HStack(spacing: Spacing.lg) {
ForEach(AIClipboardSkillCatalog.visible()) { skill in
Button {
state.submitAIClipboardSkill(skill)
} label: {
VStack(spacing: 6) {
Image(systemName: skill.systemImage)
.font(.system(size: 18, weight: .semibold))
.foregroundStyle(palette.textPrimary.opacity(0.85))
.frame(width: Layout.skillButtonSize, height: Layout.skillButtonSize)
.glassEffect(.regular.interactive(), in: Circle())
Text(clipboardSkillTitle(skill))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textSecondary)
}
}
.buttonStyle(.plain)
.disabled(!state.aiServiceAvailable || state.aiSession.isBusy)
.accessibilityLabel(Text(clipboardSkillTitle(skill)))
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
/// Translate follows the keyboard target; Reply / Summarize stay static.
private func clipboardSkillTitle(_ skill: AIClipboardSkill) -> String {
if skill.id == AIClipboardSkillCatalog.translateID {
return AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: state.translationTargetLocaleId,
uiLanguage: AppGroupStore().uiLanguage
)
}
return ExtL10n.string(skill.titleKey)
}
/// Copy-then-30s window: skill chips replace the rotating hint.
private var showsClipboardSkills: Bool {
guard showsPlaceholder, state.clipboardHistoryEnabled else { return false }
return AIHintPool.isClipboardSkillWindowActive(
clipboardHistoryEnabled: true,
newestClipboard: clipboardHistory.newestEntry
)
}
/// No draft/answer yet show the centered hint carousel instead of a scroll body. /// No draft/answer yet show the centered hint carousel instead of a scroll body.
private var showsPlaceholder: Bool { private var showsPlaceholder: Bool {
let hasDraft = !(state.aiSession.draftAnswerText?.isEmpty ?? true) let hasDraft = !(state.aiSession.draftAnswerText?.isEmpty ?? true)
@@ -367,16 +426,20 @@ struct AIKeyboardView: View {
? ExtL10n.string("keyboard.ai.listening") ? ExtL10n.string("keyboard.ai.listening")
: state.aiSession.transcript : state.aiSession.transcript
case .recognizing: case .recognizing:
return state.aiSession.transcript.isEmpty if state.aiSession.transcript.isEmpty
? ExtL10n.string("keyboard.ai.recognizing") || AIClipboardPrompt.isInternalPrompt(state.aiSession.transcript) {
: state.aiSession.transcript return ExtL10n.string("keyboard.ai.recognizing")
}
return state.aiSession.transcript
case .generating: case .generating:
if let draft = state.aiSession.draftAnswerText, !draft.isEmpty { if let draft = state.aiSession.draftAnswerText, !draft.isEmpty {
return ExtL10n.string("keyboard.ai.generating") return ExtL10n.string("keyboard.ai.generating")
} }
return state.aiSession.transcript.isEmpty if state.aiSession.transcript.isEmpty
? ExtL10n.string("keyboard.ai.thinking") || AIClipboardPrompt.isInternalPrompt(state.aiSession.transcript) {
: state.aiSession.transcript return ExtL10n.string("keyboard.ai.thinking")
}
return state.aiSession.transcript
case .failed: case .failed:
return ExtL10n.string("keyboard.ai.error.requestFailed") return ExtL10n.string("keyboard.ai.error.requestFailed")
} }
@@ -391,19 +454,18 @@ struct AIKeyboardView: View {
// MARK: - Carousel // MARK: - Carousel
/// Rebuild the pool and show a card right away, without a fade. /// Rebuild the pool and show a card right away, without a fade.
/// Skip advancing the chip while clipboard skills own the surface, so a
/// leftover clipboard sentence cannot replace the three buttons.
private func resetCarousel() { private func resetCarousel() {
reloadHintPool(resetBag: true) reloadHintPool(resetBag: true)
guard !showsClipboardSkills else { return }
showNextHint(animated: false) showNextHint(animated: false)
} }
private func reloadHintPool(resetBag: Bool) { private func reloadHintPool(resetBag: Bool) {
let locale = AIHintLocaleResolver.packLocale() let locale = AIHintLocaleResolver.packLocale()
let pack = AIHintStore.resolvedPack(locale: locale) let pack = AIHintStore.resolvedPack(locale: locale)
poolCards = AIHintPool.activeCards( poolCards = AIHintPool.activeCards(pack: pack)
pack: pack,
clipboardHistoryEnabled: state.clipboardHistoryEnabled,
newestClipboard: clipboardHistory.newestEntry
)
if resetBag { if resetBag {
carouselBag.reset() carouselBag.reset()
} }
+4 -3
View File
@@ -22,8 +22,8 @@ import OSGKeyboardShared
private enum KeyboardLayoutMetrics { private enum KeyboardLayoutMetrics {
static let micSize: CGFloat = 121 static let micSize: CGFloat = 121
static let micToButtonGap: CGFloat = 8 static let micToButtonGap: CGFloat = 8
/// Square undo key beside the mic (outer edge, aligned with delete). /// Circular undo / translation keys beside the mic (outer edge).
static let undoButtonSize: CGFloat = 44 static let undoButtonSize: CGFloat = 52
static let bottomActionRowHeight: CGFloat = KeyboardChromeLayout.actionKeyHeight static let bottomActionRowHeight: CGFloat = KeyboardChromeLayout.actionKeyHeight
static let bottomActionSpacing: CGFloat = KeyboardChromeLayout.actionKeySpacing static let bottomActionSpacing: CGFloat = KeyboardChromeLayout.actionKeySpacing
/// Gap between the top control row and the transcript / hint line. /// Gap between the top control row and the transcript / hint line.
@@ -419,7 +419,7 @@ public struct KeyboardRootView: View {
.frame(height: KeyboardLayoutMetrics.bottomActionRowHeight) .frame(height: KeyboardLayoutMetrics.bottomActionRowHeight)
} }
/// Square undo key on the outer drag pad same chrome / haptic / click /// Circular undo key on the outer drag pad same chrome / haptic / click
/// as space & return. Vertically matches the mic disc. /// as space & return. Vertically matches the mic disc.
private func undoButton(disabled: Bool, visible: Bool) -> some View { private func undoButton(disabled: Bool, visible: Bool) -> some View {
RectangularToolbarButton( RectangularToolbarButton(
@@ -427,6 +427,7 @@ public struct KeyboardRootView: View {
label: ExtL10n.string("keyboard.undoA11y"), label: ExtL10n.string("keyboard.undoA11y"),
disabled: disabled, disabled: disabled,
usesLiquidGlass: true, usesLiquidGlass: true,
usesCircleGlass: true,
hapticIntensity: state.keyboardHapticIntensity hapticIntensity: state.keyboardHapticIntensity
) { ) {
state.undoLastInsertion() state.undoLastInsertion()
+3 -10
View File
@@ -280,25 +280,18 @@ struct KeyboardTranslationMenuButton: View, Equatable {
} }
} }
} label: { } label: {
// Match the adjacent undo key: 44×44 rounded Liquid Glass control.
ZStack { ZStack {
Color.clear Color.clear
Image(systemName: isEnabled ? "character.bubble.fill" : "character.bubble") Image(systemName: isEnabled ? "character.bubble.fill" : "character.bubble")
.font(.system(size: 14, weight: .semibold)) .font(.system(size: 17, weight: .semibold))
.foregroundStyle( .foregroundStyle(
isEnabled isEnabled
? palette.accent ? palette.accent
: palette.textSecondary : palette.textSecondary
) )
} }
.contentShape(Rectangle()) .contentShape(Circle())
.glassEffect( .glassEffect(.regular.interactive(), in: Circle())
.regular.interactive(),
in: RoundedRectangle(
cornerRadius: KeyboardChromeLayout.actionKeyCornerRadius,
style: .continuous
)
)
} }
.menuStyle(.button) .menuStyle(.button)
.accessibilityLabel(Text(SharedL10n.string("keyboard.translation.a11y"))) .accessibilityLabel(Text(SharedL10n.string("keyboard.translation.a11y")))
@@ -10,6 +10,8 @@ import OSGKeyboardShared
private enum ToolbarButtonMetrics { private enum ToolbarButtonMetrics {
static let iconSize: CGFloat = 14 static let iconSize: CGFloat = 14
/// Slightly larger glyph so a 52 pt circular glass key does not look sparse.
static let circleIconSize: CGFloat = 17
static let titleSize: CGFloat = 16 static let titleSize: CGFloat = 16
static let cornerRadius: CGFloat = KeyboardChromeLayout.actionKeyCornerRadius static let cornerRadius: CGFloat = KeyboardChromeLayout.actionKeyCornerRadius
static let spaceBarCapsuleWidth: CGFloat = 31 static let spaceBarCapsuleWidth: CGFloat = 31
@@ -201,6 +203,7 @@ struct RectangularToolbarButton: View {
let disabled: Bool let disabled: Bool
let isSend: Bool let isSend: Bool
let usesLiquidGlass: Bool let usesLiquidGlass: Bool
let usesCircleGlass: Bool
/// Settings General Haptics; space / return use `.action` role. /// Settings General Haptics; space / return use `.action` role.
var hapticIntensity: KeyboardHapticIntensity = .off var hapticIntensity: KeyboardHapticIntensity = .off
let action: () -> Void let action: () -> Void
@@ -210,6 +213,7 @@ struct RectangularToolbarButton: View {
label: String, label: String,
disabled: Bool = false, disabled: Bool = false,
usesLiquidGlass: Bool = false, usesLiquidGlass: Bool = false,
usesCircleGlass: Bool = false,
hapticIntensity: KeyboardHapticIntensity = .off, hapticIntensity: KeyboardHapticIntensity = .off,
action: @escaping () -> Void action: @escaping () -> Void
) { ) {
@@ -220,6 +224,7 @@ struct RectangularToolbarButton: View {
self.disabled = disabled self.disabled = disabled
self.isSend = false self.isSend = false
self.usesLiquidGlass = usesLiquidGlass self.usesLiquidGlass = usesLiquidGlass
self.usesCircleGlass = usesCircleGlass
self.hapticIntensity = hapticIntensity self.hapticIntensity = hapticIntensity
self.action = action self.action = action
} }
@@ -239,6 +244,7 @@ struct RectangularToolbarButton: View {
self.disabled = disabled self.disabled = disabled
self.isSend = isSend self.isSend = isSend
self.usesLiquidGlass = usesLiquidGlass self.usesLiquidGlass = usesLiquidGlass
self.usesCircleGlass = false
self.hapticIntensity = hapticIntensity self.hapticIntensity = hapticIntensity
self.action = action self.action = action
self.title = title self.title = title
@@ -259,6 +265,7 @@ struct RectangularToolbarButton: View {
self.disabled = disabled self.disabled = disabled
self.isSend = false self.isSend = false
self.usesLiquidGlass = usesLiquidGlass self.usesLiquidGlass = usesLiquidGlass
self.usesCircleGlass = false
self.hapticIntensity = hapticIntensity self.hapticIntensity = hapticIntensity
self.action = action self.action = action
} }
@@ -267,7 +274,7 @@ struct RectangularToolbarButton: View {
var body: some View { var body: some View {
buttonSurface buttonSurface
.contentShape(Rectangle()) .contentShape(usesCircleGlass ? AnyShape(Circle()) : AnyShape(Rectangle()))
.gesture(pressGesture) .gesture(pressGesture)
.opacity(disabled ? 0.38 : 1) .opacity(disabled ? 0.38 : 1)
.allowsHitTesting(!disabled) .allowsHitTesting(!disabled)
@@ -282,13 +289,7 @@ struct RectangularToolbarButton: View {
Color.clear Color.clear
buttonContent buttonContent
} }
.glassEffect( .modifier(ToolbarLiquidGlass(isCircle: usesCircleGlass))
.regular.interactive(),
in: RoundedRectangle(
cornerRadius: ToolbarButtonMetrics.cornerRadius,
style: .continuous
)
)
// The custom press gesture fires on touch-down; mirror that state // The custom press gesture fires on touch-down; mirror that state
// visually while Liquid Glass supplies its native light response. // visually while Liquid Glass supplies its native light response.
.scaleEffect(isPressing ? 0.97 : 1) .scaleEffect(isPressing ? 0.97 : 1)
@@ -312,7 +313,12 @@ struct RectangularToolbarButton: View {
.frame(width: ToolbarButtonMetrics.spaceBarCapsuleWidth, height: 3) .frame(width: ToolbarButtonMetrics.spaceBarCapsuleWidth, height: 3)
} else if let systemName { } else if let systemName {
Image(systemName: systemName) Image(systemName: systemName)
.font(.system(size: ToolbarButtonMetrics.iconSize, weight: .semibold)) .font(.system(
size: usesCircleGlass
? ToolbarButtonMetrics.circleIconSize
: ToolbarButtonMetrics.iconSize,
weight: .semibold
))
.foregroundStyle(buttonForeground) .foregroundStyle(buttonForeground)
} else if let title { } else if let title {
Text(title) Text(title)
@@ -340,3 +346,21 @@ struct RectangularToolbarButton: View {
} }
} }
} }
private struct ToolbarLiquidGlass: ViewModifier {
let isCircle: Bool
func body(content: Content) -> some View {
if isCircle {
content.glassEffect(.regular.interactive(), in: Circle())
} else {
content.glassEffect(
.regular.interactive(),
in: RoundedRectangle(
cornerRadius: ToolbarButtonMetrics.cornerRadius,
style: .continuous
)
)
}
}
}
+3
View File
@@ -294,3 +294,6 @@
"keyboard.ai.error.requestTimeout" = "AI response timed out. Try again"; "keyboard.ai.error.requestTimeout" = "AI response timed out. Try again";
"keyboard.ai.error.requestFailed" = "AI response failed. Try again"; "keyboard.ai.error.requestFailed" = "AI response failed. Try again";
"keyboard.ai.error.clipboardUnavailable" = "This clipboard suggestion expired. Copy the text again"; "keyboard.ai.error.clipboardUnavailable" = "This clipboard suggestion expired. Copy the text again";
"keyboard.ai.skill.reply" = "Reply";
"keyboard.ai.skill.summarize" = "Summarize";
"keyboard.ai.skill.translate" = "Translate";
@@ -294,3 +294,6 @@
"keyboard.ai.error.requestTimeout" = "AI 回答超时,请重试"; "keyboard.ai.error.requestTimeout" = "AI 回答超时,请重试";
"keyboard.ai.error.requestFailed" = "AI 回答失败,请重试"; "keyboard.ai.error.requestFailed" = "AI 回答失败,请重试";
"keyboard.ai.error.clipboardUnavailable" = "剪贴板建议已过期,请重新复制文本"; "keyboard.ai.error.clipboardUnavailable" = "剪贴板建议已过期,请重新复制文本";
"keyboard.ai.skill.reply" = "回复";
"keyboard.ai.skill.summarize" = "总结";
"keyboard.ai.skill.translate" = "翻译";
@@ -110,6 +110,11 @@ final class KeyboardStateTests: XCTestCase {
KeyboardState.Phase.ErrorKind.fromFlowTranscription(flowError), KeyboardState.Phase.ErrorKind.fromFlowTranscription(flowError),
.hostTranscriptionFailed("asr failed") .hostTranscriptionFailed("asr failed")
) )
let discarded = FlowTranscriptionError(message: "", kind: .discardedEmpty)
XCTAssertEqual(
KeyboardState.Phase.ErrorKind.fromFlowTranscription(discarded),
.noSpeechDetected
)
} }
func testInputModeIsPolishOnly() { func testInputModeIsPolishOnly() {
@@ -1126,6 +1126,12 @@ public final class FlowContinuousCapture {
return report return report
} }
/// Copy of utterance PCM so the host can decide empty-tap skip
/// without consuming the buffer used by the normal drain path.
public func utterancePCMSnapshot() -> [Float] {
utterancePCMStore.snapshot()
}
/// Returns the utterance PCM accumulated during the last recording cycle. /// Returns the utterance PCM accumulated during the last recording cycle.
public func consumeUtteranceSamples() -> [Float] { public func consumeUtteranceSamples() -> [Float] {
utterancePCMStore.consume() utterancePCMStore.consume()
+46 -4
View File
@@ -1,11 +1,39 @@
// AIHintModels.swift // AIHintModels.swift
// OSGKeyboard · Shared // OSGKeyboard · Shared
// //
// Hint cards for the AI-mode idle carousel. Remote packs use `text`; the // Hint cards for the AI-mode idle carousel. Remote packs use `text` plus
// host compresses that into `displayText` before writing the ready pack. // optional `metadata`; the host extracts a keyword `displayText` for the chip.
import Foundation import Foundation
public struct AIHintMetadata: Codable, Equatable, Sendable {
public var title: String?
public var city: String?
public var tempC: Double?
public var soul: String?
public var name: String?
public var date: String?
public var day: String?
public init(
title: String? = nil,
city: String? = nil,
tempC: Double? = nil,
soul: String? = nil,
name: String? = nil,
date: String? = nil,
day: String? = nil
) {
self.title = title
self.city = city
self.tempC = tempC
self.soul = soul
self.name = name
self.date = date
self.day = day
}
}
public struct AIHintCard: Codable, Equatable, Identifiable, Sendable { public struct AIHintCard: Codable, Equatable, Identifiable, Sendable {
public let id: String public let id: String
/// One-line carousel label (after host keyword pass, or local catalog). /// One-line carousel label (after host keyword pass, or local catalog).
@@ -17,6 +45,7 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable {
public var source: String public var source: String
public var locale: String public var locale: String
public var conditions: [String] public var conditions: [String]
public var metadata: AIHintMetadata?
public init( public init(
id: String, id: String,
@@ -26,7 +55,8 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable {
priority: Int = 50, priority: Int = 50,
source: String = "local", source: String = "local",
locale: String = "zh", locale: String = "zh",
conditions: [String] = [] conditions: [String] = [],
metadata: AIHintMetadata? = nil
) { ) {
self.id = id self.id = id
self.displayText = displayText self.displayText = displayText
@@ -36,14 +66,24 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable {
self.source = source self.source = source
self.locale = locale self.locale = locale
self.conditions = conditions self.conditions = conditions
self.metadata = metadata
} }
public var requiresClipboard30s: Bool { public var requiresClipboard30s: Bool {
conditions.contains("clipboard_30s") || category == "clipboard" conditions.contains("clipboard_30s") || category == "clipboard"
} }
/// Keyword shown in the idle chip; prefers feed metadata over raw `text`.
public var resolvedDisplayText: String {
AIHintKeywordExtractor.displayText(for: self)
}
public var visualKind: AIHintVisualKind {
AIHintVisualKind.resolve(self)
}
enum CodingKeys: String, CodingKey { enum CodingKeys: String, CodingKey {
case id, displayText, text, prompt, category, priority, source, locale, conditions case id, displayText, text, prompt, category, priority, source, locale, conditions, metadata
} }
public init(from decoder: Decoder) throws { public init(from decoder: Decoder) throws {
@@ -55,6 +95,7 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable {
source = try container.decodeIfPresent(String.self, forKey: .source) ?? "remote" source = try container.decodeIfPresent(String.self, forKey: .source) ?? "remote"
locale = try container.decodeIfPresent(String.self, forKey: .locale) ?? "zh" locale = try container.decodeIfPresent(String.self, forKey: .locale) ?? "zh"
conditions = try container.decodeIfPresent([String].self, forKey: .conditions) ?? [] conditions = try container.decodeIfPresent([String].self, forKey: .conditions) ?? []
metadata = try container.decodeIfPresent(AIHintMetadata.self, forKey: .metadata)
if let display = try container.decodeIfPresent(String.self, forKey: .displayText), if let display = try container.decodeIfPresent(String.self, forKey: .displayText),
!display.isEmpty { !display.isEmpty {
displayText = display displayText = display
@@ -73,6 +114,7 @@ public struct AIHintCard: Codable, Equatable, Identifiable, Sendable {
try container.encode(source, forKey: .source) try container.encode(source, forKey: .source)
try container.encode(locale, forKey: .locale) try container.encode(locale, forKey: .locale)
try container.encode(conditions, forKey: .conditions) try container.encode(conditions, forKey: .conditions)
try container.encodeIfPresent(metadata, forKey: .metadata)
} }
} }
@@ -0,0 +1,58 @@
// AIHintVisualKind.swift
// OSGKeyboard · Shared
//
// Maps hint-feed category/source onto the idle-chip SF Symbol.
import Foundation
public enum AIHintVisualKind: String, Equatable, Sendable {
case calendar
case weather
case news
case stocks
case trending
case search
public var systemImage: String {
switch self {
case .calendar: return "calendar"
case .weather: return "cloud.sun.fill"
case .news: return "newspaper.fill"
case .stocks: return "chart.line.uptrend.xyaxis"
case .trending: return "flame.fill"
case .search: return "magnifyingglass"
}
}
public static func resolve(_ card: AIHintCard) -> Self {
let category = card.category.lowercased()
let source = card.source.lowercased()
let id = card.id.lowercased()
if category == "weather" || source.contains("meteo") {
return .weather
}
if category == "economy" || id.contains("stock") {
return .stocks
}
if category == "society"
|| source.contains("open-hot")
|| source.contains("tophub-open") {
return .trending
}
if category == "holiday"
|| category == "history"
|| source.contains("holiday")
|| card.conditions.contains("date") {
return .calendar
}
if category == "daily" {
if card.metadata?.soul != nil || id.contains("soul") {
return .search
}
return .news
}
// Quotes, encyclopedia, how-to, and unknown capability cards.
return .search
}
}
@@ -161,7 +161,7 @@ public struct AISessionState: Equatable, Sendable {
/// replace the previous committed `answer` until `receiveAnswer`. /// replace the previous committed `answer` until `receiveAnswer`.
public mutating func receivePartialAnswer(_ text: String, utteranceID: UUID) { public mutating func receivePartialAnswer(_ text: String, utteranceID: UUID) {
guard isActive, activeUtteranceID == utteranceID else { return } guard isActive, activeUtteranceID == utteranceID else { return }
if phase == .recognizing { if phase == .recognizing || phase == .preparing {
phase = .generating phase = .generating
} }
guard phase == .generating else { return } guard phase == .generating else { return }
@@ -25,6 +25,45 @@ public struct TranslationLanguage: Identifiable, Hashable, Sendable {
self.promptLanguageName = promptLanguageName self.promptLanguageName = promptLanguageName
self.nativeName = nativeName self.nativeName = nativeName
} }
/// One-character Chinese token for compact chips such as.
public var chineseShort: String {
switch id {
case "en": return ""
case "zh-Hans": return ""
case "zh-Hant": return ""
case "ja": return ""
case "ko": return ""
case "fr": return ""
case "de": return ""
case "es": return "西"
case "ru": return ""
case "pt": return ""
default: return nativeName
}
}
/// Country-style English code for compact chips such asTo JP.
public var englishShort: String {
switch id {
case "en": return "EN"
case "zh-Hans": return "CN"
case "zh-Hant": return "TW"
case "ja": return "JP"
case "ko": return "KR"
case "fr": return "FR"
case "de": return "DE"
case "es": return "ES"
case "ru": return "RU"
case "pt": return "PT"
default: return id.uppercased()
}
}
/// True when this target is a Chinese script ( or ).
public var isChineseScript: Bool {
id == "zh-Hans" || id == "zh-Hant"
}
} }
public enum TranslationLanguageCatalog { public enum TranslationLanguageCatalog {
@@ -48,6 +48,11 @@ public enum AIClipboardPrompt: Sendable {
return resolve(instruction: question, material: material) return resolve(instruction: question, material: material)
} }
/// True when `text` is the clipboard-AI XML envelope, not user-visible copy.
public static func isInternalPrompt(_ text: String) -> Bool {
text.contains("<clipboard_request") || text.contains("clipboard-ai-v1")
}
/// Instruction text with any inline material placeholder removed. /// Instruction text with any inline material placeholder removed.
static func strippingPlaceholder(_ prompt: String) -> String { static func strippingPlaceholder(_ prompt: String) -> String {
trimmed(prompt.replacingOccurrences(of: materialPlaceholder, with: "")) trimmed(prompt.replacingOccurrences(of: materialPlaceholder, with: ""))
@@ -0,0 +1,129 @@
// AIClipboardSkill.swift
// OSGKeyboard · Shared
//
// Built-in clipboard actions for AI idle. The catalog is an ordered list so
// Settings can later persist a subset or permutation without changing the view.
import Foundation
public struct AIClipboardSkill: Identifiable, Equatable, Sendable {
public let id: String
public let systemImage: String
/// Keyboard.strings key for the short button title.
public let titleKey: String
public init(id: String, systemImage: String, titleKey: String) {
self.id = id
self.systemImage = systemImage
self.titleKey = titleKey
}
}
public enum AIClipboardSkillCatalog: Sendable {
public static let replyID = "reply"
public static let summarizeID = "summarize"
public static let translateID = "translate"
/// Default set, in display order. Future skills append here.
public static let builtIn: [AIClipboardSkill] = [
AIClipboardSkill(
id: replyID,
systemImage: "arrowshape.turn.up.left.fill",
titleKey: "keyboard.ai.skill.reply"
),
AIClipboardSkill(
id: summarizeID,
systemImage: "doc.text.magnifyingglass",
titleKey: "keyboard.ai.skill.summarize"
),
AIClipboardSkill(
id: translateID,
systemImage: "character.bubble.fill",
titleKey: "keyboard.ai.skill.translate"
),
]
/// `enabledIDs` is the future Settings hook: `nil` keeps the built-in list.
public static func visible(enabledIDs: [String]? = nil) -> [AIClipboardSkill] {
guard let enabledIDs, !enabledIDs.isEmpty else { return builtIn }
let byID = Dictionary(uniqueKeysWithValues: builtIn.map { ($0.id, $0) })
return enabledIDs.compactMap { byID[$0] }
}
public static func instruction(
for skill: AIClipboardSkill,
locale: String,
translationTargetLocaleId: String
) -> String {
instruction(
skillID: skill.id,
locale: locale,
translationTargetLocaleId: translationTargetLocaleId
)
}
/// Compact Translate-chip label. Unset target ; Chinese UI
/// targeting / (avoids); otherwise × / To XX.
public static func translateButtonTitle(
translationTargetLocaleId: String,
uiLanguage: AppUILanguage
) -> String {
let isChineseUI = uiLanguage.resolvedLanguageCode() == "zh-Hans"
if TranslationLanguageCatalog.isOff(translationTargetLocaleId) {
return isChineseUI ? "中↔英" : "CN↔EN"
}
let target = TranslationLanguageCatalog.resolve(translationTargetLocaleId)
if isChineseUI, target.isChineseScript {
return "简↔繁"
}
if isChineseUI {
return "中译\(target.chineseShort)"
}
return "To \(target.englishShort)"
}
public static func instruction(
skillID: String,
locale: String,
translationTargetLocaleId: String
) -> String {
let zh = locale == "zh"
switch skillID {
case replyID:
return zh
? "请根据剪贴板内容起草一段礼貌、简洁的回复,语气自然,可直接发送。"
: "Draft a concise, polite reply the user can send, based on the clipboard text."
case summarizeID:
return zh
? "请概括剪贴板内容的核心意思,保留关键事实与结论,不要改写成可发送的短消息。"
: "Summarize the clipboard text: keep the key facts and conclusions; do not rewrite it as a sendable short message."
case translateID:
return translateInstruction(
locale: locale,
translationTargetLocaleId: translationTargetLocaleId
)
default:
return zh
? "请根据剪贴板内容完成用户选择的操作。"
: "Complete the selected action using the clipboard text."
}
}
/// Uses the keyboard translation target when set; otherwise Chinese English.
private static func translateInstruction(
locale: String,
translationTargetLocaleId: String
) -> String {
let zh = locale == "zh"
if !TranslationLanguageCatalog.isOff(translationTargetLocaleId) {
let language = TranslationLanguageCatalog.resolve(translationTargetLocaleId)
let name = language.promptLanguageName
return zh
? "请将剪贴板内容翻译成\(name),保留原意与语气。"
: "Translate the clipboard text into \(name), preserving meaning and tone."
}
return zh
? "请将剪贴板内容在中文与英文之间互译:若原文主要是中文则译成自然英文,若主要是英文则译成自然中文。保留原意与语气。"
: "Translate the clipboard between Chinese and English: if it is primarily Chinese, produce natural English; if primarily English, produce natural Chinese. Preserve meaning and tone."
}
}
@@ -1,8 +1,8 @@
// AIHintKeywordCompressor.swift // AIHintKeywordCompressor.swift
// OSGKeyboard · Shared // OSGKeyboard · Shared
// //
// Uses the user's polish LLM to compress remote hint titles into one-line // Optional LLM pass after deterministic keyword extraction. Failure leaves
// display labels. Failure leaves the previous ready pack untouched (caller). // the extracted labels in place (hard truncate only as a last resort).
import Foundation import Foundation
@@ -19,8 +19,15 @@ public struct AIHintKeywordCompressor: Sendable {
cards: [AIHintCard], cards: [AIHintCard],
locale: String locale: String
) async -> [AIHintCard] { ) async -> [AIHintCard] {
let candidates = cards.filter { shouldCompress($0) } let prepared = cards.map { card -> AIHintCard in
guard !candidates.isEmpty else { return cards } var copy = card
copy.displayText = AIHintKeywordExtractor.displayText(for: card)
return copy
}
let candidates = zip(cards, prepared).compactMap { original, extracted -> AIHintCard? in
shouldCompress(extracted) ? original : nil
}
guard !candidates.isEmpty else { return prepared }
do { do {
let client = try resolveClient() let client = try resolveClient()
@@ -28,6 +35,7 @@ public struct AIHintKeywordCompressor: Sendable {
[ [
"id": $0.id, "id": $0.id,
"text": $0.displayText, "text": $0.displayText,
"title": $0.metadata?.title ?? "",
"category": $0.category, "category": $0.category,
"source": $0.source, "source": $0.source,
] ]
@@ -48,32 +56,27 @@ public struct AIHintKeywordCompressor: Sendable {
return result return result
} }
let mapping = Self.parseDisplayMap(from: raw) let mapping = Self.parseDisplayMap(from: raw)
guard !mapping.isEmpty else { return cards } guard !mapping.isEmpty else { return prepared }
return cards.map { card in return prepared.map { card in
guard let display = mapping[card.id], !display.isEmpty else { return card } guard let display = mapping[card.id], !display.isEmpty else { return card }
var copy = card var copy = card
copy.displayText = Self.sanitizeDisplay(display, locale: locale) copy.displayText = AIHintKeywordExtractor.finalize(display, locale: locale)
return copy return copy
} }
} catch { } catch {
#if DEBUG #if DEBUG
print("⚠️ [AIHintKeywordCompressor] failed: \(error)") print("⚠️ [AIHintKeywordCompressor] failed: \(error)")
#endif #endif
return cards.map { card in return prepared
var copy = card
copy.displayText = Self.fallbackTruncate(card.displayText, locale: locale)
return copy
}
} }
} }
private func shouldCompress(_ card: AIHintCard) -> Bool { private func shouldCompress(_ card: AIHintCard) -> Bool {
if isHistoricalToday(card) { return false } if isHistoricalToday(card) { return false }
if card.locale == "zh" || card.displayText.contains(where: { $0.isCJKUnifiedIdeograph }) { if card.requiresClipboard30s { return false }
return card.displayText.count > 12 || card.displayText.contains("") let limit = AIHintKeywordExtractor.characterLimit(locale: card.locale)
|| card.displayText.contains("全网热点") return card.displayText.count > limit
} || card.displayText.contains("")
return card.displayText.count > 28
} }
private func isHistoricalToday(_ card: AIHintCard) -> Bool { private func isHistoricalToday(_ card: AIHintCard) -> Bool {
@@ -103,34 +106,30 @@ public struct AIHintKeywordCompressor: Sendable {
private static func systemPrompt(locale: String) -> String { private static func systemPrompt(locale: String) -> String {
if locale == "zh" { if locale == "zh" {
return """ return """
AI AI
JSON id/text/category/source JSON id/text/title/category/source
JSON {"id","displayText"} JSON {"id","displayText"}
- displayText - displayText /
- 512 -
- -
· + - 410 title
·
·
· /
·
·
- id - id
- prompt Markdown JSON - prompt Markdown JSON
""" """
} }
return """ return """
You compress AI keyboard idle hint titles. You extract keywords for AI keyboard idle hint chips.
Input: JSON array of {id,text,category,source}. Input: JSON array of {id,text,title,category,source}.
Output: JSON array of {"id","displayText"} only. Output: JSON array of {"id","displayText"} only.
Rules: Rules:
- displayText must be one line, no trailing ellipsis - displayText is the entity/keyword, not a question or action sentence
- English: 28 characters, NO "Chat"/"Chat about" prefix - No "Chat"/"Chat about" prefix
- Match intent (action / query / discuss) with a short natural label - One line, no trailing ellipsis
- Drop "On this day" / historical-today style items (omit their ids) - English: 22 characters; prefer the title field
- Drop "On this day" / historical-today items (omit their ids)
- Do not change prompts; JSON only, no Markdown - Do not change prompts; JSON only, no Markdown
""" """
} }
@@ -163,31 +162,10 @@ public struct AIHintKeywordCompressor: Sendable {
} }
public static func sanitizeDisplay(_ text: String, locale: String) -> String { public static func sanitizeDisplay(_ text: String, locale: String) -> String {
var value = text AIHintKeywordExtractor.finalize(text, locale: locale)
.replacingOccurrences(of: "\n", with: " ")
.trimmingCharacters(in: .whitespacesAndNewlines)
while value.hasSuffix("") || value.hasSuffix("...") {
if value.hasSuffix("...") {
value = String(value.dropLast(3))
} else {
value = String(value.dropLast())
}
value = value.trimmingCharacters(in: .whitespacesAndNewlines)
}
return fallbackTruncate(value, locale: locale)
} }
public static func fallbackTruncate(_ text: String, locale: String) -> String { public static func fallbackTruncate(_ text: String, locale: String) -> String {
let limit = locale == "zh" ? 12 : 28 AIHintKeywordExtractor.finalize(text, locale: locale)
guard text.count > limit else { return text }
return String(text.prefix(limit))
}
}
private extension Character {
var isCJKUnifiedIdeograph: Bool {
unicodeScalars.contains { scalar in
(0x4E00...0x9FFF).contains(scalar.value)
}
} }
} }
@@ -0,0 +1,162 @@
// AIHintKeywordExtractor.swift
// OSGKeyboard · Shared
//
// Deterministic idle-chip labels: the icon carries type, the text is the
// entity. LLM compression is only a last resort for leftovers.
import Foundation
public enum AIHintKeywordExtractor: Sendable {
public static func characterLimit(locale: String) -> Int {
locale == "zh" ? 10 : 22
}
public static func displayText(for card: AIHintCard) -> String {
let locale = card.locale == "zh" ? "zh" : "en"
let kind = AIHintVisualKind.resolve(card)
let raw = keyword(for: card, kind: kind, locale: locale)
return finalize(raw, locale: locale)
}
/// Strip prefixes, pick a fitting chunk, then enforce the character cap.
public static func finalize(_ text: String, locale: String) -> String {
var value = stripPrefixes(text, locale: locale)
.replacingOccurrences(of: "\n", with: " ")
.trimmingCharacters(in: .whitespacesAndNewlines)
value = stripTrailingPunctuation(value)
let limit = characterLimit(locale: locale)
if value.count <= limit { return value }
if let clause = firstFittingClause(value, limit: limit) { return clause }
if let latin = leadingLatinPhrase(value, limit: limit) { return latin }
if let chunk = lastFittingChunk(value, limit: limit) { return chunk }
return String(value.prefix(limit))
}
// MARK: - Keyword by kind
private static func keyword(
for card: AIHintCard,
kind: AIHintVisualKind,
locale: String
) -> String {
switch kind {
case .trending:
return firstNonEmpty(card.metadata?.title, card.displayText)
case .weather:
if let city = trimmed(card.metadata?.city) {
if let temp = card.metadata?.tempC {
return "\(city) \(Int(temp.rounded()))°"
}
return city
}
return card.displayText
case .news:
return locale == "zh" ? "今日早报" : "Today's briefing"
case .stocks:
return locale == "zh" ? "今日大盘" : "Markets"
case .calendar:
if locale != "zh", let name = trimmed(card.metadata?.name) {
return name
}
return card.displayText
case .search:
if card.metadata?.soul != nil {
return locale == "zh" ? "今日金句" : "A quote"
}
return card.displayText
}
}
// MARK: - Prefixes
public static func stripPrefixes(_ text: String, locale: String) -> String {
var value = text.trimmingCharacters(in: .whitespacesAndNewlines)
let prefixes = locale == "zh" ? zhPrefixes : enPrefixes
var changed = true
while changed {
changed = false
for prefix in prefixes where value.hasPrefix(prefix) {
value = String(value.dropFirst(prefix.count))
.trimmingCharacters(in: .whitespacesAndNewlines)
changed = true
break
}
}
return value
}
private static let zhPrefixes = [
"全网热点:", "全网热点:", "临近节日:", "临近节日:",
"历史上的今天:", "历史上的今天:", "今日一句:", "今日一句:",
"查百科:", "查百科:", "聊聊", "看看",
]
private static let enPrefixes = [
"Trending: ", "Upcoming: ", "On this day: ",
"Chat about ", "Chat ", "Weather in ",
]
// MARK: - Chunks
private static func firstFittingClause(_ text: String, limit: Int) -> String? {
let separators = CharacterSet(charactersIn: ",。;;,.!??!")
let parts = text.components(separatedBy: separators)
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
guard let first = parts.first, first.count <= limit, first.count >= 2 else {
return nil
}
return first
}
private static func lastFittingChunk(_ text: String, limit: Int) -> String? {
let parts = text.split { $0 == " " || $0 == "" || $0 == ":" }
.map(String.init)
.filter { !$0.isEmpty }
guard let last = parts.last else { return nil }
let cleaned = stripTrailingPunctuation(last)
guard (4...limit).contains(cleaned.count) else { return nil }
return cleaned
}
private static func leadingLatinPhrase(_ text: String, limit: Int) -> String? {
var scalars: [Unicode.Scalar] = []
for scalar in text.unicodeScalars {
let isLatin = (0x41...0x5A).contains(scalar.value)
|| (0x61...0x7A).contains(scalar.value)
|| (0x30...0x39).contains(scalar.value)
|| scalar == "." || scalar == "-"
let isSpace = scalar == " "
if isLatin || (isSpace && !scalars.isEmpty) {
scalars.append(scalar)
} else if !scalars.isEmpty {
break
}
}
var phrase = String(String.UnicodeScalarView(scalars))
.trimmingCharacters(in: .whitespaces)
guard phrase.count >= 2 else { return nil }
if phrase.count <= limit { return phrase }
while phrase.count > limit {
guard let lastSpace = phrase.lastIndex(of: " "), lastSpace > phrase.startIndex else {
return String(phrase.prefix(limit))
}
phrase = String(phrase[..<lastSpace])
}
return phrase
}
private static func stripTrailingPunctuation(_ text: String) -> String {
text.trimmingCharacters(in: CharacterSet(charactersIn: "…。.!??!、,"))
}
private static func firstNonEmpty(_ values: String?...) -> String {
values.compactMap(trimmed).first ?? ""
}
private static func trimmed(_ value: String?) -> String? {
guard let value else { return nil }
let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines)
return trimmed.isEmpty ? nil : trimmed
}
}
@@ -44,7 +44,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-zh-encyclopedia", id: "local-zh-encyclopedia",
displayText: "讲个有趣概念", displayText: "有趣概念",
prompt: "用通俗易懂的中文解释一个有趣但常见的概念,并给一个生活里的例子(4-6 句)。", prompt: "用通俗易懂的中文解释一个有趣但常见的概念,并给一个生活里的例子(4-6 句)。",
category: "capability", category: "capability",
priority: 40, priority: 40,
@@ -53,7 +53,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-zh-stocks", id: "local-zh-stocks",
displayText: "大盘如何", displayText: "大盘",
prompt: "请用非专业口吻概括今天 A 股/港股/美股中至少一个市场的整体表现、" prompt: "请用非专业口吻概括今天 A 股/港股/美股中至少一个市场的整体表现、"
+ "可能驱动因素,并提醒这并非投资建议(4-6 句)。", + "可能驱动因素,并提醒这并非投资建议(4-6 句)。",
category: "economy", category: "economy",
@@ -63,7 +63,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-zh-daily-brief", id: "local-zh-daily-brief",
displayText: "今日早报", displayText: "今日早报",
prompt: "请用中文写一份简洁的「今日早报」:国内外各 2–3 条要点、一条财经/科技、" prompt: "请用中文写一份简洁的「今日早报」:国内外各 2–3 条要点、一条财经/科技、"
+ "一条轻松话题;每条一句话,总计不超过 12 句。不确定处请标明。", + "一条轻松话题;每条一句话,总计不超过 12 句。不确定处请标明。",
category: "daily", category: "daily",
@@ -73,7 +73,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-zh-quote", id: "local-zh-quote",
displayText: "来句今日金句", displayText: "今日金句",
prompt: "请给一句适合今天分享的中文金句,并附上一两句简短解释。", prompt: "请给一句适合今天分享的中文金句,并附上一两句简短解释。",
category: "capability", category: "capability",
priority: 38, priority: 38,
@@ -82,7 +82,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-zh-howto", id: "local-zh-howto",
displayText: "给我一个小技巧", displayText: "生活技巧",
prompt: "分享一个实用的生活或工作效率小技巧,用中文说清步骤与适用场景(4-6 句)。", prompt: "分享一个实用的生活或工作效率小技巧,用中文说清步骤与适用场景(4-6 句)。",
category: "capability", category: "capability",
priority: 36, priority: 36,
@@ -124,7 +124,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-en-encyclopedia", id: "local-en-encyclopedia",
displayText: "Explain a concept", displayText: "A concept",
prompt: "Explain an interesting everyday concept in plain English with one real-life example (4-6 sentences).", prompt: "Explain an interesting everyday concept in plain English with one real-life example (4-6 sentences).",
category: "capability", category: "capability",
priority: 40, priority: 40,
@@ -133,7 +133,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-en-stocks", id: "local-en-stocks",
displayText: "Market pulse", displayText: "Markets",
prompt: "Summarize today's broad market mood (US or global) in plain English, " prompt: "Summarize today's broad market mood (US or global) in plain English, "
+ "note possible drivers, and add this is not financial advice (4-6 sentences).", + "note possible drivers, and add this is not financial advice (4-6 sentences).",
category: "economy", category: "economy",
@@ -153,7 +153,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-en-quote", id: "local-en-quote",
displayText: "Share a quote", displayText: "A quote",
prompt: "Share one short quote worth sending today, plus one or two sentences of context.", prompt: "Share one short quote worth sending today, plus one or two sentences of context.",
category: "capability", category: "capability",
priority: 38, priority: 38,
@@ -162,7 +162,7 @@ public enum AIHintLocalCatalog: Sendable {
), ),
AIHintCard( AIHintCard(
id: "local-en-howto", id: "local-en-howto",
displayText: "Give a tip", displayText: "A tip",
prompt: "Share one practical life or productivity tip in English, with steps and when it helps (4-6 sentences).", prompt: "Share one practical life or productivity tip in English, with steps and when it helps (4-6 sentences).",
category: "capability", category: "capability",
priority: 36, priority: 36,
+16 -19
View File
@@ -1,34 +1,21 @@
// AIHintPool.swift // AIHintPool.swift
// OSGKeyboard · Shared // OSGKeyboard · Shared
// //
// Builds the idle carousel pool: 100% clipboard cards while eligible, // Builds the idle carousel pool from local + remote cards. Clipboard
// otherwise a shuffled mix of non-clipboard local + remote cards. // sentence cards are excluded: the 30s copy window shows skill chips.
import Foundation import Foundation
public enum AIHintPool: Sendable { public enum AIHintPool: Sendable {
public static func activeCards( public static func activeCards(
pack: AIHintPack, pack: AIHintPack
clipboardHistoryEnabled: Bool,
newestClipboard: ClipboardHistoryEntry?,
now: Date = Date()
) -> [AIHintCard] { ) -> [AIHintCard] {
let clipboardEligible = clipboardHistoryEnabled
&& newestClipboard.map { ClipboardHistoryPolicy.isEligibleForAIHint($0, now: now) } == true
let clipboardCards = pack.cards.filter(\.requiresClipboard30s)
let regularCards = pack.cards.filter { !$0.requiresClipboard30s } let regularCards = pack.cards.filter { !$0.requiresClipboard30s }
.filter { !isHistoricalToday($0) } .filter { !isHistoricalToday($0) }
// Within 30s: only clipboard-related sentences. // Clipboard-conditioned sentences no longer rotate in the carousel;
if clipboardEligible { // the 30s window shows skill chips instead. Keep evergreen content
let pool = clipboardCards.isEmpty // loaded so leaving the window does not flash a leftover clipboard card.
? AIHintLocalCatalog.cards(locale: pack.locale).filter(\.requiresClipboard30s)
: clipboardCards
return pool.sorted { $0.priority > $1.priority }
}
// Otherwise: drop clipboard-conditioned cards entirely.
var merged = regularCards var merged = regularCards
let localRegular = AIHintLocalCatalog.cards(locale: pack.locale) let localRegular = AIHintLocalCatalog.cards(locale: pack.locale)
.filter { !$0.requiresClipboard30s } .filter { !$0.requiresClipboard30s }
@@ -38,6 +25,16 @@ public enum AIHintPool: Sendable {
return merged.sorted { $0.priority > $1.priority } return merged.sorted { $0.priority > $1.priority }
} }
/// Copy-then-30s window where clipboard skill chips replace the carousel.
public static func isClipboardSkillWindowActive(
clipboardHistoryEnabled: Bool,
newestClipboard: ClipboardHistoryEntry?,
now: Date = Date()
) -> Bool {
clipboardHistoryEnabled
&& newestClipboard.map { ClipboardHistoryPolicy.isEligibleForAIHint($0, now: now) } == true
}
/// Prompt for a tapped card. Clipboard cards fail closed so an expired /// Prompt for a tapped card. Clipboard cards fail closed so an expired
/// window can never send an instruction without its material. /// window can never send an instruction without its material.
public static func resolvePrompt( public static func resolvePrompt(
@@ -110,11 +110,81 @@ public enum FlowKeyboardAdoptBusyPolicy {
guard !isAwaitingFlowResult else { return .none } guard !isAwaitingFlowResult else { return .none }
guard let busyId = snapshot.busyUtteranceId else { return .none } guard let busyId = snapshot.busyUtteranceId else { return .none }
guard busyId != lastConsumedUtteranceId else { return .none } guard busyId != lastConsumedUtteranceId else { return .none }
guard busyId != lastStoppedUtteranceId else { return .none }
return .adoptProcessing(sessionId: sessionId, utteranceId: busyId) return .adoptProcessing(sessionId: sessionId, utteranceId: busyId)
default: default:
return .none return .none
} }
} }
/// Host still advertises `processing` for an utterance the keyboard already
/// acked, and the result mailbox is empty. That is a leaked gate not a
/// live ASR/LLM wait (those have no ack yet).
public static func isStaleDeliveredProcessing(
busyUtteranceId: UUID,
latestResult: FlowResult?,
latestAck: FlowAck?
) -> Bool {
guard let ack = latestAck, ack.utteranceId == busyUtteranceId else {
return false
}
if latestResult?.utteranceId == busyUtteranceId {
return false
}
return true
}
}
// MARK: - Terminal store after await
public enum FlowTerminalStorePolicy {
/// After an `await`, only the still-current, not-yet-terminal utterance
/// may write a final/error payload. Abort during LLM must not deliver.
public static func canStore(
currentUtteranceId: UUID?,
finishedUtteranceId: UUID,
alreadyTerminal: Bool
) -> Bool {
currentUtteranceId == finishedUtteranceId && !alreadyTerminal
}
}
// MARK: - Ack must drop a leaked processing gate
public enum FlowHostAckGatePolicy {
/// The keyboard acked this live utterance; the processing flag must not
/// outlive that ack (hint-card used to leak `reason=processing`).
public static func shouldDropProcessingGate(
ackUtteranceId: UUID,
currentUtteranceId: UUID?,
isUtteranceProcessing: Bool
) -> Bool {
isUtteranceProcessing && currentUtteranceId == ackUtteranceId
}
}
// MARK: - Empty double-tap skip
/// Drop a take before ASR when the press was too short to be speech.
public enum FlowEmptyTapSkipPolicy {
public static let maxDurationSeconds: TimeInterval = 0.3
public static let silencePeakThreshold: Float =
FlowCaptureTailDrainPolicy.flowDefault.silenceRMSThreshold
public static func peakAbs(_ samples: [Float]) -> Float {
samples.reduce(Float(0)) { max($0, abs($1)) }
}
/// `sampleCount == 0` or a missing peak counts as silence.
public static func shouldSkip(
durationSeconds: TimeInterval,
sampleCount: Int,
peakAmplitude: Float?
) -> Bool {
guard durationSeconds < maxDurationSeconds else { return false }
if sampleCount <= 0 { return true }
return (peakAmplitude ?? 0) < silencePeakThreshold
}
} }
// MARK: - Result matching // MARK: - Result matching
@@ -139,5 +139,7 @@ public enum FlowSessionKeys {
case audioUnavailable case audioUnavailable
case asrFailed case asrFailed
case generic case generic
/// Short silent tap discarded before ASR keyboard returns to idle quietly.
case discardedEmpty
} }
} }
@@ -239,6 +239,8 @@ public final class KeyboardState: ObservableObject {
public var sendAIAnswer: () -> Void = {} public var sendAIAnswer: () -> Void = {}
/// Sends a tapped idle hint card as the AI question (skip microphone). /// Sends a tapped idle hint card as the AI question (skip microphone).
public var submitAIHint: (AIHintCard) -> Void = { _ in } public var submitAIHint: (AIHintCard) -> Void = { _ in }
/// Sends a clipboard skill (reply / summarize / translate / future).
public var submitAIClipboardSkill: (AIClipboardSkill) -> Void = { _ in }
public var openSettings: () -> Void = {} public var openSettings: () -> Void = {}
/// Opens the host app straight to input-resource deployment. Used by the /// Opens the host app straight to input-resource deployment. Used by the
/// typing surface when Rime resources have not been deployed yet. /// typing surface when Rime resources have not been deployed yet.
@@ -303,8 +305,9 @@ public final class KeyboardState: ObservableObject {
surface == .ai && aiSession.isBusy surface == .ai && aiSession.isBusy
} }
/// Normal dictation can be discarded from initial microphone startup /// Normal dictation can be discarded from microphone startup through
/// through ASR / polish processing. Edit mode owns its separate close flow. /// ASR / polish, including the abort-wait after Cancel until the host
/// acks (coordinator keeps `phase == .processing` for that window).
public var canCancelVoiceInput: Bool { public var canCancelVoiceInput: Bool {
guard !editSession.isActive else { return false } guard !editSession.isActive else { return false }
switch phase { switch phase {
@@ -360,6 +363,8 @@ extension KeyboardState.Phase.ErrorKind {
return .hostAudioUnavailable return .hostAudioUnavailable
case .asrFailed, .generic: case .asrFailed, .generic:
return .hostTranscriptionFailed(error.message) return .hostTranscriptionFailed(error.message)
case .discardedEmpty:
return .noSpeechDetected
} }
} }
} }
@@ -36,6 +36,11 @@ public final class FlowUtterancePCMStore: @unchecked Sendable {
lock.withLock { samples.count } lock.withLock { samples.count }
} }
/// Copy of accumulated samples without clearing the store.
public func snapshot() -> [Float] {
lock.withLock { samples }
}
/// Returns accumulated samples and clears the store. /// Returns accumulated samples and clears the store.
public func consume() -> [Float] { public func consume() -> [Float] {
lock.withLock { lock.withLock {
@@ -5,6 +5,15 @@ import XCTest
@testable import OSGKeyboardShared @testable import OSGKeyboardShared
final class AIClipboardPromptTests: XCTestCase { final class AIClipboardPromptTests: XCTestCase {
func testInternalClipboardEnvelopeIsDetected() {
let prompt = AIClipboardPrompt.compose(
instruction: "请翻译剪贴板",
material: "hello"
)
XCTAssertTrue(AIClipboardPrompt.isInternalPrompt(prompt))
XCTAssertFalse(AIClipboardPrompt.isInternalPrompt("请翻译剪贴板"))
}
func testComposeKeepsInstructionAndMaterialInSeparateBlocks() { func testComposeKeepsInstructionAndMaterialInSeparateBlocks() {
let prompt = AIClipboardPrompt.compose( let prompt = AIClipboardPrompt.compose(
instruction: "请翻译剪贴板", instruction: "请翻译剪贴板",
@@ -0,0 +1,238 @@
// AIHintKeywordExtractorTests.swift
// OSGKeyboardTests
import XCTest
@testable import OSGKeyboardShared
final class AIHintKeywordExtractorTests: XCTestCase {
func testTrendingPrefersMetadataTitleOverHotPrefix() throws {
let json = """
{"id":"hot-1","text":"全网热点:朱镕基同志逝世","prompt":"请概括","category":"society","source":"tophub-open-hot","locale":"zh","metadata":{"title":"朱镕基同志逝世"}}
"""
let card = try JSONDecoder().decode(AIHintCard.self, from: Data(json.utf8))
XCTAssertEqual(card.visualKind, .trending)
XCTAssertEqual(card.visualKind.systemImage, "flame.fill")
XCTAssertEqual(card.resolvedDisplayText, "朱镕基同志逝世")
}
func testWeatherUsesCityAndTemperature() {
let card = AIHintCard(
id: "weather-zh-上海",
displayText: "上海天气速览",
prompt: "请说明天气",
category: "weather",
source: "open-meteo",
locale: "zh",
metadata: AIHintMetadata(city: "上海", tempC: 25.6)
)
XCTAssertEqual(card.visualKind, .weather)
XCTAssertEqual(card.resolvedDisplayText, "上海 26°")
}
func testHolidayStripsPrefixAndUsesChineseName() {
let card = AIHintCard(
id: "holiday-next-cn",
displayText: "临近节日:中秋节",
prompt: "介绍中秋",
category: "holiday",
source: "nager-holidays",
locale: "zh",
conditions: ["date"],
metadata: AIHintMetadata(name: "Mid-Autumn Festival")
)
XCTAssertEqual(card.visualKind, .calendar)
XCTAssertEqual(card.resolvedDisplayText, "中秋节")
}
func testEnglishHolidayUsesMetadataName() {
let card = AIHintCard(
id: "holiday-next-us",
displayText: "Upcoming: Labour Day",
prompt: "Explain Labour Day",
category: "holiday",
source: "nager-holidays",
locale: "en",
metadata: AIHintMetadata(name: "Labour Day")
)
XCTAssertEqual(card.resolvedDisplayText, "Labour Day")
}
func testLongOrgTitleFallsBackToLastChunk() {
let text = "中共中央 全国人大常委会 国务院 全国政协讣告 朱镕基同志逝世"
XCTAssertEqual(
AIHintKeywordExtractor.finalize(text, locale: "zh"),
"朱镕基同志逝世"
)
}
func testMixedTitlePrefersLeadingLatin() {
XCTAssertEqual(
AIHintKeywordExtractor.finalize(
"DeepSeek Pro 正式版已经发布,如何评价该模型?",
locale: "zh"
),
"DeepSeek"
)
}
func testDailyBriefIsNewsAndSoulQuoteIsSearch() {
let brief = AIHintCard(
id: "tophub-daily-brief-1",
displayText: "看看今日早报",
prompt: "写早报",
category: "daily",
source: "tophub-daily",
locale: "zh"
)
XCTAssertEqual(brief.visualKind, .news)
XCTAssertEqual(brief.resolvedDisplayText, "今日早报")
let soul = AIHintCard(
id: "tophub-daily-soul-1",
displayText: "今日一句:展开聊聊",
prompt: "解释这句话",
category: "daily",
source: "tophub-daily",
locale: "zh",
metadata: AIHintMetadata(soul: "为了防止我这个月又乱花钱")
)
XCTAssertEqual(soul.visualKind, .search)
XCTAssertEqual(soul.resolvedDisplayText, "今日金句")
}
func testCapabilityMapsToSearch() {
let card = AIHintLocalCatalog.cards(locale: "zh")
.first { $0.id == "local-zh-encyclopedia" }!
XCTAssertEqual(card.visualKind, .search)
XCTAssertEqual(card.resolvedDisplayText, "有趣概念")
}
func testStocksMapsToChartIcon() {
let card = AIHintLocalCatalog.cards(locale: "zh")
.first { $0.id == "local-zh-stocks" }!
XCTAssertEqual(card.visualKind, .stocks)
XCTAssertEqual(card.resolvedDisplayText, "今日大盘")
}
}
final class AIClipboardSkillTests: XCTestCase {
func testVisibleDefaultsAreReplySummarizeTranslate() {
XCTAssertEqual(
AIClipboardSkillCatalog.visible().map(\.id),
["reply", "summarize", "translate"]
)
}
func testVisibleRespectsEnabledIDsForFutureSettings() {
XCTAssertEqual(
AIClipboardSkillCatalog.visible(enabledIDs: ["translate", "reply"]).map(\.id),
["translate", "reply"]
)
XCTAssertEqual(
AIClipboardSkillCatalog.visible(enabledIDs: ["unknown"]).map(\.id),
[]
)
}
func testTranslateFollowsUserTargetLanguage() {
let ja = AIClipboardSkillCatalog.instruction(
skillID: AIClipboardSkillCatalog.translateID,
locale: "zh",
translationTargetLocaleId: "ja"
)
XCTAssertTrue(ja.contains("Japanese"))
XCTAssertFalse(ja.contains("互译"))
}
func testTranslateFallsBackToChineseEnglishWhenUnset() {
let zh = AIClipboardSkillCatalog.instruction(
skillID: AIClipboardSkillCatalog.translateID,
locale: "zh",
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId
)
XCTAssertTrue(zh.contains("中文与英文"))
let en = AIClipboardSkillCatalog.instruction(
skillID: AIClipboardSkillCatalog.translateID,
locale: "en",
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId
)
XCTAssertTrue(en.lowercased().contains("chinese"))
XCTAssertTrue(en.lowercased().contains("english"))
}
func testTranslateButtonTitleUsesDirectionPairsAndTargetShortNames() {
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId,
uiLanguage: .chinese
),
"中↔英"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId,
uiLanguage: .english
),
"CN↔EN"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "en",
uiLanguage: .chinese
),
"中译英"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "de",
uiLanguage: .chinese
),
"中译德"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "ja",
uiLanguage: .english
),
"To JP"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "zh-Hans",
uiLanguage: .english
),
"To CN"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "zh-Hant",
uiLanguage: .english
),
"To TW"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "zh-Hans",
uiLanguage: .chinese
),
"简↔繁"
)
XCTAssertEqual(
AIClipboardSkillCatalog.translateButtonTitle(
translationTargetLocaleId: "zh-Hant",
uiLanguage: .chinese
),
"简↔繁"
)
}
func testSummarizeAsksForOverviewNotShortening() {
let prompt = AIClipboardSkillCatalog.instruction(
skillID: AIClipboardSkillCatalog.summarizeID,
locale: "zh",
translationTargetLocaleId: TranslationLanguageCatalog.offLocaleId
)
XCTAssertTrue(prompt.contains("概括"))
XCTAssertTrue(prompt.contains("不要改写成可发送的短消息"))
}
}
+7 -9
View File
@@ -12,19 +12,21 @@ final class AIHintPoolTests: XCTestCase {
XCTAssertEqual(AIHintLocaleResolver.packLocale(preferredLanguages: ["en-US"]), "en") XCTAssertEqual(AIHintLocaleResolver.packLocale(preferredLanguages: ["en-US"]), "en")
} }
func testClipboardWindowForcesOnlyClipboardCards() { func testClipboardWindowNeverPutsClipboardCardsInCarousel() {
let pack = AIHintPack( let pack = AIHintPack(
locale: "zh", locale: "zh",
cards: AIHintLocalCatalog.cards(locale: "zh") cards: AIHintLocalCatalog.cards(locale: "zh")
) )
let recent = ClipboardHistoryEntry(text: "hello", createdAt: Date()) let recent = ClipboardHistoryEntry(text: "hello", createdAt: Date())
let cards = AIHintPool.activeCards( XCTAssertTrue(
pack: pack, AIHintPool.isClipboardSkillWindowActive(
clipboardHistoryEnabled: true, clipboardHistoryEnabled: true,
newestClipboard: recent newestClipboard: recent
) )
)
let cards = AIHintPool.activeCards(pack: pack)
XCTAssertFalse(cards.isEmpty) XCTAssertFalse(cards.isEmpty)
XCTAssertTrue(cards.allSatisfy(\.requiresClipboard30s)) XCTAssertTrue(cards.allSatisfy { !$0.requiresClipboard30s })
} }
func testClipboardDisabledDropsClipboardCards() { func testClipboardDisabledDropsClipboardCards() {
@@ -32,11 +34,7 @@ final class AIHintPoolTests: XCTestCase {
locale: "zh", locale: "zh",
cards: AIHintLocalCatalog.cards(locale: "zh") cards: AIHintLocalCatalog.cards(locale: "zh")
) )
let cards = AIHintPool.activeCards( let cards = AIHintPool.activeCards(pack: pack)
pack: pack,
clipboardHistoryEnabled: false,
newestClipboard: ClipboardHistoryEntry(text: "hello")
)
XCTAssertFalse(cards.isEmpty) XCTAssertFalse(cards.isEmpty)
XCTAssertTrue(cards.allSatisfy { !$0.requiresClipboard30s }) XCTAssertTrue(cards.allSatisfy { !$0.requiresClipboard30s })
} }
@@ -188,6 +188,28 @@ final class FlowKeyboardPoliciesTests: XCTestCase {
), ),
.none .none
) )
let processing = FlowReadySnapshot(
sessionId: sessionId,
ready: false,
reason: .processing,
engineMode: "cloud",
localeId: "zh-Hans",
busyUtteranceId: utteranceId,
hostGeneration: "gen-1"
)
XCTAssertEqual(
FlowKeyboardAdoptBusyPolicy.decide(
snapshot: processing,
currentHostGeneration: "gen-1",
isFlowRecording: false,
isAwaitingFlowResult: false,
lastConsumedUtteranceId: nil,
lastStoppedUtteranceId: utteranceId
),
.none,
"an aborted utterance must not come back as 识别中 on the next open"
)
} }
func testIgnoresDeadHostGenerationSnapshot() { func testIgnoresDeadHostGenerationSnapshot() {
@@ -234,6 +256,138 @@ final class FlowKeyboardPoliciesTests: XCTestCase {
) )
} }
func testStaleDeliveredProcessingRequiresMatchingAckWithoutResult() {
let busyId = UUID()
let sessionId = UUID()
XCTAssertFalse(
FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing(
busyUtteranceId: busyId,
latestResult: nil,
latestAck: nil
),
"live ASR/LLM has no result and no ack yet — must wait, not abort"
)
let otherAck = FlowAck(
sessionId: sessionId,
utteranceId: UUID(),
commandSeq: 1
)
XCTAssertFalse(
FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing(
busyUtteranceId: busyId,
latestResult: nil,
latestAck: otherAck
),
"ack for a previous utterance must not abort the live one"
)
let matchingAck = FlowAck(
sessionId: sessionId,
utteranceId: busyId,
commandSeq: 1
)
XCTAssertTrue(
FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing(
busyUtteranceId: busyId,
latestResult: nil,
latestAck: matchingAck
),
"acked + empty mailbox + still processing is a leaked gate"
)
let liveResult = FlowResult(
sessionId: sessionId,
utteranceId: busyId,
commandSeq: 1,
status: .streaming,
text: "draft"
)
XCTAssertFalse(
FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing(
busyUtteranceId: busyId,
latestResult: liveResult,
latestAck: matchingAck
),
"result still sitting for this utterance is waitable, not stale"
)
XCTAssertFalse(
FlowKeyboardAdoptBusyPolicy.isStaleDeliveredProcessing(
busyUtteranceId: UUID(),
latestResult: liveResult,
latestAck: matchingAck
)
)
}
func testTerminalStorePolicyRejectsAbortAndReplacement() {
let live = UUID()
XCTAssertTrue(
FlowTerminalStorePolicy.canStore(
currentUtteranceId: live,
finishedUtteranceId: live,
alreadyTerminal: false
)
)
XCTAssertFalse(
FlowTerminalStorePolicy.canStore(
currentUtteranceId: nil,
finishedUtteranceId: live,
alreadyTerminal: false
),
"abort cleared currentUtteranceId — do not deliver"
)
XCTAssertFalse(
FlowTerminalStorePolicy.canStore(
currentUtteranceId: UUID(),
finishedUtteranceId: live,
alreadyTerminal: false
),
"a newer utterance owns the gate"
)
XCTAssertFalse(
FlowTerminalStorePolicy.canStore(
currentUtteranceId: live,
finishedUtteranceId: live,
alreadyTerminal: true
),
"already terminal — abort already claimed"
)
}
func testAckGateDropsProcessingOnlyForTheLiveUtterance() {
let live = UUID()
XCTAssertTrue(
FlowHostAckGatePolicy.shouldDropProcessingGate(
ackUtteranceId: live,
currentUtteranceId: live,
isUtteranceProcessing: true
)
)
XCTAssertFalse(
FlowHostAckGatePolicy.shouldDropProcessingGate(
ackUtteranceId: live,
currentUtteranceId: live,
isUtteranceProcessing: false
)
)
XCTAssertFalse(
FlowHostAckGatePolicy.shouldDropProcessingGate(
ackUtteranceId: live,
currentUtteranceId: UUID(),
isUtteranceProcessing: true
),
"must not clobber a newer utterance"
)
XCTAssertFalse(
FlowHostAckGatePolicy.shouldDropProcessingGate(
ackUtteranceId: live,
currentUtteranceId: nil,
isUtteranceProcessing: true
)
)
}
// MARK: - Result matcher // MARK: - Result matcher
func testMatchingResultRequiresAlignedSessionAndUtterance() { func testMatchingResultRequiresAlignedSessionAndUtterance() {
@@ -371,3 +525,58 @@ final class FlowKeyboardPoliciesTests: XCTestCase {
) )
} }
} }
// MARK: - Empty tap skip
extension FlowKeyboardPoliciesTests {
func testEmptyTapSkipRequiresShortDuration() {
XCTAssertTrue(
FlowEmptyTapSkipPolicy.shouldSkip(
durationSeconds: 0.2,
sampleCount: 0,
peakAmplitude: nil
)
)
XCTAssertFalse(
FlowEmptyTapSkipPolicy.shouldSkip(
durationSeconds: 0.35,
sampleCount: 0,
peakAmplitude: nil
)
)
}
func testEmptyTapSkipTreatsMissingSamplesAsSilence() {
XCTAssertTrue(
FlowEmptyTapSkipPolicy.shouldSkip(
durationSeconds: 0.1,
sampleCount: 16,
peakAmplitude: nil
)
)
}
func testEmptyTapSkipKeepsShortSpeech() {
XCTAssertFalse(
FlowEmptyTapSkipPolicy.shouldSkip(
durationSeconds: 0.2,
sampleCount: 3_200,
peakAmplitude: 0.2
)
)
}
func testEmptyTapSkipDropsShortSilence() {
XCTAssertTrue(
FlowEmptyTapSkipPolicy.shouldSkip(
durationSeconds: 0.2,
sampleCount: 3_200,
peakAmplitude: 0.001
)
)
XCTAssertEqual(
FlowEmptyTapSkipPolicy.peakAbs([0.001, -0.004, 0.002]),
0.004
)
}
}
@@ -20,4 +20,12 @@ final class FlowUtterancePCMStoreTests: XCTestCase {
store.append([1, 2, 3, 4, 5]) store.append([1, 2, 3, 4, 5])
XCTAssertEqual(store.consume(), [2, 3, 4, 5]) XCTAssertEqual(store.consume(), [2, 3, 4, 5])
} }
func testSnapshotDoesNotClear() {
let store = FlowUtterancePCMStore(maxSampleCount: 100)
store.append([0.1, -0.2])
XCTAssertEqual(store.snapshot(), [0.1, -0.2])
XCTAssertEqual(store.sampleCount, 2)
XCTAssertEqual(store.consume(), [0.1, -0.2])
}
} }
+1 -1
View File
@@ -7,7 +7,7 @@ Voice input for iPhone, iPad, and Mac. Speak in any app — polished text lands
![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple) ![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple)
![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple) ![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple)
![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift) ![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift)
![Version](https://img.shields.io/badge/version-1.7.0-3aa05a) ![Version](https://img.shields.io/badge/version-1.7.5-3aa05a)
![License](https://img.shields.io/badge/license-Source%20Available-blue) ![License](https://img.shields.io/badge/license-Source%20Available-blue)
[Website](https://hkgood.github.io/OSGKeyboard/) · [中文 README](./README.md) · [Privacy Policy](https://hkgood.github.io/OSGKeyboard/privacy/) [Website](https://hkgood.github.io/OSGKeyboard/) · [中文 README](./README.md) · [Privacy Policy](https://hkgood.github.io/OSGKeyboard/privacy/)
+1 -1
View File
@@ -7,7 +7,7 @@
![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple) ![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple)
![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple) ![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple)
![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift) ![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift)
![Version](https://img.shields.io/badge/version-1.7.0-3aa05a) ![Version](https://img.shields.io/badge/version-1.7.5-3aa05a)
![License](https://img.shields.io/badge/license-Source%20Available-blue) ![License](https://img.shields.io/badge/license-Source%20Available-blue)
[官网](https://hkgood.github.io/OSGKeyboard/) · [English](./README.en.md) · [隐私政策](https://hkgood.github.io/OSGKeyboard/privacy/) [官网](https://hkgood.github.io/OSGKeyboard/) · [English](./README.en.md) · [隐私政策](https://hkgood.github.io/OSGKeyboard/privacy/)
+19 -16
View File
@@ -1,4 +1,4 @@
# App Store Connect — OSGKeyboard 1.7.0 (build 65) # App Store Connect — OSGKeyboard 1.7.5 (build 66)
> Current metadata baseline for the iOS/iPadOS App Store build. Version and build > Current metadata baseline for the iOS/iPadOS App Store build. Version and build
> numbers come from `project.yml`. The repository also contains a separate > numbers come from `project.yml`. The repository also contains a separate
@@ -11,7 +11,7 @@
| App name | `OSGKeyboard` | ≤ 30 characters | | App name | `OSGKeyboard` | ≤ 30 characters |
| Subtitle | `Voice input, everywhere` | ≤ 30 characters | | Subtitle | `Voice input, everywhere` | ≤ 30 characters |
| Bundle ID | `com.osgkeyboard.ios` | iOS host target | | Bundle ID | `com.osgkeyboard.ios` | iOS host target |
| Version / build | `1.7.0` / `65` | `MARKETING_VERSION` / `CURRENT_PROJECT_VERSION` | | Version / build | `1.7.5` / `66` | `MARKETING_VERSION` / `CURRENT_PROJECT_VERSION` |
| Minimum system | iOS/iPadOS 26 | iPhone and iPad | | Minimum system | iOS/iPadOS 26 | iPhone and iPad |
| Primary locale | `en-US` | Simplified Chinese is also bundled | | Primary locale | `en-US` | Simplified Chinese is also bundled |
| Primary category | Utilities | | | Primary category | Utilities | |
@@ -101,22 +101,25 @@ Voice input anywhere, with on-device recognition by default. Add your own AI key
keyboard,voice,dictation,speech,transcribe,AI,pinyin,Chinese,English,polish,typing,productivity keyboard,voice,dictation,speech,transcribe,AI,pinyin,Chinese,English,polish,typing,productivity
``` ```
## What's new in 1.7.0 ## What's new in 1.7.5
```text ```text
NEW NEW
AI keyboard mode turns spoken questions into reviewable answers, with Optional clipboard history (latest 15 plain-text items) with a top-bar
explicit Insert and Send actions. panel and suggestion strip; AI idle hints can use a fresh copy for ~30s.
Provider-supported web search is available for AI questions, with a AI mode empty state rotates one-line suggestions; tap sends the card
no-search fallback if the provider rejects the request. prompt without using the mic.
Long-press the microphone to describe an edit to the last verified Spoken clipboard requests in AI mode attach stored clipboard text when
OSGKeyboard insertion. you name the clipboard out loud.
• Settings → AI Agent adds Response length (Short / Medium / Detailed).
• AI answers stream into the keyboard as the model writes.
CHANGED CHANGED
Polish and AI mode now use only the API key you configure; the built-in History and Personal dictionary move from the tab bar onto Home preview
DeepSeek fallback has been removed. cards; engine status scrolls with the page.
iPad voice and typing surfaces include the system globe key, and the Translation control sits beside the mic (mirrored with Undo); the former
typing layout adds iPad-specific sizing and editing controls. top-bar slot is the Clipboard button.
• Undo covers clipboard pastes as well as dictation, AI answers, and edits.
``` ```
## App Review information ## App Review information
@@ -198,10 +201,10 @@ standard HTTPS. Re-evaluate this answer if non-exempt cryptography is added.
## Submission checklist ## Submission checklist
- [ ] Confirm `project.yml` still reads version 1.7.0 / build 65 - [ ] Confirm `project.yml` still reads version 1.7.5 / build 66
- [ ] Generate the project with `./Scripts/generate-xcodeproj.sh` - [ ] Open the existing Xcode project (do not regenerate unless needed)
- [ ] Run the release build and test suites on macOS with Xcode 26 - [ ] Run the release build and test suites on macOS with Xcode 26
- [ ] Replace screenshots with captures from the submitted build - [ ] Replace screenshots with captures from the submitted build
- [ ] Verify the privacy answers against the submitted provider features - [ ] Verify the privacy answers against the submitted provider features
- [ ] Confirm the tip product remains optional and unlocks no feature - [ ] Confirm the tip product remains optional and unlocks no feature
- [ ] Upload, select build 65, add review notes, and submit - [ ] Upload, select build 66, add review notes, and submit
+4 -4
View File
@@ -68,7 +68,7 @@
"image": "https://hkgood.github.io/OSGKeyboard/assets/app-icon.png", "image": "https://hkgood.github.io/OSGKeyboard/assets/app-icon.png",
"screenshot": "https://hkgood.github.io/OSGKeyboard/assets/hero-devices.png", "screenshot": "https://hkgood.github.io/OSGKeyboard/assets/hero-devices.png",
"description": "Source-available voice input for iPhone, iPad, and Mac. On-device speech recognition by default, BYOK AI polish, personal dictionary, and Mac global hotkey dictation.", "description": "Source-available voice input for iPhone, iPad, and Mac. On-device speech recognition by default, BYOK AI polish, personal dictionary, and Mac global hotkey dictation.",
"softwareVersion": "1.7.0", "softwareVersion": "1.7.5",
"isAccessibleForFree": true, "isAccessibleForFree": true,
"offers": { "offers": {
"@type": "Offer", "@type": "Offer",
@@ -1280,7 +1280,7 @@
<footer> <footer>
<div class="wrap footer-inner"> <div class="wrap footer-inner">
<span data-i18n="footer.copy">© OSGKeyboard · v1.7.0 · 源码可见(非开源)</span> <span data-i18n="footer.copy">© OSGKeyboard · v1.7.5 · 源码可见(非开源)</span>
<span class="footer-links"> <span class="footer-links">
<a href="#download" data-i18n="footer.download">Download</a> <a href="#download" data-i18n="footer.download">Download</a>
<a href="#start" data-i18n="footer.install">Install</a> <a href="#start" data-i18n="footer.install">Install</a>
@@ -1403,7 +1403,7 @@
"download.subtitle": "Get iPhone / iPad from the App Store. The Mac DMG below is the signed and notarized historical version 1.1.", "download.subtitle": "Get iPhone / iPad from the App Store. The Mac DMG below is the signed and notarized historical version 1.1.",
"download.install": "Installation steps", "download.install": "Installation steps",
"download.source": "GitHub source", "download.source": "GitHub source",
"footer.copy": "© OSGKeyboard · v1.7.0 · source available (not open source)", "footer.copy": "© OSGKeyboard · v1.7.5 · source available (not open source)",
"footer.download": "Download", "footer.download": "Download",
"footer.install": "Setup", "footer.install": "Setup",
"footer.compare": "Compare", "footer.compare": "Compare",
@@ -1520,7 +1520,7 @@
"download.subtitle": "iPhone / iPad 从 App Store 安装;下方 Mac DMG 为已签名并公证的历史版本 1.1。", "download.subtitle": "iPhone / iPad 从 App Store 安装;下方 Mac DMG 为已签名并公证的历史版本 1.1。",
"download.install": "查看安装步骤", "download.install": "查看安装步骤",
"download.source": "GitHub 源码", "download.source": "GitHub 源码",
"footer.copy": "© OSGKeyboard · v1.7.0 · 源码可见(非开源)", "footer.copy": "© OSGKeyboard · v1.7.5 · 源码可见(非开源)",
"footer.download": "下载", "footer.download": "下载",
"footer.install": "安装", "footer.install": "安装",
"footer.compare": "对比", "footer.compare": "对比",
+1 -1
View File
@@ -2,7 +2,7 @@
> Source-available voice input for iPhone, iPad, and Mac. Speak in any app — text lands at the cursor, with optional BYOK polish. Recognition is on-device by default. The iOS keyboard also types Chinese (pinyin / double pinyin) and English (autocomplete, autocorrect, next-word). > Source-available voice input for iPhone, iPad, and Mac. Speak in any app — text lands at the cursor, with optional BYOK polish. Recognition is on-device by default. The iOS keyboard also types Chinese (pinyin / double pinyin) and English (autocomplete, autocorrect, next-word).
OSGKeyboard 1.7.0 (build 64) is a custom keyboard for iOS/iPadOS 26+ and a menu-bar global dictation app for macOS 15+. iOS uses Apple SpeechAnalyzer locally. Mac local mode defaults to catalog model qwen3-mlx-0.6b-4bit with Qwen3 MLX streaming and Apple Speech fallback. Cloud ASR/LLM is used only when the user opts in and configures credentials. OSGKeyboard 1.7.5 (build 66) is a custom keyboard for iOS/iPadOS 26+ and a menu-bar global dictation app for macOS 15+. iOS uses Apple SpeechAnalyzer locally. Mac local mode defaults to catalog model qwen3-mlx-0.6b-4bit with Qwen3 MLX streaming and Apple Speech fallback. Cloud ASR/LLM is used only when the user opts in and configures credentials.
## Product ## Product
+3 -3
View File
@@ -176,9 +176,9 @@
<p class="empty" id="empty-zh">暂无与当前版本匹配的更新说明。</p> <p class="empty" id="empty-zh">暂无与当前版本匹配的更新说明。</p>
<p class="empty" id="empty-en">No release notes match this app version.</p> <p class="empty" id="empty-en">No release notes match this app version.</p>
<!-- ========== 1.7.0 ========== --> <!-- ========== 1.7.5 ========== -->
<article class="version-block" data-version="1.7.0"> <article class="version-block" data-version="1.7.5">
<div class="version-chip">1.7.0</div> <div class="version-chip">1.7.5</div>
<div class="lang-block" data-lang="zh"> <div class="lang-block" data-lang="zh">
<section class="section"> <section class="section">
+2 -2
View File
@@ -48,8 +48,8 @@ settings:
ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES
STRING_CATALOG_GENERATE_SYMBOLS: YES STRING_CATALOG_GENERATE_SYMBOLS: YES
CLANG_CXX_LANGUAGE_STANDARD: c++17 CLANG_CXX_LANGUAGE_STANDARD: c++17
MARKETING_VERSION: "1.7.0" MARKETING_VERSION: "1.7.5"
CURRENT_PROJECT_VERSION: "65" CURRENT_PROJECT_VERSION: "67"
# 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖) # 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖)
# 项目级签名 xcconfig,适用于所有 target # 项目级签名 xcconfig,适用于所有 target