merge: bring keyboard touch accuracy and mood-emoji polish into main

This commit is contained in:
Rocky
2026-08-06 15:18:40 +08:00
57 changed files with 2860 additions and 489 deletions
+24
View File
@@ -7,9 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **Volcengine ASR API Key auth**: settings can switch to the new-console single `X-Api-Key` mode while keeping legacy APP ID + Access Token as the default; SAUC resource is fixed to Doubao streaming 2.0 (`volc.seedasr.sauc.duration`). / **火山 ASR API Key 鉴权**:设置可切换到新控制台单字段 `X-Api-Key`,默认仍为旧版 APP ID + Access TokenSAUC 资源固定为豆包流式 2.0(`volc.seedasr.sauc.duration`)。
- **Custom style mood emoji**: custom polish styles can opt in (default off) to allow emotion-matched emoji; the prompt overrides R5 and post-processing keeps them on screen. Paste-only prompts that declare emoji opt-in are detected automatically. / **自定义风格情绪 emoji**:自定义润色风格可单独开启(默认关)按情绪点缀 emoji;提示词覆盖 R5,后处理保留上屏。仅粘贴声明允许 emoji 的 prompt 也会自动识别。
### Fixed
- **PiP post-utterance yellow flash**: after a voice turn, the mic no longer briefly shows yellow「正在启动画中画」— hold ready once the session proved live, refresh host ready on ack, and avoid labeling an already-active PiP as `.starting`. / **PiP 句末黄色闪烁**:语音说完后麦克风不再短暂变黄并提示「正在启动画中画」——会话曾就绪后保持绿灯、ack 后立即刷新 host ready,且已激活的 PiP 不再标成 `.starting`
- **English Shift after Return**: sentence autocapitalization treats newline as a new-line boundary (system keyboard behavior), so Notes-style multi-line Return arms Shift again. / **回车后英文 Shift**:句首自动大写将换行视为新行边界(对齐系统键盘),备忘录等多行回车后会重新点亮 Shift。
- **English Shift with stale proxy**: after our own insert/delete, autocap merges a local caret-prefix shadow when `documentContextBeforeInput` lags (e.g. period / Return in Notes). / **滞后前文下的英文 Shift**:自身插删后若 `documentContextBeforeInput` 滞后,自动大写会合并本地光标前文影子(如备忘录中的句号/回车)。
## [1.6.2] - 2026-08-06
### Changed
- **ASR capture voice processing**: iOS App Flow / preview capture switches from `.measurement` to `.voiceChat`, enables `AVAudioEngine` Voice Processing, and prefers a near-talk built-in mic pattern so competing talkers in the same room are suppressed more strongly (Control Center Voice Isolation remains available once VP is on). / **ASR 采音人声处理**iOS App 的 Flow / 预览采音从 `.measurement` 改为 `.voiceChat`,打开 `AVAudioEngine` Voice Processing,并优先近讲内置麦指向,以更好压制同房间旁人说话(开启 VP 后控制中心仍可选人声隔离)。
### Fixed
- **Voice toolbar haptics**: delete / space / return on the voice keyboard follow Settings → General → Haptics (mic unchanged). / **语音底栏震动**:语音键盘的删除 / 空格 / 回车跟随设置 → 通用 → 震动(麦克风大圆钮不变)。
- **Haptics after app switch**: re-prepare Taptic when the keyboard appears so feedback does not go cold after switching host apps. / **切 App 后震动消失**:键盘每次出现时重新预热触觉引擎,避免从 A 切到 B 后触感变冷。
- **Typing switch after cold start**: a sticky App Group `hostHeavy` flag no longer silently blocks 中文/EN when the host died mid-warmup; the flag now expires and is cleared on Flow state reset / host relaunch. / **冷启动后无法切拼音/英文**:宿主中途被杀留下的 `hostHeavy` 不再静默挡住中文/EN;该标志会过期,并在 Flow 状态清理与宿主重启时清除。
- **Chinese Shift uppercase**: with Shift / Caps Lock / Shift-hold on in Pinyin mode, letter keys insert Latin uppercase directly (system-keyboard style) instead of sending uppercase keycodes to Rime, which rejects them. / **拼音 Shift 大写**:拼音模式下开启 Shift / Caps Lock / 按住 Shift 时,字母键直接插入大写拉丁字母(对齐系统键盘),不再把大写键码送给会拒绝的 Rime。
## [1.6.1] - 2026-08-05
### Added
- **Personal dictionary → Chinese Pinyin**: adding or deleting terms redeploys an `osg_personal` Rime sidecar (local pinyin from the bundled dict; same-code pin-to-top; Latin names also surface in Chinese mode); English typing and ASR keep using the same dictionary automatically. Takes effect the next time the keyboard opens. / **个性词库 → 中文拼音**:增删词条会重部署 `osg_personal` Rime 旁路词表(复用打包词典本地注音;同码置顶;拉丁专名在中文键盘也可出候选);英文打字与 ASR 继续自动共用同一词库。下次打开键盘生效。
- **Typing key sound & haptics**: letter / modifier / space / return / delete keys play system click sounds on press-down; Settings → General → Haptics offers Off / Light (default) / Strong role-based feedback. / **打字按键音效与震动**:字母 / 修饰 / 空格 / 回车 / 删除键按下即播系统咔嗒音;设置 → 通用 → 震动提供关 / 轻(默认) / 强 的角色分层触感。
- **Typing touch accuracy (Phase 13)**: gap-filling hit regions (no dead seams), grid-level down → move → up tracking with slide-to-reselect, release-to-commit for letters/space/return, press-and-repeat delete, and a light upward touch intent offset. / **打字触控准确率(Phase 1–3)**:热区填缝(无死区)、网格级按下→滑动改选→松手确认、字母/空格/回车松手提交、删除按下连删,以及轻微向上的触点意图偏移。
- **Pinyin next-key bias (Phase 4)**: during full-pinyin composition, ambiguous seam hits prefer legal next letters (capped boost/shrink); clear on-key hits and English / double-pinyin stay unbiased. / **拼音下一键偏心(Phase 4**:全拼组词中,缝隙歧义命中优先合法后续字母(有上下限);明确落在键上的点击以及英文/双拼不受偏置。
### Changed
- **Rime personal-dict import**: SharedSupport now imports `osg_personal` into `osg_pinyin` (resource version `2.3.0`). / **Rime 个性词导入**SharedSupport 将 `osg_personal` 导入 `osg_pinyin`(资源版本 `2.3.0`)。
@@ -1,22 +0,0 @@
<svg width="1024" height="1024" viewBox="0 0 1024 1024" fill="none" xmlns="http://www.w3.org/2000/svg">
<rect x="103" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="103" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="225.632" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="225.632" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="348.265" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="348.265" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="225.632" y="827.837" width="572.017" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="470.897" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="470.897" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="716.162" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="716.162" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="593.529" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="593.529" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="838.794" y="555" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<rect x="838.794" y="677.633" width="81.4876" height="81.4876" rx="40.7438" stroke="white" stroke-opacity="0.8" style="stroke:white;stroke-opacity:0.8;" stroke-width="10"/>
<path d="M578.033 284.581C578.033 248.262 548.706 218.82 512.53 218.82C476.353 218.82 447.026 248.262 447.026 284.581C447.026 320.9 476.353 350.343 512.53 350.343V356.163C473.151 356.163 441.229 324.115 441.229 284.581C441.229 245.048 473.151 213 512.53 213C551.908 213 583.83 245.048 583.83 284.581C583.83 324.115 551.908 356.163 512.53 356.163V350.343C548.706 350.343 578.033 320.9 578.033 284.581Z" fill="#3AA05A" style="fill:#3AA05A;fill:color(display-p3 0.2267 0.6268 0.3534);fill-opacity:1;"/>
<path d="M252 284.581C252 324.115 283.922 356.163 323.301 356.163C362.679 356.163 394.601 324.115 394.601 284.581C394.601 245.048 362.679 213 323.301 213C283.922 213 252 245.048 252 284.581Z" fill="#3AA05A" style="fill:#3AA05A;fill:color(display-p3 0.2267 0.6268 0.3534);fill-opacity:1;"/>
<path d="M701.197 213.561C739.824 213.561 771.275 244.276 772.461 282.615L701.197 282.616V288.229H772.418C770.659 326.044 739.446 356.162 701.197 356.162C661.819 356.162 629.897 324.24 629.896 284.861C629.896 245.483 661.819 213.561 701.197 213.561ZM772.498 284.861C772.498 284.912 772.497 284.964 772.497 285.015V284.707C772.497 284.758 772.498 284.81 772.498 284.861Z" fill="#3AA05A" style="fill:#3AA05A;fill:color(display-p3 0.2267 0.6268 0.3534);fill-opacity:1;"/>
<path d="M447.966 312.933L447.966 307.319L512.53 307.319L512.53 312.933L447.966 312.933Z" fill="#3AA05A" style="fill:#3AA05A;fill:color(display-p3 0.2267 0.6268 0.3534);fill-opacity:1;"/>
<path d="M512.53 268.018V262.404H577.093V268.018H512.53Z" fill="#3AA05A" style="fill:#3AA05A;fill:color(display-p3 0.2267 0.6268 0.3534);fill-opacity:1;"/>
</svg>

Before

Width:  |  Height:  |  Size: 4.2 KiB

@@ -0,0 +1,58 @@
<svg width="1024" height="1024" viewBox="0 0 1024 1024" fill="none" xmlns="http://www.w3.org/2000/svg">
<rect width="1024" height="1024" fill="white"/>
<rect width="1024" height="1024" fill="url(#paint0_linear_2178_78)" fill-opacity="0.2"/>
<g filter="url(#filter0_dd_2178_78)">
<path d="M603.944 512.419C603.944 461.795 563.066 420.756 512.64 420.756C462.215 420.756 421.337 461.795 421.337 512.419C421.337 563.043 462.215 604.081 512.64 604.081V612.193C457.752 612.193 413.257 567.523 413.257 512.419C413.257 457.315 457.752 412.644 512.64 412.644C567.528 412.644 612.023 457.315 612.023 512.419C612.023 567.523 567.528 612.193 512.64 612.193V604.081C563.066 604.081 603.944 563.043 603.944 512.419Z" fill="#00AB11"/>
<path d="M603.944 512.419C603.944 461.795 563.066 420.756 512.64 420.756C462.215 420.756 421.337 461.795 421.337 512.419C421.337 563.043 462.215 604.081 512.64 604.081V612.193C457.752 612.193 413.257 567.523 413.257 512.419C413.257 457.315 457.752 412.644 512.64 412.644C567.528 412.644 612.023 457.315 612.023 512.419C612.023 567.523 567.528 612.193 512.64 612.193V604.081C563.066 604.081 603.944 563.043 603.944 512.419Z" fill="url(#paint1_linear_2178_78)" fill-opacity="0.2"/>
<path d="M149.498 512.419C149.498 567.523 193.993 612.193 248.881 612.193C303.769 612.193 348.264 567.523 348.264 512.419C348.264 457.315 303.769 412.644 248.881 412.644C193.993 412.644 149.498 457.315 149.498 512.419Z" fill="#00AB11"/>
<path d="M149.498 512.419C149.498 567.523 193.993 612.193 248.881 612.193C303.769 612.193 348.264 567.523 348.264 512.419C348.264 457.315 303.769 412.644 248.881 412.644C193.993 412.644 149.498 457.315 149.498 512.419Z" fill="url(#paint2_linear_2178_78)" fill-opacity="0.2"/>
<path d="M422.647 551.937L422.647 544.112L512.64 544.112L512.64 551.937L422.647 551.937Z" fill="#00AB11"/>
<path d="M422.647 551.937L422.647 544.112L512.64 544.112L512.64 551.937L422.647 551.937Z" fill="url(#paint3_linear_2178_78)" fill-opacity="0.2"/>
<path d="M512.641 489.332V481.506H602.633V489.332H512.641Z" fill="#00AB11"/>
<path d="M512.641 489.332V481.506H602.633V489.332H512.641Z" fill="url(#paint4_linear_2178_78)" fill-opacity="0.2"/>
<path d="M775.617 413.425C829.458 413.425 873.296 456.239 874.949 509.679H775.617V517.504H874.89C872.438 570.212 828.931 612.191 775.617 612.191C720.729 612.191 676.233 567.696 676.233 512.809C676.233 457.921 720.729 413.425 775.617 413.425ZM875 517.504H874.89C874.962 515.948 875 514.383 875 512.809C875 511.761 874.981 510.718 874.949 509.679H875V517.504Z" fill="#00AB11"/>
<path d="M775.617 413.425C829.458 413.425 873.296 456.239 874.949 509.679H775.617V517.504H874.89C872.438 570.212 828.931 612.191 775.617 612.191C720.729 612.191 676.233 567.696 676.233 512.809C676.233 457.921 720.729 413.425 775.617 413.425ZM875 517.504H874.89C874.962 515.948 875 514.383 875 512.809C875 511.761 874.981 510.718 874.949 509.679H875V517.504Z" fill="url(#paint5_linear_2178_78)" fill-opacity="0.2"/>
</g>
<defs>
<filter id="filter0_dd_2178_78" x="124.498" y="402.644" width="775.502" height="249.549" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB">
<feFlood flood-opacity="0" result="BackgroundImageFix"/>
<feColorMatrix in="SourceAlpha" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0" result="hardAlpha"/>
<feOffset dy="4"/>
<feGaussianBlur stdDeviation="2"/>
<feComposite in2="hardAlpha" operator="out"/>
<feColorMatrix type="matrix" values="0 0 0 0 0 0 0 0 0 0.382061 0 0 0 0 0.0382061 0 0 0 0.3 0"/>
<feBlend mode="normal" in2="BackgroundImageFix" result="effect1_dropShadow_2178_78"/>
<feColorMatrix in="SourceAlpha" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0" result="hardAlpha"/>
<feOffset dy="15"/>
<feGaussianBlur stdDeviation="12.5"/>
<feComposite in2="hardAlpha" operator="out"/>
<feColorMatrix type="matrix" values="0 0 0 0 0 0 0 0 0 0.6 0 0 0 0 0.06 0 0 0 0.3 0"/>
<feBlend mode="normal" in2="effect1_dropShadow_2178_78" result="effect2_dropShadow_2178_78"/>
<feBlend mode="normal" in="SourceGraphic" in2="effect2_dropShadow_2178_78" result="shape"/>
</filter>
<linearGradient id="paint0_linear_2178_78" x1="512" y1="1207" x2="131" y2="56.9999" gradientUnits="userSpaceOnUse">
<stop/>
<stop offset="1" stop-color="#666666" stop-opacity="0"/>
</linearGradient>
<linearGradient id="paint1_linear_2178_78" x1="443.12" y1="400.5" x2="615.798" y2="422.58" gradientUnits="userSpaceOnUse">
<stop/>
<stop offset="1" stop-color="#666666"/>
</linearGradient>
<linearGradient id="paint2_linear_2178_78" x1="179.361" y1="400.5" x2="352.039" y2="422.58" gradientUnits="userSpaceOnUse">
<stop/>
<stop offset="1" stop-color="#666666"/>
</linearGradient>
<linearGradient id="paint3_linear_2178_78" x1="499.119" y1="552.413" x2="474.127" y2="515.519" gradientUnits="userSpaceOnUse">
<stop/>
<stop offset="1" stop-color="#666666"/>
</linearGradient>
<linearGradient id="paint4_linear_2178_78" x1="589.112" y1="489.808" x2="564.121" y2="452.913" gradientUnits="userSpaceOnUse">
<stop/>
<stop offset="1" stop-color="#666666"/>
</linearGradient>
<linearGradient id="paint5_linear_2178_78" x1="706.097" y1="401.328" x2="878.752" y2="423.493" gradientUnits="userSpaceOnUse">
<stop/>
<stop offset="1" stop-color="#666666"/>
</linearGradient>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 5.1 KiB

+2 -5
View File
@@ -8,11 +8,8 @@
"hidden" : false,
"layers" : [
{
"fill" : "none",
"hidden" : false,
"image-name" : "App Icon Template.svg",
"name" : "App Icon Template",
"opacity" : 1
"image-name" : "Group 29.svg",
"name" : "Group 29"
}
],
"lighting" : "individual",
@@ -787,6 +787,11 @@ final class FlowSessionManager: ObservableObject {
reason = .audioEngineNotLive
} else if !usesPiPKeepAlive, !hasRecentAudio {
reason = .waitingForAudioProof
} else if usesPiPKeepAlive, pipController.isPictureInPictureActive {
// PiP is already up transient gates (!polling / interruption)
// are not a cold start. Prefer awaitingDelivery-adjacent idle over
// `.starting` so the keyboard never flashes.
reason = .awaitingDelivery
} else {
reason = .starting
}
@@ -1331,6 +1336,10 @@ final class FlowSessionManager: ObservableObject {
return
}
FlowSessionBridge.clearResult()
// Ack clears the terminal result; republish ready immediately so the
// keyboard does not linger on awaitingDelivery / starting between the
// 500 ms poll and the next heartbeat.
refreshHostReady()
}
private func hasUnacknowledgedTerminalResult() -> Bool {
+50 -22
View File
@@ -58,6 +58,34 @@ struct ASRSettingsCard: View {
private var volcengineRows: some View {
Group {
Toggle(isOn: volcengineAPIKeyModeBinding) {
VStack(alignment: .leading, spacing: Spacing.xxs) {
Text("settings.asr.volcengine.apiKeyMode.title")
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
Text("settings.asr.volcengine.apiKeyMode.subtitle")
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.fixedSize(horizontal: false, vertical: true)
}
}
.tint(palette.accent)
.settingsListRow()
rowDivider
if volcengineFields.usesAPIKeyAuth {
SettingsCredentialRow(
title: AppL10n.string("settings.asr.volcengine.apiKey"),
placeholder: "API Key",
text: Binding(
get: { volcengineFields.apiKeyCredential },
set: { updateVolcengine(apiKeyCredential: $0) }
),
isSecret: true,
isMonospaced: true
)
} else {
SettingsCredentialRow(
title: AppL10n.string("settings.asr.volcengine.appId"),
placeholder: "APP ID",
@@ -79,20 +107,13 @@ struct ASRSettingsCard: View {
isSecret: true,
isMonospaced: true
)
rowDivider
SettingsCredentialRow(
title: AppL10n.string("settings.asr.volcengine.resourceId"),
placeholder: CloudASRModelCatalog.defaultModel(for: "volcengine"),
text: Binding(
get: { volcengineFields.resourceID },
set: { updateVolcengine(resourceID: $0) }
),
isMonospaced: true,
defaultValue: CloudASRModelCatalog.defaultModel(for: "volcengine")
)
}
rowDivider
SettingsProviderRow(title: AppL10n.string("settings.provider.note")) {
Text("settings.asr.volcengine.note")
Text(volcengineFields.usesAPIKeyAuth
? "settings.asr.volcengine.note.apiKey"
: "settings.asr.volcengine.note.appToken")
.font(TypeStyle.caption)
.foregroundStyle(palette.textTertiary)
.fixedSize(horizontal: false, vertical: true)
@@ -105,22 +126,29 @@ struct ASRSettingsCard: View {
}
private var volcengineFields: VolcengineASRFields {
VolcengineASRFields.parse(
apiKey: config.asrApiKey,
resourceFallback: config.asrModel.isEmpty
? CloudASRModelCatalog.defaultModel(for: "volcengine")
: config.asrModel
VolcengineASRFields.parse(apiKey: config.asrApiKey)
}
private var volcengineAPIKeyModeBinding: Binding<Bool> {
Binding(
get: { volcengineFields.usesAPIKeyAuth },
set: { updateVolcengine(authMode: $0 ? .apiKey : .appToken) }
)
}
private func updateVolcengine(appID: String? = nil, accessToken: String? = nil, resourceID: String? = nil) {
private func updateVolcengine(
authMode: VolcengineASRAuthMode? = nil,
appID: String? = nil,
accessToken: String? = nil,
apiKeyCredential: String? = nil
) {
var fields = volcengineFields
if let authMode { fields.authMode = authMode }
if let appID { fields.appID = appID }
if let accessToken { fields.accessToken = accessToken }
if let resourceID {
fields.resourceID = resourceID
config.asrModel = resourceID
}
if let apiKeyCredential { fields.apiKeyCredential = apiKeyCredential }
// Keep store model aligned with the locked SAUC 2.0 resource.
config.asrModel = VolcengineASRFields.fixedResourceID
config.asrApiKey = fields.encodedAPIKey
}
+19 -1
View File
@@ -239,7 +239,8 @@ struct PolishStylesView: View {
format: AppL10n.string("polishStyles.copyName"),
pack.displayName(language: config.uiLanguage)
),
prompt: pack.prompt
prompt: pack.prompt,
allowsAddedEmoji: pack.allowsAddedEmoji
)
showEditor = true
}
@@ -310,12 +311,14 @@ private struct PolishStyleEditorSheet: View {
@Environment(\.themePalette) private var palette
@State private var name: String
@State private var prompt: String
@State private var allowsAddedEmoji: Bool
init(pack: PolishStylePack?, onSave: @escaping (PolishStylePack) -> Void) {
self.pack = pack
self.onSave = onSave
_name = State(initialValue: pack?.name ?? "")
_prompt = State(initialValue: pack?.prompt ?? PolishStylePackCatalog.newUserPromptTemplate)
_allowsAddedEmoji = State(initialValue: pack?.allowsAddedEmoji ?? false)
}
var body: some View {
@@ -324,10 +327,23 @@ private struct PolishStyleEditorSheet: View {
Section("polishStyles.editor.name") {
TextField("polishStyles.editor.namePlaceholder", text: $name)
}
Section {
Toggle("polishStyles.editor.allowsAddedEmoji", isOn: $allowsAddedEmoji)
} footer: {
Text("polishStyles.editor.allowsAddedEmoji.hint")
}
Section {
TextEditor(text: $prompt)
.font(.body.monospaced())
.frame(minHeight: 320)
.onChange(of: prompt) { _, newValue in
// Paste-only custom prompts that declare emoji opt-in
// should flip the toggle so post-processing keeps them.
if !allowsAddedEmoji,
PolishStylePack.promptDeclaresAddedEmojiOptIn(newValue) {
allowsAddedEmoji = true
}
}
} header: {
HStack {
Text("polishStyles.editor.prompt")
@@ -355,6 +371,8 @@ private struct PolishStyleEditorSheet: View {
id: pack?.id ?? "user.\(UUID().uuidString.lowercased())",
name: name,
prompt: prompt,
allowsAddedEmoji: allowsAddedEmoji
|| PolishStylePack.promptDeclaresAddedEmojiOptIn(prompt),
kind: .user,
createdAt: pack?.createdAt ?? Date()
)
+7 -2
View File
@@ -137,8 +137,11 @@
"settings.provider.note" = "Note";
"settings.asr.volcengine.appId" = "APP ID";
"settings.asr.volcengine.accessToken" = "Access Token";
"settings.asr.volcengine.resourceId" = "Resource ID";
"settings.asr.volcengine.note" = "Secret Key is not required. Resource ID defaults to volc.seedasr.sauc.duration.";
"settings.asr.volcengine.apiKey" = "API Key";
"settings.asr.volcengine.apiKeyMode.title" = "Use new API Key auth";
"settings.asr.volcengine.apiKeyMode.subtitle" = "Turn on for the new Volcengine console. Keep off if you already use APP ID + Access Token.";
"settings.asr.volcengine.note.appToken" = "Legacy console: enter APP ID and Access Token. Secret Key is not required. Resource is fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration).";
"settings.asr.volcengine.note.apiKey" = "New console: enter only the API Key. Resource is fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration).";
"provider.openai" = "OpenAI";
"provider.deepseek" = "DeepSeek";
"provider.qwen" = "Qwen (DashScope)";
@@ -404,6 +407,8 @@
"polishStyles.copyName" = "%@ Copy";
"polishStyles.editor.name" = "Name";
"polishStyles.editor.namePlaceholder" = "Style name";
"polishStyles.editor.allowsAddedEmoji" = "Allow mood emojis";
"polishStyles.editor.allowsAddedEmoji.hint" = "When on, polish may add a few emojis that match the drafts emotion, and keeps them on screen. Off by default.";
"polishStyles.editor.prompt" = "Complete prompt";
"polishStyles.editor.hint" = "Describe only the role, tone, style boundaries, and examples. The system appends the personal dictionary, safety contract, and output rules.";
"polishStyles.error.title" = "Couldnt save style";
@@ -137,8 +137,11 @@
"settings.provider.note" = "提示";
"settings.asr.volcengine.appId" = "APP ID";
"settings.asr.volcengine.accessToken" = "Access Token";
"settings.asr.volcengine.resourceId" = "Resource ID";
"settings.asr.volcengine.note" = "Secret Key 当前无需填写。Resource ID 默认使用 volc.seedasr.sauc.duration。";
"settings.asr.volcengine.apiKey" = "API Key";
"settings.asr.volcengine.apiKeyMode.title" = "使用新版 API Key 鉴权";
"settings.asr.volcengine.apiKeyMode.subtitle" = "新控制台请打开此开关;已有 AppID + Token 可保持关闭。";
"settings.asr.volcengine.note.appToken" = "旧版控制台:填写 APP ID 与 Access Token。Secret Key 无需填写。识别资源固定为豆包流式 2.0volc.seedasr.sauc.duration)。";
"settings.asr.volcengine.note.apiKey" = "新版控制台:只需填写 API Key。识别资源固定为豆包流式 2.0volc.seedasr.sauc.duration)。";
"provider.openai" = "OpenAI";
"provider.deepseek" = "DeepSeek";
"provider.qwen" = "通义千问";
@@ -403,6 +406,8 @@
"polishStyles.copyName" = "%@副本";
"polishStyles.editor.name" = "名称";
"polishStyles.editor.namePlaceholder" = "风格名称";
"polishStyles.editor.allowsAddedEmoji" = "允许按情绪添加 emoji";
"polishStyles.editor.allowsAddedEmoji.hint" = "开启后,润色可按原文情绪点缀少量 emoji,并保留上屏。默认关闭。";
"polishStyles.editor.prompt" = "完整提示词";
"polishStyles.editor.hint" = "只需描述角色、语气、风格边界和示例。系统会自动追加个人词典、安全边界和输出契约。";
"polishStyles.error.title" = "无法保存风格";
@@ -165,6 +165,9 @@ public final class KeyboardViewController: UIInputViewController {
configSync.refreshConfigFromAppGroup()
// Settings may have changed while the extension stayed alive.
applyPreferredSurfaceOnOpen()
// Re-warm Taptic after host app switches: SwiftUI `onAppear` often
// skips when the extension process is reused, leaving generators cold.
KeyboardHapticFeedback.prepare()
if state.surface == .typing {
OSGDiag.log("KVC.viewWillAppear enterTypingMode", category: "boot")
typingSession.enterTypingMode()
@@ -41,6 +41,9 @@ final class KeyboardFlowCoordinator {
/// stale sample never flashes the mic orange while the session is healthy.
private var lastHostReadyAt: TimeInterval = 0
private static let hostReadyGrace: TimeInterval = 4
/// Once the host has published ready for this session, hold green through
/// brief inter-utterance ready flaps instead of flashing preparingSession.
private var sessionProvenReady = false
private var flowSessionMonitorTask: Task<Void, Never>?
private var isAwaitingFlowResult = false
private var activeSessionId: UUID?
@@ -168,9 +171,17 @@ final class KeyboardFlowCoordinator {
// show red/white instead of a fake orange "starting" state.
adoptHostBusyStateIfNeeded(snapshot: readySnapshot)
let hostReady = readySnapshot?.ready == true && FlowSessionBridge.isHostReady()
let hostReadyRaw = readySnapshot?.ready == true && FlowSessionBridge.isHostReady()
let sessionActive = FlowSessionBridge.isSessionActive()
let now = Date().timeIntervalSince1970
if hostReady { lastHostReadyAt = now }
if hostReadyRaw {
lastHostReadyAt = now
sessionProvenReady = true
}
if !sessionActive {
sessionProvenReady = false
lastHostReadyAt = 0
}
// Grace window: the host was ready very recently, so treat a momentary
// stale heartbeat read as "still warming" rather than an outright
// failure. `isSessionActive` is heartbeat-independent, so it stays true
@@ -181,19 +192,30 @@ final class KeyboardFlowCoordinator {
// it as preparingSession was the orange-stuck bug after cold start:
// host utt.rec=1 ready=false keyboard forever "".
let hostBusy = FlowKeyboardHostWarming.isHostBusy(reason: readySnapshot?.reason)
// Hold green after the session already proved ready PiP mic release /
// ack lag must not flash yellow».
let holdReady = FlowKeyboardHostWarming.shouldHoldReady(
hostReady: hostReadyRaw,
hostBusy: hostBusy,
sessionActive: sessionActive,
sessionProvenReady: sessionProvenReady,
isPendingFlowStart: isPendingFlowStart,
snapshotReason: readySnapshot?.reason
)
let hostReady = hostReadyRaw || holdReady
// PiP sessions publish `reason=.starting` while the small window is
// coming up treat that as warming so the mic stays orange (wait)
// instead of jumping into another cold start.
let hostWarming = FlowKeyboardHostWarming.isHostWarming(
hostReady: hostReady,
hostBusy: hostBusy,
sessionActive: FlowSessionBridge.isSessionActive(),
sessionActive: sessionActive,
hostReachable: FlowSessionBridge.isHostReachable(),
isPendingFlowStart: isPendingFlowStart,
withinReadyGrace: withinReadyGrace,
snapshotReason: readySnapshot?.reason
)
state.flowSessionActive = FlowSessionBridge.isSessionActive()
state.flowSessionActive = sessionActive
state.debugPendingFlowStart = isPendingFlowStart
state.debugFlowRecording = isFlowRecording
state.debugAwaitingFlowResult = isAwaitingFlowResult
@@ -626,6 +648,7 @@ final class KeyboardFlowCoordinator {
"utterance=\(result.utteranceId.uuidString.prefix(8)) "
+ "commandSeq=\(result.commandSeq) warning=\(result.warning == nil ? 0 : 1)"
)
recomputeMicVoiceAvailability()
return
}
if let result = matchingResult(), isTerminalFailure(result) {
@@ -0,0 +1,270 @@
// TypingKeyTouchPad.swift
// OSGKeyboard · Keyboard Extension
//
// Grid-level UIKit touch tracking for the typing surface:
// Down highlight Move reselect Up commit (letters / space / return),
// delete repeats on down, Shift holds while the gesture owns it.
import SwiftUI
import UIKit
import OSGKeyboardShared
struct TypingKeyTouchPad: UIViewRepresentable {
var layout: TypingKeyLayout
var hapticIntensity: KeyboardHapticIntensity
var onHighlightChange: (String?) -> Void
var onCommit: (TypingKeyHitTarget) -> Void
var onDeleteFire: () -> Void
var onShiftBegan: () -> Void
var onShiftEnded: () -> Void
func makeUIView(context: Context) -> TypingKeyTouchPadUIView {
let view = TypingKeyTouchPadUIView()
view.coordinator = context.coordinator
return view
}
func updateUIView(_ uiView: TypingKeyTouchPadUIView, context: Context) {
context.coordinator.parent = self
uiView.layoutModel = layout
}
func makeCoordinator() -> Coordinator {
Coordinator(parent: self)
}
@MainActor
final class Coordinator {
var parent: TypingKeyTouchPad
private var activeKeyID: String?
private var gestureOwnsShift = false
private var deleteRepeatTask: Task<Void, Never>?
private var deleteRepeatStartedAt: Date?
private var isDeleteRepeating = false
init(parent: TypingKeyTouchPad) {
self.parent = parent
}
func handleBegan(at point: CGPoint) {
resetDeleteRepeat()
gestureOwnsShift = false
guard let key = hit(at: point) else {
setHighlight(nil)
return
}
activate(key)
}
func handleMoved(at point: CGPoint) {
let key = hit(at: point)
if key?.id == activeKeyID { return }
if activeBehavior == .deleteRepeat {
stopDeleteRepeat()
}
if let key {
activate(key)
} else {
// Outside plane: clear highlight; Shift stays held until ended.
setHighlight(nil)
activeKeyID = nil
}
}
func handleEnded(at point: CGPoint) {
// Outside the key plane cancel (no commit), per accuracy plan.
let key = hit(at: point)
stopDeleteRepeat()
defer {
setHighlight(nil)
activeKeyID = nil
finishShiftIfNeeded()
}
guard let key else { return }
switch key.behavior {
case .commitOnRelease:
parent.onCommit(key)
case .deleteRepeat:
// Already fired on down / while held.
break
case .shiftHold:
// endShiftHold decides tap vs hold-with-type.
break
}
}
func handleCancelled() {
stopDeleteRepeat()
setHighlight(nil)
activeKeyID = nil
finishShiftIfNeeded()
}
// MARK: - Internals
private var activeBehavior: TypingKeyTouchBehavior? {
guard let activeKeyID else { return nil }
return parent.layout.key(id: activeKeyID)?.behavior
}
private func hit(at point: CGPoint) -> TypingKeyHitTarget? {
let layout = parent.layout
return KeyHitTesting.hitTarget(
rawTouch: point,
targets: layout.keys,
keyPlaneBounds: layout.keyPlaneBounds,
horizontalGap: layout.horizontalGap,
verticalGap: layout.verticalGap,
hitWeights: layout.hitWeights
)
}
private func activate(_ key: TypingKeyHitTarget) {
activeKeyID = key.id
setHighlight(key.id)
playFeedback(for: key)
switch key.behavior {
case .commitOnRelease:
break
case .deleteRepeat:
parent.onDeleteFire()
startDeleteRepeat()
case .shiftHold:
if !gestureOwnsShift {
gestureOwnsShift = true
parent.onShiftBegan()
}
}
}
private func setHighlight(_ id: String?) {
parent.onHighlightChange(id)
}
private func playFeedback(for key: TypingKeyHitTarget) {
let role: KeyboardHapticKeyRole
let isDelete: Bool
switch key.behavior {
case .deleteRepeat:
role = .delete
isDelete = true
case .shiftHold:
role = .modifier
isDelete = false
case .commitOnRelease:
isDelete = false
if key.id.hasPrefix("bottom.") {
role = key.id == TypingKeyLayoutBuilder.BottomKeyID.pageSwitch.rawValue
? .modifier
: .action
} else if ["123", "#+=", "ABC"].contains(key.label) {
role = .modifier
} else {
role = .character
}
}
TypingKeyFeedback.play(
role: role,
intensity: parent.hapticIntensity,
isDelete: isDelete
)
}
private func startDeleteRepeat() {
stopDeleteRepeat()
isDeleteRepeating = true
deleteRepeatStartedAt = Date()
deleteRepeatTask = Task { @MainActor in
try? await Task.sleep(
nanoseconds: UInt64(RepeatingDeleteTiming.initialDelay * 1_000_000_000)
)
guard !Task.isCancelled, self.isDeleteRepeating else { return }
let anchor = self.deleteRepeatStartedAt ?? Date()
while !Task.isCancelled, self.isDeleteRepeating {
// Match RepeatingPressButton: sound/haptic then action each tick.
TypingKeyFeedback.play(
role: .delete,
intensity: self.parent.hapticIntensity,
isDelete: true
)
self.parent.onDeleteFire()
let elapsed = Date().timeIntervalSince(anchor)
let wait = RepeatingDeleteTiming.interval(for: elapsed)
try? await Task.sleep(nanoseconds: UInt64(wait * 1_000_000_000))
}
}
}
private func stopDeleteRepeat() {
isDeleteRepeating = false
deleteRepeatStartedAt = nil
deleteRepeatTask?.cancel()
deleteRepeatTask = nil
}
private func resetDeleteRepeat() {
stopDeleteRepeat()
}
private func finishShiftIfNeeded() {
guard gestureOwnsShift else { return }
gestureOwnsShift = false
parent.onShiftEnded()
}
}
}
final class TypingKeyTouchPadUIView: UIView {
weak var coordinator: TypingKeyTouchPad.Coordinator?
var layoutModel = TypingKeyLayout(
keys: [],
keyPlaneBounds: .zero,
horizontalGap: 6,
verticalGap: 7,
bottomRowMinY: 0
)
/// Same trick as CursorDragPad: non-zero alpha so SwiftUI hosting does
/// not treat the pad as pass-through.
private static let padTint = UIColor { traits in
UIColor.systemGray4.resolvedColor(with: traits).withAlphaComponent(0.02)
}
override init(frame: CGRect) {
super.init(frame: frame)
backgroundColor = Self.padTint
isMultipleTouchEnabled = false
isExclusiveTouch = true
isUserInteractionEnabled = true
}
@available(*, unavailable)
required init?(coder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
guard let touch = touches.first else { return }
coordinator?.handleBegan(at: touch.location(in: self))
}
override func touchesMoved(_ touches: Set<UITouch>, with event: UIEvent?) {
guard let touch = touches.first else { return }
coordinator?.handleMoved(at: touch.location(in: self))
}
override func touchesEnded(_ touches: Set<UITouch>, with event: UIEvent?) {
guard let touch = touches.first else { return }
coordinator?.handleEnded(at: touch.location(in: self))
}
override func touchesCancelled(_ touches: Set<UITouch>, with event: UIEvent?) {
coordinator?.handleCancelled()
}
}
+168 -213
View File
@@ -42,6 +42,8 @@ struct TypingRootView: View {
/// After the first expand, keep the panel tree mounted and only toggle
/// opacity so subsequent / taps stay cheap.
@State private var candidatePanelMounted = false
/// Key currently under the finger (grid-level touch pad).
@State private var highlightedKeyID: String?
static let totalHeight: CGFloat = TypingLayoutMetrics.totalHeight
@@ -58,14 +60,8 @@ struct TypingRootView: View {
// Keep the QWERTY tree mounted; only toggle visibility so /
// does not destroy GeometryReader key rows every time.
ZStack(alignment: .top) {
VStack(spacing: 0) {
keyGrid
typingKeySurface
.padding(.top, TypingLayoutMetrics.verticalKeySpacing)
bottomRow
.frame(height: TypingLayoutMetrics.bottomRowHeight)
.padding(.top, TypingLayoutMetrics.keyRowSpacing)
}
.opacity(expanded ? 0 : 1)
.allowsHitTesting(!expanded)
.accessibilityHidden(expanded)
@@ -99,6 +95,8 @@ struct TypingRootView: View {
if !hasContent { candidatePanelMounted = false }
}
.onAppear {
// Primary warm-up lives in KVC.viewWillAppear (covers app switch).
// Keep this for first mount / surface flip into typing.
KeyboardHapticFeedback.prepare()
}
}
@@ -277,149 +275,109 @@ struct TypingRootView: View {
// MARK: - Keys
private var keyGrid: some View {
VStack(spacing: TypingLayoutMetrics.keyRowSpacing) {
ForEach(Array(typing.keyRows.enumerated()), id: \.offset) { rowIndex, row in
/// Visual keys + grid-level touch pad (Phase 1 gap fill + Phase 2
/// down move up). Geometry is shared via ``TypingKeyLayoutBuilder``.
private var typingKeySurface: some View {
GeometryReader { proxy in
let inset = rowIndex == 1 ? TypingLayoutMetrics.secondRowInset : 0
let weights = row.enumerated().map {
keyWeight(label: $0.element, index: $0.offset, rowIndex: rowIndex)
}
let spacing = TypingLayoutMetrics.keyHorizontalSpacing
* CGFloat(max(0, row.count - 1))
let availableWidth = proxy.size.width - inset * 2 - spacing
let unitWidth = availableWidth / max(1, weights.reduce(0, +))
let layout = makeTypingKeyLayout(size: proxy.size)
ZStack(alignment: .topLeading) {
TypingKeyTouchPad(
layout: layout,
hapticIntensity: state.keyboardHapticIntensity,
onHighlightChange: { highlightedKeyID = $0 },
onCommit: { commitTypingKey($0) },
onDeleteFire: { apply(typing.handleKey("")) },
onShiftBegan: { typing.beginShiftHold() },
onShiftEnded: { typing.endShiftHold() }
)
HStack(spacing: TypingLayoutMetrics.keyHorizontalSpacing) {
ForEach(Array(row.enumerated()), id: \.offset) { keyIndex, key in
keyButton(key)
.frame(width: unitWidth * weights[keyIndex])
ForEach(layout.keys) { key in
visualTypingKey(key)
.frame(width: key.visualFrame.width, height: key.visualFrame.height)
.position(
x: key.visualFrame.midX,
y: key.visualFrame.midY
)
// Touches go to the UIKit pad; visuals stay for VoiceOver.
.allowsHitTesting(false)
}
}
.padding(.horizontal, inset)
}
.frame(height: TypingLayoutMetrics.keyRowHeight)
}
}
private func makeTypingKeyLayout(size: CGSize) -> TypingKeyLayout {
let pageLabel = typing.page == .letters ? "123" : "ABC"
let spaceLabel = typing.language == .chinese ? "空格" : "space"
let returnLabel: String = {
switch state.returnKeyRole {
case .newline: return "return"
case .send: return ExtL10n.string(state.returnKeyRole.titleKey)
}
}()
let layout = TypingKeyLayoutBuilder.build(
size: size,
letterRows: typing.keyRows,
pageSwitchLabel: pageLabel,
spaceLabel: spaceLabel,
returnLabel: returnLabel,
metrics: TypingKeyLayoutBuilder.Metrics(
keyRowHeight: TypingLayoutMetrics.keyRowHeight,
keyRowSpacing: TypingLayoutMetrics.keyRowSpacing,
keyHorizontalSpacing: TypingLayoutMetrics.keyHorizontalSpacing,
secondRowInset: TypingLayoutMetrics.secondRowInset,
bottomRowHeight: TypingLayoutMetrics.bottomRowHeight,
bottomActionSpacing: TypingLayoutMetrics.bottomActionSpacing,
gridToBottomSpacing: TypingLayoutMetrics.keyRowSpacing
),
keyWeight: { label, index, rowIndex in
keyWeight(label: label, index: index, rowIndex: rowIndex)
}
)
let validNext = PinyinNextKeyResolver.validNextKeys(
rawInput: typing.composition.rawInput,
schema: typing.schema,
language: typing.language,
page: typing.page
)
return layout.withHitWeights(
PinyinNextKeyResolver.hitWeights(for: layout.keys, validNext: validNext)
)
}
@ViewBuilder
private func keyButton(_ label: String) -> some View {
if label == "" {
typingDeleteKey()
} else if label == "" {
typingShiftKey()
} else {
let isSpecial = ["123", "#+=", "ABC"].contains(label)
let role: KeyboardHapticKeyRole = isSpecial ? .modifier : .character
PressDownKeyButton {
TypingKeyFeedback.play(
role: role,
intensity: state.keyboardHapticIntensity
)
apply(typing.handleKey(label))
} label: { isPressed in
NativeKeyboardKeySurface(
isPressed: isPressed,
fill: keyFill,
pressedFill: keyPressedFill,
border: palette.divider,
cornerRadius: TypingLayoutMetrics.keyCornerRadius
) {
keyLabel(label, isSpecial: isSpecial)
}
}
.accessibilityLabel(Text(label))
}
}
private func visualTypingKey(_ key: TypingKeyHitTarget) -> some View {
let pressed = highlightedKeyID == key.id
let isBottom = key.id.hasPrefix("bottom.")
let isShift = key.label == ""
let shiftLit = isShift && typing.isShiftEnabled
let showPressed = pressed || shiftLit
/// Hold = continuous uppercase while pressed; tap = one-shot / Caps Lock cycle.
private func typingShiftKey() -> some View {
let lit = typing.isShiftEnabled
return keyLabel("", isSpecial: true)
.frame(maxWidth: .infinity, maxHeight: .infinity)
.background(
RoundedRectangle(
cornerRadius: TypingLayoutMetrics.keyCornerRadius,
style: .continuous
)
.fill(lit ? keyPressedFill : keyFill)
)
.overlay(
RoundedRectangle(
cornerRadius: TypingLayoutMetrics.keyCornerRadius,
style: .continuous
)
.stroke(palette.divider, lineWidth: 0.5)
)
.shadow(
color: Color.black.opacity(lit ? 0.04 : 0.13),
radius: lit ? 0.5 : 1,
y: lit ? 0 : 1
)
.contentShape(Rectangle())
.gesture(
DragGesture(minimumDistance: 0)
.onChanged { _ in
if !typing.shiftHeld {
TypingKeyFeedback.play(
role: .modifier,
intensity: state.keyboardHapticIntensity
)
typing.beginShiftHold()
}
}
.onEnded { _ in
typing.endShiftHold()
}
)
.accessibilityLabel(Text("shift"))
.accessibilityAddTraits(.isButton)
}
/// Shares ``RepeatingPressButton`` with the voice toolbar delete key.
private func typingDeleteKey() -> some View {
RepeatingPressButton(hapticIntensity: state.keyboardHapticIntensity) {
apply(typing.handleKey(""))
} label: { isPressed in
Group {
if key.label == "" {
Image(systemName: "delete.left")
.font(.system(size: 20, weight: .medium))
.foregroundStyle(keyTextColor)
.frame(maxWidth: .infinity, maxHeight: .infinity)
.background(
RoundedRectangle(
cornerRadius: TypingLayoutMetrics.keyCornerRadius,
style: .continuous
)
.fill(isPressed ? keyPressedFill : keyFill)
)
.overlay(
RoundedRectangle(
cornerRadius: TypingLayoutMetrics.keyCornerRadius,
style: .continuous
)
.stroke(palette.divider, lineWidth: 0.5)
)
.shadow(
color: Color.black.opacity(isPressed ? 0.04 : 0.13),
radius: isPressed ? 0.5 : 1,
y: isPressed ? 0 : 1
)
.scaleEffect(isPressed ? 0.98 : 1)
.animation(.easeOut(duration: 0.08), value: isPressed)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
@ViewBuilder
private func keyLabel(_ label: String, isSpecial: Bool) -> some View {
switch label {
case "":
} else if key.label == "" {
Image(systemName: typing.isShiftEnabled ? "shift.fill" : "shift")
.font(.system(size: 19, weight: .medium))
.foregroundStyle(keyTextColor)
default:
Text(label)
} else if key.id == TypingKeyLayoutBuilder.BottomKeyID.return.rawValue {
returnKeyLabel
.foregroundStyle(returnKeyTextColor)
} else if isBottom {
let isSpecial = key.id == TypingKeyLayoutBuilder.BottomKeyID.pageSwitch.rawValue
Text(key.label)
.font(
.system(
size: isSpecial ? 15 : 17,
weight: isSpecial ? .semibold : .regular
)
)
.foregroundStyle(keyTextColor)
} else {
let isSpecial = ["123", "#+=", "ABC"].contains(key.label)
Text(key.label)
.font(
.system(
size: isSpecial ? 15 : 22,
@@ -429,90 +387,83 @@ struct TypingRootView: View {
.foregroundStyle(keyTextColor)
}
}
private var bottomRow: some View {
GeometryReader { proxy in
let widths = KeyboardChromeLayout.actionKeyWidths(
availableWidth: proxy.size.width
.frame(maxWidth: .infinity, maxHeight: .infinity)
.background(
RoundedRectangle(
cornerRadius: TypingLayoutMetrics.keyCornerRadius,
style: .continuous
)
HStack(spacing: TypingLayoutMetrics.bottomActionSpacing) {
PressDownKeyButton {
TypingKeyFeedback.play(
role: .modifier,
intensity: state.keyboardHapticIntensity
.fill(visualKeyFill(for: key, pressed: showPressed))
)
.overlay(
RoundedRectangle(
cornerRadius: TypingLayoutMetrics.keyCornerRadius,
style: .continuous
)
.stroke(visualKeyBorder(for: key), lineWidth: 0.5)
)
.shadow(
color: Color.black.opacity(showPressed ? 0.04 : 0.13),
radius: showPressed ? 0.5 : 1,
y: showPressed ? 0 : 1
)
.scaleEffect(pressed ? 0.98 : 1)
.animation(.easeOut(duration: 0.08), value: pressed)
.accessibilityElement()
.accessibilityAddTraits(.isButton)
.accessibilityLabel(Text(accessibilityLabel(for: key)))
.accessibilityAction {
commitTypingKey(key)
}
}
private func visualKeyFill(for key: TypingKeyHitTarget, pressed: Bool) -> Color {
if key.id == TypingKeyLayoutBuilder.BottomKeyID.return.rawValue {
return pressed ? returnKeyPressedFill : returnKeyFill
}
return pressed ? keyPressedFill : keyFill
}
private func visualKeyBorder(for key: TypingKeyHitTarget) -> Color {
if key.id == TypingKeyLayoutBuilder.BottomKeyID.return.rawValue {
return returnKeyBorder
}
return palette.divider
}
private func accessibilityLabel(for key: TypingKeyHitTarget) -> String {
switch key.id {
case TypingKeyLayoutBuilder.BottomKeyID.return.rawValue:
return returnKeyAccessibilityLabel
case TypingKeyLayoutBuilder.BottomKeyID.space.rawValue:
return key.label
default:
if key.label == "" { return "shift" }
if key.label == "" { return "delete" }
return key.label
}
}
private func commitTypingKey(_ key: TypingKeyHitTarget) {
switch key.id {
case TypingKeyLayoutBuilder.BottomKeyID.pageSwitch.rawValue:
typing.setPage(typing.page == .letters ? .numbers : .letters)
} label: { isPressed in
NativeKeyboardKeySurface(
isPressed: isPressed,
fill: keyFill,
pressedFill: keyPressedFill,
border: palette.divider,
cornerRadius: TypingLayoutMetrics.keyCornerRadius
) {
Text(typing.page == .letters ? "123" : "ABC")
.font(.system(size: 15, weight: .semibold))
.foregroundStyle(keyTextColor)
}
.frame(
width: widths.side,
height: TypingLayoutMetrics.bottomRowHeight
)
}
.accessibilityLabel(Text(typing.page == .letters ? "123" : "ABC"))
PressDownKeyButton {
TypingKeyFeedback.play(
role: .action,
intensity: state.keyboardHapticIntensity
)
case TypingKeyLayoutBuilder.BottomKeyID.space.rawValue:
apply(typing.handleSpace())
} label: { isPressed in
NativeKeyboardKeySurface(
isPressed: isPressed,
fill: keyFill,
pressedFill: keyPressedFill,
border: palette.divider,
cornerRadius: TypingLayoutMetrics.keyCornerRadius
) {
Text(typing.language == .chinese ? "空格" : "space")
.font(.system(size: 17, weight: .regular))
.foregroundStyle(keyTextColor)
}
.frame(
width: widths.center,
height: TypingLayoutMetrics.bottomRowHeight
)
}
.accessibilityLabel(Text(typing.language == .chinese ? "空格" : "space"))
PressDownKeyButton {
TypingKeyFeedback.play(
role: .action,
intensity: state.keyboardHapticIntensity
)
case TypingKeyLayoutBuilder.BottomKeyID.return.rawValue:
apply(typing.handleReturn())
} label: { isPressed in
NativeKeyboardKeySurface(
isPressed: isPressed,
fill: returnKeyFill,
pressedFill: returnKeyPressedFill,
border: returnKeyBorder,
cornerRadius: TypingLayoutMetrics.keyCornerRadius
) {
returnKeyLabel
.foregroundStyle(returnKeyTextColor)
}
.frame(
width: widths.side,
height: TypingLayoutMetrics.bottomRowHeight
)
}
.accessibilityLabel(Text(returnKeyAccessibilityLabel))
default:
switch key.behavior {
case .commitOnRelease:
apply(typing.handleKey(key.label))
case .deleteRepeat:
apply(typing.handleKey(""))
case .shiftHold:
// Mirror a completed Shift tap for VoiceOver / accessibility.
typing.beginShiftHold()
typing.endShiftHold()
}
}
.frame(height: TypingLayoutMetrics.bottomRowHeight)
}
@ViewBuilder
@@ -540,8 +491,12 @@ struct TypingRootView: View {
if !output.text.isEmpty {
onInsert(output.text)
}
// Proxy context is up to date after inserts/deletes refresh Shift.
typing.syncAutocapitalization()
// Notes / some hosts lag `documentContextBeforeInput` pass the edit we
// just applied so sentence Shift can arm after Return / "." immediately.
typing.syncAutocapitalization(
accountingForInsert: output.text,
deleteCount: output.deleteCount
)
}
private func keyWeight(label: String, index: Int, rowIndex: Int) -> CGFloat {
+12 -3
View File
@@ -234,14 +234,22 @@ public struct KeyboardRootView: View {
}
private func bottomDeleteButton(disabled: Bool) -> some View {
RepeatingDeleteButton(disabled: disabled) {
RepeatingDeleteButton(
disabled: disabled,
hapticIntensity: state.keyboardHapticIntensity
) {
state.deleteBackward()
}
.frame(height: KeyboardLayoutMetrics.bottomActionRowHeight)
}
private func bottomSpaceButton(disabled: Bool) -> some View {
RectangularToolbarButton(spaceStyle: true, label: "space", disabled: disabled) {
RectangularToolbarButton(
spaceStyle: true,
label: "space",
disabled: disabled,
hapticIntensity: state.keyboardHapticIntensity
) {
state.insertSpace()
}
.frame(height: KeyboardLayoutMetrics.bottomActionRowHeight)
@@ -253,7 +261,8 @@ public struct KeyboardRootView: View {
title: title,
label: title,
disabled: disabled,
isSend: state.returnKeyRole == .send
isSend: state.returnKeyRole == .send,
hapticIntensity: state.keyboardHapticIntensity
) {
state.insertNewline()
}
@@ -154,7 +154,10 @@ struct KeyboardTopControls: View {
return
}
onInsert(output.text)
typing.syncAutocapitalization()
typing.syncAutocapitalization(
accountingForInsert: output.text,
deleteCount: output.deleteCount
)
}
private func accessibilityLabel(for tab: KeyboardInputTab) -> String {
@@ -94,7 +94,7 @@ struct RepeatingPressButton<Label: View>: View {
var disabled: Bool = false
/// Plays the system delete click on each fire (matches stock keyboard).
var playsDeleteSound: Bool = true
/// Typing-grid haptic strength; `.off` skips haptics (voice toolbar default).
/// Settings General Haptics; `.off` skips impact feedback.
var hapticIntensity: KeyboardHapticIntensity = .off
let action: () -> Void
@ViewBuilder let label: (_ isPressed: Bool) -> Label
@@ -166,10 +166,16 @@ struct RepeatingDeleteButton: View {
@Environment(\.colorScheme) private var colorScheme
let disabled: Bool
/// Mirrors Settings General Haptics (same as typing delete).
var hapticIntensity: KeyboardHapticIntensity = .off
let action: () -> Void
var body: some View {
RepeatingPressButton(disabled: disabled, action: action) { isPressed in
RepeatingPressButton(
disabled: disabled,
hapticIntensity: hapticIntensity,
action: action
) { isPressed in
ToolbarKeySurface(
isPressed: isPressed,
cornerRadius: ToolbarButtonMetrics.cornerRadius,
@@ -227,15 +233,24 @@ struct RectangularToolbarButton: View {
let label: String
let disabled: Bool
let isSend: Bool
/// Settings General Haptics; space / return use `.action` role.
var hapticIntensity: KeyboardHapticIntensity = .off
let action: () -> Void
init(systemName: String, label: String, disabled: Bool = false, action: @escaping () -> Void) {
init(
systemName: String,
label: String,
disabled: Bool = false,
hapticIntensity: KeyboardHapticIntensity = .off,
action: @escaping () -> Void
) {
self.systemName = systemName
self.spaceStyle = false
self.title = nil
self.label = label
self.disabled = disabled
self.isSend = false
self.hapticIntensity = hapticIntensity
self.action = action
}
@@ -244,6 +259,7 @@ struct RectangularToolbarButton: View {
label: String,
disabled: Bool = false,
isSend: Bool = false,
hapticIntensity: KeyboardHapticIntensity = .off,
action: @escaping () -> Void
) {
self.systemName = nil
@@ -251,17 +267,25 @@ struct RectangularToolbarButton: View {
self.label = label
self.disabled = disabled
self.isSend = isSend
self.hapticIntensity = hapticIntensity
self.action = action
self.title = title
}
init(spaceStyle: Bool, label: String, disabled: Bool = false, action: @escaping () -> Void) {
init(
spaceStyle: Bool,
label: String,
disabled: Bool = false,
hapticIntensity: KeyboardHapticIntensity = .off,
action: @escaping () -> Void
) {
self.systemName = nil
self.spaceStyle = spaceStyle
self.title = nil
self.label = label
self.disabled = disabled
self.isSend = false
self.hapticIntensity = hapticIntensity
self.action = action
}
@@ -299,13 +323,14 @@ struct RectangularToolbarButton: View {
isSend ? .white : NativeKeyboardKeyColors.text(for: colorScheme)
}
// Button
// / Button
private var pressGesture: some Gesture {
DragGesture(minimumDistance: 0)
.onChanged { _ in
guard !disabled, !isPressing else { return }
isPressing = true
KeyboardSoundFeedback.keyClick()
KeyboardHapticFeedback.play(role: .action, intensity: hapticIntensity)
action()
}
.onEnded { _ in
@@ -61,6 +61,15 @@ final class EnglishTypingTests: XCTestCase {
XCTAssertTrue(
TypingAutocapitalization.shouldCapitalize(precedingText: "Hello!\n", mode: .sentences)
)
// System keyboard (.sentences): Return starts a new line capitalize.
// Notes-style document context after Return is typically "\n".
XCTAssertTrue(
TypingAutocapitalization.shouldCapitalize(precedingText: "Hello\n", mode: .sentences),
"Return/newline must arm Shift (Notes multi-line)"
)
XCTAssertTrue(
TypingAutocapitalization.shouldCapitalize(precedingText: "Hello\n\n", mode: .sentences)
)
XCTAssertFalse(
TypingAutocapitalization.shouldCapitalize(precedingText: "Hello ", mode: .sentences)
)
@@ -72,6 +81,55 @@ final class EnglishTypingTests: XCTestCase {
)
}
@MainActor
func testEnglishShiftArmsAfterReturnLikeNotes() {
// Simulates Notes: proxy preceding text gains a trailing newline after Return.
var preceding = "Hello"
let typing = TypingSessionController()
typing.precedingTextProvider = { preceding }
typing.autocapitalizationModeProvider = { .sentences }
_ = typing.setLanguage(.english)
XCTAssertFalse(typing.shiftActive, "mid-word should not arm Shift")
_ = typing.handleReturn()
preceding = "Hello\n" // host document after inserting newline
typing.syncAutocapitalization(accountingForInsert: "\n")
XCTAssertTrue(
typing.shiftActive,
"after Return, Shift must light for next line (system sentences behavior)"
)
XCTAssertEqual(typing.keyRows.first?.first, "Q")
}
@MainActor
func testEnglishShiftArmsWhenProxyLagsAfterReturn() {
// Notes often still reports pre-Return context right after insertText("\n").
var preceding = "Hello"
let typing = TypingSessionController()
typing.precedingTextProvider = { preceding }
typing.autocapitalizationModeProvider = { .sentences }
_ = typing.setLanguage(.english)
_ = typing.handleReturn()
// Proxy intentionally stale still "Hello" without "\n".
typing.syncAutocapitalization(accountingForInsert: "\n")
XCTAssertTrue(typing.shiftActive)
XCTAssertEqual(typing.keyRows.first?.first, "Q")
}
@MainActor
func testEnglishShiftArmsWhenProxyLagsAfterPeriod() {
var preceding = "Hello"
let typing = TypingSessionController()
typing.precedingTextProvider = { preceding }
typing.autocapitalizationModeProvider = { .sentences }
_ = typing.setLanguage(.english)
_ = typing.handleKey(".")
typing.syncAutocapitalization(accountingForInsert: ".")
XCTAssertTrue(typing.shiftActive)
}
@MainActor
func testEnglishIdleShowsNoCandidatesUntilLetterTyped() {
let typing = TypingSessionController()
@@ -0,0 +1,184 @@
// KeyHitTestingTests.swift
// OSGKeyboard · Ext unit tests
//
// Phase 1 / 3: gap fill, nearest-center, intent offset, edge expansion.
import XCTest
@testable import OSGKeyboardShared
final class KeyHitTestingTests: XCTestCase {
private func makePairTargets(
left: CGRect = CGRect(x: 0, y: 0, width: 40, height: 50),
gap: CGFloat = 6
) -> (targets: [TypingKeyHitTarget], plane: CGRect) {
let right = CGRect(
x: left.maxX + gap,
y: left.minY,
width: left.width,
height: left.height
)
let targets = [
TypingKeyHitTarget(
id: "L",
label: "A",
visualFrame: left,
behavior: .commitOnRelease
),
TypingKeyHitTarget(
id: "R",
label: "S",
visualFrame: right,
behavior: .commitOnRelease
)
]
let plane = left.union(right)
return (targets, plane)
}
func testGapMidpointHitsNearestKey() {
let pair = makePairTargets()
let mid = CGPoint(x: 43, y: 25) // center of 6pt gap between 40 and 46
let hit = KeyHitTesting.hitTarget(
at: mid,
targets: pair.targets,
keyPlaneBounds: pair.plane,
horizontalGap: 6,
verticalGap: 7,
edgeExpansion: 0
)
// Midpoint is equidistant; either is acceptable, but must not miss.
XCTAssertNotNil(hit)
}
func testGapCloserToLeftSelectsLeft() {
let pair = makePairTargets()
// Just right of left key visual edge, still in gap, closer to left center.
let point = CGPoint(x: 41, y: 25)
let hit = KeyHitTesting.hitTarget(
at: point,
targets: pair.targets,
keyPlaneBounds: pair.plane,
horizontalGap: 6,
verticalGap: 7,
edgeExpansion: 0
)
XCTAssertEqual(hit?.id, "L")
}
func testGapCloserToRightSelectsRight() {
let pair = makePairTargets()
let point = CGPoint(x: 45, y: 25)
let hit = KeyHitTesting.hitTarget(
at: point,
targets: pair.targets,
keyPlaneBounds: pair.plane,
horizontalGap: 6,
verticalGap: 7,
edgeExpansion: 0
)
XCTAssertEqual(hit?.id, "R")
}
func testOutsidePlaneReturnsNil() {
let pair = makePairTargets()
let hit = KeyHitTesting.hitTarget(
at: CGPoint(x: 200, y: 200),
targets: pair.targets,
keyPlaneBounds: pair.plane,
horizontalGap: 6,
verticalGap: 7,
edgeExpansion: 0
)
XCTAssertNil(hit)
}
func testIntentOffsetShiftsHitUpward() {
// Key occupies y 1060. Raw touch at y=62 is below the key; with a
// 4pt upward intent offset it maps to y=58 and should still hit.
let target = TypingKeyHitTarget(
id: "K",
label: "M",
visualFrame: CGRect(x: 0, y: 10, width: 40, height: 50),
behavior: .commitOnRelease
)
let plane = target.visualFrame
let raw = CGPoint(x: 20, y: 62)
let withoutOffset = KeyHitTesting.hitTarget(
at: raw,
targets: [target],
keyPlaneBounds: plane,
horizontalGap: 0,
verticalGap: 0,
edgeExpansion: 0
)
let withOffset = KeyHitTesting.hitTarget(
rawTouch: raw,
targets: [target],
keyPlaneBounds: plane,
horizontalGap: 0,
verticalGap: 0,
intentOffsetY: 4,
edgeExpansion: 0
)
XCTAssertNil(withoutOffset)
XCTAssertEqual(withOffset?.id, "K")
}
func testEdgeExpansionExtendsOuterHit() {
let target = TypingKeyHitTarget(
id: "Q",
label: "Q",
visualFrame: CGRect(x: 10, y: 0, width: 40, height: 50),
behavior: .commitOnRelease
)
let plane = target.visualFrame
// Point just left of the visual key, inside edge expansion.
let point = CGPoint(x: 7, y: 25)
let hit = KeyHitTesting.hitTarget(
at: point,
targets: [target],
keyPlaneBounds: plane,
horizontalGap: 0,
verticalGap: 0,
edgeExpansion: 5
)
XCTAssertEqual(hit?.id, "Q")
}
func testLayoutBuilderCoversGapsBetweenKeys() {
let layout = TypingKeyLayoutBuilder.build(
size: CGSize(width: 300, height: 200),
letterRows: [
["Q", "W", "E"],
["A", "S", "D"],
["Z", "X", "C"]
],
pageSwitchLabel: "123",
spaceLabel: "space",
returnLabel: "return",
keyWeight: { _, _, _ in 1 }
)
XCTAssertEqual(layout.keys.count, 12) // 9 letters + 3 bottom
// Mid-gap between Q and W on first row should hit something.
let q = layout.keys.first { $0.label == "Q" }!
let w = layout.keys.first { $0.label == "W" }!
let mid = CGPoint(x: (q.visualFrame.maxX + w.visualFrame.minX) / 2, y: q.center.y)
let hit = KeyHitTesting.hitTarget(
at: mid,
targets: layout.keys,
keyPlaneBounds: layout.keyPlaneBounds,
horizontalGap: layout.horizontalGap,
verticalGap: layout.verticalGap
)
XCTAssertNotNil(hit)
XCTAssertTrue(hit?.label == "Q" || hit?.label == "W")
}
func testBehaviorResolver() {
XCTAssertEqual(TypingKeyBehaviorResolver.behavior(for: ""), .deleteRepeat)
XCTAssertEqual(TypingKeyBehaviorResolver.behavior(for: ""), .shiftHold)
XCTAssertEqual(TypingKeyBehaviorResolver.behavior(for: "A"), .commitOnRelease)
XCTAssertEqual(TypingKeyBehaviorResolver.behavior(for: "123"), .commitOnRelease)
}
}
@@ -154,4 +154,120 @@ final class KeyboardSurfaceStateTests: XCTestCase {
XCTAssertFalse(typing.shiftHeld)
XCTAssertFalse(typing.capsLock)
}
func testChineseShiftInsertsUppercaseLatinBypassingRime() {
let engine = TrackingStubRimeEngine()
let typing = TypingSessionController(engine: { engine })
_ = typing.handleKey("")
XCTAssertTrue(typing.shiftActive)
let output = typing.handleKey("N")
XCTAssertEqual(output, .insert("N"))
XCTAssertEqual(engine.processCharacterCallCount, 0)
XCTAssertFalse(typing.shiftActive, "one-shot Shift clears after Latin insert")
XCTAssertTrue(typing.composition.preedit.isEmpty)
}
func testChineseShiftPreservesExistingComposition() {
let engine = TrackingStubRimeEngine()
let typing = TypingSessionController(engine: { engine })
// Seed session composition via a lowercase letter (goes to Rime).
_ = typing.handleKey("n")
XCTAssertEqual(typing.composition.preedit, "n")
_ = typing.handleKey("")
let output = typing.handleKey("A")
XCTAssertEqual(output, .insert("A"))
XCTAssertEqual(engine.processCharacterCallCount, 1, "only the unshifted letter hits Rime")
XCTAssertEqual(typing.composition.preedit, "n", "Shift Latin must not clear preedit")
}
func testChineseCapsLockKeepsInsertingUppercaseLatin() {
let engine = TrackingStubRimeEngine()
let typing = TypingSessionController(engine: { engine })
_ = typing.handleKey("")
_ = typing.handleKey("") // second tap Caps Lock
XCTAssertTrue(typing.capsLock)
XCTAssertEqual(typing.handleKey("O"), .insert("O"))
XCTAssertEqual(typing.handleKey("S"), .insert("S"))
XCTAssertEqual(engine.processCharacterCallCount, 0)
XCTAssertTrue(typing.capsLock)
}
func testChineseUnshiftedLetterStillComposes() {
let engine = TrackingStubRimeEngine()
let typing = TypingSessionController(engine: { engine })
let output = typing.handleKey("n")
XCTAssertEqual(output, .none)
XCTAssertEqual(engine.processCharacterCallCount, 1)
XCTAssertEqual(engine.lastProcessedCharacter, "n")
XCTAssertEqual(typing.composition.preedit, "n")
}
}
/// Stub that records `processCharacter` calls for Chinese Shift bypass tests.
@MainActor
private final class TrackingStubRimeEngine: RimeEngineBridging {
var composition: TypingComposition = .empty
var isReady: Bool = true
var schema: TypingInputSchema = .fullPinyin
private(set) var processCharacterCallCount = 0
private(set) var lastProcessedCharacter: Character?
func prepare() async throws {}
func teardown() { composition = .empty }
func setLanguage(_ language: TypingInputLanguage) {
if language == .english { composition = .empty }
}
@discardableResult
func setSchema(_ schema: TypingInputSchema) -> Bool {
self.schema = schema
return true
}
func processCharacter(_ character: Character) -> String? {
processCharacterCallCount += 1
lastProcessedCharacter = character
composition = TypingComposition(
preedit: String(character).lowercased(),
candidates: [TypingCandidate(text: "")]
)
return nil
}
func processBackspace() -> String? {
composition = .empty
return nil
}
func processSpace() -> String? {
composition = .empty
return " "
}
func processReturn() -> String? {
composition = .empty
return "\n"
}
func selectCandidate(at index: Int) -> String {
composition = .empty
return ""
}
func flushPreedit() -> String {
let raw = composition.preedit
composition = .empty
return raw
}
func clearComposition() {
composition = .empty
}
}
@@ -0,0 +1,181 @@
// PinyinNextKeyResolverTests.swift
// OSGKeyboard · Ext unit tests
//
// Phase 4: legal next-key sets and weighted ambiguous hit resolution.
import XCTest
@testable import OSGKeyboardShared
final class PinyinNextKeyResolverTests: XCTestCase {
func testZhongPrefixAllowsG() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "zhon",
schema: .fullPinyin,
language: .chinese,
page: .letters
)
XCTAssertEqual(keys, ["g"])
}
func testCompleteZhongAllowsNewSyllableInitials() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "zhong",
schema: .fullPinyin,
language: .chinese,
page: .letters
)
XCTAssertNotNil(keys)
XCTAssertTrue(keys?.contains("g") == true) // zhongguo
XCTAssertTrue(keys?.contains("w") == true)
}
func testMultiSyllableTrailingGAllowsU() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "zhongg",
schema: .fullPinyin,
language: .chinese,
page: .letters
)
XCTAssertTrue(keys?.contains("u") == true) // gu / guo
}
func testNiAllowsExtensionAndNewInitial() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "ni",
schema: .fullPinyin,
language: .chinese,
page: .letters
)
XCTAssertTrue(keys?.contains("a") == true) // nia
XCTAssertTrue(keys?.contains("h") == true) // nihao
}
func testEmptyRawDisablesBias() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "",
schema: .fullPinyin,
language: .chinese,
page: .letters
)
XCTAssertNil(keys)
}
func testEnglishDisablesBias() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "ni",
schema: .fullPinyin,
language: .english,
page: .letters
)
XCTAssertNil(keys)
}
func testDoublePinyinDisablesBias() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "nihk",
schema: .microsoftDoublePinyin,
language: .chinese,
page: .letters
)
XCTAssertNil(keys)
}
func testNumbersPageDisablesBias() {
let keys = PinyinNextKeyResolver.validNextKeys(
rawInput: "ni",
schema: .fullPinyin,
language: .chinese,
page: .numbers
)
XCTAssertNil(keys)
}
func testHitWeightsBoostLegalLetters() {
let keys = [
TypingKeyHitTarget(
id: "grid.0.0",
label: "G",
visualFrame: .zero,
behavior: .commitOnRelease
),
TypingKeyHitTarget(
id: "grid.0.1",
label: "H",
visualFrame: .zero,
behavior: .commitOnRelease
),
TypingKeyHitTarget(
id: "grid.0.2",
label: "",
visualFrame: .zero,
behavior: .deleteRepeat
)
]
let weights = PinyinNextKeyResolver.hitWeights(for: keys, validNext: ["g"])
XCTAssertEqual(weights["grid.0.0"], KeyHitBiasMetrics.legalBoost)
XCTAssertEqual(weights["grid.0.1"], KeyHitBiasMetrics.illegalShrink)
XCTAssertNil(weights["grid.0.2"])
}
func testWeightedNearestPrefersLegalKeyInGap() {
let left = TypingKeyHitTarget(
id: "L",
label: "F",
visualFrame: CGRect(x: 0, y: 0, width: 40, height: 50),
behavior: .commitOnRelease
)
let right = TypingKeyHitTarget(
id: "R",
label: "G",
visualFrame: CGRect(x: 46, y: 0, width: 40, height: 50),
behavior: .commitOnRelease
)
let plane = left.visualFrame.union(right.visualFrame)
// Shared expanded edge (half of 6pt gap) both frames contain x=43.
let point = CGPoint(x: 43, y: 25)
let weighted = KeyHitTesting.hitTarget(
at: point,
targets: [left, right],
keyPlaneBounds: plane,
horizontalGap: 6,
verticalGap: 7,
edgeExpansion: 0,
hitWeights: [
"L": KeyHitBiasMetrics.illegalShrink,
"R": KeyHitBiasMetrics.legalBoost
]
)
XCTAssertEqual(weighted?.id, "R")
}
func testClearSingleHitIgnoresBias() {
// Point clearly inside F must not jump to boosted G.
let left = TypingKeyHitTarget(
id: "L",
label: "F",
visualFrame: CGRect(x: 0, y: 0, width: 40, height: 50),
behavior: .commitOnRelease
)
let right = TypingKeyHitTarget(
id: "R",
label: "G",
visualFrame: CGRect(x: 46, y: 0, width: 40, height: 50),
behavior: .commitOnRelease
)
let plane = left.visualFrame.union(right.visualFrame)
let point = CGPoint(x: 20, y: 25)
let hit = KeyHitTesting.hitTarget(
at: point,
targets: [left, right],
keyPlaneBounds: plane,
horizontalGap: 6,
verticalGap: 7,
edgeExpansion: 0,
hitWeights: [
"L": KeyHitBiasMetrics.illegalShrink,
"R": KeyHitBiasMetrics.legalBoost
]
)
XCTAssertEqual(hit?.id, "L")
}
}
@@ -29,19 +29,15 @@ struct VolcengineCloudASRClient: CloudASRTranscribing, CloudASRStreamingCapable
onPartial: @escaping @Sendable (String) -> Void
) async throws -> any CloudASRStreamingSession {
_ = locale
let credentials = try VolcengineCredentials.parse(
apiKey: apiKey,
fallbackResourceID: resolvedResourceID
)
_ = resourceID // Product is locked to Doubao streaming 2.0 duration.
let credentials = VolcengineASRFields.parse(apiKey: apiKey)
guard credentials.hasUsableCredentials else { throw CloudASRError.noAPIKey }
let url = try resolvedEndpointURL()
let connectID = UUID().uuidString
var request = URLRequest(url: url)
request.timeoutInterval = 8
request.setValue(credentials.appID, forHTTPHeaderField: "X-Api-App-Key")
request.setValue(credentials.accessToken, forHTTPHeaderField: "X-Api-Access-Key")
request.setValue(credentials.resourceID, forHTTPHeaderField: "X-Api-Resource-Id")
request.setValue(connectID, forHTTPHeaderField: "X-Api-Connect-Id")
credentials.applyWebSocketAuthHeaders(to: &request, connectID: connectID)
let task = session.webSocketTask(with: request)
task.resume()
@@ -90,12 +86,6 @@ struct VolcengineCloudASRClient: CloudASRTranscribing, CloudASRStreamingCapable
live.cancel()
}
private var resolvedResourceID: String {
resourceID.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
? CloudASRModelCatalog.volcengineDefaultResourceID
: resourceID.trimmingCharacters(in: .whitespacesAndNewlines)
}
private func resolvedEndpointURL() throws -> URL {
let raw = endpoint.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
? CloudASRModelCatalog.volcengineEndpoint
@@ -424,46 +414,6 @@ private final class VolcengineStreamingSession: CloudASRStreamingSession, @unche
}
}
private struct VolcengineCredentials {
let appID: String
let accessToken: String
let resourceID: String
static func parse(apiKey: String, fallbackResourceID: String) throws -> VolcengineCredentials {
let trimmed = apiKey.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { throw CloudASRError.noAPIKey }
if let data = trimmed.data(using: .utf8),
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] {
let appID = string(json, keys: ["app_id", "appId", "appid"])
let token = string(json, keys: ["access_token", "accessToken", "token"])
let resourceID = string(json, keys: ["resource_id", "resourceId", "resource"])
?? fallbackResourceID
guard let appID, let token, !resourceID.isEmpty else { throw CloudASRError.noAPIKey }
return VolcengineCredentials(appID: appID, accessToken: token, resourceID: resourceID)
}
let separators = CharacterSet(charactersIn: ":\n,")
let parts = trimmed
.components(separatedBy: separators)
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
guard parts.count >= 2 else { throw CloudASRError.noAPIKey }
let resourceID = parts.count >= 3 ? parts[2] : fallbackResourceID
return VolcengineCredentials(appID: parts[0], accessToken: parts[1], resourceID: resourceID)
}
private static func string(_ json: [String: Any], keys: [String]) -> String? {
for key in keys {
if let value = json[key] as? String {
let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty { return trimmed }
}
}
return nil
}
}
enum VolcengineMessageType: UInt8 {
case fullClientRequest = 0b0001
case audioOnlyRequest = 0b0010
@@ -68,10 +68,13 @@ public final class FlowAudioSessionCoordinator: @unchecked Sendable {
public func activateCapture() async throws -> FlowAudioSessionSnapshot {
let activation: CaptureActivation = try await perform {
if self.mode != .capture {
// `.voiceChat` turns on system speech DSP (noise suppression /
// AGC). `.measurement` feeds near-raw PCM and is weak against
// a competing talker in the same room.
try self.session.setCategory(
.playAndRecord,
mode: .measurement,
options: [.defaultToSpeaker, .allowBluetoothHFP, .mixWithOthers]
mode: FlowCaptureVoiceProcessing.captureMode,
options: FlowCaptureVoiceProcessing.captureOptions
)
}
if !self.active {
@@ -95,6 +98,9 @@ public final class FlowAudioSessionCoordinator: @unchecked Sendable {
category: "flow"
)
}
} else {
// Built-in near-talk preference (cardioid when available).
FlowCaptureVoiceProcessing.preferNearTalkBuiltInMic(on: self.session)
}
self.mode = .capture
return CaptureActivation(
@@ -0,0 +1,108 @@
// FlowCaptureVoiceProcessing.swift
// OSGKeyboard · Host Support
//
// Capture-side speech front-end for local ASR: Apple Voice Processing
// (noise suppression / AGC / AEC) plus near-talk built-in mic preference.
// `.measurement` delivers near-raw PCM and is a poor fit for competing
// talkers in the same room; `.voiceChat` + VP is the system path that
// also unlocks Control Center Mic Modes (incl. Voice Isolation).
import AVFoundation
import Foundation
import OSGKeyboardShared
public enum FlowCaptureVoiceProcessing {
/// Speech-oriented session mode. Prefer over `.measurement` for ASR.
public static let captureMode: AVAudioSession.Mode = .voiceChat
public static let captureOptions: AVAudioSession.CategoryOptions = [
.defaultToSpeaker, .allowBluetoothHFP, .mixWithOthers
]
/// Enable Apple Voice Processing on a stopped engine. Format may change
/// afterward callers must re-read `inputNode` formats before `installTap`.
@discardableResult
public static func enableVoiceProcessing(on engine: AVAudioEngine) -> Bool {
let input = engine.inputNode
if input.isVoiceProcessingEnabled { return true }
do {
try input.setVoiceProcessingEnabled(true)
OSGDiag.log("voiceProcessing enabled", category: "flow")
logActiveMicrophoneMode()
return true
} catch {
OSGDiag.log(
"voiceProcessing enable failed: \(error.localizedDescription)",
category: "flow"
)
return false
}
}
/// Prefer a near-field built-in data source (front/lower + cardioid when
/// available). No-op while Bluetooth HFP is preferred or active keep
/// the existing headset route.
public static func preferNearTalkBuiltInMic(on session: AVAudioSession) {
if session.currentRoute.inputs.first?.portType == .bluetoothHFP { return }
if session.preferredInput?.portType == .bluetoothHFP { return }
guard let builtIn = session.availableInputs?.first(where: {
$0.portType == .builtInMic
}) else { return }
let sources = builtIn.dataSources ?? []
let preferred =
sources.first(where: { $0.orientation == .front })
?? sources.first(where: { $0.location == .lower })
?? sources.first
if let preferred {
if preferred.supportedPolarPatterns?.contains(.cardioid) == true {
do {
try preferred.setPreferredPolarPattern(.cardioid)
} catch {
OSGDiag.log(
"cardioid polar pattern failed: \(error.localizedDescription)",
category: "flow"
)
}
}
do {
try builtIn.setPreferredDataSource(preferred)
} catch {
OSGDiag.log(
"preferred data source failed: \(error.localizedDescription)",
category: "flow"
)
}
}
do {
try session.setPreferredInput(builtIn)
} catch {
OSGDiag.log(
"preferred built-in mic failed: \(error.localizedDescription)",
category: "flow"
)
}
}
public static func logActiveMicrophoneMode() {
let preferred = micModeLabel(AVCaptureDevice.preferredMicrophoneMode)
let active = micModeLabel(AVCaptureDevice.activeMicrophoneMode)
OSGDiag.log(
"micMode preferred=\(preferred) active=\(active)",
category: "flow"
)
}
private static func micModeLabel(_ mode: AVCaptureDevice.MicrophoneMode) -> String {
switch mode {
case .standard: return "standard"
case .wideSpectrum: return "wideSpectrum"
case .voiceIsolation: return "voiceIsolation"
@unknown default: return "unknown(\(mode.rawValue))"
}
}
}
@@ -604,13 +604,25 @@ public final class FlowContinuousCapture {
FlowAudioEngineHandle(audioEngine)
)
// Voice Processing must be enabled while the engine is stopped, and
// it can change the RemoteIO format enable first, then sync rates.
var candidateEngine = AVAudioEngine()
var voiceProcessingOn = FlowCaptureVoiceProcessing.enableVoiceProcessing(
on: candidateEngine
)
var inputNode = candidateEngine.inputNode
var hardwareFormat = inputNode.inputFormat(forBus: 0)
var outputFormat = inputNode.outputFormat(forBus: 0)
for _ in 0..<3 where abs(hardwareFormat.sampleRate - sessionSnapshot.sampleRate) >= 1 {
var resolvedSession = sessionSnapshot
for _ in 0..<3 where abs(hardwareFormat.sampleRate - resolvedSession.sampleRate) >= 1 {
try? await Task.sleep(nanoseconds: 50_000_000)
// Re-enter capture so the session rate tracks VP / route shifts.
resolvedSession = (try? await FlowAudioSessionCoordinator.shared.activateCapture())
?? resolvedSession
candidateEngine = AVAudioEngine()
voiceProcessingOn = FlowCaptureVoiceProcessing.enableVoiceProcessing(
on: candidateEngine
)
inputNode = candidateEngine.inputNode
hardwareFormat = inputNode.inputFormat(forBus: 0)
outputFormat = inputNode.outputFormat(forBus: 0)
@@ -619,8 +631,9 @@ public final class FlowContinuousCapture {
"audioSession.active",
"hwRate=\(Int(hardwareFormat.sampleRate)) hwChannels=\(hardwareFormat.channelCount) "
+ "outputRate=\(Int(outputFormat.sampleRate)) "
+ "sessionRate=\(Int(sessionSnapshot.sampleRate)) "
+ "route=\(sessionSnapshot.inputPortType)"
+ "sessionRate=\(Int(resolvedSession.sampleRate)) "
+ "route=\(resolvedSession.inputPortType) "
+ "voiceProcessing=\(voiceProcessingOn ? 1 : 0)"
)
guard hardwareFormat.sampleRate > 0, hardwareFormat.channelCount > 0 else {
FlowTrace.warn(
@@ -632,8 +645,8 @@ public final class FlowContinuousCapture {
channels: Int(hardwareFormat.channelCount)
)
}
guard sessionSnapshot.sampleRate <= 0
|| abs(hardwareFormat.sampleRate - sessionSnapshot.sampleRate) < 1 else {
guard resolvedSession.sampleRate <= 0
|| abs(hardwareFormat.sampleRate - resolvedSession.sampleRate) < 1 else {
throw StartError.invalidHardwareFormat(
sampleRate: hardwareFormat.sampleRate,
channels: Int(hardwareFormat.channelCount)
@@ -703,7 +716,7 @@ public final class FlowContinuousCapture {
throw StartError.engineStartFailed(error.localizedDescription)
}
audioEngine = candidateEngine
activeRouteSnapshot = sessionSnapshot
activeRouteSnapshot = resolvedSession
engineActivationCount += 1
lastActivationAt = Date()
FlowTrace.capture("engine.started", "running=\(audioEngine.isRunning ? 1 : 0)")
@@ -177,30 +177,27 @@ public final class LiveDictationController: ObservableObject {
// 3. Audio session only configure once per process.
//
// Category is `.record` (not `.playAndRecord`) because the
// preview never plays back audio it just records from the
// mic and hands the buffers to `SpeechAnalyzer`. On the
// iOS Simulator, `.playAndRecord` requires the
// `AURemoteIO` Audio Unit's *output* side to also be
// enabled, but the simulator's "speaker" reports a 0 Hz
// hardware format, so `AURemoteIO::enable` fails with
// `kAudioUnitErr_FormatNotSupported` (-10851) and any
// subsequent `installTap` traps with "Failed to create tap
// due to format mismatch". `.record` skips the output
// side entirely, so the simulator can record.
//
// The real keyboard extension (`OSGKeyboardExt`) keeps
// `.playAndRecord` because it runs on a real device where
// the output side has a real hardware format, and may want
// to play click sounds / haptic feedback. Only the preview
// needs the simulator-friendly category.
// On device we use `.playAndRecord` + `.voiceChat` so Apple Voice
// Processing can suppress competing talkers. The iOS Simulator's
// speaker reports a 0 Hz output format, so `.playAndRecord` fails
// with `kAudioUnitErr_FormatNotSupported` (-10851); keep `.record`
// there so preview still works under CoreSimulator.
if !didConfigureAudioSession {
do {
let session = AVAudioSession.sharedInstance()
try session.setCategory(.record,
mode: .measurement,
options: [])
#if targetEnvironment(simulator)
try session.setCategory(.record, mode: .default, options: [])
#else
try session.setCategory(
.playAndRecord,
mode: FlowCaptureVoiceProcessing.captureMode,
options: FlowCaptureVoiceProcessing.captureOptions
)
#endif
try session.setActive(true, options: .notifyOthersOnDeactivation)
#if !targetEnvironment(simulator)
FlowCaptureVoiceProcessing.preferNearTalkBuiltInMic(on: session)
#endif
didConfigureAudioSession = true
} catch {
debug("audio session failed: \(error.localizedDescription)")
@@ -214,6 +211,8 @@ public final class LiveDictationController: ObservableObject {
// The route may have changed while the preview was closed. A fresh
// engine created after session activation avoids a stale input node.
audioEngine = AVAudioEngine()
// Enable VP before reading hardware format RemoteIO rate can shift.
_ = FlowCaptureVoiceProcessing.enableVoiceProcessing(on: audioEngine)
// 4. Spin up the engine + ASR.
phase = .recording
+21 -3
View File
@@ -146,7 +146,8 @@ struct MacPolishStylesView: View {
duplicate: {
editingPack = PolishStylePack(
name: "\(pack.displayName(language: lang)) \(MacL10n.string("mac.styles.copy", language: lang))",
prompt: pack.prompt
prompt: pack.prompt,
allowsAddedEmoji: pack.allowsAddedEmoji
)
showEditor = true
},
@@ -347,6 +348,7 @@ private struct MacPolishStyleEditor: View {
@Environment(\.themePalette) private var palette
@State private var name: String
@State private var prompt: String
@State private var allowsAddedEmoji: Bool
init(
pack: PolishStylePack?,
@@ -358,6 +360,7 @@ private struct MacPolishStyleEditor: View {
self.onSave = onSave
_name = State(initialValue: pack?.name ?? "")
_prompt = State(initialValue: pack?.prompt ?? PolishStylePackCatalog.newUserPromptTemplate)
_allowsAddedEmoji = State(initialValue: pack?.allowsAddedEmoji ?? false)
}
var body: some View {
@@ -366,6 +369,13 @@ private struct MacPolishStyleEditor: View {
.font(TypeStyle.title2)
TextField(MacL10n.string("mac.styles.name", language: language), text: $name)
.textFieldStyle(.roundedBorder)
Toggle(
MacL10n.string("mac.styles.allowsAddedEmoji", language: language),
isOn: $allowsAddedEmoji
)
Text(MacL10n.string("mac.styles.allowsAddedEmoji.hint", language: language))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
HStack {
Text(MacL10n.string("mac.styles.prompt", language: language))
.font(MacSettingsType.sectionTitle)
@@ -380,13 +390,19 @@ private struct MacPolishStyleEditor: View {
}
TextEditor(text: $prompt)
.font(.body.monospaced())
.frame(minHeight: 360)
.frame(minHeight: 320)
.padding(4)
.background(palette.surface, in: RoundedRectangle(cornerRadius: Radius.medium))
.overlay(
RoundedRectangle(cornerRadius: Radius.medium)
.stroke(palette.divider, lineWidth: 1)
)
.onChange(of: prompt) { _, newValue in
if !allowsAddedEmoji,
PolishStylePack.promptDeclaresAddedEmojiOptIn(newValue) {
allowsAddedEmoji = true
}
}
Text(MacL10n.string("mac.styles.hint", language: language))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
@@ -399,6 +415,8 @@ private struct MacPolishStyleEditor: View {
id: pack?.id ?? "user.\(UUID().uuidString.lowercased())",
name: name,
prompt: prompt,
allowsAddedEmoji: allowsAddedEmoji
|| PolishStylePack.promptDeclaresAddedEmojiOptIn(prompt),
kind: .user,
createdAt: pack?.createdAt ?? Date()
)
@@ -414,7 +432,7 @@ private struct MacPolishStyleEditor: View {
}
}
.padding(Spacing.xl)
.frame(width: 680, height: 590)
.frame(width: 680, height: 640)
.background(palette.background)
}
}
+37 -20
View File
@@ -188,6 +188,25 @@ struct MacSettingsView: View {
@ViewBuilder
private var volcengineAsrRows: some View {
MacProviderThinkingRow(
title: MacL10n.string("mac.settings.volcengineApiKeyMode", language: lang),
subtitle: MacL10n.string("mac.settings.volcengineApiKeyModeSubtitle", language: lang),
isOn: Binding(
get: { macVolcengineFields.usesAPIKeyAuth },
set: { updateMacVolcengine(authMode: $0 ? .apiKey : .appToken) }
)
)
if macVolcengineFields.usesAPIKeyAuth {
MacCredentialField(
title: MacL10n.string("mac.settings.volcengineApiKey", language: lang),
placeholder: "API Key",
text: Binding(
get: { macVolcengineFields.apiKeyCredential },
set: { updateMacVolcengine(apiKeyCredential: $0) }
),
isSecret: true
)
} else {
MacCredentialField(
title: MacL10n.string("mac.settings.volcengineAppId", language: lang),
placeholder: "APP ID",
@@ -206,16 +225,15 @@ struct MacSettingsView: View {
),
isSecret: true
)
MacCredentialField(
title: MacL10n.string("mac.settings.volcengineResourceId", language: lang),
placeholder: CloudASRModelCatalog.defaultModel(for: "volcengine"),
text: Binding(
get: { macVolcengineFields.resourceID },
set: { updateMacVolcengine(resourceID: $0) }
),
defaultValue: CloudASRModelCatalog.defaultModel(for: "volcengine")
}
MacProviderNoteRow(
text: MacL10n.string(
macVolcengineFields.usesAPIKeyAuth
? "mac.settings.volcengineNoteApiKey"
: "mac.settings.volcengineNoteAppToken",
language: lang
)
)
MacProviderNoteRow(text: MacL10n.string("mac.settings.volcengineNote", language: lang))
}
@ViewBuilder
@@ -407,22 +425,21 @@ struct MacSettingsView: View {
}
private var macVolcengineFields: VolcengineASRFields {
VolcengineASRFields.parse(
apiKey: viewModel.config.asrApiKey,
resourceFallback: viewModel.config.asrModel.isEmpty
? CloudASRModelCatalog.defaultModel(for: "volcengine")
: viewModel.config.asrModel
)
VolcengineASRFields.parse(apiKey: viewModel.config.asrApiKey)
}
private func updateMacVolcengine(appID: String? = nil, accessToken: String? = nil, resourceID: String? = nil) {
private func updateMacVolcengine(
authMode: VolcengineASRAuthMode? = nil,
appID: String? = nil,
accessToken: String? = nil,
apiKeyCredential: String? = nil
) {
var fields = macVolcengineFields
if let authMode { fields.authMode = authMode }
if let appID { fields.appID = appID }
if let accessToken { fields.accessToken = accessToken }
if let resourceID {
fields.resourceID = resourceID
viewModel.config.asrModel = resourceID
}
if let apiKeyCredential { fields.apiKeyCredential = apiKeyCredential }
viewModel.config.asrModel = VolcengineASRFields.fixedResourceID
viewModel.config.asrApiKey = fields.encodedAPIKey
}
@@ -140,7 +140,13 @@ public struct AppGroupConfiguration: Sendable, Equatable {
public var isCloudASRKeyMissing: Bool {
guard engineMode == "cloud" else { return false }
return asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
let key = asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines)
guard !key.isEmpty else { return true }
// Volcengine may store auth_mode JSON before credentials are filled.
if asrProviderId == "volcengine" {
return !VolcengineASRFields.parse(apiKey: key).hasUsableCredentials
}
return false
}
public var isPolishKeyMissing: Bool {
+1 -1
View File
@@ -191,7 +191,7 @@ public struct LLMProvider: Identifiable, Codable, Hashable, Sendable {
defaultBaseURL: "wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async",
defaultModel: "volc.seedasr.sauc.duration",
apiKeyURL: URL(string: "https://console.volcengine.com/speech"),
blurb: "流式大模型 ASR · API Key 填 appId:accessToken[:resourceId]",
blurb: "流式大模型 ASR · 旧版 AppID+Token / 新版 API Key",
isUserSelectable: false
),
.init(
@@ -15,6 +15,9 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable {
public let id: String
public var name: String
public var prompt: String
/// When true, polish may keep model-added emoji and the prompt overrides R5.
/// Defaults off so existing / builtin styles stay emoji-strict.
public var allowsAddedEmoji: Bool
public let kind: Kind
public let createdAt: Date
public var updatedAt: Date
@@ -23,6 +26,7 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable {
id: String = "user.\(UUID().uuidString.lowercased())",
name: String,
prompt: String,
allowsAddedEmoji: Bool = false,
kind: Kind = .user,
createdAt: Date = Date(),
updatedAt: Date? = nil
@@ -30,6 +34,7 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable {
self.id = id
self.name = name
self.prompt = prompt
self.allowsAddedEmoji = allowsAddedEmoji
self.kind = kind
self.createdAt = createdAt
self.updatedAt = updatedAt ?? createdAt
@@ -39,6 +44,46 @@ public struct PolishStylePack: Codable, Equatable, Identifiable, Sendable {
guard kind == .builtin else { return name }
return SharedL10n.string("polishStyle.\(id.dropFirst("builtin.".count))", language: language)
}
/// Effective emoji policy for polish: explicit toggle, or a custom prompt that
/// clearly opts in (so paste-only custom styles still keep model-added emoji).
public var effectiveAllowsAddedEmoji: Bool {
if allowsAddedEmoji { return true }
guard kind == .user else { return false }
return Self.promptDeclaresAddedEmojiOptIn(prompt)
}
/// Heuristic for custom prompts that declare add mood emoji themselves.
public static func promptDeclaresAddedEmojiOptIn(_ prompt: String) -> Bool {
let markers = [
"允许新增 emoji",
"允许新增emoji",
"按情绪点缀",
"按原文情绪",
"Emoji 覆盖",
"outranks global R5",
"may add emojis",
"allow mood emoji",
"allowsAddedEmoji",
]
return markers.contains { prompt.localizedCaseInsensitiveContains($0) }
}
private enum CodingKeys: String, CodingKey {
case id, name, prompt, allowsAddedEmoji, kind, createdAt, updatedAt
}
public init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
id = try container.decode(String.self, forKey: .id)
name = try container.decode(String.self, forKey: .name)
prompt = try container.decode(String.self, forKey: .prompt)
// Older synced packs omit the key stay emoji-strict.
allowsAddedEmoji = try container.decodeIfPresent(Bool.self, forKey: .allowsAddedEmoji) ?? false
kind = try container.decode(Kind.self, forKey: .kind)
createdAt = try container.decode(Date.self, forKey: .createdAt)
updatedAt = try container.decode(Date.self, forKey: .updatedAt)
}
}
public enum PolishStyleLimits {
@@ -92,6 +137,7 @@ public struct PolishStyleCatalog: Codable, Equatable, Sendable {
var updated = pack
updated.name = name
updated.prompt = prompt
updated.allowsAddedEmoji = pack.allowsAddedEmoji
updated.updatedAt = date
entries[index] = updated
} else {
@@ -101,6 +147,7 @@ public struct PolishStyleCatalog: Codable, Equatable, Sendable {
var created = pack
created.name = name
created.prompt = prompt
created.allowsAddedEmoji = pack.allowsAddedEmoji
created.updatedAt = date
entries.append(created)
}
@@ -313,7 +313,14 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
public var isASRConfigured: Bool {
guard !isLocalEngine else { return true }
return !asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
let key = asrApiKey.trimmingCharacters(in: .whitespacesAndNewlines)
let hasKey: Bool = {
if asrProviderId == "volcengine" {
return VolcengineASRFields.parse(apiKey: key).hasUsableCredentials
}
return !key.isEmpty
}()
return hasKey
&& (!asrBaseURL.isEmpty || CloudASRModelCatalog.strategy(for: asrProviderId) != .prompt)
}
@@ -2,65 +2,155 @@
// OSGKeyboard · Shared
//
// Parse / encode Volcengine SAUC credentials stored in the ASR API key field.
// Supports legacy AppID+AccessToken (old console) and single API Key (new console).
import Foundation
/// Volcengine speech console auth style for SAUC streaming ASR.
public enum VolcengineASRAuthMode: String, Sendable, Equatable {
/// Old console: `X-Api-App-Key` + `X-Api-Access-Key`.
case appToken = "app_token"
/// New console: single `X-Api-Key`.
case apiKey = "api_key"
}
public struct VolcengineASRFields: Sendable, Equatable {
public var authMode: VolcengineASRAuthMode
public var appID: String
public var accessToken: String
public var resourceID: String
/// New-console API Key (`X-Api-Key`). Kept alongside app-token fields so the
/// settings toggle can switch modes without wiping the other credential set.
public var apiKeyCredential: String
/// Locked product: Doubao streaming ASR 2.0 · duration billing.
public static let fixedResourceID = CloudASRModelCatalog.volcengineDefaultResourceID
public init(
authMode: VolcengineASRAuthMode = .appToken,
appID: String = "",
accessToken: String = "",
resourceID: String = CloudASRModelCatalog.defaultModel(for: "volcengine")
apiKeyCredential: String = ""
) {
self.authMode = authMode
self.appID = appID
self.accessToken = accessToken
self.resourceID = resourceID
self.apiKeyCredential = apiKeyCredential
}
/// Always `volc.seedasr.sauc.duration` not user-editable.
public var resourceID: String { Self.fixedResourceID }
public var usesAPIKeyAuth: Bool { authMode == .apiKey }
public var hasUsableCredentials: Bool {
switch authMode {
case .appToken:
return !appID.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
&& !accessToken.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
case .apiKey:
return !apiKeyCredential.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
}
}
public var encodedAPIKey: String {
let object = [
"app_id": appID,
"access_token": accessToken,
"resource_id": resourceID,
var object: [String: String] = [
"auth_mode": authMode.rawValue,
"resource_id": Self.fixedResourceID,
]
// Persist both credential sets so toggling auth mode is non-destructive.
if !appID.isEmpty { object["app_id"] = appID }
if !accessToken.isEmpty { object["access_token"] = accessToken }
if !apiKeyCredential.isEmpty { object["api_key"] = apiKeyCredential }
guard let data = try? JSONSerialization.data(withJSONObject: object),
let string = String(data: data, encoding: .utf8) else {
return [appID, accessToken, resourceID].joined(separator: ":")
switch authMode {
case .appToken:
return [appID, accessToken, Self.fixedResourceID].joined(separator: ":")
case .apiKey:
return apiKeyCredential
}
}
return string
}
public static func parse(apiKey: String, resourceFallback: String) -> VolcengineASRFields {
let trimmed = apiKey.trimmingCharacters(in: .whitespacesAndNewlines)
var fields = VolcengineASRFields(
appID: "",
accessToken: "",
resourceID: resourceFallback.isEmpty
? CloudASRModelCatalog.defaultModel(for: "volcengine")
: resourceFallback
/// Apply SAUC WebSocket handshake headers for the active auth mode.
public func applyWebSocketAuthHeaders(to request: inout URLRequest, connectID: String) {
request.setValue(Self.fixedResourceID, forHTTPHeaderField: "X-Api-Resource-Id")
request.setValue(connectID, forHTTPHeaderField: "X-Api-Connect-Id")
switch authMode {
case .apiKey:
request.setValue(
apiKeyCredential.trimmingCharacters(in: .whitespacesAndNewlines),
forHTTPHeaderField: "X-Api-Key"
)
case .appToken:
request.setValue(
appID.trimmingCharacters(in: .whitespacesAndNewlines),
forHTTPHeaderField: "X-Api-App-Key"
)
request.setValue(
accessToken.trimmingCharacters(in: .whitespacesAndNewlines),
forHTTPHeaderField: "X-Api-Access-Key"
)
}
}
/// - Parameter resourceFallback: Ignored; resource is always `fixedResourceID`.
/// Kept so call sites stay source-compatible.
public static func parse(apiKey: String, resourceFallback: String = "") -> VolcengineASRFields {
_ = resourceFallback
let trimmed = apiKey.trimmingCharacters(in: .whitespacesAndNewlines)
var fields = VolcengineASRFields()
guard !trimmed.isEmpty else { return fields }
if let data = trimmed.data(using: .utf8),
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] {
fields.appID = string(json, keys: ["app_id", "appId", "appid"]) ?? ""
fields.accessToken = string(json, keys: ["access_token", "accessToken", "token"]) ?? ""
fields.resourceID = string(json, keys: ["resource_id", "resourceId", "resource"]) ?? fields.resourceID
fields.apiKeyCredential = string(json, keys: ["api_key", "apiKey"]) ?? ""
fields.authMode = resolveAuthMode(
raw: string(json, keys: ["auth_mode", "authMode"]),
hasAPIKey: !fields.apiKeyCredential.isEmpty,
hasAppToken: !fields.appID.isEmpty && !fields.accessToken.isEmpty
)
return fields
}
// Legacy colon form is always old-console app-token auth.
let parts = trimmed
.components(separatedBy: CharacterSet(charactersIn: ":\n,"))
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
if parts.indices.contains(0) { fields.appID = parts[0] }
if parts.indices.contains(1) { fields.accessToken = parts[1] }
if parts.indices.contains(2) { fields.resourceID = parts[2] }
fields.authMode = .appToken
return fields
}
private static func resolveAuthMode(
raw: String?,
hasAPIKey: Bool,
hasAppToken: Bool
) -> VolcengineASRAuthMode {
if let raw {
let normalized = raw.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
if normalized == VolcengineASRAuthMode.apiKey.rawValue || normalized == "apikey" {
return .apiKey
}
if normalized == VolcengineASRAuthMode.appToken.rawValue
|| normalized == "apptoken"
|| normalized == "app_id_token" {
return .appToken
}
}
// Legacy JSON without auth_mode: prefer app-token when present.
if hasAppToken { return .appToken }
if hasAPIKey { return .apiKey }
return .appToken
}
private static func string(_ json: [String: Any], keys: [String]) -> String? {
for key in keys {
if let value = json[key] as? String {
@@ -14,8 +14,39 @@ public enum FlowKeyboardHostWarming {
reason == .recording || reason == .processing || reason == .awaitingDelivery
}
/// Keep the mic green after the session has already proven ready.
///
/// Inter-utterance PiP flaps (mic release, ack lag, brief `reason=.starting`)
/// used to flash yelloweven though Picture in Picture was
/// already running. Hold ready through those windows; real cold starts still
/// go through `isHostWarming` while `sessionProvenReady` is false.
public static func shouldHoldReady(
hostReady: Bool,
hostBusy: Bool,
sessionActive: Bool,
sessionProvenReady: Bool,
isPendingFlowStart: Bool,
snapshotReason: FlowReadySnapshot.Reason?
) -> Bool {
guard !hostReady,
sessionProvenReady,
sessionActive,
!isPendingFlowStart else {
return false
}
// After insert the host may still publish awaitingDelivery until it
// consumes the ack that is not a PiP restart.
if hostBusy {
return snapshotReason == .awaitingDelivery
}
return true
}
/// Session lives but ready contract is not fresh keep mic orange (wait)
/// instead of launching another cold start.
///
/// `withinReadyGrace` is intentionally unused for warming: a recent ready
/// must hold green via `shouldHoldReady`, not flash preparingSession.
public static func isHostWarming(
hostReady: Bool,
hostBusy: Bool,
@@ -25,13 +56,13 @@ public enum FlowKeyboardHostWarming {
withinReadyGrace: Bool,
snapshotReason: FlowReadySnapshot.Reason?
) -> Bool {
!hostReady
_ = withinReadyGrace
return !hostReady
&& !hostBusy
&& sessionActive
&& (
hostReachable
|| isPendingFlowStart
|| withinReadyGrace
|| snapshotReason == .starting
)
}
@@ -671,6 +671,8 @@ public enum FlowSessionBridge {
store.removeObject(forKey: FlowSessionKeys.audioLevels)
store.removeObject(forKey: FlowSessionKeys.lastActivityAt)
clearHostReady(defaults: store, notify: false)
// Previous generation may have died mid Rime/CLM/ASR with hostHeavy=1.
clearHostHeavy(defaults: store)
flush(store)
}
@@ -701,13 +703,47 @@ public enum FlowSessionBridge {
/// avoid stacking typing-engine RSS on top.
public static func setHostHeavy(_ heavy: Bool, defaults: UserDefaults? = nil) {
let store = resolvedDefaults(defaults)
store.set(heavy, forKey: FlowSessionKeys.hostHeavy)
if heavy {
store.set(true, forKey: FlowSessionKeys.hostHeavy)
store.set(Date().timeIntervalSince1970, forKey: FlowSessionKeys.hostHeavyAt)
} else {
clearHostHeavy(defaults: store)
}
flush(store)
OSGDiag.log("hostHeavy=\(heavy ? 1 : 0) \(OSGDiag.memoryTag())", category: "flow")
}
/// True only while the host recently marked itself busy. A sticky `true`
/// left by a dead host (no `setHostHeavy(false)`) expires after
/// `hostHeavyMaxAge` so typing /EN is not silently blocked forever.
public static func isHostHeavy(defaults: UserDefaults? = nil) -> Bool {
resolvedDefaults(defaults).bool(forKey: FlowSessionKeys.hostHeavy)
let store = resolvedDefaults(defaults)
guard store.bool(forKey: FlowSessionKeys.hostHeavy) else { return false }
let markedAt = store.double(forKey: FlowSessionKeys.hostHeavyAt)
// Legacy writes had the bool but no timestamp treat as stale so a
// pre-fix sticky flag cannot brick typing after upgrade.
guard markedAt > 0 else {
clearHostHeavy(defaults: store)
flush(store)
OSGDiag.log("hostHeavy stale missingAt — cleared \(OSGDiag.memoryTag())", category: "flow")
return false
}
let age = Date().timeIntervalSince1970 - markedAt
guard age >= 0, age <= FlowSessionKeys.hostHeavyMaxAge else {
clearHostHeavy(defaults: store)
flush(store)
OSGDiag.log(
"hostHeavy stale age=\(Int(age))s — cleared \(OSGDiag.memoryTag())",
category: "flow"
)
return false
}
return true
}
private static func clearHostHeavy(defaults: UserDefaults) {
defaults.set(false, forKey: FlowSessionKeys.hostHeavy)
defaults.removeObject(forKey: FlowSessionKeys.hostHeavyAt)
}
/// True when the host has published a fresh ready contract (stricter than heartbeat alone).
@@ -948,6 +984,7 @@ public enum FlowSessionBridge {
store.removeObject(forKey: FlowSessionKeys.pendingHostBundleId)
store.removeObject(forKey: FlowSessionKeys.lastActivityAt)
clearHostReady(defaults: store, notify: false)
clearHostHeavy(defaults: store)
flush(store)
}
@@ -43,6 +43,15 @@ public enum FlowSessionKeys {
/// Host is mid heavy work (Rime/CLM/ASR). Extension should stay on voice
/// and skip typing engine prepare until this clears.
public static let hostHeavy = "flow.hostHeavy.v1"
/// Wall-clock timestamp paired with `hostHeavy` (seconds since 1970).
/// Lets the keyboard ignore a sticky flag left behind when the host died
/// mid-warmup without ever clearing App Group state.
public static let hostHeavyAt = "flow.hostHeavyAt.v1"
/// `hostHeavy` older than this is treated as stale (host likely jetsammed
/// or force-quit before `setHostHeavy(false)`). Rime/CLM/ASR bursts are
/// expected well under this window.
public static let hostHeavyMaxAge: TimeInterval = 120
/// Heartbeat older than this host is not actively reachable for recording.
public static let heartbeatStaleInterval: TimeInterval = 3
@@ -211,6 +211,10 @@ public enum PolishPromptComposer {
dictionaryBlock,
useChineseGuidance: useChineseGuidance
)
let emojiOverride = emojiOverrideSection(
allowsAddedEmoji: style.effectiveAllowsAddedEmoji,
useChineseGuidance: useChineseGuidance
)
if usesHeavyFunPipeline {
let formatting = useChineseGuidance
? chineseFunFormattingPrompt
@@ -226,6 +230,7 @@ public enum PolishPromptComposer {
\(personality)
\(outputInstruction)
\(emojiOverride)
"""
}
@@ -262,6 +267,7 @@ public enum PolishPromptComposer {
fieldHints: context.fieldHints,
useChineseGuidance: true
))
\(emojiOverride)
"""
}
@@ -284,6 +290,7 @@ public enum PolishPromptComposer {
fieldHints: context.fieldHints,
useChineseGuidance: false
))The user message is the transcript to process. Output the processed text only.
\(emojiOverride)
"""
}
@@ -346,6 +353,8 @@ public enum PolishPromptComposer {
useChineseGuidance: Bool
) -> String {
let body = PolishStylePackCatalog.runtimePersonality(for: style)
// Emoji override is appended at the end of compose(), not here
// it must outrank core R5 and later context premises.
if style.kind == .user {
return useChineseGuidance
? """
@@ -370,6 +379,30 @@ public enum PolishPromptComposer {
"""
}
/// When a style opts in, override global R5 so the model may add emoji.
/// Placed last in the composed prompt so it wins over core R5.
private static func emojiOverrideSection(
allowsAddedEmoji: Bool,
useChineseGuidance: Bool
) -> String {
guard allowsAddedEmoji else { return "" }
return useChineseGuidance
? """
# Emoji ·
emoji R5 emoji emoji
02 emoji
emoji emoji
"""
: """
# Emoji override (enabled for this style · final priority)
This style may add emojis and outranks global R5 (add no emojis) and any earlier do not add emojis guidance.
Add 02 tone-matching emojis only when the draft already expresses emotion; skip neutral schedules, formal notices, and ultra-short acks.
If the draft already has emojis, keep them and do not replace or stack. No decorative spam or emoji walls.
"""
}
/// Neutralize envelope-breaking tags inside user-controlled transcript text.
internal static func sanitizeEnvelopeContent(_ text: String) -> String {
let maxCharacters = 16_000
@@ -285,7 +285,15 @@ public actor PolishingService {
// One prompt, one model request. Deterministic validation may reject a
// result locally, but it never starts a second polish request.
let firstCandidate = TranscriptPostProcessor.process(original: trimmed, llmOutput: first)
let activeStyle = PolishStylePackCatalog.resolve(
id: store.activePolishStyleId,
userCatalog: store.polishStyleCatalog
)
let firstCandidate = TranscriptPostProcessor.process(
original: trimmed,
llmOutput: first,
allowsAddedEmoji: activeStyle.effectiveAllowsAddedEmoji
)
let firstViolations = PolishOutputValidator.validate(
input: trimmed,
output: firstCandidate,
@@ -169,9 +169,17 @@ public enum TranscriptPostProcessor: Sendable {
// MARK: - Post-LLM pipeline
/// Apply deterministic cleanup and quality gate to LLM output.
public static func process(original: String, llmOutput: String) -> String {
public static func process(
original: String,
llmOutput: String,
allowsAddedEmoji: Bool = false
) -> String {
let trimmedOriginal = original.trimmingCharacters(in: .whitespacesAndNewlines)
let decision = qualityGate(original: trimmedOriginal, candidate: llmOutput)
let decision = qualityGate(
original: trimmedOriginal,
candidate: llmOutput,
allowsAddedEmoji: allowsAddedEmoji
)
switch decision {
case .accept(let text):
return text
@@ -191,7 +199,11 @@ public enum TranscriptPostProcessor: Sendable {
/// back when the model returned genuinely unusable output (empty, or
/// pure explanation), and even then we prefer a cleaned candidate
/// over the raw transcript.
public static func qualityGate(original: String, candidate: String) -> GateDecision {
public static func qualityGate(
original: String,
candidate: String,
allowsAddedEmoji: Bool = false
) -> GateDecision {
var text = candidate.trimmingCharacters(in: .whitespacesAndNewlines)
if text.isEmpty {
@@ -201,7 +213,9 @@ public enum TranscriptPostProcessor: Sendable {
text = stripExplanatoryPrefix(from: text)
text = stripPauseMarkers(from: text)
text = unwrapSurroundingQuotes(text)
if !allowsAddedEmoji {
text = stripAddedEmojis(original: original, output: text)
}
text = repairMidSentenceLineBreaks(text)
text = normalizeWhitespaceAndPunctuation(text)
text = normalizeNumberedLists(text)
@@ -0,0 +1,204 @@
// KeyHitTesting.swift
// OSGKeyboard · Shared
//
// Pure geometry for typing-key hit testing: invisible gap fill, edge
// expansion, and a light upward intent offset (Phase 1 + Phase 3).
import CoreGraphics
import Foundation
/// How a key should respond once the finger is tracked onto it.
public enum TypingKeyTouchBehavior: Equatable, Sendable {
/// Highlight on down / move; commit on finger-up (letters, space, return).
case commitOnRelease
/// Fire on down and repeat while held (delete).
case deleteRepeat
/// Hold-to-shift while the gesture owns Shift ().
case shiftHold
}
/// One hittable key in surface coordinates.
public struct TypingKeyHitTarget: Equatable, Identifiable, Sendable {
public let id: String
public let label: String
public let visualFrame: CGRect
public let behavior: TypingKeyTouchBehavior
public init(
id: String,
label: String,
visualFrame: CGRect,
behavior: TypingKeyTouchBehavior
) {
self.id = id
self.label = label
self.visualFrame = visualFrame
self.behavior = behavior
}
public var center: CGPoint {
CGPoint(x: visualFrame.midX, y: visualFrame.midY)
}
}
/// Tunables for gap-filling hit regions and finger intent correction.
public enum KeyHitTestingMetrics: Sendable {
/// Shift the reported touch slightly upward thumbs contact below the
/// visual aim point.
public static let intentOffsetY: CGFloat = 4
/// Extra expansion on the outer edges of the key plane (Q / P / ).
public static let edgeExpansion: CGFloat = 5
}
public enum KeyHitTesting {
/// Map a raw touch into an intent point (Phase 3).
public static func intentPoint(
from point: CGPoint,
offsetY: CGFloat = KeyHitTestingMetrics.intentOffsetY
) -> CGPoint {
CGPoint(x: point.x, y: point.y - offsetY)
}
/// Expand a visual key frame so neighboring keys meet at the mid-gap
/// (no dead zone). Outer keys grow further past the plane edge.
public static func expandedHitFrame(
for visualFrame: CGRect,
keyPlaneBounds: CGRect,
horizontalGap: CGFloat,
verticalGap: CGFloat,
edgeExpansion: CGFloat = KeyHitTestingMetrics.edgeExpansion
) -> CGRect {
var frame = visualFrame.insetBy(
dx: -horizontalGap / 2,
dy: -verticalGap / 2
)
let epsilon: CGFloat = 0.5
if visualFrame.minX <= keyPlaneBounds.minX + epsilon {
frame.origin.x -= edgeExpansion
frame.size.width += edgeExpansion
}
if visualFrame.maxX >= keyPlaneBounds.maxX - epsilon {
frame.size.width += edgeExpansion
}
if visualFrame.minY <= keyPlaneBounds.minY + epsilon {
frame.origin.y -= edgeExpansion
frame.size.height += edgeExpansion
}
if visualFrame.maxY >= keyPlaneBounds.maxY - epsilon {
frame.size.height += edgeExpansion
}
return frame
}
/// Resolve which key owns `point` (already intent-corrected, or raw).
///
/// - Returns `nil` when the point is outside the key plane (cancel).
/// - Inside the plane: prefer expanded frames; fall back to nearest center
/// so mid-gap touches never miss.
public static func hitTarget(
at point: CGPoint,
targets: [TypingKeyHitTarget],
keyPlaneBounds: CGRect,
horizontalGap: CGFloat,
verticalGap: CGFloat,
edgeExpansion: CGFloat = KeyHitTestingMetrics.edgeExpansion,
hitWeights: [String: CGFloat] = [:]
) -> TypingKeyHitTarget? {
guard !targets.isEmpty else { return nil }
let activePlane = keyPlaneBounds.insetBy(
dx: -edgeExpansion,
dy: -edgeExpansion
)
guard activePlane.contains(point) else { return nil }
let expanded = targets.map { target in
(
target,
expandedHitFrame(
for: target.visualFrame,
keyPlaneBounds: keyPlaneBounds,
horizontalGap: horizontalGap,
verticalGap: verticalGap,
edgeExpansion: edgeExpansion
)
)
}
let containing = expanded.compactMap { target, frame -> TypingKeyHitTarget? in
frame.contains(point) ? target : nil
}
// Clear single-key hit always wins bias only breaks ties / nearest.
if containing.count == 1 {
return containing[0]
}
if containing.count > 1 {
return nearest(to: point, among: containing, hitWeights: hitWeights)
}
return nearest(to: point, among: targets, hitWeights: hitWeights)
}
/// Convenience: apply intent offset then hit-test.
public static func hitTarget(
rawTouch point: CGPoint,
targets: [TypingKeyHitTarget],
keyPlaneBounds: CGRect,
horizontalGap: CGFloat,
verticalGap: CGFloat,
intentOffsetY: CGFloat = KeyHitTestingMetrics.intentOffsetY,
edgeExpansion: CGFloat = KeyHitTestingMetrics.edgeExpansion,
hitWeights: [String: CGFloat] = [:]
) -> TypingKeyHitTarget? {
hitTarget(
at: intentPoint(from: point, offsetY: intentOffsetY),
targets: targets,
keyPlaneBounds: keyPlaneBounds,
horizontalGap: horizontalGap,
verticalGap: verticalGap,
edgeExpansion: edgeExpansion,
hitWeights: hitWeights
)
}
private static func nearest(
to point: CGPoint,
among targets: [TypingKeyHitTarget],
hitWeights: [String: CGFloat]
) -> TypingKeyHitTarget? {
targets.min { lhs, rhs in
weightedDistanceSquared(point, lhs, hitWeights)
< weightedDistanceSquared(point, rhs, hitWeights)
}
}
private static func weightedDistanceSquared(
_ point: CGPoint,
_ target: TypingKeyHitTarget,
_ hitWeights: [String: CGFloat]
) -> CGFloat {
let weight = max(0.01, hitWeights[target.id] ?? 1.0)
return distanceSquared(point, target.center) / weight
}
private static func distanceSquared(_ a: CGPoint, _ b: CGPoint) -> CGFloat {
let dx = a.x - b.x
let dy = a.y - b.y
return dx * dx + dy * dy
}
}
/// Resolve touch behavior from a visible key label.
public enum TypingKeyBehaviorResolver {
public static func behavior(for label: String) -> TypingKeyTouchBehavior {
switch label {
case "":
return .deleteRepeat
case "":
return .shiftHold
default:
return .commitOnRelease
}
}
}
@@ -144,8 +144,10 @@ public final class LibrimeEngine: RimeEngineBridging {
return nil
}
let preedit = snapshot.preedit
let raw = bridge?.rawInput() ?? ""
composition = TypingComposition(
preedit: preedit,
rawInput: raw,
candidates: snapshot.candidates.enumerated().map { displayIndex, candidate in
TypingCandidate(
id: "\(preedit)|\(displayIndex)|\(candidate.index)|\(candidate.text)",
@@ -0,0 +1,102 @@
// PinyinNextKeyResolver.swift
// OSGKeyboard · Shared
//
// Phase 4: legal next letters during full-pinyin composition.
// Double-pinyin schemas return nil (no bias) until a dedicated FSM exists.
import CoreGraphics
import Foundation
public enum KeyHitBiasMetrics: Sendable {
/// Legal next letters are slightly sticky in ambiguous hit tests.
public static let legalBoost: CGFloat = 1.20
/// Illegal letters shrink a little but remain reachable.
public static let illegalShrink: CGFloat = 0.90
public static let neutral: CGFloat = 1.0
}
public enum PinyinNextKeyResolver {
/// Returns legal next key characters, or `nil` when bias should be off.
public static func validNextKeys(
rawInput: String,
schema: TypingInputSchema,
language: TypingInputLanguage,
page: TypingKeyPage
) -> Set<Character>? {
guard language == .chinese, page == .letters else { return nil }
// Phase 4a: full pinyin only. Double-pinyin stays neutral.
guard schema == .fullPinyin else { return nil }
let normalized = normalize(rawInput)
guard !normalized.isEmpty else { return nil }
let segment = lastSpellingSegment(normalized)
var result = Set<Character>()
collectNext(prefix: segment, into: &result)
return result.isEmpty ? nil : result
}
/// Maps layout keys hit weights for ambiguous nearest-key resolution.
public static func hitWeights(
for keys: [TypingKeyHitTarget],
validNext: Set<Character>?
) -> [String: CGFloat] {
guard let validNext, !validNext.isEmpty else { return [:] }
var weights: [String: CGFloat] = [:]
for key in keys {
guard key.id.hasPrefix("grid.") else { continue }
guard key.behavior == .commitOnRelease else { continue }
let label = key.label.lowercased()
guard label.count == 1, let char = label.first, char.isLetter else {
continue
}
weights[key.id] = validNext.contains(char)
? KeyHitBiasMetrics.legalBoost
: KeyHitBiasMetrics.illegalShrink
}
return weights
}
// MARK: - Internals
private static func normalize(_ raw: String) -> String {
raw.lowercased().replacingOccurrences(of: "ü", with: "v")
}
private static func lastSpellingSegment(_ input: String) -> String {
let lettersAndDelim = input.filter { $0.isLetter || $0 == "'" || $0 == " " }
let parts = lettersAndDelim.split { $0 == "'" || $0 == " " }
return parts.last.map(String.init) ?? ""
}
private static func collectNext(prefix: String, into result: inout Set<Character>) {
var canExtend = false
for syllable in PinyinSyllableTable.syllables
where syllable.hasPrefix(prefix) && syllable.count > prefix.count
{
let index = syllable.index(syllable.startIndex, offsetBy: prefix.count)
result.insert(syllable[index])
canExtend = true
}
// Complete syllable (or empty) also allow starting a new syllable.
if prefix.isEmpty || PinyinSyllableTable.syllables.contains(prefix) {
for syllable in PinyinSyllableTable.syllables {
if let first = syllable.first {
result.insert(first)
}
}
}
// Multi-syllable undelimited input (e.g. "zhongg" "zhong" + "g").
// Only peel when the whole prefix cannot extend as one syllable.
if !canExtend, !prefix.isEmpty {
let longest = PinyinSyllableTable.longestSyllablePrefix(of: prefix)
if !longest.isEmpty, longest.count < prefix.count {
let remainder = String(prefix.dropFirst(longest.count))
collectNext(prefix: remainder, into: &result)
}
}
}
}
@@ -0,0 +1,77 @@
// PinyinSyllableTable.swift
// OSGKeyboard · Shared
//
// Mandarin pinyin syllables (no tones) for Phase 4 next-key bias.
// Uses ASCII `v` for ü so it matches keyboard / Rime raw input.
import Foundation
public enum PinyinSyllableTable {
/// Complete syllables accepted by full-pinyin next-key logic.
public static let syllables: Set<String> = [
"a", "ai", "an", "ang", "ao",
"ba", "bai", "ban", "bang", "bao", "bei", "ben", "beng", "bi", "bian",
"biao", "bie", "bin", "bing", "bo", "bu",
"ca", "cai", "can", "cang", "cao", "ce", "cen", "ceng", "cha", "chai",
"chan", "chang", "chao", "che", "chen", "cheng", "chi", "chong", "chou",
"chu", "chua", "chuai", "chuan", "chuang", "chui", "chun", "chuo",
"ci", "cong", "cou", "cu", "cuan", "cui", "cun", "cuo",
"da", "dai", "dan", "dang", "dao", "de", "dei", "den", "deng", "di",
"dia", "dian", "diao", "die", "ding", "diu", "dong", "dou", "du",
"duan", "dui", "dun", "duo",
"e", "ei", "en", "eng", "er",
"fa", "fan", "fang", "fei", "fen", "feng", "fiao", "fo", "fou", "fu",
"ga", "gai", "gan", "gang", "gao", "ge", "gei", "gen", "geng", "gong",
"gou", "gu", "gua", "guai", "guan", "guang", "gui", "gun", "guo",
"ha", "hai", "han", "hang", "hao", "he", "hei", "hen", "heng", "hong",
"hou", "hu", "hua", "huai", "huan", "huang", "hui", "hun", "huo",
"ji", "jia", "jian", "jiang", "jiao", "jie", "jin", "jing", "jiong",
"jiu", "ju", "juan", "jue", "jun",
"ka", "kai", "kan", "kang", "kao", "ke", "ken", "keng", "kong", "kou",
"ku", "kua", "kuai", "kuan", "kuang", "kui", "kun", "kuo",
"la", "lai", "lan", "lang", "lao", "le", "lei", "leng", "li", "lia",
"lian", "liang", "liao", "lie", "lin", "ling", "liu", "lo", "long",
"lou", "lu", "luan", "lue", "lun", "luo", "lv", "lve",
"ma", "mai", "man", "mang", "mao", "me", "mei", "men", "meng", "mi",
"mian", "miao", "mie", "min", "ming", "miu", "mo", "mou", "mu",
"na", "nai", "nan", "nang", "nao", "ne", "nei", "nen", "neng", "ni",
"nian", "niang", "niao", "nie", "nin", "ning", "niu", "nong", "nou",
"nu", "nuan", "nue", "nun", "nuo", "nv", "nve",
"o", "ou",
"pa", "pai", "pan", "pang", "pao", "pei", "pen", "peng", "pi", "pian",
"piao", "pie", "pin", "ping", "po", "pou", "pu",
"qi", "qia", "qian", "qiang", "qiao", "qie", "qin", "qing", "qiong",
"qiu", "qu", "quan", "que", "qun",
"ran", "rang", "rao", "re", "ren", "reng", "ri", "rong", "rou", "ru",
"ruan", "rui", "run", "ruo",
"sa", "sai", "san", "sang", "sao", "se", "sen", "seng", "sha", "shai",
"shan", "shang", "shao", "she", "shei", "shen", "sheng", "shi", "shou",
"shu", "shua", "shuai", "shuan", "shuang", "shui", "shun", "shuo",
"si", "song", "sou", "su", "suan", "sui", "sun", "suo",
"ta", "tai", "tan", "tang", "tao", "te", "tei", "ten", "teng", "ti",
"tian", "tiao", "tie", "ting", "tong", "tou", "tu", "tuan", "tui",
"tun", "tuo",
"wa", "wai", "wan", "wang", "wei", "wen", "weng", "wo", "wu",
"xi", "xia", "xian", "xiang", "xiao", "xie", "xin", "xing", "xiong",
"xiu", "xu", "xuan", "xue", "xun",
"ya", "yan", "yang", "yao", "ye", "yi", "yin", "ying", "yo", "yong",
"you", "yu", "yuan", "yue", "yun",
"za", "zai", "zan", "zang", "zao", "ze", "zei", "zen", "zeng", "zha",
"zhai", "zhan", "zhang", "zhao", "zhe", "zhei", "zhen", "zheng",
"zhi", "zhong", "zhou", "zhu", "zhua", "zhuai", "zhuan", "zhuang",
"zhui", "zhun", "zhuo", "zi", "zong", "zou", "zu", "zuan", "zui",
"zun", "zuo"
]
public static func longestSyllablePrefix(of input: String) -> String {
let maxLen = min(6, input.count)
guard maxLen > 0 else { return "" }
for len in stride(from: maxLen, through: 1, by: -1) {
let prefix = String(input.prefix(len))
if syllables.contains(prefix) {
return prefix
}
}
return ""
}
}
@@ -45,10 +45,18 @@ public struct TypingCandidate: Identifiable, Equatable, Sendable {
/// Snapshot the UI observes while composing.
public struct TypingComposition: Equatable, Sendable {
public var preedit: String
/// Raw key sequence from the engine (ASCII). Prefer this over `preedit`
/// for spelling / next-key logic preedit may include separators.
public var rawInput: String
public var candidates: [TypingCandidate]
public init(preedit: String = "", candidates: [TypingCandidate] = []) {
public init(
preedit: String = "",
rawInput: String = "",
candidates: [TypingCandidate] = []
) {
self.preedit = preedit
self.rawInput = rawInput
self.candidates = candidates
}
@@ -42,20 +42,22 @@ public enum TypingAutocapitalization: Sendable {
private static func needsSentenceCapitalization(_ preceding: String?) -> Bool {
guard let preceding, !preceding.isEmpty else { return true }
// Walk backward past trailing whitespace / newlines; capitalize when
// the field is empty or the previous visible character ends a sentence.
// Walk backward from the caret. Trailing spaces are ignored; a newline
// itself starts a new line (system .sentences behavior, e.g. Notes
// after Return). Otherwise capitalize only after a sentence terminator.
var index = preceding.endIndex
var sawContent = false
while index > preceding.startIndex {
index = preceding.index(before: index)
let character = preceding[index]
if character.isWhitespace || character.isNewline {
if character.isNewline {
return true
}
if character.isWhitespace {
continue
}
sawContent = true
return isSentenceTerminator(character)
}
return !sawContent
return true
}
private static func isSentenceTerminator(_ character: Character) -> Bool {
@@ -0,0 +1,178 @@
// TypingKeyLayout.swift
// OSGKeyboard · Shared
//
// Builds visual frames for the typing grid + bottom action row so hit
// testing and rendering share one geometry source.
import CoreGraphics
import Foundation
public struct TypingKeyLayout: Equatable, Sendable {
public let keys: [TypingKeyHitTarget]
/// Union of all visual key frames (letter grid + bottom row).
public let keyPlaneBounds: CGRect
public let horizontalGap: CGFloat
public let verticalGap: CGFloat
public let bottomRowMinY: CGFloat
/// Phase 4: per-key hit weights for ambiguous nearest resolution.
/// Empty / missing id neutral (`1.0`).
public let hitWeights: [String: CGFloat]
public init(
keys: [TypingKeyHitTarget],
keyPlaneBounds: CGRect,
horizontalGap: CGFloat,
verticalGap: CGFloat,
bottomRowMinY: CGFloat,
hitWeights: [String: CGFloat] = [:]
) {
self.keys = keys
self.keyPlaneBounds = keyPlaneBounds
self.horizontalGap = horizontalGap
self.verticalGap = verticalGap
self.bottomRowMinY = bottomRowMinY
self.hitWeights = hitWeights
}
public func key(id: String) -> TypingKeyHitTarget? {
keys.first { $0.id == id }
}
public func withHitWeights(_ weights: [String: CGFloat]) -> TypingKeyLayout {
TypingKeyLayout(
keys: keys,
keyPlaneBounds: keyPlaneBounds,
horizontalGap: horizontalGap,
verticalGap: verticalGap,
bottomRowMinY: bottomRowMinY,
hitWeights: weights
)
}
}
public enum TypingKeyLayoutBuilder {
public struct Metrics: Equatable, Sendable {
public var keyRowHeight: CGFloat
public var keyRowSpacing: CGFloat
public var keyHorizontalSpacing: CGFloat
public var secondRowInset: CGFloat
public var bottomRowHeight: CGFloat
public var bottomActionSpacing: CGFloat
/// Gap between the last letter row and the bottom action row.
public var gridToBottomSpacing: CGFloat
public init(
keyRowHeight: CGFloat = 50,
keyRowSpacing: CGFloat = 7,
keyHorizontalSpacing: CGFloat = 6,
secondRowInset: CGFloat = 18,
bottomRowHeight: CGFloat = KeyboardChromeLayout.actionKeyHeight,
bottomActionSpacing: CGFloat = KeyboardChromeLayout.actionKeySpacing,
gridToBottomSpacing: CGFloat = 7
) {
self.keyRowHeight = keyRowHeight
self.keyRowSpacing = keyRowSpacing
self.keyHorizontalSpacing = keyHorizontalSpacing
self.secondRowInset = secondRowInset
self.bottomRowHeight = bottomRowHeight
self.bottomActionSpacing = bottomActionSpacing
self.gridToBottomSpacing = gridToBottomSpacing
}
}
/// Bottom-row semantic labels used by the touch pad (not always the glyph).
public enum BottomKeyID: String, Sendable {
case pageSwitch = "bottom.page"
case space = "bottom.space"
case `return` = "bottom.return"
}
public static func build(
size: CGSize,
letterRows: [[String]],
pageSwitchLabel: String,
spaceLabel: String,
returnLabel: String,
metrics: Metrics = Metrics(),
keyWeight: (_ label: String, _ index: Int, _ rowIndex: Int) -> CGFloat
) -> TypingKeyLayout {
var keys: [TypingKeyHitTarget] = []
var cursorY: CGFloat = 0
for (rowIndex, row) in letterRows.enumerated() {
let inset = rowIndex == 1 ? metrics.secondRowInset : 0
let weights = row.enumerated().map { keyWeight($0.element, $0.offset, rowIndex) }
let spacingTotal = metrics.keyHorizontalSpacing * CGFloat(max(0, row.count - 1))
let availableWidth = size.width - inset * 2 - spacingTotal
let unitWidth = availableWidth / max(1, weights.reduce(0, +))
var x = inset
for (keyIndex, label) in row.enumerated() {
let width = unitWidth * weights[keyIndex]
let frame = CGRect(
x: x,
y: cursorY,
width: width,
height: metrics.keyRowHeight
)
keys.append(
TypingKeyHitTarget(
id: "grid.\(rowIndex).\(keyIndex)",
label: label,
visualFrame: frame,
behavior: TypingKeyBehaviorResolver.behavior(for: label)
)
)
x += width + metrics.keyHorizontalSpacing
}
cursorY += metrics.keyRowHeight
if rowIndex < letterRows.count - 1 {
cursorY += metrics.keyRowSpacing
}
}
cursorY += metrics.gridToBottomSpacing
let bottomY = cursorY
let widths = KeyboardChromeLayout.actionKeyWidths(availableWidth: size.width)
let bottomFrames: [(String, String, CGFloat)] = [
(BottomKeyID.pageSwitch.rawValue, pageSwitchLabel, widths.side),
(BottomKeyID.space.rawValue, spaceLabel, widths.center),
(BottomKeyID.return.rawValue, returnLabel, widths.side)
]
var bottomX: CGFloat = 0
for (index, item) in bottomFrames.enumerated() {
let frame = CGRect(
x: bottomX,
y: bottomY,
width: item.2,
height: metrics.bottomRowHeight
)
keys.append(
TypingKeyHitTarget(
id: item.0,
label: item.1,
visualFrame: frame,
behavior: .commitOnRelease
)
)
bottomX += item.2
if index < bottomFrames.count - 1 {
bottomX += metrics.bottomActionSpacing
}
}
let plane = keys.reduce(CGRect.null) { $0.union($1.visualFrame) }
return TypingKeyLayout(
keys: keys,
keyPlaneBounds: plane.isNull ? .zero : plane,
horizontalGap: metrics.keyHorizontalSpacing,
// Between letter rows use keyRowSpacing; between grid and bottom
// use gridToBottomSpacing. Hit-test uses the larger gap so the
// mid-gap is always covered.
verticalGap: max(metrics.keyRowSpacing, metrics.gridToBottomSpacing),
bottomRowMinY: bottomY
)
}
}
@@ -68,6 +68,10 @@ public final class TypingSessionController: ObservableObject {
private var shiftPrimedByUser = false
/// True if any key was typed while the current Shift hold was active.
private var typedWhileShiftHeld = false
/// Local caret-prefix mirror so autocap survives stale `documentContextBeforeInput`
/// (common in Notes). Capped; reseeds from the proxy when it looks fresh.
private var precedingShadow = ""
private static let precedingShadowLimit = 400
public init(
engine: (@MainActor () -> RimeEngineBridging)? = nil,
@@ -287,6 +291,14 @@ public final class TypingSessionController: ObservableObject {
return handleEnglishCharacter(ch)
}
// Chinese + Shift: insert Latin directly (iOS-style mix-in), leave Rime
// composition untouched. Rime's alphabet is lowercase-only, so uppercase
// keycodes would otherwise be rejected with no output.
if isShiftEnabled, ch.isLetter {
clearOneShotShiftIfNeeded()
return .insert(String(ch))
}
// Chinese letters compose
let committed = engine.processCharacter(ch) ?? ""
composition = engine.composition
@@ -490,17 +502,94 @@ public final class TypingSessionController: ObservableObject {
/// Arms Shift for sentence / word starts using the host field traits.
/// Manual one-shot, hold, and Caps Lock always win over autocapitalization.
public func syncAutocapitalization() {
/// - Parameters:
/// - insert: Text just written through the document proxy (may not be
/// reflected in `documentContextBeforeInput` yet).
/// - deleteCount: Characters just deleted before `insert` (replace path).
public func syncAutocapitalization(
accountingForInsert insert: String = "",
deleteCount: Int = 0
) {
guard language == .english, page == .letters else { return }
guard !capsLock, !shiftHeld, !shiftPrimedByUser else { return }
let mode = autocapitalizationModeProvider?() ?? .sentences
let preceding = precedingTextProvider?()
let preceding = resolvedPrecedingText(
accountingForInsert: insert,
deleteCount: deleteCount
)
shiftActive = TypingAutocapitalization.shouldCapitalize(
precedingText: preceding,
mode: mode
)
}
/// Prefer a fresh proxy; when the host lags (Notes), merge our just-applied edit.
private func resolvedPrecedingText(
accountingForInsert insert: String,
deleteCount: Int
) -> String? {
// Host-driven refresh (appear / textDidChange): reseed from proxy.
if deleteCount == 0, insert.isEmpty {
if let proxy = precedingTextProvider?() {
return storePrecedingShadow(proxy)
}
return precedingShadow.isEmpty ? nil : precedingShadow
}
let proxy = precedingTextProvider?()
if deleteCount > 0, !insert.isEmpty {
if let proxy {
if proxy.hasSuffix(insert) {
return storePrecedingShadow(proxy)
}
if proxy.count >= deleteCount {
return storePrecedingShadow(String(proxy.dropLast(deleteCount)) + insert)
}
}
trimPrecedingShadow(by: deleteCount)
return storePrecedingShadow(precedingShadow + insert)
}
if deleteCount > 0 {
if let proxy, proxy.count + deleteCount == precedingShadow.count
|| (precedingShadow.count >= deleteCount
&& String(precedingShadow.dropLast(deleteCount)) == proxy) {
return storePrecedingShadow(proxy)
}
if precedingShadow.count >= deleteCount {
return storePrecedingShadow(String(precedingShadow.dropLast(deleteCount)))
}
if let proxy { return storePrecedingShadow(proxy) }
precedingShadow = ""
return ""
}
// insert only
if let proxy {
if proxy.hasSuffix(insert) {
return storePrecedingShadow(proxy)
}
return storePrecedingShadow(proxy + insert)
}
return storePrecedingShadow(precedingShadow + insert)
}
@discardableResult
private func storePrecedingShadow(_ value: String) -> String {
precedingShadow = String(value.suffix(Self.precedingShadowLimit))
return precedingShadow
}
private func trimPrecedingShadow(by count: Int) {
guard count > 0 else { return }
if precedingShadow.count >= count {
precedingShadow.removeLast(count)
} else {
precedingShadow = ""
}
}
// MARK: - Shift
/// Tap cycle: off one-shot Caps Lock off (iOS-style second tap).
@@ -536,6 +625,7 @@ public final class TypingSessionController: ObservableObject {
shiftHeld = false
shiftPrimedByUser = false
typedWhileShiftHeld = false
precedingShadow = ""
}
private func clearEnglishWordState(keepPrevious: Bool) {
+7 -2
View File
@@ -158,6 +158,8 @@
"mac.styles.error" = "Couldnt Save Style";
"mac.styles.validation" = "Check the name, prompt length, and the 8-style limit.";
"mac.styles.name" = "Style name";
"mac.styles.allowsAddedEmoji" = "Allow mood emojis";
"mac.styles.allowsAddedEmoji.hint" = "When on, polish may add a few emojis that match the drafts emotion, and keeps them on screen. Off by default.";
"mac.styles.prompt" = "Complete prompt";
"mac.styles.hint" = "Use {{DICTIONARY}} to place the personal dictionary. System rules are appended automatically.";
"mac.section.settings" = "Settings";
@@ -262,8 +264,11 @@
"mac.settings.translationOff" = "Don't translate";
"mac.settings.volcengineAppId" = "APP ID";
"mac.settings.volcengineAccessToken" = "Access Token";
"mac.settings.volcengineResourceId" = "Resource ID";
"mac.settings.volcengineNote" = "Secret Key is not required. Resource ID defaults to volc.seedasr.sauc.duration.";
"mac.settings.volcengineApiKey" = "API Key";
"mac.settings.volcengineApiKeyMode" = "New API Key auth";
"mac.settings.volcengineApiKeyModeSubtitle" = "On for the new console; off keeps APP ID + Access Token.";
"mac.settings.volcengineNoteAppToken" = "Legacy console: APP ID + Access Token. Secret Key not required. Resource fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration).";
"mac.settings.volcengineNoteApiKey" = "New console: API Key only. Resource fixed to Doubao streaming 2.0 (volc.seedasr.sauc.duration).";
"mac.settings.recognition" = "RECOGNITION METHOD";
"mac.settings.cloudEngine" = "Cloud Engine & AI Refinement";
"mac.settings.cloudEngineDesc" = "Premium transcription via your provider's API, plus AI grammar and style polishing.";
@@ -157,6 +157,8 @@
"mac.styles.error" = "无法保存风格";
"mac.styles.validation" = "请检查名称、提示词长度及 8 个风格的数量上限。";
"mac.styles.name" = "风格名称";
"mac.styles.allowsAddedEmoji" = "允许按情绪添加 emoji";
"mac.styles.allowsAddedEmoji.hint" = "开启后,润色可按原文情绪点缀少量 emoji,并保留上屏。默认关闭。";
"mac.styles.prompt" = "完整提示词";
"mac.styles.hint" = "使用 {{DICTIONARY}} 指定个人词典位置;系统规则会自动追加。";
"mac.section.settings" = "设置";
@@ -261,8 +263,11 @@
"mac.settings.translationOff" = "不翻译";
"mac.settings.volcengineAppId" = "APP ID";
"mac.settings.volcengineAccessToken" = "Access Token";
"mac.settings.volcengineResourceId" = "Resource ID";
"mac.settings.volcengineNote" = "Secret Key 当前无需填写。Resource ID 默认使用 volc.seedasr.sauc.duration。";
"mac.settings.volcengineApiKey" = "API Key";
"mac.settings.volcengineApiKeyMode" = "使用新版 API Key 鉴权";
"mac.settings.volcengineApiKeyModeSubtitle" = "新控制台请打开;已有 AppID + Token 可保持关闭。";
"mac.settings.volcengineNoteAppToken" = "旧版控制台:填写 APP ID 与 Access Token。Secret Key 无需填写。识别资源固定为豆包流式 2.0volc.seedasr.sauc.duration)。";
"mac.settings.volcengineNoteApiKey" = "新版控制台:只需填写 API Key。识别资源固定为豆包流式 2.0volc.seedasr.sauc.duration)。";
"mac.settings.recognition" = "识别方式";
"mac.settings.cloudEngine" = "云端引擎与 AI 润色";
"mac.settings.cloudEngineDesc" = "通过服务商 API 进行高质量转写,并自动润色语法与风格。";
+67 -2
View File
@@ -130,7 +130,10 @@ final class CloudASRTests: XCTestCase {
let fields = VolcengineASRFields.parse(apiKey: json, resourceFallback: "")
XCTAssertEqual(fields.appID, "app-1")
XCTAssertEqual(fields.accessToken, "tok-2")
XCTAssertEqual(fields.resourceID, "res-3")
XCTAssertEqual(fields.authMode, .appToken)
// Custom resource IDs are ignored; product is locked to SAUC 2.0 duration.
XCTAssertEqual(fields.resourceID, VolcengineASRFields.fixedResourceID)
XCTAssertTrue(fields.hasUsableCredentials)
}
func testVolcengineASRFieldsColonParsing() {
@@ -140,8 +143,70 @@ final class CloudASRTests: XCTestCase {
)
XCTAssertEqual(fields.appID, "app-1")
XCTAssertEqual(fields.accessToken, "tok-2")
XCTAssertEqual(fields.resourceID, "res-3")
XCTAssertEqual(fields.authMode, .appToken)
XCTAssertEqual(fields.resourceID, VolcengineASRFields.fixedResourceID)
XCTAssertTrue(fields.encodedAPIKey.contains("app-1"))
XCTAssertTrue(fields.encodedAPIKey.contains("auth_mode"))
}
func testVolcengineASRFieldsAPIKeyModeParsing() {
let json = #"{"auth_mode":"api_key","api_key":"vk-new-console"}"#
let fields = VolcengineASRFields.parse(apiKey: json)
XCTAssertEqual(fields.authMode, .apiKey)
XCTAssertEqual(fields.apiKeyCredential, "vk-new-console")
XCTAssertTrue(fields.hasUsableCredentials)
XCTAssertEqual(fields.resourceID, VolcengineASRFields.fixedResourceID)
var request = URLRequest(url: URL(string: "wss://example.invalid")!)
fields.applyWebSocketAuthHeaders(to: &request, connectID: "conn-1")
XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-Key"), "vk-new-console")
XCTAssertNil(request.value(forHTTPHeaderField: "X-Api-App-Key"))
XCTAssertNil(request.value(forHTTPHeaderField: "X-Api-Access-Key"))
XCTAssertEqual(
request.value(forHTTPHeaderField: "X-Api-Resource-Id"),
VolcengineASRFields.fixedResourceID
)
XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-Connect-Id"), "conn-1")
}
func testVolcengineASRFieldsAppTokenHeaders() {
let fields = VolcengineASRFields(
authMode: .appToken,
appID: "app-1",
accessToken: "tok-2"
)
var request = URLRequest(url: URL(string: "wss://example.invalid")!)
fields.applyWebSocketAuthHeaders(to: &request, connectID: "conn-2")
XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-App-Key"), "app-1")
XCTAssertEqual(request.value(forHTTPHeaderField: "X-Api-Access-Key"), "tok-2")
XCTAssertNil(request.value(forHTTPHeaderField: "X-Api-Key"))
XCTAssertEqual(
request.value(forHTTPHeaderField: "X-Api-Resource-Id"),
VolcengineASRFields.fixedResourceID
)
}
func testVolcengineASRFieldsTogglePreservesBothCredentialSets() {
var fields = VolcengineASRFields(
authMode: .appToken,
appID: "app-1",
accessToken: "tok-2",
apiKeyCredential: "vk-keep"
)
fields.authMode = .apiKey
let encoded = fields.encodedAPIKey
let parsed = VolcengineASRFields.parse(apiKey: encoded)
XCTAssertEqual(parsed.authMode, .apiKey)
XCTAssertEqual(parsed.apiKeyCredential, "vk-keep")
XCTAssertEqual(parsed.appID, "app-1")
XCTAssertEqual(parsed.accessToken, "tok-2")
}
func testVolcengineASRFieldsEmptyAPIKeyModeIsNotUsable() {
let json = #"{"auth_mode":"api_key"}"#
let fields = VolcengineASRFields.parse(apiKey: json)
XCTAssertEqual(fields.authMode, .apiKey)
XCTAssertFalse(fields.hasUsableCredentials)
}
func testPersonalDictionaryASRHotwordsDedupesTerms() {
@@ -42,6 +42,90 @@ final class FlowKeyboardPoliciesTests: XCTestCase {
XCTAssertTrue(warming)
}
func testRecentReadyGraceDoesNotForcePreparingSession() {
// withinReadyGrace used to OR into warming and flash yellow after each
// utterance; sticky ready must hold green instead.
let warming = FlowKeyboardHostWarming.isHostWarming(
hostReady: false,
hostBusy: false,
sessionActive: true,
hostReachable: false,
isPendingFlowStart: false,
withinReadyGrace: true,
snapshotReason: .starting
)
// Still warming via reason=.starting when not yet proven ready on the
// keyboard side; grace alone must not be the trigger.
XCTAssertTrue(warming)
XCTAssertTrue(
FlowKeyboardHostWarming.shouldHoldReady(
hostReady: false,
hostBusy: false,
sessionActive: true,
sessionProvenReady: true,
isPendingFlowStart: false,
snapshotReason: .starting
),
"proven-ready session must hold green through starting flaps"
)
XCTAssertTrue(
FlowKeyboardHostWarming.shouldHoldReady(
hostReady: false,
hostBusy: true,
sessionActive: true,
sessionProvenReady: true,
isPendingFlowStart: false,
snapshotReason: .awaitingDelivery
),
"ack lag awaitingDelivery must not drop to preparingSession"
)
XCTAssertFalse(
FlowKeyboardHostWarming.shouldHoldReady(
hostReady: false,
hostBusy: true,
sessionActive: true,
sessionProvenReady: true,
isPendingFlowStart: false,
snapshotReason: .recording
),
"live recording must not be masked as sticky ready"
)
XCTAssertFalse(
FlowKeyboardHostWarming.shouldHoldReady(
hostReady: false,
hostBusy: false,
sessionActive: true,
sessionProvenReady: false,
isPendingFlowStart: true,
snapshotReason: .starting
),
"cold start pending must still show preparing"
)
}
func testHeldReadySuppressesWarming() {
let hold = FlowKeyboardHostWarming.shouldHoldReady(
hostReady: false,
hostBusy: false,
sessionActive: true,
sessionProvenReady: true,
isPendingFlowStart: false,
snapshotReason: .starting
)
XCTAssertTrue(hold)
let warming = FlowKeyboardHostWarming.isHostWarming(
hostReady: true, // effective ready after hold
hostBusy: false,
sessionActive: true,
hostReachable: true,
isPendingFlowStart: false,
withinReadyGrace: true,
snapshotReason: .starting
)
XCTAssertFalse(warming)
}
// MARK: - Adopt busy
func testReAdoptsRecordingAfterExtensionProcessLoss() {
@@ -220,4 +220,14 @@ final class FlowReliabilityTests: XCTestCase {
)
)
}
func testCaptureVoiceProcessingUsesSpeechOrientedSessionMode() {
XCTAssertEqual(FlowCaptureVoiceProcessing.captureMode, .voiceChat)
XCTAssertTrue(
FlowCaptureVoiceProcessing.captureOptions.contains(.allowBluetoothHFP)
)
XCTAssertTrue(
FlowCaptureVoiceProcessing.captureOptions.contains(.defaultToSpeaker)
)
}
}
@@ -428,6 +428,41 @@ final class FlowSessionBridgeTests: XCTestCase {
)
}
func testHostHeavyClearsOnFlowStateReset() {
let defaults = makeDefaults()
FlowSessionBridge.setHostHeavy(true, defaults: defaults)
XCTAssertTrue(FlowSessionBridge.isHostHeavy(defaults: defaults))
FlowSessionBridge.clearFlowState(defaults: defaults)
XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults))
FlowSessionBridge.setHostHeavy(true, defaults: defaults)
FlowSessionBridge.setPendingHostBundleId("com.example.host", defaults: defaults)
FlowSessionBridge.clearFlowStateOnHostLaunch(defaults: defaults)
XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults))
XCTAssertEqual(
FlowSessionBridge.pendingHostBundleId(defaults: defaults),
"com.example.host"
)
}
func testStaleHostHeavyDoesNotBlockTyping() {
let defaults = makeDefaults()
// Legacy sticky bool with no timestamp must not brick /EN.
defaults.set(true, forKey: FlowSessionKeys.hostHeavy)
XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults))
XCTAssertFalse(defaults.bool(forKey: FlowSessionKeys.hostHeavy))
FlowSessionBridge.setHostHeavy(true, defaults: defaults)
XCTAssertTrue(FlowSessionBridge.isHostHeavy(defaults: defaults))
// Expired timestamp treat as clear so cold keyboard can switch.
let expired = Date().timeIntervalSince1970 - FlowSessionKeys.hostHeavyMaxAge - 1
defaults.set(expired, forKey: FlowSessionKeys.hostHeavyAt)
XCTAssertFalse(FlowSessionBridge.isHostHeavy(defaults: defaults))
XCTAssertFalse(defaults.bool(forKey: FlowSessionKeys.hostHeavy))
}
func testRotateHostGenerationReturnsPreviousToken() {
let defaults = makeDefaults()
XCTAssertNil(FlowSessionBridge.rotateHostGeneration(defaults: defaults))
@@ -340,6 +340,54 @@ final class IntelligentPolishTests: XCTestCase {
XCTAssertTrue(result.contains("完成"))
}
func testPolishServiceKeepsAddedEmojiWhenStyleAllows() async throws {
store.setEngineMode("local")
var catalog = PolishStyleCatalog()
let pack = PolishStylePack(
id: "user.emoji",
name: "Emoji",
prompt: "保持口语,可按情绪加 emoji。",
allowsAddedEmoji: true
)
try catalog.upsert(pack)
store.setPolishStyleCatalog(catalog)
store.setActivePolishStyleId(pack.id)
let emojiClient = FixedResponseLLMClient(response: "今天太开心了,终于搞定了😆")
let service = PolishingService(store: store, client: emojiClient)
let result = try await service.polish(
"今天太开心了终于搞定了",
context: PolishContext()
)
XCTAssertTrue(result.contains("😆"), "Allowed-emoji styles must keep model-added emoji. Got: \(result)")
XCTAssertTrue(result.contains("开心"))
}
func testPolishServiceKeepsAddedEmojiWhenPromptOptsInWithoutToggle() async throws {
store.setEngineMode("local")
var catalog = PolishStyleCatalog()
let pack = PolishStylePack(
id: "user.paste-emoji",
name: "PasteEmoji",
prompt: "本风格允许新增 emoji。按情绪点缀合适表情。",
allowsAddedEmoji: false
)
try catalog.upsert(pack)
store.setPolishStyleCatalog(catalog)
store.setActivePolishStyleId(pack.id)
let emojiClient = FixedResponseLLMClient(response: "辛苦你了,真的谢谢🙏")
let service = PolishingService(store: store, client: emojiClient)
let result = try await service.polish(
"辛苦你了真的谢谢",
context: PolishContext()
)
XCTAssertTrue(
result.contains("🙏"),
"Prompt opt-in must keep emoji even when toggle is off. Got: \(result)"
)
}
func testPolishServiceFallsBackWhenOutputEmpty() async throws {
store.setEngineMode("local")
let emptyClient = FixedResponseLLMClient(response: " ")
@@ -410,6 +458,30 @@ final class IntelligentPolishTests: XCTestCase {
XCTAssertEqual(result, "好的")
}
func testQualityGateKeepsAddedEmojiWhenAllowed() {
let decision = TranscriptPostProcessor.qualityGate(
original: "今天太开心了",
candidate: "今天太开心了😆",
allowsAddedEmoji: true
)
guard case .accept(let text) = decision else {
return XCTFail("Expected accept")
}
XCTAssertTrue(text.contains("😆"))
}
func testQualityGateStripsAddedEmojiByDefault() {
let decision = TranscriptPostProcessor.qualityGate(
original: "今天太开心了",
candidate: "今天太开心了😆",
allowsAddedEmoji: false
)
guard case .accept(let text) = decision else {
return XCTFail("Expected accept")
}
XCTAssertFalse(text.contains("😆"))
}
func testQualityGateStripsResidualPauseMarkers() {
let result = TranscriptPostProcessor.process(
original: "第一段 ⟨0.8s⟩ 第二段",
@@ -206,6 +206,90 @@ final class PolishStylePackTests: XCTestCase {
XCTAssertTrue(prompt.contains("词典命中优先于同音猜测"))
XCTAssertTrue(prompt.contains("风格接入(纠错之后)"))
XCTAssertFalse(prompt.contains("原始内容"))
XCTAssertFalse(prompt.contains("Emoji 覆盖"))
}
func testComposerInjectsEmojiOverrideWhenStyleAllows() {
let style = PolishStylePack(
id: "user.emoji",
name: "Emoji",
prompt: "ROLE",
allowsAddedEmoji: true
)
let prompt = PolishPromptComposer.compose(
text: "今天太开心了",
style: style,
context: PolishContext(),
dictionaryBlock: "",
useChineseGuidance: true
)
XCTAssertTrue(prompt.contains("Emoji 覆盖(本风格开启 · 最终优先级)"))
XCTAssertTrue(prompt.contains("优先级高于全局 R5"))
// Override must appear after core R5 so it wins.
let r5 = prompt.range(of: "R5 不新增 emoji")
let override = prompt.range(of: "Emoji 覆盖(本风格开启 · 最终优先级)")
XCTAssertNotNil(r5)
XCTAssertNotNil(override)
XCTAssertLessThan(r5!.lowerBound, override!.lowerBound)
}
func testPromptOptInEnablesEmojiWithoutToggle() async throws {
let style = PolishStylePack(
id: "user.paste",
name: "Paste",
prompt: """
#
emoji emoji
""",
allowsAddedEmoji: false
)
XCTAssertTrue(style.effectiveAllowsAddedEmoji)
let composed = PolishPromptComposer.compose(
text: "今天太开心了",
style: style,
context: PolishContext(),
dictionaryBlock: "",
useChineseGuidance: true
)
XCTAssertTrue(composed.contains("Emoji 覆盖(本风格开启 · 最终优先级)"))
}
func testLegacyPackDecodeDefaultsAllowsAddedEmojiToFalse() throws {
let encoder = JSONEncoder()
encoder.dateEncodingStrategy = .iso8601
// Encode without the new key by decoding a minimal legacy payload.
let legacyJSON = """
{
"id": "user.legacy",
"name": "Legacy",
"prompt": "ROLE",
"kind": "user",
"createdAt": 0,
"updatedAt": 0
}
"""
let decoder = JSONDecoder()
decoder.dateDecodingStrategy = .secondsSince1970
let pack = try decoder.decode(PolishStylePack.self, from: Data(legacyJSON.utf8))
XCTAssertFalse(pack.allowsAddedEmoji)
}
func testUpsertPreservesAllowsAddedEmoji() throws {
var catalog = PolishStyleCatalog()
let pack = PolishStylePack(
name: "Emoji",
prompt: "ROLE",
allowsAddedEmoji: true
)
try catalog.upsert(pack)
XCTAssertEqual(catalog.entries.first?.allowsAddedEmoji, true)
var updated = pack
updated.prompt = "ROLE 2"
try catalog.upsert(updated)
XCTAssertEqual(catalog.entries.first?.allowsAddedEmoji, true)
XCTAssertEqual(catalog.entries.first?.prompt, "ROLE 2")
}
func testComposerKeepsHomophoneRepairWhenDictionaryPresent() {
+4 -1
View File
@@ -1,8 +1,9 @@
# 打字键盘输入准确率提升计划
> **文档状态**:产品与工程规划(已讨论对齐,待进入实现
> **文档状态**:产品与工程规划(**实现中**:Phase 1~4 已落地,待真机试打
> **适用范围**iOS 键盘扩展打字面(`OSGKeyboardExt` 打字键网格)+ 共享布局度量(`OSGKeyboardShared`
> **关联基线**`docs/TYPING_KEYBOARD_BASELINE.md`
> **实现分支**`feat/keyboard-touch-accuracy`
> **创建日期**2026-08-03
> **目标版本**:待 Phase 0 基线试打后确定发布节奏
@@ -345,3 +346,5 @@ Week 34 Phase 5 气泡等加分项(可选)
| 日期 | 说明 |
|------|------|
| 2026-08-03 | 初稿:基于系统键盘分层分析与产品讨论冻结 Phase 0~5 |
| 2026-08-05 | 开分支 `feat/keyboard-touch-accuracy`;落地 Phase 1(无死区)+ Phase 2(网格 Down/Move/Up+ Phase 3(触点上偏);纯逻辑见 `KeyHitTesting` / `TypingKeyLayout`,触控见 `TypingKeyTouchPad` |
| 2026-08-05 | Phase 4:全拼合法下一键偏心(`PinyinNextKeyResolver` + `rawInput`);双拼/英文保持中性;歧义最近键加权,单键明确命中不受偏置 |
+4 -1
View File
@@ -13,7 +13,10 @@ wiring; jetsam behavior is device-only.
- **Do not** stack CLM + Rime + ASR in the same second after onboarding.
- ASR warmup runs on **first mic press** (`beginUtterance`), gated by
`HostMemoryBudget` (~260 MB RSS). `hostHeavy` is set only while heavy work
actually runs, then cleared.
actually runs, then cleared. A sticky `hostHeavy` (host died mid-work)
expires after `hostHeavyMaxAge` (~120 s) and is cleared on
`clearFlowState` / host-launch reconciliation so typing 中文/EN is not
permanently blocked.
## Console checklist
+2 -2
View File
@@ -51,8 +51,8 @@ settings:
ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES
STRING_CATALOG_GENERATE_SYMBOLS: YES
CLANG_CXX_LANGUAGE_STANDARD: c++17
MARKETING_VERSION: "1.6.0"
CURRENT_PROJECT_VERSION: "44"
MARKETING_VERSION: "1.6.2"
CURRENT_PROJECT_VERSION: "47"
# 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖)
# 项目级签名 xcconfig,适用于所有 target