feat(keyboard): improve typing, voice flow, and polish reliability
Reduce extension memory pressure and delivery races while adding richer candidates, tactile feedback, and safer two-level creative polishing.
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
# Flow Bluetooth HFP release gate
|
||||
|
||||
Run this checklist on a physical iPhone before every release that changes Flow,
|
||||
PiP, ASR warmup, or audio-session code. Simulator audio is not an HFP substitute.
|
||||
|
||||
## Required devices
|
||||
|
||||
- Built-in iPhone microphone
|
||||
- One Bluetooth HFP headset (AirPods or equivalent)
|
||||
|
||||
## Pass criteria
|
||||
|
||||
1. Start a PiP Flow session and record 50 consecutive utterances with the
|
||||
built-in microphone.
|
||||
2. Repeat 50 utterances with the Bluetooth headset already connected.
|
||||
3. Switch Prompt in the host app, return to the original text field, and record
|
||||
20 more Bluetooth utterances.
|
||||
4. During recording, connect and disconnect the headset once. The app must
|
||||
recover automatically before speech starts, or stop with an explicit
|
||||
audio-device-changed error after speech has started.
|
||||
5. Recreate the keyboard extension once while processing and once after the
|
||||
Host writes the final result.
|
||||
|
||||
The run fails if any of the following occurs:
|
||||
|
||||
- `-10868` or `formats don't match`
|
||||
- `capture.engine.startFailed`
|
||||
- A completed utterance reports zero input frames
|
||||
- `host.delivered` has no matching `keyboard.insert`
|
||||
- The same utterance is inserted more than once
|
||||
- A background task exceeds 30 seconds
|
||||
- AVAudioSession activation blocks the main thread
|
||||
|
||||
Archive the filtered `[trace]` log with the release test record.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Testing
|
||||
|
||||
OSGKeyboard keeps **one hermetic XCTest suite**, organized so you can run **all** or **a subset** without duplicating cases.
|
||||
|
||||
## Single source of truth
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| [`Tests/suite-manifest.json`](../Tests/suite-manifest.json) | Atomic groups + presets. Each `*Tests` class appears in **exactly one** group. |
|
||||
| [`Scripts/resolve_test_suite.py`](../Scripts/resolve_test_suite.py) | Expand presets, validate membership vs on-disk files. |
|
||||
| [`Scripts/run-tests.sh`](../Scripts/run-tests.sh) | Resolve → `xcodebuild test -only-testing:…` (iOS and/or Mac). |
|
||||
|
||||
Helpers such as `FakeUbiquitousKeyValueStore.swift` are not listed (they are not XCTest classes).
|
||||
|
||||
## Quick start (Mac + Xcode)
|
||||
|
||||
```bash
|
||||
./Scripts/generate-xcodeproj.sh # once / after project.yml changes
|
||||
./Scripts/run-tests.sh list # presets + groups
|
||||
./Scripts/run-tests.sh validate # every on-disk *Tests.swift is in the manifest once
|
||||
./Scripts/run-tests.sh pr # default CI / PR gate
|
||||
```
|
||||
|
||||
### Presets
|
||||
|
||||
| Preset | Contains | When to use |
|
||||
|--------|----------|-------------|
|
||||
| `pr` | config, sync, polish, cloud_asr, local_asr, utterance, flow, keyboard | PR / default CI |
|
||||
| `all` | `pr` + `host_misc` + `pipeline_perf` | Full iOS+Ext hermetic run before release |
|
||||
| `api` | cloud_asr, polish | Online API contracts + polish/LLM stubs |
|
||||
| `asr` | cloud_asr, local_asr, utterance | Transcription path |
|
||||
| `polish` | polish | Polish only |
|
||||
| `keyboard` | keyboard, **flow** | Keyboard + handoff/mic |
|
||||
| `flow` | flow | Flow session only |
|
||||
| `sync` | sync | iCloud sync only |
|
||||
| `perf` | pipeline_perf | Hermetic voice→polish stage timings (stub ASR/LLM) |
|
||||
| `mac` | mac | macOS host only (separate scheme) |
|
||||
|
||||
`live_api` is reserved for optional live-network smoke and is **empty / excluded** from `all` and `pr` by design.
|
||||
|
||||
### Atomic groups or mixes
|
||||
|
||||
```bash
|
||||
./Scripts/run-tests.sh polish
|
||||
./Scripts/run-tests.sh cloud_asr utterance
|
||||
./Scripts/run-tests.sh perf
|
||||
./Scripts/run-tests.sh api keyboard
|
||||
DRY_RUN=1 ./Scripts/run-tests.sh all
|
||||
```
|
||||
|
||||
### Environment
|
||||
|
||||
| Variable | Meaning |
|
||||
|----------|---------|
|
||||
| `DESTINATION` | iOS Simulator destination (default: `platform=iOS Simulator,name=iPhone 17`) |
|
||||
| `MAC_DESTINATION` | macOS destination (default: `platform=macOS`) |
|
||||
| `CONFIGURATION` | `Debug` (default) or `Release` |
|
||||
| `DRY_RUN=1` | Print `xcodebuild` only |
|
||||
| `SKIP_GENERATE=1` | Do not call `generate-xcodeproj.sh` even if the project is missing |
|
||||
|
||||
## CI
|
||||
|
||||
`.github/workflows/ci.yml` runs `./Scripts/run-tests.sh pr` (critical path, including `OSGKeyboardExtTests`).
|
||||
|
||||
## Adding a new test
|
||||
|
||||
1. Add the XCTest class under the correct target folder (`OSGKeyboardTests`, `OSGKeyboardExtTests`, or `OSGKeyboardMacTests`).
|
||||
2. Register it in **exactly one** group in `Tests/suite-manifest.json`.
|
||||
3. Run `./Scripts/run-tests.sh validate`.
|
||||
4. Prefer extending an existing group; only add a new group when the domain is genuinely new.
|
||||
|
||||
## Layer note (avoid false “duplicates”)
|
||||
|
||||
- `ChunkedUtterancePipelineTests` (HostSupport pipeline orchestration) and `FinalChunkRecoveryTests` (Ext short/empty final recovery) cover **different layers** — both stay, both live in `utterance`.
|
||||
- Cloud ASR **runtime** WebSocket clients and `live_api` smoke are future work inside `cloud_asr` / `live_api`, not a second parallel suite.
|
||||
- Flow keyboard mic regressions (orange stuck / jetsam re-adopt / command seq) are covered by `FlowKeyboardPolicies` helpers in the `flow` group.
|
||||
- Streaming provider event JSON / Volcengine frames are covered by `CloudASRStreamingEventParsingTests` in `cloud_asr`.
|
||||
- Translation chip App Group poll clobber is covered by `KeyboardTranslationConfigProtectionTests` in `keyboard`.
|
||||
|
||||
## Pipeline performance (`perf` / `pipeline_perf`)
|
||||
|
||||
Hermetic **voice → chunk ASR → transcript guard → (optional batch fallback) → polish → App Group bridge deliver** timings live in `VoicePipelinePerformanceTests`.
|
||||
|
||||
- Synthetic PCM + stub ASR/LLM only — **no mic, no live network**.
|
||||
- Each run attaches a stage report (`pcm_feed`, `chunk_asr`, `transcript_guard`, `batch_fallback?`, `polish`, `bridge_deliver`, `total_e2e`) to the xcresult.
|
||||
- Run alone: `./Scripts/run-tests.sh perf`. Included in `all`, **not** in `pr` (keeps the PR gate free of timing-sensitive ceilings).
|
||||
|
||||
These measure harness/orchestration cost under stubs — not on-device SpeechAnalyzer or real LLM latency.
|
||||
|
||||
## Physical-device Flow audio gate
|
||||
|
||||
Bluetooth HFP and PiP route transitions cannot be validated by the simulator.
|
||||
Before release, run [FLOW_BLUETOOTH_TESTING.md](FLOW_BLUETOOTH_TESTING.md) and
|
||||
retain the filtered trace log with the release record.
|
||||
|
||||
## Manual / non-XCTest checklists
|
||||
|
||||
Some product surfaces still use manual docs (StoreKit, personal-dictionary iCloud, etc.). Those are complementary; they are not duplicated into XCTest.
|
||||
@@ -0,0 +1,347 @@
|
||||
# 打字键盘输入准确率提升计划
|
||||
|
||||
> **文档状态**:产品与工程规划(已讨论对齐,待进入实现)
|
||||
> **适用范围**:iOS 键盘扩展打字面(`OSGKeyboardExt` 打字键网格)+ 共享布局度量(`OSGKeyboardShared`)
|
||||
> **关联基线**:`docs/TYPING_KEYBOARD_BASELINE.md`
|
||||
> **创建日期**:2026-08-03
|
||||
> **目标版本**:待 Phase 0 基线试打后确定发布节奏
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
### 1.1 目标
|
||||
|
||||
在拼音打字主路径上,将点按准确率与按键契约提升到**接近系统键盘的日常体感**:
|
||||
|
||||
- 漏触(按了没反应)接近 0
|
||||
- 邻键误触明显下降
|
||||
- 「不准」不再成为用户槽点
|
||||
|
||||
**成功定义(产品语言)**:同设备、同姿势对比系统键盘时,内部试打主观分接近;键缝无空响应;字母键支持预览与滑动改选。
|
||||
|
||||
### 1.2 核心结论(冻结)
|
||||
|
||||
| 决策 | 选择 |
|
||||
|------|------|
|
||||
| 准确率策略 | 对齐系统**触控契约 + 分层纠错**,不复刻私有算法 |
|
||||
| 字母 / 数字 / 符号 | 按下高亮 → 可滑动改选 → **松手确认** |
|
||||
| 删除 | **按下即删** + 长按连删(与现状及系统一致) |
|
||||
| 空格 / 回车 | 与字母一致:**松手确认** |
|
||||
| 键缝 | 视觉保留间距;触控**不允许无响应**(最近键中心) |
|
||||
| 滑出键平面再松手 | **取消本次输入**(不出字) |
|
||||
| 视觉 | 键位外观不因热区/智能变形(热区隐形) |
|
||||
| 智能边界 | 只微调;用户明确落点或滑动可覆盖 |
|
||||
| 语言策略 | 中文拼音过程可偏心;英文先求稳 |
|
||||
| 反馈最低标准 | Phase 1~2:**高亮 + 按键音**;气泡为加分项 |
|
||||
| 实施顺序 | 无死区 → 按键契约 → 触点微调 → 拼音偏心 → 气泡等 |
|
||||
|
||||
### 1.3 非目标(本期不做)
|
||||
|
||||
- 绝对准确率 100% 追平系统键盘
|
||||
- QuickPath / 滑行输入
|
||||
- 依赖 `UIKeyboardLayoutStar` 等系统私有 hit API
|
||||
- 完整双 strike zone 个性化、键盘整体漂移重定位(re-spotting)
|
||||
- 英文词级 predictive hit 放大(后置评估)
|
||||
- 重写 Rime / 抛光引擎(触控层与词级兜底分工,不互相替代)
|
||||
|
||||
### 1.4 现实预期
|
||||
|
||||
| 目标 | 是否现实 |
|
||||
|------|----------|
|
||||
| 漏触接近系统 | ✅ Phase 1~2 可达 |
|
||||
| 邻键误触主观接近系统 | ✅ Phase 2~4 可达 |
|
||||
| 中文拼音连打体感不输系统 | ✅ 相对最有机会(拼音偏心) |
|
||||
| 全场景绝对准确率 = 系统 | ❌ 不作为本期 KPI |
|
||||
|
||||
---
|
||||
|
||||
## 2. 背景与现状差距
|
||||
|
||||
### 2.1 问题定义(第一性原理)
|
||||
|
||||
用户打字时只关心两件事:
|
||||
|
||||
1. **我以为我点了谁**(视觉反馈)
|
||||
2. **机器最后认了谁**(实际输入)
|
||||
|
||||
系统键盘准,是因为整条链路在缩小二者差距,并在不一致时提供改选与词级兜底。OSG 当前布局已对齐系统键位,但触控模型仍接近「普通按钮网格」。
|
||||
|
||||
### 2.2 系统键盘能力分层(参考模型)
|
||||
|
||||
系统做法是多层流水线(专利 + 私有头文件命名 + 可观察行为),不是单一「热区 +N pt」:
|
||||
|
||||
| 层 | 含义 | 本期态度 |
|
||||
|----|------|----------|
|
||||
| ① 触点估计 | 触点中心 ≠ 意图点,可做偏移/加权 | Phase 3 轻量做 |
|
||||
| ② 几何热区 | 视觉 ≠ hit;缝可命中 | Phase 1 必做 |
|
||||
| ③ 动态偏心 | 下一键更可能则更好点 | Phase 4 拼音合法优先 |
|
||||
| ④ 触摸生命周期 | Down 反馈 / Move 改选 / Up 提交 | Phase 2 必做 |
|
||||
| ⑤ 个性化双区 | 按节奏/习惯自适应 | 后置 |
|
||||
| ⑥ 词级兜底 | 自动更正 / 候选 | 已有 Rime + 抛光,需保住 |
|
||||
|
||||
### 2.3 现状(代码事实)
|
||||
|
||||
| 区域 | 现状 | 影响 |
|
||||
|------|------|------|
|
||||
| 字符键 | SwiftUI `Button`,视觉 frame ≈ hit | 键间 6~8pt 为死区;松手且需仍在键内 |
|
||||
| 删除 / 部分工具栏 | `DragGesture(minimumDistance: 0)` 按下即响 | 已接近系统删除契约 |
|
||||
| 主动扩大 hit | 基本仅候选栏展开按钮(34 视觉 / 44 hit) | 字母区无 fat-finger 扩展 |
|
||||
| 滑动改选 | 无行级/网格级跟踪 | 邻键误触难在松手前纠正 |
|
||||
| 点击音 | 字符键未统一走 `KeyboardSoundFeedback` | 反馈弱于系统 |
|
||||
| 词级 | Rime 候选 + 抛光 | 后段较强;前段触控不稳会浪费兜底 |
|
||||
|
||||
主要相关位置:
|
||||
|
||||
- `OSGKeyboardExt/Typing/TypingRootView.swift`(键网格、间距、`keyButton`)
|
||||
- `OSGKeyboardExt/Views/ToolbarActionButtons.swift`(按下即响模板)
|
||||
- `OSGKeyboardExt/Utilities/KeyboardSoundFeedback.swift`
|
||||
- `OSGKeyboardShared/Typing/TypingLayoutProviding.swift`(键位布局)
|
||||
|
||||
### 2.4 用户失败模式
|
||||
|
||||
| 体感 | 主因 | 主要靠哪一层治 |
|
||||
|------|------|----------------|
|
||||
| 点到缝没反应 | 间距死区 | Phase 1 |
|
||||
| 经常打成邻键 | 无改选窗口 + 无触点/上下文加权 | Phase 2~4 |
|
||||
| 滑出键外丢输入 | per-key Button 生命周期 | Phase 2 |
|
||||
|
||||
---
|
||||
|
||||
## 3. 产品宪法(全程遵守)
|
||||
|
||||
1. **字母 / 数字 / 符号**:按下高亮 → 可滑动改选 → **松手确认**
|
||||
2. **删除**:**按下即删**,长按连删
|
||||
3. **键平面内不允许无响应**(缝必命中最近键中心)
|
||||
4. **视觉键位不因智能变形**(热区与偏心均隐形)
|
||||
5. **智能只微调**;用户滑动或明确落点可覆盖
|
||||
6. **中文拼音过程可以更聪明;英文先求稳**
|
||||
|
||||
对内沟通口径(Phase 2):
|
||||
|
||||
> 不是变慢,是把确认点从按下挪到松开,换来改选能力——和系统键盘一样。
|
||||
|
||||
---
|
||||
|
||||
## 4. 分阶段计划
|
||||
|
||||
### Phase 0 — 基线与验收设计
|
||||
|
||||
**目的**:能量化「准不准」,避免凭感觉争论。
|
||||
|
||||
| 事项 | 说明 |
|
||||
|------|------|
|
||||
| 记录现状问题 | 漏触 / 邻键错 / 滑出丢键的典型场景 |
|
||||
| 定试打脚本 | 至少 3 类:缝点击、邻键偏触、常见拼音连打(如 `zhongguo`、`pingguo`) |
|
||||
| 定对比方式 | 同设备、同姿势;系统键盘 vs OSG;主观 1~5 分 + 错字/漏触次数 |
|
||||
| 定反馈最低标准 | Phase 1~2:高亮 + 按键音;气泡单列加分 |
|
||||
|
||||
**工期**:0.5~1 天
|
||||
**产出**:试打清单 + 打分表(可放飞书/Notion;关键结论可回写本文「附录」)
|
||||
|
||||
**出口**:脚本固定、打分表可用,再进入开发。
|
||||
|
||||
---
|
||||
|
||||
### Phase 1 — 无死区热区(P0)
|
||||
|
||||
**产品一句话**:缝也算键,别点空。
|
||||
|
||||
**范围**
|
||||
|
||||
- 字母 / 数字 / 符号区:视觉间距保留,触控无空洞
|
||||
- 缝内命中:**最近键中心**
|
||||
- 边缘键(如 Q/P、A/L)外侧略扩,降低边缘难点
|
||||
- 不改变键的视觉大小与间距观感
|
||||
|
||||
**工期**:可与 Phase 2 合并开发,但**验收分开**
|
||||
**建议**:约短迭代内完成几何命中;若与 Phase 2 同 PR,须分 commit 或分验收项
|
||||
|
||||
**验收**
|
||||
|
||||
- [ ] 故意点键缝 20 次,0 次无响应
|
||||
- [ ] 视觉观感与改前一致
|
||||
- [ ] 无明显「看在 A 上却出 S」的跳键
|
||||
|
||||
---
|
||||
|
||||
### Phase 2 — 系统同款按键契约(P0)
|
||||
|
||||
**产品一句话**:先告诉你按了谁,松手再算数。
|
||||
|
||||
**范围**
|
||||
|
||||
| 键类 | 行为 |
|
||||
|------|------|
|
||||
| 字母 / 数字 / 符号 | Down 高亮 → Move 改选 → Up 提交 |
|
||||
| 删除 | Down 即删 + 长按连删(保持) |
|
||||
| 空格 / 回车 | 松手确认 |
|
||||
| 滑出键平面再松手 | 取消本次输入 |
|
||||
| 反馈 | 高亮 + `KeyboardSoundFeedback.keyClick()` |
|
||||
|
||||
**本阶段不做**:按键气泡、长按附加符号、滑行输入。
|
||||
|
||||
**工期**:约 1 个迭代(含内部试打调参)
|
||||
|
||||
**验收**
|
||||
|
||||
- [ ] 按下 F 滑到 G 松开 → 只出 G,高亮跟随
|
||||
- [ ] 按下后滑出键盘再松 → 不出字
|
||||
- [ ] 删除仍为按下即删、可连删
|
||||
- [ ] 内部试打:「手感接近系统」主观分显著高于改前
|
||||
|
||||
**工程注意(规划级)**
|
||||
|
||||
- 宜收敛为网格/行级触摸跟踪,而非继续堆 per-key `Button`
|
||||
- 可复用 `RepeatingPressButton` / `CursorDragPad` 的 UIKit 触控经验,但字母键生命周期以 Up 确认为准
|
||||
- 需单列与候选栏、Shift、123 切换的手势边界用例
|
||||
|
||||
---
|
||||
|
||||
### Phase 3 — 触点微调(P1)
|
||||
|
||||
**产品一句话**:按手指意图认键,不只认触点中心。
|
||||
|
||||
**范围**
|
||||
|
||||
- 对命中点做轻微、稳定的向上偏移(或按触点大小微调)
|
||||
- 规则保持可预期;偏移量建议可配置便于试打
|
||||
- 中英文均可生效(与语言无关的几何修正)
|
||||
|
||||
**工期**:0.5~1 天(可挂在 Phase 2 尾声)
|
||||
|
||||
**验收**
|
||||
|
||||
- [ ] 拇指自然打字时,偏下误触主观减少
|
||||
- [ ] 不引入新的「明显认错键」投诉
|
||||
- [ ] 偏移可调,便于内测收敛
|
||||
|
||||
---
|
||||
|
||||
### Phase 4 — 拼音下一键偏心(P1)
|
||||
|
||||
**产品一句话**:拼到一半时,更可能的下一家更好点。
|
||||
|
||||
**范围(档位 1:合法字母优先)**
|
||||
|
||||
- 仅拼音组词过程中启用
|
||||
- 依据当前拼音/Rime 状态:合法后续字母热区略增,非法略减
|
||||
- **偏心有上限**(微调,不霸道)
|
||||
- 用户滑动改选 / 明确落点可覆盖
|
||||
- 英文模式:关闭或明显减弱
|
||||
- 上屏或开始新音节:偏心状态重置
|
||||
|
||||
**后置**:英文词预测式放大、重度概率模型、个性化学习。
|
||||
|
||||
**工期**:约 1 个迭代
|
||||
**依赖**:必须在 Phase 2 稳定之后(无改选窗口时,偏心误判会被感知为「自作主张」)
|
||||
|
||||
**验收**
|
||||
|
||||
- [ ] 常见拼音连打,邻键误触下降
|
||||
- [ ] 故意打冷门 / 非法后续字母,仍能打上
|
||||
- [ ] 视觉上看不出键变大;体感更顺
|
||||
- [ ] 可用试打脚本对比「偏心开 / 关」
|
||||
|
||||
---
|
||||
|
||||
### Phase 5 — 体验加分(P2,不阻塞主目标)
|
||||
|
||||
| 项 | 说明 | 优先级 |
|
||||
|----|------|--------|
|
||||
| 按键气泡预览 | 强化所见即所得 | 高(体验) |
|
||||
| 候选 / 抛光联动检查 | 触控改善后词级兜底不回退 | 高(质量) |
|
||||
| 长按附加符号 | 系统能力对齐,非准确率核心 | 中 |
|
||||
| 个性化热区 | 长期数据,后置 | 低 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 推荐排期
|
||||
|
||||
```text
|
||||
Week 0 Phase 0 基线试打脚本 + 打分
|
||||
Week 1 Phase 1+2 无死区 + 松手确认/滑动改选(可合并开发,分开验收)
|
||||
Week 1 末 Phase 3 触点微调
|
||||
Week 2 内部试打 / 调参 / 边界修复(滑出取消、宽键、Shift、候选冲突)
|
||||
Week 3 Phase 4 拼音合法下一键偏心
|
||||
Week 3~4 Phase 5 气泡等加分项(可选)
|
||||
```
|
||||
|
||||
**底线**:Week 1 必须交出 Phase 1 + 2;没有这两项,不宣称「接近系统准确率」。
|
||||
|
||||
---
|
||||
|
||||
## 6. 范围边界
|
||||
|
||||
| 做 | 不做(本项目) |
|
||||
|----|----------------|
|
||||
| 点按准确率与系统同款契约 | QuickPath |
|
||||
| 拼音过程热区偏心(合法优先) | 系统私有 hit API |
|
||||
| 高亮 + 按键音 | 完整自动更正引擎重写 |
|
||||
| 与现有 Rime / 候选协同 | 键盘整体自动平移 / 缩放 |
|
||||
| 可配置调参(偏移、偏心强度) | 为准确率牺牲视觉键位一致性 |
|
||||
|
||||
---
|
||||
|
||||
## 7. 风险与对策
|
||||
|
||||
| 风险 | 对策 |
|
||||
|------|------|
|
||||
| 「松手确认」被觉得变钝 | 对内对齐系统口径;用试打错字率对比说话 |
|
||||
| 热区重叠导致跳键 | 最近中心 + 偏心上限;内测可关偏心 |
|
||||
| 拼音偏心误伤冷门输入 | 合法优先 + 可覆盖;可选调试开关 |
|
||||
| 与删除 / 空格 / 候选手势冲突 | Phase 2 单列边界用例;滑出取消 |
|
||||
| 扩展内难自动化 UI 测 | Phase 0 人工脚本为主;命中几何/偏心纯逻辑补单测 |
|
||||
| 改动面集中在打字面 | 语音面工具栏契约保持;避免一次改两套手感却无验收 |
|
||||
|
||||
---
|
||||
|
||||
## 8. 角色与出口标准
|
||||
|
||||
| 角色 | 职责 |
|
||||
|------|------|
|
||||
| 产品 | 守宪法、主观验收、决定气泡是否进本期 |
|
||||
| 设计 | 高亮态;(可选)气泡视觉 |
|
||||
| 工程 | Phase 1~4 实现;关键常量可配置 |
|
||||
| 内测 | 每 Phase 用同一试打脚本回归 |
|
||||
|
||||
**每阶段出口**:试打清单通过 + 无 P0「无响应 / 乱出字」再进入下一阶段。
|
||||
|
||||
---
|
||||
|
||||
## 9. 与版本发布的关系
|
||||
|
||||
- 本能力属用户可感知的打字体验提升,合并 `main` 并准备发布时:
|
||||
- 按 `AGENTS.md` 用 Conventional Commits 评估版本 bump(预期含 `feat` → **MINOR**)
|
||||
- `CHANGELOG.md` 在 `[Unreleased]` 下补双语条目(实现时再写,不在本文提前编造版本号)
|
||||
- 功能分支开发期间**不**提前 bump `project.yml` 版本号
|
||||
|
||||
---
|
||||
|
||||
## 10. 一句话执行顺序
|
||||
|
||||
> **先量基线 → 无死区 + 系统按键契约 → 触点微调 → 拼音偏心 → 气泡等加分。**
|
||||
|
||||
---
|
||||
|
||||
## 附录 A. 试打脚本草案(Phase 0 细化)
|
||||
|
||||
| 编号 | 场景 | 操作 | 记录 |
|
||||
|------|------|------|------|
|
||||
| T1 | 键缝 | 在同行相邻键缝点击 20 次 | 无响应次数 |
|
||||
| T2 | 邻键偏触 | 瞄准某键但故意偏向邻键 20 次 | 错键次数;有/无滑动改选 |
|
||||
| T3 | 滑动改选 | 按下 A 滑到 S 松开(Phase 2+) | 是否只出 S |
|
||||
| T4 | 滑出取消 | 按下后滑出键盘松开(Phase 2+) | 是否不出字 |
|
||||
| T5 | 删除契约 | 点按 / 长按删除 | 是否按下即删、可连删 |
|
||||
| T6 | 拼音连打 | `zhongguo` / `pingguo` 等各 5 遍 | 错字次数、主观 1~5 |
|
||||
| T7 | 偏心开关 | Phase 4 后 T6 对比开/关 | 错字差、误伤冷门键与否 |
|
||||
|
||||
对比时尽量:同一 iPhone、竖屏、单手拇指、系统键盘与 OSG 各跑一轮。
|
||||
|
||||
---
|
||||
|
||||
## 附录 B. 修订记录
|
||||
|
||||
| 日期 | 说明 |
|
||||
|------|------|
|
||||
| 2026-08-03 | 初稿:基于系统键盘分层分析与产品讨论冻结 Phase 0~5 |
|
||||
@@ -0,0 +1,50 @@
|
||||
# Keyboard memory budget — acceptance (OSGDiag)
|
||||
|
||||
Phase 0 / 2 memory work is validated on a **physical iPhone** with Console
|
||||
filtering for `OSGDiag`. Unit tests and `xcodebuild` cover compile-time
|
||||
wiring; jetsam behavior is device-only.
|
||||
|
||||
## Hybrid Flow (product default)
|
||||
|
||||
- Foreground host is **light by default**: orphan Live Activity cleanup only —
|
||||
**no** auto `startSession` / continuous capture on appear.
|
||||
- Capture starts only on explicit Start / `osgkeyboard://startflow` / mic press.
|
||||
- Idle background capture is stopped so a parked host does not jetsam the keyboard.
|
||||
- **Do not** stack CLM + Rime + ASR in the same second after onboarding.
|
||||
- ASR warmup runs on **first mic press** (`beginUtterance`), gated by
|
||||
`HostMemoryBudget` (~260 MB RSS). `hostHeavy` is set only while heavy work
|
||||
actually runs, then cleared.
|
||||
|
||||
## Console checklist
|
||||
|
||||
Filter Console by process separately: host `OSGKeyboard` vs extension
|
||||
`com.osgkeyboard.ios.keyboard`. Host-only filter will never show `KVC.*`.
|
||||
|
||||
| Scenario | Expect |
|
||||
|----------|--------|
|
||||
| **Force-quit host**, open Notes, switch to OSG | First `dyld.constructor`, then `KVC.init` → `viewDidLoad` → `viewDidAppear` |
|
||||
| Host foreground right after launch | `skip capture` + `postOnboardingWarmup scheduled … delay=45s` (no immediate Rime/CLM) |
|
||||
| ~45 s later, host still active | Serial `rime.installIfNeeded` then `clm.prepare` |
|
||||
| First mic press | `scheduleASRWarmup`; `hostHeavy` only while work runs |
|
||||
| Switch to typing | Single `rime.prepare` / `englishPrepare` |
|
||||
|
||||
If neither `dyld.constructor` nor `KVC.init` appears after force-quitting the host,
|
||||
the extension is dying in dyld (Shared≈9.4 MB + librime) — next lever is splitting
|
||||
Rime out of Shared.
|
||||
|
||||
## RSS comparison (optional)
|
||||
|
||||
Record `OSGDiag` `rss=` tags for:
|
||||
|
||||
1. Cold voice surface only
|
||||
2. Cold typing after prepare
|
||||
3. Host foreground + extension
|
||||
|
||||
Target: typing peak below the old “voice + eager Librime construct” baseline.
|
||||
|
||||
## Structural split
|
||||
|
||||
- Extension links **OSGKeyboardShared** only (no Charts / StoreKit / Speech / HostSupport).
|
||||
- Host embeds **OSGKeyboardHostSupport** (ASR, CLM, CloudASR, tip/charts UI).
|
||||
- Heavy assets (`osg_pinyin.dict.yaml`, CLM bin, licenses, local-asr catalog)
|
||||
ship in the **host app** bundle; extension reads Rime from App Group after deploy.
|
||||
@@ -62,17 +62,17 @@
|
||||
| `4ab60ba` | 删除手动场景 UI/模型(~871 行),改依赖自动 `AppContext` |
|
||||
| 残留 | `polishScenario.*` 等本地化字符串仍在;`config.polishScenarioId` / `config.systemPrompt` 可能仍在升级用户设备上 |
|
||||
|
||||
### 2.2 当前润色路径(问题)
|
||||
### 2.2 当前润色路径
|
||||
|
||||
```text
|
||||
ASR 文本
|
||||
→ PolishingService.polish
|
||||
→ buildPrompt:
|
||||
globalOutputContract
|
||||
+ Task1 纠错 + Task2 结构
|
||||
+ Task3:AppContext.polishGuideline + Intensity
|
||||
+ 词典 + 上文 + 原文
|
||||
→ TranscriptPostProcessor → 插入
|
||||
→ PolishPromptComposer:
|
||||
Core + Dictionary + Personality + Policy
|
||||
+ AppContext + Intensity + QuestionGuard + RuntimeContext
|
||||
→ 单次 LLM
|
||||
→ 一次 TranscriptPostProcessor + 硬校验
|
||||
→ 插入
|
||||
```
|
||||
|
||||
| 缺口 | 说明 |
|
||||
@@ -225,19 +225,17 @@ flowchart TB
|
||||
装配顺序:
|
||||
|
||||
```text
|
||||
1. [可选] AppContext 前提(短;unknown 可省略)
|
||||
2. StylePack.prompt
|
||||
- 含 {{DICTIONARY}} → 替换为词典块
|
||||
- 无占位符且词典非空 → 追加词典块(兼容用户删占位符)
|
||||
3. Intensity.promptGuideline(短)
|
||||
4. globalOutputContract(强制尾部,用户包不可关闭)
|
||||
5. precedingText(若有)
|
||||
6. 「原文」+ transcript
|
||||
1. Core 全局输出契约与 T1–T5
|
||||
2. Dictionary 独立词典块
|
||||
3. StylePack.prompt(仅人格)
|
||||
4. StylePolicy + Intensity
|
||||
5. AppContext + QuestionGuard + RuntimeContext
|
||||
6. 用户消息中的 transcript
|
||||
```
|
||||
|
||||
| 保留 | 由 Composer 接管 / 替换 |
|
||||
|------|-------------------------|
|
||||
| API key / 超时 / skipLLM | 旧 Task3「风格要求」行(`AppContext.polishGuideline` 作为人格) |
|
||||
| API key / 超时 / skipLLM | 旧 Task3「风格要求」行(场景提示改由 Composer 管理) |
|
||||
| `globalOutputContract` | 旧「角色 + Task1/2/3」整段骨架(人格改由 pack 提供) |
|
||||
| Intensity 追加 | 平行 `ScenarioPrompt` |
|
||||
| 词典注入 | `systemPrompt` 作为第三种风格旁路 |
|
||||
@@ -246,11 +244,7 @@ flowchart TB
|
||||
|
||||
**自定义 = 编辑 user pack 的 `prompt`**,不再单独暴露「系统提示」设置页。
|
||||
|
||||
占位符常量:
|
||||
|
||||
```swift
|
||||
public static let dictionaryPlaceholder = "{{DICTIONARY}}"
|
||||
```
|
||||
词典由 Composer 独立注入,Style Pack 不再携带占位符。
|
||||
|
||||
### 3.5 存储与云同步
|
||||
|
||||
@@ -263,7 +257,7 @@ public static let dictionaryPlaceholder = "{{DICTIONARY}}"
|
||||
|
||||
规则:
|
||||
|
||||
- `activePolishStyleId`:学 `polishIntensity` 进 V2(`decodeIfPresent`,**不 bump schemaVersion**)
|
||||
- `activePolishStyleId`:使用 `decodeIfPresent` 进入 V2(**不 bump schemaVersion**)
|
||||
- Catalog sync:镜像 `PersonalDictionaryCloudSync`(tombstone、clearedAt、payload 上限、跟随 `settingsICloudSyncEnabled`)
|
||||
- `AppCloudSync.pullAll` / `syncNow` **各加一行**
|
||||
- Extension:**只读**;主 App / Mac:**读写**
|
||||
|
||||
Reference in New Issue
Block a user