merge: bring local uncommitted work onto the cloud-agent branch

Merges the wip/local-snapshot-20260901 safety commit (94 uncommitted files:
in-flight Account/Mac/Tip work, clipboard-semantics improvements, and the
feature-preview recording tooling) into the newer cloud-agent line.

Conflict resolutions:
- docs/*: honour the branch's deliberate removal in eac44da; drop the now
  dangling docs/archive/README.md that only pointed at the deleted files.
- CHANGELOG: both sides describe separate [Unreleased] features; keep both.
- ClipboardSkillSemanticRanker: take the branch's reply routing, which retires
  ranking playfulReply/acceptInvitation/... as separate skills in favour of
  ordinary/formal/playful variants inside one reply action. Keep the local
  link/phone/CJK-density work, which is orthogonal and still referenced.
- OfficialSkillCatalogTests: keep the local "installed exactly once" assertion;
  it matches the test's own comment and does not hardcode the default set.
- ClipboardSkillSemanticRankerTests: keep both sides' disjoint test functions;
  retarget the two link/phone cases at the branch's row, which no longer
  surfaces the retired specialised reply skills.

Verified: xcodebuild build succeeds and the 77 ranker/catalog/analyzer tests
pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-01 12:34:23 -07:00
co-authored by Claude Opus 5
79 changed files with 3459 additions and 1178 deletions
+145
View File
@@ -0,0 +1,145 @@
# 功能预览片录制 / Feature preview recording
四段中文 App Store 预览片的录制与合成流程。成片位于
`docs/assets/app-preview/zh/OSGKeyboard-<slug>-6.7-zh.mp4`。
| slug | 讲什么 | 素材来源 |
|------|--------|----------|
| `voice-polish` | 语音输入与自动润色,收在趣味润色风格 | `--edit-demo` + 润色风格页 |
| `personal-style` | 从历史听写学习你的表达习惯,生成专属风格 | `--polish-styles-screenshot` |
| `clipboard-agent` | 剪贴板 AI Agent,一键回复消息 | `--clipboard-demo` |
| `ask-ai` | 轻点听写、长按问 AI | `--ai-demo` |
## 为什么不用键盘扩展录
`WhatsNewDemoDriver` 那条路(`--whats-new-host`)会让**真实键盘扩展**盖在
Notes 宿主上,画面最真。但把第三方键盘设成**当前**输入法必须点一下地球键,
`simctl` 没有对应命令,`.GlobalPreferences` 里的 `AppleKeyboards` 只决定
"已启用"而不是"当前使用"。
所以这套流程改走 app 内的 DEBUG demo host:它们渲染的是**同一批真实键盘视图**
(`AIKeyboardView`、`LastInputEditView`、`ClipboardHistoryPanelView`……,见
`project.yml` 中 app target 的 `OSGKeyboardExt/Views/*` 源),由脚本化的
`KeyboardState` 驱动,不走 ASR、不走 LLM、不联网,也就不需要任何点击。
`--preview-fullscreen` 会在键盘上方补一块 Notes / Messages 宿主文档
(`FeaturePreviewHostDocument`),把原本留给 What's New 裁剪的空白填满,
让整帧可以直接当全屏预览片用。不传这个 flag 时,各 demo view 的画面与
What's New 工作流完全一致。
## 设备要求
- **iPhone 16 Plus**:原生 1290×2796,正好是 App Store Connect 6.9" 预览位
接受的尺寸之一,零缩放。
- **iOS 26.5 或更新**:app 用到的 `SwiftUI.View.glassEffect(_:in:)` 在 iOS 26.0
运行时里不存在,装上去会在启动时 `Symbol not found` 崩溃。
Xcode 预置的 iPhone 16 Plus 挂在 iOS 26.0 下,需要自建一台:
```bash
xcrun simctl create "OSG-Preview-16Plus" \
com.apple.CoreSimulator.SimDeviceType.iPhone-16-Plus \
com.apple.CoreSimulator.SimRuntime.iOS-26-5
```
## 完整流程
```bash
# 1) 编译(必须带签名,否则没有 entitlements → App Group 不可用 → 启动崩溃)
xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
-configuration Debug -destination "id=$UDID" \
-onlyUsePackageVersionsFromResolvedFile CODE_SIGNING_ALLOWED=YES build
# 2) 干净安装。覆盖安装不会刷新 App Group 容器,
# 容器缺失时 app 会渲染 AppGroupErrorView,录出来就是一屏报错。
xcrun simctl uninstall "$UDID" com.osgkeyboard.ios
xcrun simctl install "$UDID" \
~/Library/Developer/Xcode/DerivedData/OSGKeyboard-*/Build/Products/Debug-iphonesimulator/OSGKeyboard.app
# 3) 录制(脚本会先校验 App Group 容器,缺失直接报错退出)
Scripts/record_feature_previews.sh "$UDID" # 全部四段
Scripts/record_feature_previews.sh "$UDID" ask-ai # 只录一段
# 4) 合成。需要 Pillow —— 这版 ffmpeg 没有 libfreetype / libass,
# 所有文字都由 PIL 渲染成 RGBA PNG 再 overlay。
python3 -m venv .venv && .venv/bin/pip install Pillow
.venv/bin/python Scripts/compose_feature_previews.py
```
## 已知的坑
**变帧率**。`simctl io recordVideo` 录的是 VFR,时间轴不按墙钟走:直接
`-ss` 会定位到错误位置,`-t` 截出来的长度也不对(实测 24s 的窗口出来 27.4s,
足以让成片超出 App Store 的 30 秒上限)。`compose_feature_previews.py` 会先把
每段原始素材转成 30fps CFR 再裁,所以配置里的 `trim_start` / `trim_duration`
是真实秒数。
**静态页面录不出长片**。VFR 只在画面变化时出帧,纯静态页面录 22 秒也只得到
约 5 秒素材。`Source(freeze_at=…)` 会抽一帧定格,配合 `ken_burns` 用运镜提供
动势 —— `personal-style` 就是这么做的。
**字幕 PNG 必须 `-loop 1`**。单帧输入的 pts 恒为 0,`fade` 的时间轴永远停在
起点,alpha 被钉死,字幕完全不显示。
**色彩标签会丢**。原始素材是 `tv` range / bt709 / sRGB transfer,重编码不显式
带上就变成 `unknown`,播放器猜错会明显发灰。合成脚本每一次编码都显式打标签。
**抽帧检查会骗你**。用 ffmpeg 抽出 PNG 再看,色彩管理与实际播放不同,画面会
偏灰;判断压暗层强度请量像素值,不要看缩略图。
## App 内卡片版(无字幕短循环)
同一批原始素材还会产出第二种成品:给 app 内卡片用的短循环,位于
`docs/assets/feature-cards/<slug>.mp4`,另附 `<slug>-poster.png` 作占位图。
```bash
.venv/bin/python Scripts/compose_feature_cards.py
```
与预览片的区别:**没有字幕、没有片头片尾、没有音轨**,并且裁到键盘区
(`1290×944`),画面里只有真实键盘 chrome,不含任何为讲故事补的宿主界面。
窗口都选在首尾状态相近的位置,再叠一次 0.5 秒的尾接头淡入,所以可以无限循环
播放而看不出接缝(实测首末帧平均像素差 0.19–0.57 / 255)。`personal-style` 是
静态设置页,改用推近再拉回的对称运镜,天然闭合。
裁切区取 `y=1780` 起、高 944:剪贴板面板是键盘区里最高的元素(顶边 y=1779),
屏幕最底部 72px 是键盘下方的空白内边距,一并去掉。
### 技能行必须由真实排序器产生
`AIKeyboardView` 的技能行读 `ClipboardHistoryStore.shared` 与
`ClipboardSemanticRankingStore.shared` 两个**单例**,并交给
`ClipboardSkillSemanticRanker.recommended(...)` 按剪贴板内容排序。demo view 早期
用私有 store + `AIKeyboardView.debugPreviewSkills` 强塞了一排回复风格,结果录出
一个真实 app 里**不可能出现**的键盘——排序器在通用「回复」之外最多只放
`maximumReplyRecommendations = 2` 个专门回复技能。
现在 `ClipboardHistoryDemoView` 直接喂真实单例:把要回复的那条消息 ingest 成
最新剪贴板项,调 `ranking.analyze(entry)` 并**等它出 snapshot**(分析在主 actor 外
跑,抢跑会让行退化成孤零零一个「回复」),然后由排序器决定行内容。当前这条
消息排出的是「译为英语 / 回复 / 澄清追问 / 日程」。
演示脚本只从排出来的行里挑技能点——先点通用「回复」,再点行内第一个
`supportsReplyStyle` 的技能——所以画面里点的永远是屏幕上真实存在的 chip。
app 里**没有**「点一次回复,弹出多条候选」这种交互:点一个技能生成一条。
多风格是靠技能行里并列的多个回复技能实现的,先选风格再生成。
一个容易踩的坑:`AISessionState` 的 `beginGenerating` / `receivePartialAnswer` /
`receiveAnswer` 都用 `activeUtteranceID == utteranceID` 做守卫,而这个 ID 只由
`beginPreparing` 设置。跳过 `beginPreparing` 直接 `beginGenerating`,整串调用会
**静默变成空操作**,键盘区从头到尾不出现任何答案。每一轮都要
`enter()` → `beginPreparing` → `beginGenerating`。
两个 ffmpeg 细节:`zoompan` 的 `d=1` 是按输入帧走的,而 `-loop 1` 读 PNG 默认
**25fps**,不加 `-framerate 30` 出来的片长会短 25/30;另外推拉运镜不要用单条
`zoompan` 表达式同时做进和出——它的实际出帧数对不上 `duration * FPS`,片子会
停在半路,循环处肉眼可见地跳。现在的做法是只渲染推近的一半再 `reverse` 拼接。
## 改动了 demo 时间轴之后
`Scripts/feature_preview_clips.py` 里的裁切点和字幕时间是照着实际录像量出来的。
只要改了 `EditDemoView` / `AIKeyboardDemoView` / `ClipboardHistoryDemoView` 的
时间轴,就要重录并重新对时。合成脚本会校验成片是否落在 App Store 要求的
15–30 秒、1290×2796、H.264 之内,不合规会在输出里标 `!!`。
-109
View File
@@ -1,109 +0,0 @@
# OSGKeyboard 1.8.0 更新说明
> 以下为 OSGKeyboard 从 1.6.6 升级至 1.8.0 的关键变化。
## 新功能
### AI Agent
- 语音听写与 AI 问答合并为全新的助手模式:轻点麦克风开始听写,长按麦克风可直接语音提问。
- 新增热点与情境热词,麦克风区域会展示天气、节日、新闻、热搜等建议;点按即可向 AI 获取相关信息。
- AI 回答会实时显示在键盘中,确认后可插入当前输入框;支持的服务商可联网搜索,并在失败时自动回退。
- 仅当原输入框与光标上下文仍然匹配时自动插入 AI 回答;上下文变化时会保留回答,等待用户明确插入或丢弃。
- 设置 → AI Agent 可调节回复篇幅:简短、中等或详细。
### 剪贴板与 AI 技能
- 新增剪贴板历史。在设置 → 剪贴板开启“历史记录”后,可在本机保存最近 15 条纯文本;该功能默认关闭。
- 新增剪贴板建议条。可从键盘顶栏打开历史并点按插入;开启建议条后,最新复制内容会显示在键盘上方。
- 复制文字后约 30 秒内,助手会显示回复、总结、翻译等快捷技能;技能支持分页浏览,并可在技能页调整顺序。
- 新增技能页,最多可将 8 个技能放入键盘。内置技能可提取待办、添加日历日程、存入备忘录,或识别地址后打开高德、百度与 Apple 地图导航。
- 新增自定义 AI 技能,可设置名称、图标和提示词,也可选择连接自己的 iCloud 快捷指令。
### 编辑上次输入
- 编辑入口移至键盘右下角。点按后可口述修改要求、对比原文与编辑结果,并选择替换原文或追加内容。
- 编辑只使用最近一次由 OSGKeyboard 插入且仍可验证的文字,不会读取输入框中的无关内容。
## 全新 App 界面
- iPhone 底部导航重做为“首页 / 技能 / 风格 / 设置”四栏 Liquid Glass Dock,并以绿色胶囊标示当前页面。
- 首页改为信息总览,将使用统计、听写历史和个性词库集中显示为资料卡。
- 技能与润色风格采用统一的卡片式管理:点按选择、右上角编辑、右下角显示启用状态;技能支持长按拖动排序。
- 设置入口重新整理,AI Agent、剪贴板、语音与文字输入等页面更容易找到,并直接显示当前设置摘要。
- iPad 改为侧栏 + 内容区布局;键盘横竖屏均充分利用可用宽度,并加入系统地球键、撤销、重做、复制和剪切等操作。
## 中英文输入全面升级
- 中英文输入均可在本机学习用户的输入习惯、候选频率和选择偏好;设置中可单独清除打字习惯,不会删除个性词库。
- 拼音优化混合简拼排序,并支持 `zh`、`ch`、`sh` 两键简拼。
- 英文键盘新增类 QuickType 候选栏,同时显示原词、纠错和补全;约 4 万词离线词表会结合系统词库、通讯录名称与文本替换提供建议。
- 英文邻键纠错更贴合真实键位,并减少对专名、短词和全大写词的误改;拒绝纠错后会学习保留原词。
- 支持叠指连打、英文双空格句号,以及根据输入框显示前往、搜索、发送、完成、下一项等操作。
## 稳定性、隐私与重要变化
- 听写、AI 回答、编辑结果和剪贴板粘贴可通过统一的撤销操作回滚。
- 修复切换键盘崩溃、中文输入初始化后无法恢复、通用剪贴板导致键盘卡住,以及 AI 会话结束后麦克风不可用等问题。
- 全部内置与自定义润色风格都会保留问句原意,不再把用户的问题改写成回答。
- 设备端语音识别仍为默认选项;AI 使用用户自己配置的服务商与 API Key,不再提供内置 DeepSeek 回退。
- 剪贴板历史默认关闭,内容保存在本机,并仅在用户主动调用技能后发送给所配置的 AI 服务商。
- 长按麦克风现用于向 AI 提问;编辑上次输入改用右下角编辑按钮。旧的长按剪贴板语音指令已由剪贴板历史和快捷技能取代。
- 已安装旧中文名称“待办”或“日程”快捷指令的用户,需要从技能页重新安装 `OSGExtractTodos` 和 `OSGExtractEvents`。
- 已移除空白区域滑动光标及其设置开关。
---
# OSGKeyboard 1.8.0 Release Notes
> Key changes when upgrading OSGKeyboard from 1.6.6 to 1.8.0.
## New Features
### AI Agent
- Voice dictation and AI questions now share one Assistant mode: tap the microphone to dictate, or hold it to ask a question by voice.
- New trending and contextual hotwords surface suggestions for weather, holidays, news, trending topics, and more; tap one to ask AI about it.
- AI answers stream directly into the keyboard for review and insertion. Supported providers can search the web and fall back automatically when needed.
- AI answers insert automatically only while the original field and cursor context still match. If the context changes, the answer is retained for explicit insertion or discard.
- Settings → AI Agent now offers Short, Medium, and Detailed response lengths.
### Clipboard and AI Skills
- New Clipboard History keeps the latest 15 plain-text copies on this device when enabled in Settings → Clipboard. It is off by default.
- A new Clipboard suggestion strip lets you open history from the keyboard, insert an item, or keep the latest copy above the keyboard.
- For about 30 seconds after copying text, Assistant shows quick Reply, Summarize, Translate, and other skills. Skills support paging and can be reordered from the Skills page.
- The new Skills page lets you place up to eight skills on the keyboard. Built-in skills extract reminders, add calendar events, save to Notes, or open navigation in Amap, Baidu Maps, or Apple Maps.
- Custom AI Skills support your own name, icon, and prompt, with an optional iCloud Shortcut connection.
### Edit Last Input
- Edit moves to the bottom-right keyboard button. Describe the change by voice, compare the original and edited text, then replace or append the result.
- Editing uses only the latest verifiable text inserted by OSGKeyboard and does not read unrelated field content.
## A New App Interface
- The iPhone dock is rebuilt around four Liquid Glass destinations: Home, Skills, Styles, and Settings, with a green capsule marking the current page.
- Home becomes an overview of usage statistics, dictation history, and Personal Dictionary cards.
- Skills and polish styles share a card-based interaction: tap to select, edit from the top-right, see enabled status at the bottom-right, and long-press skills to reorder.
- Settings navigation is reorganized so AI Agent, Clipboard, Speech, and Text Input are easier to find, with current summaries shown beside each row.
- iPad adopts a sidebar + detail layout. The keyboard fills the available portrait and landscape width and adds the system globe key, undo, redo, copy, and cut.
## Chinese and English Typing, Rebuilt
- Chinese and English typing learn candidate frequency and selection preferences locally. Typing habits can be cleared without deleting Personal Dictionary entries.
- Pinyin improves mixed abbreviations and supports two-key `zh`, `ch`, and `sh` abbreviations.
- The English keyboard adds a QuickType-style bar for the typed word, correction, and completion. An offline list of about 40,000 words works with the system lexicon, contact names, and text replacements.
- Neighbor-key correction follows the physical keyboard while avoiding unwanted changes to names, short words, and all-caps text. Rejecting a correction teaches the original.
- Overlapping key presses, the English double-space period, and contextual Go, Search, Send, Done, and Next actions make typing feel closer to the system keyboard.
## Stability, Privacy, and Important Changes
- One Undo action now rolls back dictation, AI answers, edits, and clipboard pastes.
- Fixes include keyboard-switch crashes, Chinese input recovery after setup, Universal Clipboard freezes, and a microphone that could remain unavailable after AI sessions.
- Every built-in and custom polish style preserves questions instead of rewriting them as answers.
- On-device speech recognition remains the default. AI uses your configured provider and API key; the built-in DeepSeek fallback has been removed.
- Clipboard History stays off by default, keeps its contents on device, and sends text to the configured AI provider only after you invoke a skill.
- Holding the microphone now asks AI, while Edit Last Input moves to the bottom-right Edit button. Clipboard History and quick skills replace the old hold-to-command clipboard flow.
- If you installed the older Chinese-named Tasks or Events Shortcuts, reinstall `OSGExtractTodos` and `OSGExtractEvents` from the Skills page.
- Blank-area cursor sliding and its Settings toggle have been removed.
Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 170 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 153 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.
+57
View File
@@ -178,6 +178,63 @@
<p class="empty" id="empty-zh">暂无与当前版本匹配的更新说明。</p>
<p class="empty" id="empty-en">No release notes match this app version.</p>
<!-- ========== 2.0.3 ========== -->
<article class="version-block" data-version="2.0.3">
<div class="version-chip">2.0.3</div>
<div class="lang-block" data-lang="zh">
<section class="section">
<h2>英文个性词推荐</h2>
<ul>
<li>除中文输入习惯外,键盘现在也会学习你反复使用的英文生词,并推荐加入个人词库</li>
<li>中英文推荐词都需要你确认后才会加入,原有的中文学习记录不受影响</li>
</ul>
</section>
<section class="section">
<h2>界面一致性优化</h2>
<ul>
<li>统一卡片间距、设置页字体和菜单对齐方式,账号与购买记录的排版更整齐</li>
<li>一级页面与详情页之间的标签栏过渡更自然</li>
</ul>
</section>
<section class="section">
<h2>问题修复</h2>
<ul>
<li>重装 App 或重新回到前台后,Apple 账号登录状态恢复更可靠,登录过程中的加载提示保持一致</li>
<li>在不同 App 之间切换后,中英文候选词可靠恢复,且不会残留上一个输入框的内容</li>
</ul>
</section>
</div>
<div class="lang-block" data-lang="en">
<section class="section">
<h2>English Personal-Term Suggestions</h2>
<ul>
<li>Alongside Chinese typing habits, the keyboard now learns rare English words you use repeatedly and suggests adding them to your Personal Dictionary</li>
<li>Suggestions in either language are added only after you confirm, and existing Chinese learning history is preserved</li>
</ul>
</section>
<section class="section">
<h2>Interface Consistency</h2>
<ul>
<li>Card spacing, settings typography, menu alignment, and the account and purchase-history rows are now consistent</li>
<li>Tab-bar transitions between primary pages and detail pages feel smoother</li>
</ul>
</section>
<section class="section">
<h2>Fixes</h2>
<ul>
<li>Apple sign-in is restored more reliably after a reinstall or when returning to the foreground, with a consistent loading state</li>
<li>Chinese and English candidates recover reliably after switching apps, without leaking state from the previous text field</li>
</ul>
</section>
</div>
</article>
<!-- ========== 2.0.2 ========== -->
<article class="version-block" data-version="2.0.2">
<div class="version-chip">2.0.2</div>