rocky/OSGKeyboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(app): add crash visibility, memory relief, and clipboard reply work

Browse Source 
Squashes the working tree that had accumulated on cloud-agent. Two
strands of work, kept in one commit at the maintainer's request.

Crash and stability (this session):

- Subscribe to MetricKit so crash, hang, CPU, and disk-write diagnostics
  are recorded on device. Keyboard-extension memory kills are the most
  likely way a user loses the keyboard mid-sentence and iOS does not
  classify them as crashes, so they never reach App Store Connect.
- Upload those reports to POST /v1/analytics/diagnostics from internal
  builds only (local Debug and TestFlight), reusing the analytics
  installation identifier and skipping entirely when analytics are off.
  App Store builds keep everything on device.
- Shed the Chinese engine, English lexicon, and in-flight pipeline work
  at the 40 MiB and 48 MiB budget thresholds instead of waiting for a
  system memory warning that jetsam often preempts.
- Install audio taps with the input node's live format through one
  shared adaptive resampler, so an audio-route change mid-recording no
  longer raises an uncatchable format-mismatch exception. Covers the
  dictation preview and the Mac recorder, which still used the pattern
  the Flow capture path had already moved away from.
- Harden the librime bridge: never pass a nil string across a nonnull
  boundary when a user dictionary holds malformed UTF-8, null-check the
  engine API vector, and distrust a candidate count without its array.
- Guard the expanded candidate grid against a list that shrinks while
  the collection view is still laying out against the previous count.
- Resolve localized text without the App Group so the screen shown when
  that container is missing renders its message instead of crashing on
  launch, which is the one situation it exists to handle.
- Gate CI on the App Group and Keychain wiring that the keyboard
  extension hard-depends on, with a self-test proving the check fails.

Clipboard, reply, and semantics (pre-existing working tree):

- Unify the reply center, refresh keyboard AI features, and add native
  event and reminder exporters.
- Expand clipboard semantic routing with authored corpora and a
  product-blind holdout, plus the scripts that build and validate them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
rocky
2026-09-18 17:31:29 -07:00
co-authored by Claude Opus 5
parent 628e983794
commit 0023741455
210 changed files with 14731 additions and 2940 deletions
Download Patch File Download Diff File Expand all files Collapse all files
OSGKeyboardShared/Services/AIClipboardSkill.swift
+213 -23
View File
@@ -17,9 +17,9 @@ public enum AIClipboardSkillKind: String, Codable, Sendable {
case export
}
/// Optional user-owned style distilled from dictation history or authored in
/// the Styles page. Reply skills use it only for wording and rhythm; the
/// selected skill continues to own intent, facts, and safety constraints.
/// The user's distilled personal style. Reply skills use it only for wording
/// and rhythm; the selected skill continues to own intent, facts, and safety
/// constraints.
public struct AIClipboardReplyStyleContext: Equatable, Sendable {
public let styleID: String
public let prompt: String
@@ -29,15 +29,19 @@ public struct AIClipboardReplyStyleContext: Equatable, Sendable {
self.prompt = prompt
}
/// Only user-owned packs are eligible. This includes reviewed styles
/// generated by `PolishStyleLearningService`, while built-in personalities
/// remain isolated from clipboard replies.
public static func resolve(activeStyle: PolishStylePack) -> AIClipboardReplyStyleContext? {
guard activeStyle.kind == .user else { return nil }
let prompt = PolishStylePackCatalog.runtimePersonality(for: activeStyle)
/// Only a distilled personal style is eligible, and it arrives from
/// `AppGroupStore.personalReplyStyle` — never from the voice-polish
/// selection. Hand-written packs stay on the Styles page, where they shape
/// dictation only, and built-in personalities never reach replies at all.
public static func resolve(
personalReplyStyle: PolishStylePack?
) -> AIClipboardReplyStyleContext? {
guard let personalReplyStyle,
PolishStylePackCatalog.isPersonalReplyStyle(personalReplyStyle) else { return nil }
let prompt = PolishStylePackCatalog.runtimePersonality(for: personalReplyStyle)
.trimmingCharacters(in: .whitespacesAndNewlines)
guard !prompt.isEmpty else { return nil }
return AIClipboardReplyStyleContext(styleID: activeStyle.id, prompt: prompt)
return AIClipboardReplyStyleContext(styleID: personalReplyStyle.id, prompt: prompt)
}
}
@@ -50,8 +54,12 @@ public enum AIClipboardReplyScene: Equatable, Sendable {
case clarification
case complaint
case negativeQuestion
case yesNoQuestion
public static func resolve(from analysis: ClipboardSemanticAnalysis) -> Self? {
public static func resolve(
from analysis: ClipboardSemanticAnalysis,
sourceText: String? = nil
) -> Self? {
if analysis.complaint.isDetected,
analysis.complaint.isApprovedForAutomaticRouting {
return .complaint
@@ -73,6 +81,13 @@ public enum AIClipboardReplyScene: Equatable, Sendable {
}) {
return .task
}
// A yes/no question wants both an affirmative and a negative answer, so
// route it ahead of the single-stance negative-question / clarification
// scenes. Detected from the text's own interrogative shape, so it works
// even when the model's `question` label is weak or gets suppressed.
if let sourceText, isYesNoQuestion(sourceText) {
return .yesNoQuestion
}
if analysis.sentiment == .negative,
analysis.question.isDetected,
analysis.question.isApprovedForAutomaticRouting {
@@ -89,9 +104,33 @@ public enum AIClipboardReplyScene: Equatable, Sendable {
return nil
}
/// Deterministic yes/no interrogative detector. Kept text-based so a clear
/// "是否 / 能不能 / …吗？ / can you …?" always earns both-stance variants.
public static func isYesNoQuestion(_ text: String) -> Bool {
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty, trimmed.count <= 400 else { return false }
let lower = trimmed.lowercased()
let markers = [
// Chinese A-not-A and yes/no interrogatives.
"是否", "是不是", "能不能", "能否", "可不可以", "可以吗", "可以么",
"有没有", "要不要", "会不会", "行不行", "行吗", "对不对", "对吗",
"好不好", "好吗", "好么", "愿不愿意", "方不方便", "需不需要",
"有无", "能吗", "吗？", "吗?", "么？", "么?",
// English yes/no question openers.
"can you", "could you", "would you", "will you", "do you", "did you",
"does ", "are you", "is it", "is there", "was it", "were you",
"have you", "has ", "should i", "should we", "may i", "whether "
]
if markers.contains(where: { lower.contains($0) }) {
return true
}
// A Chinese "…吗" ending is a yes/no question even without punctuation.
return trimmed.hasSuffix("吗") || trimmed.hasSuffix("么")
}
public var requiresIntentVariants: Bool {
switch self {
case .invitation, .task, .blessing, .clarification:
case .invitation, .task, .blessing, .clarification, .yesNoQuestion:
return true
case .complaint, .negativeQuestion:
return false
@@ -173,6 +212,18 @@ public enum AIClipboardReplyScene: Equatable, Sendable {
Scene modifier: the question carries urgency, frustration, or concern. Briefly acknowledge that feeling, then answer directly or state the next step. A negative tone alone does not prove the user is at fault, so do not apologize, accept blame, or promise anything without source support. In multi-reply mode every variant must stay measured and considerate; playful must not joke, tease the sender, or use the playful emotion.
</reply_scene>
"""
case .yesNoQuestion:
return zh
? """
<reply_scene type="yes_no">
场景修饰：对方在问一个是非/能否类问题。四个候选必须分别表达：肯定回答、否定回答、有条件的回答（说明在什么前提下成立），以及需要先确认再答复。正向还是负向取决于原文本身是否表明存在问题或障碍，不要一律给正面答复；不得虚构事实、能力、时间或承诺，缺少依据时用"需要确认"的候选说明。每条都要简短直接、可直接发送。
</reply_scene>
"""
: """
<reply_scene type="yes_no">
Scene modifier: the sender is asking a yes/no or can/can't question. The four variants must respectively give an affirmative answer, a negative answer, a conditional answer (stating the condition under which it holds), and one that needs to confirm before answering. Whether the stance is positive or negative depends on what the source itself implies about problems or blockers — do not default to a positive answer. Invent no facts, capability, timing, or commitments; when unsupported, use the confirm-first variant to say so. Keep each short, direct, and ready to send.
</reply_scene>
"""
}
}
}
@@ -275,8 +326,35 @@ public enum AIClipboardSkillCatalog: Sendable {
/// Legacy ID consolidated into `clarifyRequestID`.
public static let askForDetailsID = "askForDetails"
public static let businessReplyID = "businessReply"
/// Auto-only: drafts a single, email-formatted reply. Not in `catalog`, so it
/// never appears as a chip, in ranking, or in the host Skills tab — it is
/// invoked only by the auto-email-reply behavior. Its id is not `replyID`, so
/// the submit path yields one answer rather than reply variants.
public static let emailReplyID = "emailReply"
/// Standalone descriptor for `emailReplyID` (built outside `catalog`). The
/// localization keys are reused from Reply and never rendered.
public static let emailReplySkill = AIClipboardSkill(
id: emailReplyID,
systemImage: "envelope.fill",
titleKey: "keyboard.ai.skill.reply",
cardTitleKey: "skills.reply.name",
descriptionKey: "skills.reply.description",
kind: .transform,
isDefault: false
)
public static let organizeListID = "organizeList"
/// Rewrites the clipboard in the user's distilled personal style. Unlike
/// Reply it does not answer the clipboard. It ships enabled, but stays
/// unavailable until a personal style exists — see
/// `requiresPersonalReplyStyleIDs`.
public static let speakAsMeID = "speakAsMe"
/// Skills whose generated reply is *modified* by the personal style.
/// `speakAsMeID` is absent on purpose: there the style is the task itself,
/// so it takes a dedicated instruction instead of `replyInstruction`.
public static let replyStyleSkillIDs: Set<String> = [replyID]
/// Skills that cannot run without a distilled personal style. The keyboard
/// drops these from the enabled list when none is active.
public static let requiresPersonalReplyStyleIDs: Set<String> = [speakAsMeID]
/// Contextual system actions remain available to semantic ranking but are
/// not user-managed entries in the host app's Skills catalog.
public static let hiddenFromSkillManagementIDs: Set<String> = [
@@ -284,13 +362,20 @@ public enum AIClipboardSkillCatalog: Sendable {
callPhoneID,
createContactID
]
/// Everything the installed / available lists must not show. Distinct from
/// `hiddenFromSkillManagementIDs` only in *why* it is hidden: `speakAsMeID`
/// carries a prerequisite those always-on system actions do not, so it is
/// gated by `requiresPersonalReplyStyleIDs` rather than by a user toggle.
/// Generating a personal style is the single act that turns it on.
public static let managedOutsideSkillListIDs: Set<String> =
hiddenFromSkillManagementIDs.union([speakAsMeID])
/// Reminders are created natively via EventKit in the host app; this skill
/// no longer ships or requires a companion Shortcut. See `AIReminderExporter`.
public static let extractTodosID = "extractTodos"
public static let extractTodosShortcutName = "OSGExtractTodos"
public static let extractTodosResourceName = "OSGExtractTodos"
/// Calendar events are created natively via EventKit in the host app; this
/// skill no longer ships or requires a companion Shortcut. See `AIEventExporter`.
public static let extractEventsID = "extractEvents"
public static let extractEventsShortcutName = "OSGExtractEvents"
public static let extractEventsResourceName = "OSGExtractEvents"
public static let saveToNotesID = "saveToNotes"
public static let saveToNotesShortcutName = "OSGSaveToNotes"
@@ -309,6 +394,19 @@ public enum AIClipboardSkillCatalog: Sendable {
kind: .transform,
isDefault: true
),
AIClipboardSkill(
id: speakAsMeID,
systemImage: "quote.bubble.fill",
titleKey: "keyboard.ai.skill.speakAsMe",
cardTitleKey: "skills.speakAsMe.name",
descriptionKey: "skills.speakAsMe.description",
kind: .transform,
// On by default: the personal style *is* the prerequisite, and
// `requiresPersonalReplyStyleIDs` already withholds the chip until
// one exists. A second opt-in toggle only re-asked a question the
// distillation had already answered.
isDefault: true
),
AIClipboardSkill(
id: translateID,
systemImage: "character.bubble.fill",
@@ -379,9 +477,7 @@ public enum AIClipboardSkillCatalog: Sendable {
cardTitleKey: "skills.extractTodos.name",
descriptionKey: "skills.extractTodos.description",
kind: .export,
isDefault: true,
shortcutName: extractTodosShortcutName,
shortcutResourceName: extractTodosResourceName
isDefault: true
),
AIClipboardSkill(
id: extractEventsID,
@@ -390,9 +486,9 @@ public enum AIClipboardSkillCatalog: Sendable {
cardTitleKey: "skills.extractEvents.name",
descriptionKey: "skills.extractEvents.description",
kind: .export,
isDefault: true,
shortcutName: extractEventsShortcutName,
shortcutResourceName: extractEventsResourceName
isDefault: true
// Events are created natively via EventKit in the host app (see
// AIEventExporter); no companion Shortcut is shipped or required.
),
AIClipboardSkill(
id: saveToNotesID,
@@ -492,6 +588,41 @@ public enum AIClipboardSkillCatalog: Sendable {
return merged
}
/// Skills the keyboard may surface right now. The keyboard ranks its chips
/// from the whole catalog rather than from the enabled list, so a skill that
/// is unavailable has to be removed here or it still reaches the surface.
///
/// Only `requiresPersonalReplyStyleIDs` members are filtered: the rest are
/// always-available actions whose enabled state governs ordering elsewhere.
public static func availableForKeyboard(
_ skills: [AIClipboardSkill],
layout: AIAgentSkillLayout,
hasPersonalReplyStyle: Bool
) -> [AIClipboardSkill] {
skills.filter { !isUnavailableOptInSkill($0.id, layout: layout, hasPersonalReplyStyle: hasPersonalReplyStyle) }
}
public static func availableEnabledIDs(
_ enabledIDs: [String],
layout: AIAgentSkillLayout,
hasPersonalReplyStyle: Bool
) -> [String] {
enabledIDs.filter { !isUnavailableOptInSkill($0, layout: layout, hasPersonalReplyStyle: hasPersonalReplyStyle) }
}
/// An opt-in skill is unavailable when its prerequisite is missing *or* the
/// user has not turned it on. Unlike always-available skills, shipping it
/// unasked would put a permanently failing chip on the keyboard.
private static func isUnavailableOptInSkill(
_ id: String,
layout: AIAgentSkillLayout,
hasPersonalReplyStyle: Bool
) -> Bool {
let canonical = canonicalID(for: id)
guard requiresPersonalReplyStyleIDs.contains(canonical) else { return false }
return !hasPersonalReplyStyle || !layout.isEnabled(canonical)
}
public static func skill(
id: String,
officialCatalog: OfficialSkillCatalog = .empty,
@@ -561,6 +692,13 @@ public enum AIClipboardSkillCatalog: Sendable {
now: now
)
}
if canonicalID(for: skill.id) == speakAsMeID {
return speakAsMeInstruction(
baseInstruction,
locale: locale,
style: replyStyle
)
}
guard skill.supportsReplyStyle else { return baseInstruction }
return replyInstruction(
baseInstruction,
@@ -596,7 +734,7 @@ public enum AIClipboardSkillCatalog: Sendable {
let zh = locale == "zh"
let instructionID: String
switch skillID {
case playfulReplyID, businessReplyID:
case playfulReplyID, businessReplyID, emailReplyID:
instructionID = skillID
default:
instructionID = canonicalID(for: skillID)
@@ -610,6 +748,10 @@ public enum AIClipboardSkillCatalog: Sendable {
return zh
? "请根据剪贴板内容，用原文的主要语言写一段俏皮、有梗、可直接发送的回复，像一个懂分寸的脱口秀演员接话。包袱要短，通常 1～2 句；优先调侃情境，不攻击对方，不拿身份、外貌、隐私、疾病或创伤开玩笑，不编造事实。遇到严肃或敏感内容时收住幽默，改为轻松但尊重的表达。"
: "Write a playful, witty, sendable reply in the clipboard text's primary language, like a tactful stand-up comic joining the conversation. Keep the punchline short, usually 1–2 sentences. Joke about the situation, never attack the person or mock identity, appearance, privacy, illness, or trauma, and invent no facts. For serious or sensitive content, dial back the humor and stay light but respectful."
case speakAsMeID:
return zh
? "请把剪贴板文本改写成这位用户自己会怎么说。保留全部事实、数字、日期、金额、人名、机构名、链接和代码标识符，并保留原文正在完成的交际动作：陈述仍是陈述，请求仍是请求，问句仍是问句。只改变表达方式——用词、句长、节奏和语域。输出长度贴近原文（± 20% 以内），不扩写、不摘要、不补背景，也不加标题、引号或说明。"
: "Rewrite the clipboard text the way this user would say it. Preserve every fact, number, date, amount, personal name, organization, URL, and code identifier, and preserve the communicative act the text performs: a statement stays a statement, a request stays a request, a question stays a question. Change only the expression — wording, sentence length, rhythm, and register. Keep the output close to the source length (within ±20%): never expand, summarize, add background, or add a title, quotation marks, or commentary."
case summarizeID:
return zh
? "请根据内容类型总结剪贴板文字，提炼核心意思、关键事实、决定、结论和下一步；没有的内容不要补充。使用清晰、简短的段落或要点，不要改写成可发送的聊天回复。"
@@ -627,6 +769,10 @@ public enum AIClipboardSkillCatalog: Sendable {
return zh
? "请写一段专业但不官腔的商务聊天回复，表达直接、自然，保留人名、组织名、时间和承诺边界，可直接发送。不要套用正式邮件开场和结尾。"
: "Write a professional but conversational business reply. Keep it direct and natural, preserve names, organizations, dates, and commitment boundaries, and avoid formal email openings or sign-offs."
case emailReplyID:
return zh
? "剪贴板里是一封收到的邮件。请先理解发件人、诉求、语气和你们的关系，再用邮件原文的主要语言写一封【完整、可直接发送的回复邮件】：以合适的称呼开头，逐条回应对方的要点，保留人名、机构名、日期、金额和承诺边界，语气与原邮件的正式程度匹配，最后用得体的结尾与落款收束。只写一版，不要提供多个选项。不要翻译、复述或概括原邮件，也不要输出解释或标题。"
: "The clipboard contains a received email. First understand the sender, their request, the tone, and your relationship, then write ONE complete, ready-to-send reply email in the original email's primary language: open with an appropriate salutation, address each of the sender's points, preserve names, organizations, dates, amounts, and commitment boundaries, match the formality of the original, and close with a fitting sign-off. Produce a single version, not multiple options. Do not translate, restate, or summarize the original, and do not output any explanation or title."
case organizeListID:
return zh
? "请把剪贴板中的清单、议程或步骤整理成结构清晰、顺序合理的列表。合并重复项，保留原意，不新增任务。"
@@ -731,6 +877,50 @@ public enum AIClipboardSkillCatalog: Sendable {
: "It is now \(stamp) (device local timezone)."
}
/// "Speak as me" rewrites the user's own draft, so the personal style is the
/// task rather than a modifier. It deliberately skips `replyInstruction`:
/// that path appends a generic chatty baseline capped at "usually 1–3
/// sentences", which would both fight a formal personal style and truncate a
/// long clipboard. It also needs the never-answer boundary that Reply omits.
private static func speakAsMeInstruction(
_ baseInstruction: String,
locale: String,
style: AIClipboardReplyStyleContext?
) -> String {
let zh = locale == "zh"
let boundary = zh
? PolishPromptComposer.chineseNeverAnswerContract
: PolishPromptComposer.englishNeverAnswerContract
guard let style,
!style.prompt.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty else {
// Unreachable in production: the keyboard drops this skill when no
// personal style is active. Never silently degrade into generic
// polish — the skill's whole premise is the user's own voice.
return "\(baseInstruction)\n\(boundary)"
}
let boundedStyle = String(
style.prompt
.trimmingCharacters(in: .whitespacesAndNewlines)
.prefix(PolishStyleLimits.maximumPromptCharacters)
)
let personalStyle = zh
? """
<user_reply_style id="\(style.styleID)">
\(boundedStyle)
</user_reply_style>
以上是这位用户本人的表达习惯。本技能的任务就是让输出读起来像这个人写的，因此要主动采用其中的用词、句长、节奏和语域。
但风格不得改变事实、交际意图、说话人或输出语言；两者冲突时以保真为准。
"""
: """
<user_reply_style id="\(style.styleID)">
\(boundedStyle)
</user_reply_style>
The block above describes this user's own way of writing. Making the output read as if they wrote it is this skill's task, so actively adopt its wording, sentence length, rhythm, and register.
Style may never change facts, communicative intent, speaker, or output language; on conflict, fidelity wins.
"""
return "\(baseInstruction)\n\(boundary)\n\(personalStyle)"
}
private static func replyInstruction(
_ baseInstruction: String,
skillID: String,