feat(app): add crash visibility, memory relief, and clipboard reply work

Squashes the working tree that had accumulated on cloud-agent. Two strands of work, kept in one commit at the maintainer's request. Crash and stability (this session): - Subscribe to MetricKit so crash, hang, CPU, and disk-write diagnostics are recorded on device. Keyboard-extension memory kills are the most likely way a user loses the keyboard mid-sentence and iOS does not classify them as crashes, so they never reach App Store Connect. - Upload those reports to POST /v1/analytics/diagnostics from internal builds only (local Debug and TestFlight), reusing the analytics installation identifier and skipping entirely when analytics are off. App Store builds keep everything on device. - Shed the Chinese engine, English lexicon, and in-flight pipeline work at the 40 MiB and 48 MiB budget thresholds instead of waiting for a system memory warning that jetsam often preempts. - Install audio taps with the input node's live format through one shared adaptive resampler, so an audio-route change mid-recording no longer raises an uncatchable format-mismatch exception. Covers the dictation preview and the Mac recorder, which still used the pattern the Flow capture path had already moved away from. - Harden the librime bridge: never pass a nil string across a nonnull boundary when a user dictionary holds malformed UTF-8, null-check the engine API vector, and distrust a candidate count without its array. - Guard the expanded candidate grid against a list that shrinks while the collection view is still laying out against the previous count. - Resolve localized text without the App Group so the screen shown when that container is missing renders its message instead of crashing on launch, which is the one situation it exists to handle. - Gate CI on the App Group and Keychain wiring that the keyboard extension hard-depends on, with a self-test proving the check fails. Clipboard, reply, and semantics (pre-existing working tree): - Unify the reply center, refresh keyboard AI features, and add native event and reminder exporters. - Expand clipboard semantic routing with authored corpora and a product-blind holdout, plus the scripts that build and validate them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 17:31:29 -07:00
parent 628e983794
commit 0023741455
210 changed files with 14731 additions and 2940 deletions
@@ -88,6 +88,10 @@ final class AIKeyboardCoordinator {
        pendingStructuredReplyResult = false
        conversationInsertionFingerprint = nil
        hasConversationInsertionTarget = false
+        // The reply session is gone; drop any pending auto-reply surface restore
+        // so the next presentation's surface is chosen normally.
+        state.autoReplyReturnSurface = nil
+        state.autoResultReadOnly = false
    }

    func toggleMicrophone() {
@@ -116,12 +120,19 @@ final class AIKeyboardCoordinator {
        }
    }

-    /// Tap a clipboard skill chip: same fail-closed material path as hint cards.
+    /// Tap a clipboard skill capsule: same fail-closed material path as hint cards.
    func submitClipboardSkill(
        _ skill: AIClipboardSkill,
        replyScene: AIClipboardReplyScene? = nil
    ) {
-        guard canAcceptIdleSubmit else { return }
+        // Block only while a request is actively running. A result that is merely
+        // displayed (.ready) is replaceable: the user tapped another skill capsule
+        // to switch what the AI does with this copy, so clear it and continue.
+        guard !state.aiSession.isBusy else { return }
+        if state.aiSession.phase == .ready {
+            discardPendingReplyFeedback()
+            state.aiSession.discardReadyAnswer()
+        }
        guard !skill.requiresShortcut
                || state.confirmedClipboardShortcutIDs.contains(skill.id) else {
            state.skillTipText = ExtL10n.string("keyboard.ai.skill.shortcutMissing")
@@ -167,6 +178,9 @@ final class AIKeyboardCoordinator {
        }

        prepareConversationForRequest()
+        // Remember which skill this result came from so the result panel can
+        // offer the *other* applicable skills without repeating this one.
+        state.activeClipboardSkillID = skill.id
        requestReplyFeedbackSource = skill.id == AIClipboardSkillCatalog.replyID
                && oobeFeature == nil
            ? material
@@ -198,15 +212,35 @@ final class AIKeyboardCoordinator {
        if skill.kind == .export {
            instruction += "\nPreserve the source language, addresses, names, and proper nouns."
        }
+        // Recent same-chat context: only for a real reply, only when we have the
+        // incoming message, and only from exchanges inside the 5-minute window.
+        // iOS gives a keyboard no source-app identity, so time proximity is the
+        // approximation of "the same conversation".
+        let conversationContext: String? = {
+            guard skill.id == AIClipboardSkillCatalog.replyID,
+                  oobeFeature == nil,
+                  let material else { return nil }
+            let turns = replyFeedbackStore.recentConversationTurns(
+                excludingIncoming: material
+            )
+            return replyConversationContext(from: turns)
+        }()
+        if conversationContext != nil {
+            instruction += "\n" + Self.replyConversationContextGuide
+        }
        AIAgentShortcutRun.trace("keyboard.submit skill=\(skill.id) kind=\(skill.kind)")
        if let material {
            AIAgentShortcutRun.traceBody("keyboard.clipboard", material)
        } else {
            AIAgentShortcutRun.trace("keyboard.clipboard missing")
        }
+        if conversationContext != nil {
+            AIAgentShortcutRun.trace("keyboard.replyContext attached")
+        }
        let resolution = AIClipboardPrompt.resolve(
            instruction: instruction,
-            material: material
+            material: material,
+            conversationContext: conversationContext
        )
        if case .materialUnavailable = resolution {
            AIAgentShortcutRun.trace("keyboard.submit rejected clipboardUnavailable skill=\(skill.id)")
@@ -337,6 +371,7 @@ final class AIKeyboardCoordinator {
            conversationInsertionFingerprint = captureInsertionFingerprint()
            hasConversationInsertionTarget = true
            resetStructuredReplyConversationIfNeeded()
+            restoreAutoReplySurfaceIfNeeded()
        }
    }

@@ -353,6 +388,7 @@ final class AIKeyboardCoordinator {
        conversationInsertionFingerprint = captureInsertionFingerprint()
        hasConversationInsertionTarget = true
        resetStructuredReplyConversationIfNeeded()
+        restoreAutoReplySurfaceIfNeeded()
    }

    func discardPendingAnswer() {
@@ -361,6 +397,28 @@ final class AIKeyboardCoordinator {
        requestInsertionFingerprint = nil
        requestOOBEFeature = nil
        resetStructuredReplyConversationIfNeeded()
+        // Tapping the result's X means the user is done with this clipboard
+        // generation — retire the whole assistant affordance for it. This routes
+        // through the shared dismiss so it both marks the skill top bar dismissed
+        // (`dismissedClipboardSkillEntryID`) AND clears the standalone paste
+        // suggestion (`clipboardSuggestionText`). Setting only the skill marker
+        // left the paste capsule + X alive on the surface we return to below,
+        // because the two are independent state.
+        state.dismissClipboardSuggestion()
+        restoreAutoReplySurfaceIfNeeded()
+    }
+
+    /// If auto mode moved the user onto the assistant surface to show a reply,
+    /// finishing it — using a reply or closing — returns them to the keyboard
+    /// they were last using instead of leaving them on voice input.
+    private func restoreAutoReplySurfaceIfNeeded() {
+        // A read-only auto result (auto-translate) is finished once dismissed.
+        state.autoResultReadOnly = false
+        // The result panel is closing, so its "other skills" row retires too.
+        state.activeClipboardSkillID = nil
+        guard let surface = state.autoReplyReturnSurface else { return }
+        state.autoReplyReturnSurface = nil
+        state.setSurface(surface)
    }

    func performCurrentFieldAction() {
@@ -482,6 +540,12 @@ final class AIKeyboardCoordinator {
            beginReplyFeedback(answer: answer)
        }
        defer { requestInsertionFingerprint = nil }
+        // Auto-translate is staged like a reply: show the result on the keyboard
+        // and let the user drop it into the field with the glass Insert button,
+        // never straight into the host field.
+        if state.autoResultReadOnly {
+            return
+        }
        guard state.aiSession.canInsert,
              let expected = requestInsertionFingerprint,
              captureInsertionFingerprint() == expected,
@@ -530,6 +594,9 @@ final class AIKeyboardCoordinator {
    }

    private func prepareConversationForRequest() {
+        // Any fresh request drops the previous result's "active skill" marker;
+        // submitClipboardSkill re-sets it right after calling this.
+        state.activeClipboardSkillID = nil
        requestExpectsReplyVariants = false
        requestReplyVariantSet = .generic
        requestReplySourceText = nil
@@ -548,6 +615,33 @@ final class AIKeyboardCoordinator {
        requestInsertionFingerprint = currentFingerprint
    }

+    /// Tells the model how to read the untrusted `<conversation_context>` block.
+    /// English to match the other inline directives in this file.
+    private static let replyConversationContextGuide =
+        "A <conversation_context> block precedes the clipboard text: it is the "
+            + "earlier back-and-forth in this same chat, where \"Them:\" is the "
+            + "other person and \"You:\" is the user. Use it only as background to "
+            + "keep the reply consistent and on-topic; reply to the latest message "
+            + "in clipboard_text, and never quote or answer the context lines "
+            + "themselves."
+
+    /// Formats recent turns into the plain `Them:/You:` transcript embedded in
+    /// the prompt. Each line is length-capped so a long history can't crowd out
+    /// the message being replied to. Returns nil when there is nothing to add.
+    private func replyConversationContext(
+        from turns: [ClipboardReplyFeedbackStore.ConversationTurn]
+    ) -> String? {
+        guard !turns.isEmpty else { return nil }
+        let cap = 500
+        let lines = turns.flatMap { turn -> [String] in
+            [
+                "Them: " + String(turn.incoming.prefix(cap)),
+                "You: " + String(turn.reply.prefix(cap))
+            ]
+        }
+        return lines.joined(separator: "\n")
+    }
+
    private func beginReplyFeedback(variants: [AIReplyVariant]) {
        guard let sourceText = requestReplyFeedbackSource else { return }
        requestReplyFeedbackSource = nil
@@ -660,12 +754,19 @@ final class AIKeyboardCoordinator {
            clarificationQuestion: ask one essential missing question.
            clarificationConfirm: briefly confirm understanding, then ask the key question.
            """
+        case .yesNoQuestion:
+            roleGuidance = """
+            answerAffirmative: answer yes / it works, only when the source supports it.
+            answerNegative: answer no / it does not, when the source implies a problem or blocker.
+            answerConditional: answer that it depends, stating the specific condition.
+            answerDefer: say you need to confirm first, without inventing when.
+            """
        }
        return """
        MULTI-REPLY OUTPUT CONTRACT (highest priority):
        Return only one valid JSON object with exactly this shape and no Markdown fence or extra keys:
        {"variants":[\(items)]}
-        Include exactly these three kinds in the shown order. Every text must be a complete reply in the source language.
+        Include exactly these \(variantSet.kinds.count) kinds in the shown order. Every text must be a complete reply in the source language.
        \(roleGuidance)
        Every item must advance the conversation with a reaction, answer, question, decision, or next step. Never restate, paraphrase, summarize, or synonymically rewrite the clipboard text. In particular, do not begin a reply by repeating the source's subject and event. For a declarative update, react to its implication or emotion instead of reporting the update back to its sender.
        Apply any <reply_scene> constraint to every item. It overrides the kind-specific tone guidance below when they conflict.
@@ -786,6 +887,10 @@ final class AIKeyboardCoordinator {
        switch skillID {
        case AIClipboardSkillCatalog.navigateID:
            tipKey = "keyboard.ai.skill.openingMaps"
+        case AIClipboardSkillCatalog.extractTodosID:
+            tipKey = "keyboard.ai.skill.savingReminders"
+        case AIClipboardSkillCatalog.extractEventsID:
+            tipKey = "keyboard.ai.skill.savingEvents"
        default:
            tipKey = "keyboard.ai.skill.runningShortcut"
        }
@@ -42,6 +42,9 @@ public struct AppGroupPersistor {
        state.handednessPreference = store.handednessPreference
        state.clipboardHistoryEnabled = store.clipboardHistoryEnabled
        state.clipboardCandidateBarEnabled = store.clipboardCandidateBarEnabled
+        state.clipboardAutoModeEnabled = store.clipboardAutoModeEnabled
+        state.clipboardAutoTranslateEnabled = store.clipboardAutoTranslateEnabled
+        state.clipboardAutoEmailReplyEnabled = store.clipboardAutoEmailReplyEnabled
        applySkillSnapshot(store: store, into: state)
        state.keyboardHapticIntensity = store.keyboardHapticIntensity
        applyAPIKeyAvailability(store: store, into: state)
@@ -98,6 +101,9 @@ public struct AppGroupPersistor {
        state.handednessPreference = store.handednessPreference
        state.clipboardHistoryEnabled = store.clipboardHistoryEnabled
        state.clipboardCandidateBarEnabled = store.clipboardCandidateBarEnabled
+        state.clipboardAutoModeEnabled = store.clipboardAutoModeEnabled
+        state.clipboardAutoTranslateEnabled = store.clipboardAutoTranslateEnabled
+        state.clipboardAutoEmailReplyEnabled = store.clipboardAutoEmailReplyEnabled
        applySkillSnapshot(store: store, into: state)
        state.keyboardHapticIntensity = store.keyboardHapticIntensity
        applyAPIKeyAvailability(store: store, into: state)
@@ -112,24 +118,39 @@ public struct AppGroupPersistor {
        let layout = store.agentSkillLayout
        let language = store.uiLanguage
        state.uiLanguage = language
-        state.enabledClipboardSkillIDs = layout.enabledIDs
+        let personalReplyStyle = store.personalReplyStyle
+        let hasPersonalReplyStyle = personalReplyStyle != nil
+        let enabledIDs = AIClipboardSkillCatalog.availableEnabledIDs(
+            layout.enabledIDs,
+            layout: layout,
+            hasPersonalReplyStyle: hasPersonalReplyStyle
+        )
+        state.enabledClipboardSkillIDs = enabledIDs
        state.confirmedClipboardShortcutIDs = layout.confirmedShortcutIDs
        state.enabledClipboardSkills = AIClipboardSkillCatalog.visible(
-            enabledIDs: layout.enabledIDs,
+            enabledIDs: enabledIDs,
            officialCatalog: store.officialSkillCatalog,
            userCatalog: store.agentUserSkillCatalog,
            uiLanguage: language
        )
-        state.clipboardSkillCatalog = AIClipboardSkillCatalog.all(
-            officialCatalog: store.officialSkillCatalog,
-            userCatalog: store.agentUserSkillCatalog,
-            uiLanguage: language
+        // The keyboard's chips are ranked from this full catalog, not from the
+        // enabled list, so an unavailable skill has to be removed here or it
+        // still reaches the surface (and `AIKeyboardCoordinator` could run it).
+        state.clipboardSkillCatalog = AIClipboardSkillCatalog.availableForKeyboard(
+            AIClipboardSkillCatalog.all(
+                officialCatalog: store.officialSkillCatalog,
+                userCatalog: store.agentUserSkillCatalog,
+                uiLanguage: language
+            ),
+            layout: layout,
+            hasPersonalReplyStyle: hasPersonalReplyStyle
        )
-        let activeStyle = store.activePolishStyle
-        // Built-in formal/corporate personalities must not make ordinary
-        // clipboard replies sound less like the user.
+        // Independent of the voice-polish selection: the Styles page shapes
+        // dictation, this shapes AI replies. Sharing one selector previously
+        // made "clean dictation" and "replies that sound like me" mutually
+        // exclusive.
        state.clipboardReplyStyle = AIClipboardReplyStyleContext.resolve(
-            activeStyle: activeStyle
+            personalReplyStyle: personalReplyStyle
        )
    }

@@ -4,6 +4,7 @@
 // Samples the general pasteboard on keyboard appear and while visible
 // (changeCount-driven). Writes accepted text into ClipboardHistoryStore.

+import Combine
 import Foundation
 import OSGKeyboardShared
 import UIKit
@@ -45,6 +46,14 @@ final class ClipboardCaptureCoordinator {
    private var isSampling = false
    private var forcesNextSample = true
    private var isKeyboardVisible = false
+    /// OOBE reply / translate practice auto-fires once per feature so the user
+    /// sees the keyboard draft the result without tapping the chip. Tracked per
+    /// feature (the host keeps one session id across every practice step).
+    private var oobeAutoFiredSessionID: UUID?
+    private var oobeAutoFiredFeatures: Set<ManagedGatewayOOBEFeature> = []
+    /// Fires the auto-reply evaluation whenever a fresh semantic analysis lands,
+    /// independent of which keyboard surface (voice / Chinese / English) is shown.
+    private var snapshotObserver: AnyCancellable?

    init(
        state: KeyboardState,
@@ -56,6 +65,17 @@ final class ClipboardCaptureCoordinator {
        self.history = history
        self.semanticRanking = semanticRanking
        self.pasteboard = pasteboard
+        snapshotObserver = semanticRanking.$snapshot
+            .sink { [weak self] snapshot in
+                // `@Published` fires in willSet, so the store's property is not
+                // updated yet; hop to the main actor so the trigger reads the
+                // published snapshot (and satisfies actor isolation).
+                guard snapshot != nil else { return }
+                Task { @MainActor in
+                    self?.autoTriggerActionIfNeeded()
+                    self?.presentAutoReplyGuideIfNeeded()
+                }
+            }
    }

    func configure(
@@ -86,6 +106,7 @@ final class ClipboardCaptureCoordinator {
            semanticRanking.analyze(newest)
        }
        forcesNextSample = true
+        autoTriggerOOBESkillIfNeeded()
        // Delay the system pasteboard read until the first poll tick. A
        // Universal Clipboard fetch or paste alert during the appear sequence
        // can otherwise freeze the keyboard before SwiftUI draws.
@@ -104,12 +125,10 @@ final class ClipboardCaptureCoordinator {
    }

    func refreshFlagsFromStore() {
-        // Settings changes may hide the active suggestion, but enabling the
-        // strip must wait for a new pasteboard generation.
-        if !state.clipboardHistoryEnabled || !state.clipboardCandidateBarEnabled {
-            endCurrentSuggestion()
-        }
+        // Paste is available whenever clipboard history is on; only turning
+        // history off hides it (the old candidate-bar toggle is gone).
        if !state.clipboardHistoryEnabled {
+            endCurrentSuggestion()
            semanticRanking.clear()
        }
    }
@@ -150,6 +169,11 @@ final class ClipboardCaptureCoordinator {
    }

    func dismissSuggestion() {
+        // Dismissing from either surface must suppress both the paste capsule
+        // AND the AI skill top bar for this clipboard generation. The typing
+        // surface's X only routes here, so set the skill marker centrally
+        // rather than relying on the AI surface's own dismiss handler.
+        state.dismissedClipboardSkillEntryID = history.newestEntry?.id
        endCurrentSuggestion()
    }

@@ -186,6 +210,7 @@ final class ClipboardCaptureCoordinator {
        let timer = Timer(timeInterval: Self.pollInterval, repeats: true) { [weak self] _ in
            Task { @MainActor in
                self?.captureIfNeeded()
+                self?.autoTriggerOOBESkillIfNeeded()
            }
        }
        timer.tolerance = Self.pollInterval / 4
@@ -342,8 +367,9 @@ final class ClipboardCaptureCoordinator {
    }

    private func updateSuggestion(with entry: ClipboardHistoryEntry, changeCount: Int) {
+        // The one-tap Paste capsule is always on (its old opt-in toggle was
+        // removed), so this only depends on clipboard history + a secure field.
        guard state.canShowClipboardEntry,
-              state.clipboardCandidateBarEnabled,
              changeCount != secureFieldSuppressedChangeCount
        else {
            clearSuggestion()
@@ -372,4 +398,273 @@ final class ClipboardCaptureCoordinator {
        state.clipboardSuggestionText = nil
        state.clipboardSuggestionChangeCount = nil
    }
+
+    // MARK: - Auto mode
+
+    /// Auto mode: on every fresh semantic analysis, pick at most one automatic
+    /// action for the newest clipboard and route it into the assistant surface
+    /// with no tap. Surface-independent (voice / Chinese / English) and fired at
+    /// most once per pasteboard generation, persisted so a reopen never repeats.
+    ///
+    /// Precedence when several toggles apply to the same paste:
+    /// email reply (a detected email should be answered, not just translated —
+    /// even in another language) → translate (foreign, non-email) → generic reply.
+    private func autoTriggerActionIfNeeded() {
+        guard state.clipboardHistoryEnabled,
+              state.aiServiceAvailable,
+              state.oobePracticeSession == nil,
+              isKeyboardVisible,
+              isRestingForAutoReply else {
+            return
+        }
+        guard state.clipboardAutoModeEnabled
+                || state.clipboardAutoTranslateEnabled
+                || state.clipboardAutoEmailReplyEnabled else {
+            return
+        }
+        guard let newest = history.newestEntry,
+              let changeCount = newest.changeCount else {
+            return
+        }
+        // One auto action per copy, even across keyboard close/reopen.
+        guard history.lastAutoRepliedChangeCount != changeCount else { return }
+        guard AIHintPool.isClipboardSkillWindowActive(
+            clipboardHistoryEnabled: state.clipboardHistoryEnabled,
+            newestClipboard: newest
+        ) else {
+            return
+        }
+        guard let snapshot = semanticRanking.snapshot,
+              snapshot.entryID == newest.id else {
+            return
+        }
+        let catalog = state.clipboardSkillCatalog
+
+        // 1. Auto-draft a single, email-formatted reply for a detected email.
+        //    Checked before translate so a foreign-language email is answered
+        //    rather than merely translated. Staged like auto-translate: shown on
+        //    the keyboard, inserted only on the glass button.
+        if state.clipboardAutoEmailReplyEnabled,
+           ClipboardEmailDetector.isEmail(newest.text) {
+            fireAutoAction(
+                AIClipboardSkillCatalog.emailReplySkill,
+                scene: nil,
+                changeCount: changeCount,
+                readOnlyResult: true,
+                insertLabelKey: "keyboard.assistant.insertReply"
+            )
+            return
+        }
+        // 2. Auto-translate a non-system-language, non-email paste (staged).
+        if state.clipboardAutoTranslateEnabled,
+           ClipboardSkillSemanticRanker.isForeignLanguage(snapshot.analysis),
+           let translate = catalog.first(where: { $0.id == AIClipboardSkillCatalog.translateID }) {
+            fireAutoAction(translate, scene: nil, changeCount: changeCount, readOnlyResult: true)
+            return
+        }
+        // 3. Auto-reply for any interpersonal message worth answering — a task,
+        //    question, invitation, complaint, follow-up, or an explicit "please
+        //    reply". Gated on reply intent (not strict #1 ranking) so a message
+        //    that also scores Events / Todos / Summary still auto-replies. Bare
+        //    links / phones / foreign text keep their own actions.
+        if state.clipboardAutoModeEnabled,
+           ClipboardSkillSemanticRanker.isAutoReplyEligible(
+               sourceText: newest.text,
+               analysis: snapshot.analysis
+           ),
+           let reply = catalog.first(where: { $0.id == AIClipboardSkillCatalog.replyID }) {
+            fireAutoAction(
+                reply,
+                scene: AIClipboardReplyScene.resolve(
+                    from: snapshot.analysis,
+                    sourceText: newest.text
+                ),
+                changeCount: changeCount,
+                readOnlyResult: false
+            )
+        }
+    }
+
+    /// Submit the chosen auto action, remembering the keyboard to return to and
+    /// taking over the assistant surface so the result is visible.
+    private func fireAutoAction(
+        _ skill: AIClipboardSkill,
+        scene: AIClipboardReplyScene?,
+        changeCount: Int,
+        readOnlyResult: Bool,
+        insertLabelKey: String = "keyboard.assistant.insertTranslation"
+    ) {
+        // Mark handled before submitting so re-entrant publishes cannot double-fire.
+        history.lastAutoRepliedChangeCount = changeCount
+        // Remember the keyboard to return to once the result is used or closed:
+        // the surface the user is actively on, or — when the keyboard opened
+        // straight onto voice — the one they last left, so finishing never
+        // strands them on voice input. nil means "already where we'd land".
+        let returnSurface: KeyboardState.Surface = state.surface != .voice
+            ? state.surface
+            : TypingInputConfiguration.lastLeftSurface()
+        state.autoReplyReturnSurface = returnSurface == .voice ? nil : returnSurface
+        // Staged auto results (translate / email reply) wait for a glass-button
+        // tap; generic replies use their own variant surface.
+        state.autoResultReadOnly = readOnlyResult
+        state.autoResultInsertLabelKey = insertLabelKey
+        // The result UI only renders on the assistant surface, so take it over.
+        if state.surface != .voice {
+            state.setSurface(.voice)
+        }
+        state.submitAIClipboardSkill(skill, scene)
+    }
+
+    // MARK: - Auto-reply guide (one-time nudge)
+
+    /// Persisted "the user has already responded to the nudge once" flag. Stored
+    /// in the extension's own defaults; the guide is a keyboard-local affordance.
+    private static let autoReplyGuideTriedKey = "keyboard.assistant.autoReplyGuidanceTried"
+    private static var autoReplyGuideTried: Bool {
+        get { UserDefaults.standard.bool(forKey: autoReplyGuideTriedKey) }
+        set { UserDefaults.standard.set(newValue, forKey: autoReplyGuideTriedKey) }
+    }
+    /// Only auto-present once per pasteboard generation, so dismissing the layer
+    /// does not immediately resurface it for the same copy.
+    private var lastGuidedChangeCount: Int?
+
+    /// For users who never turned auto mode on: when a freshly copied message
+    /// reads as replyable, raise the clean full-keyboard guide instead of firing
+    /// a reply. Shown on every replyable copy until the user acts on it once.
+    private func presentAutoReplyGuideIfNeeded() {
+        guard !state.clipboardAutoModeEnabled,
+              !Self.autoReplyGuideTried,
+              state.clipboardHistoryEnabled,
+              state.aiServiceAvailable,
+              state.oobePracticeSession == nil,
+              state.canShowClipboardEntry,
+              state.clipboardOverlay == .none,
+              isKeyboardVisible,
+              isRestingForAutoReply else {
+            return
+        }
+        guard let newest = history.newestEntry,
+              let changeCount = newest.changeCount,
+              lastGuidedChangeCount != changeCount else {
+            return
+        }
+        guard AIHintPool.isClipboardSkillWindowActive(
+            clipboardHistoryEnabled: state.clipboardHistoryEnabled,
+            newestClipboard: newest
+        ) else {
+            return
+        }
+        guard let snapshot = semanticRanking.snapshot,
+              snapshot.entryID == newest.id else {
+            return
+        }
+        let recommended = ClipboardSkillSemanticRanker.recommended(
+            skills: state.clipboardSkillCatalog,
+            sourceText: newest.text,
+            analysis: snapshot.analysis,
+            uiLanguage: state.uiLanguage,
+            limit: 5
+        )
+        guard recommended.first?.id == AIClipboardSkillCatalog.replyID else { return }
+        lastGuidedChangeCount = changeCount
+        state.clipboardOverlay = .autoReplyGuide
+    }
+
+    /// The guide's primary action: opt into auto mode (persisted so it sticks),
+    /// close the layer, and draft the reply for the copy that prompted it — the
+    /// same submission every future auto reply now makes on its own.
+    func confirmAutoReplyGuide() {
+        Self.autoReplyGuideTried = true
+        AppGroupStore().setClipboardAutoModeEnabled(true)
+        state.clipboardAutoModeEnabled = true
+        state.clipboardOverlay = .none
+        guard let newest = history.newestEntry,
+              let snapshot = semanticRanking.snapshot,
+              snapshot.entryID == newest.id,
+              let reply = state.clipboardSkillCatalog.first(where: {
+                  $0.id == AIClipboardSkillCatalog.replyID
+              }) else {
+            return
+        }
+        let scene = AIClipboardReplyScene.resolve(
+            from: snapshot.analysis,
+            sourceText: newest.text
+        )
+        fireAutoAction(
+            reply,
+            scene: scene,
+            changeCount: newest.changeCount ?? history.lastObservedChangeCount,
+            readOnlyResult: false
+        )
+    }
+
+    /// OOBE reply / translate practice: once the host has seeded the demo
+    /// message, run the skill automatically — the same submission the chip makes
+    /// — so the user experiences the auto skill before any setting is turned on.
+    /// Runs off the poll tick (OOBE material is not semantic-ranked), and fires
+    /// at most once per practice feature within the host session. The translate
+    /// result is staged (glass Insert button) exactly like the live auto flow.
+    private func autoTriggerOOBESkillIfNeeded() {
+        guard let session = state.oobePracticeSession,
+              state.aiServiceAvailable,
+              isKeyboardVisible,
+              isRestingForAutoReply else {
+            return
+        }
+        let feature = session.expectedFeature
+        let skillID: String
+        switch feature {
+        case .clipboardReply:
+            skillID = AIClipboardSkillCatalog.replyID
+        case .clipboardTranslate:
+            skillID = AIClipboardSkillCatalog.translateID
+        case .voiceInput, .askAI:
+            return
+        }
+        // The host keeps one session id across steps, so gate per feature.
+        if oobeAutoFiredSessionID != session.sessionID {
+            oobeAutoFiredSessionID = session.sessionID
+            oobeAutoFiredFeatures = []
+        }
+        guard !oobeAutoFiredFeatures.contains(feature) else { return }
+        guard KeyboardSetupBridge.oobeClipboardMaterial(
+            sessionID: session.sessionID
+        ) != nil else {
+            return
+        }
+        guard let skill = state.clipboardSkillCatalog.first(where: {
+            $0.id == skillID
+        }) else {
+            return
+        }
+        // Mark handled before submitting so a re-entrant tick cannot double-fire.
+        oobeAutoFiredFeatures.insert(feature)
+        // Stage the translation like the live auto-translate: shown on the
+        // keyboard, inserted only when the user taps the glass button.
+        if feature == .clipboardTranslate {
+            state.autoResultReadOnly = true
+            state.autoResultInsertLabelKey = "keyboard.assistant.insertTranslation"
+        }
+        // The result UI only renders on the assistant surface, so take it over.
+        if state.surface != .voice {
+            state.setSurface(.voice)
+        }
+        state.submitAIClipboardSkill(skill, nil)
+    }
+
+    /// The voice pipeline must be idle before auto mode takes over the surface.
+    private var isRestingForAutoReply: Bool {
+        guard !state.aiSession.isBusy,
+              !state.aiSession.canInsert,
+              !state.aiSession.canSelectReplyVariant,
+              !state.editSession.isActive else {
+            return false
+        }
+        switch state.phase {
+        case .idle, .error, .denied:
+            return true
+        case .requestingPermissions, .recording, .processing:
+            return false
+        }
+    }
 }
@@ -39,6 +39,14 @@ final class KeyboardFlowCoordinator {

    private var isPendingFlowStart = false
    private var flowStartDeadline: TimeInterval = 0
+    /// Absolute cap for a cold-start jump. The soft `flowStartDeadline` is
+    /// pushed forward while the host keeps acknowledging it is warming, but
+    /// never past this — bounds the wait when the host is stuck in the
+    /// background and the user never brings it forward.
+    private var flowStartHardDeadline: TimeInterval = 0
+    /// One-shot guard so the "bring the app forward" hint is shown once per
+    /// start attempt instead of on every 200 ms watchdog poll.
+    private var coldStartForegroundHintShown = false
    private var isFlowRecording = false
    private var flowWatchdogTask: Task<Void, Never>?
    private var utteranceTimerTask: Task<Void, Never>?
@@ -995,6 +1003,7 @@ final class KeyboardFlowCoordinator {
            clearUnissuedUtterance()
            return .rejected(.missingAPIKey)
        case .unavailable(.noFullAccess):
+            persistStartupFailure("noFullAccess")
            let msg = ExtL10n.string("keyboard.error.fullAccessRequired")
            state.phase = .error(.fullAccessRequired, message: msg)
            scheduleAutoClearError()
@@ -1002,6 +1011,9 @@ final class KeyboardFlowCoordinator {
            clearUnissuedUtterance()
            return .rejected(.noFullAccess)
        case .unavailable(.appGroupUnavailable):
+            // Falls back to this extension's own container — the App Group is
+            // exactly what is missing here — so it will not show in Settings.
+            persistStartupFailure("appGroupUnavailable")
            let msg = ExtL10n.string("keyboard.error.appGroupCommunication")
            state.phase = .error(.appGroupUnavailable, message: msg)
            scheduleAutoClearError()
@@ -1031,6 +1043,9 @@ final class KeyboardFlowCoordinator {
            if isPendingFlowStart || recordWhenHostReady {
                return .waitingForHost(utteranceID)
            }
+            // Availability said ready, yet no command was issued and no wait is
+            // pending: the press is dropped with no error UI at all.
+            persistStartupFailure("pressDroppedHostUnavailable")
            clearUnissuedUtterance()
            return .rejected(.hostUnavailable)
        case .waitForHostReady(let recordWhenReady):
@@ -1087,6 +1102,7 @@ final class KeyboardFlowCoordinator {

    func beginFlowStart(recordAfterHandoff: Bool = false) {
        guard voiceSetupReady else {
+            persistStartupFailure("voiceSetupIncomplete")
            promptFinishSetupInApp()
            return
        }
@@ -1099,8 +1115,15 @@ final class KeyboardFlowCoordinator {
        coldStartDebouncer.reset()
        isPendingFlowStart = true
        isFlowRecording = false
+        coldStartForegroundHintShown = false
        let now = Date().timeIntervalSince1970
-        flowStartDeadline = currentStartDeadlineAt ?? (now + FlowWatchdog.startTimeout)
+        // This path always jumps to the host (`openHostApp("startflow")`), so
+        // the budget must cover host foreground + PiP arm + first ready publish.
+        // Anchor it at the jump (not the mic press) with the cold-launch budget;
+        // reusing the press-anchored 8 s charged app-launch latency against the
+        // recording budget and timed out legitimate cold starts.
+        flowStartDeadline = now + FlowSessionKeys.coldStartHostLaunchBudget
+        flowStartHardDeadline = now + FlowSessionKeys.coldStartHostLaunchHardCap
        currentStartDeadlineAt = flowStartDeadline
        state.lastTranscript = ""
        recomputeMicVoiceAvailability()
@@ -1131,6 +1154,9 @@ final class KeyboardFlowCoordinator {
            flowStartDeadline = 0
            stopFlowWatchdog()
            traceState("openHostApp.failed", extra: "path=startflow cancelPending=1")
+            // iOS refused the jump (no Full Access, or the open was blocked):
+            // the host process never even had a chance to record anything.
+            persistStartupFailure("hostOpenRejected")
            if currentUtteranceRequest?.isEdit == true {
                onEditFailure(ExtL10n.string("keyboard.error.manualOpenForFlow"))
                resetEditTransportState()
@@ -1645,6 +1671,10 @@ final class KeyboardFlowCoordinator {
    }

    private func failHostDisconnected() {
+        // Host was alive enough to be asked, then went away (jetsam, force-quit,
+        // suspended mid-utterance). Its own process is gone, so this side is the
+        // only one that can record it.
+        persistStartupFailure("hostDisconnected")
        if let id = currentUtteranceId,
           cancelledDictationUtteranceIDs.remove(id) != nil {
            stopUtteranceCountdown()
@@ -1769,6 +1799,7 @@ final class KeyboardFlowCoordinator {
    }

    private func showFlowSessionExpiredHint() {
+        persistStartupFailure("sessionExpired")
        let message = ExtL10n.string("keyboard.flow.sessionExpired")
        state.phase = .error(.flowSessionExpired, message: message)
        scheduleAutoClearError()
@@ -1804,6 +1835,9 @@ final class KeyboardFlowCoordinator {
    private func startFlowRecording() {
        if let deadline = currentStartDeadlineAt,
           Date().timeIntervalSince1970 >= deadline {
+            // The 8 s budget started at the mic press and was spent waiting for
+            // the host; recording never even got attempted.
+            persistStartupFailure("startBudgetExpiredBeforeRecord")
            if currentUtteranceRequest?.isEdit == true {
                onEditFailure(ExtL10n.string("keyboard.edit.error.startTimeout"))
                abortEditRecording()
@@ -2002,11 +2036,41 @@ final class KeyboardFlowCoordinator {
                    return
                }
                let now = Date().timeIntervalSince1970
+                // The host acknowledged the request and is alive & warming
+                // (cold launch / foreground handoff). Extend the soft budget up
+                // to the hard cap so a legitimate cold start is not failed at
+                // the initial launch budget while the host is provably working.
+                if let ackReason = FlowSessionBridge.hostStartInProgress() {
+                    let extended = min(
+                        now + FlowSessionKeys.hostStartAckExtension,
+                        self.flowStartHardDeadline
+                    )
+                    if extended > self.flowStartDeadline {
+                        self.flowStartDeadline = extended
+                        self.currentStartDeadlineAt = extended
+                    }
+                    // Host is up but parked waiting for the user to bring it to
+                    // the foreground — PiP cannot arm from the background. Prompt
+                    // once so the user can tap back instead of watching a silent
+                    // spinner run out the clock. Edit mode has no manual-open UI,
+                    // so keep letting it wait out the hard cap.
+                    if ackReason == .waitingForForeground,
+                       !self.coldStartForegroundHintShown,
+                       self.currentUtteranceRequest?.isEdit != true {
+                        self.coldStartForegroundHintShown = true
+                        self.showManualOpenHint(path: "startflow")
+                    }
+                }
                if self.flowStartDeadline > 0, now > self.flowStartDeadline {
                    self.isPendingFlowStart = false
                    self.recordAfterHandoff = false
-                    self.flowStartDeadline = 0
                    self.traceState("startWatchdog.timeout")
+                    // The single most common user-visible failure: the host was
+                    // asked to start and never published ready inside the
+                    // budget. Persist before clearing the deadline so the
+                    // report still carries it.
+                    self.persistStartupFailure("startTimeout")
+                    self.flowStartDeadline = 0
                    if self.currentUtteranceRequest?.isEdit == true {
                        self.onEditFailure(
                            ExtL10n.string("keyboard.edit.error.startTimeout")
@@ -2027,6 +2091,8 @@ final class KeyboardFlowCoordinator {
        isPendingFlowStart = false
        recordAfterHandoff = false
        flowStartDeadline = 0
+        flowStartHardDeadline = 0
+        coldStartForegroundHintShown = false
        stopFlowWatchdog()
        state.lastTranscript = ""
        refreshSessionState()
@@ -2322,9 +2388,44 @@ final class KeyboardFlowCoordinator {
    }

    private func debug(_ message: String) {
+        // Also feeds the shared breadcrumb window, so a terminal start failure
+        // on this side ships with the keyboard's own timeline instead of only
+        // whatever the host happened to record.
+        FlowFailureDiagnostics.record(message)
        OSGLog.keyboardExt.info("\(message, privacy: .public)")
    }

+    /// Writes a startup-failure report for a keyboard-observed dead end and
+    /// asks the host for its companion window. This is the capture path that
+    /// survives the host never launching at all.
+    private func persistStartupFailure(_ reason: String) {
+        FlowFailureDiagnostics.persistKeyboardStartupFailure(
+            reason: reason,
+            context: keyboardStartupFailureContext()
+        )
+    }
+
+    private func keyboardStartupFailureContext() -> [String: String] {
+        let now = Date().timeIntervalSince1970
+        return [
+            "pendingFlowStart": isPendingFlowStart ? "true" : "false",
+            "startDeadlineRemainingSeconds": flowStartDeadline > 0
+                ? String(format: "%.3f", flowStartDeadline - now)
+                : "nil",
+            "startBudgetSeconds": String(format: "%.1f", FlowWatchdog.startTimeout),
+            "recordAfterHandoff": recordAfterHandoff ? "true" : "false",
+            "flowRecording": isFlowRecording ? "true" : "false",
+            "awaitingResult": isAwaitingFlowResult ? "true" : "false",
+            "sessionProvenReady": sessionProvenReady ? "true" : "false",
+            "fullAccess": hasFullAccess() ? "true" : "false",
+            "voiceSetupReady": voiceSetupReady ? "true" : "false",
+            "requestKind": currentUtteranceRequest?.isEdit == true
+                ? "edit"
+                : (currentUtteranceRequest?.isAIQuestion == true ? "aiQuestion" : "dictation"),
+            "keyboardPhase": String(describing: state.phase)
+        ]
+    }
+
    private func traceState(_ event: String, extra: String? = nil) {
        let staleness = FlowSessionBridge.heartbeatStaleness().map { String(format: "%.1f", $0) } ?? "nil"
        let sessionId = activeSessionId?.uuidString ?? "nil"