feat(flow): on-demand host handoff with reliable return and a back-tap tip
Keyboard
- Only jump to the app when the voice surface is shown (tab tap / opened
there); typing tabs no longer leave the host on keyboard open.
- Wait up to 0.4 s for the arbiter to report the host app before leaving,
and late-bind an id that arrives within 6 s of the jump.
- Empty arbiter callbacks no longer wipe an already captured host id;
install the capture handler in every build configuration.
App
- Return to the previous app with up to four retries using iOS's real
open result, and honor the skip-app-switch preference.
- Keep the pending host id across cold-start zombie reconciliation, and
expire it after two minutes (legacy values without a timestamp stay valid).
- Show a light tip pointing at the system back control when we cannot
return automatically ("点击返回,立即使用").
Tests: TTL behaviour and return retry policy.
This commit is contained in:
@@ -7,7 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **“Tap back” tip when we cannot return for you**: after the keyboard sends you to OSG to start voice, and OSG cannot jump back on its own (the app you came from is unknown, or the jump failed), a light veil appears with a small arrow pointing at iOS's own back control at the top-left and one line, “点击返回,立即使用” / “Tap back, start right away”. It works even when OSG could not tell which app you were in, because the system control already names it. Tap anywhere to dismiss; it also disappears as soon as you leave. / **无法替你返回时的「点击返回」提示**:键盘带你进入 OSG 启动语音后,如果 OSG 没能自动跳回(不知道来自哪个 App,或跳转失败),会出现一层淡淡的遮罩,一个指向 iOS 左上角系统返回按钮的小箭头,以及一句「点击返回,立即使用」。即使没识别出你来自哪个 App 也会显示,因为系统按钮本身就写着来源 App 的名字。点任意处关闭,你一离开也会自动消失。
|
||||
|
||||
### Changed
|
||||
- **The keyboard only opens OSG when you need it**: opening the keyboard on the typing tabs (拼音 / English) no longer jumps to OSG. The jump now happens when you land on the voice tab, or when you start dictation, ask AI, or run a clipboard skill (including automatic ones). A short cool-down keeps flipping between tabs from reopening the app. / **只在需要时才打开 OSG**:在拼音 / 英文标签页打开键盘不再跳转到 OSG。切到语音标签页,或开始听写、提问 AI、使用剪贴板技能(包括自动触发的)时才会跳转;短暂的冷却时间避免来回切换标签页反复拉起 App。
|
||||
- **Auto-return now retries and respects your setting**: going back to the previous app is tried up to four times, waiting for voice to be ready and for a late app identification, instead of a single attempt, and it now follows the “return to previous app after starting” preference instead of ignoring it. / **自动返回会重试,并遵循你的设置**:返回上一个 App 最多尝试四次,会等待语音就绪和迟到的 App 识别,不再只试一次;同时遵循「启动后返回上一个 App」设置,而不是忽略它。
|
||||
|
||||
### Fixed
|
||||
- **Host app identification no longer flickers**: the keyboard waits up to 0.4 s for the system to report which app you are in before it leaves, keeps a correct identification when a later empty system callback arrives, and still records an identification that arrives a few seconds after the jump. / **宿主 App 识别不再忽有忽无**:键盘离开前最多等待 0.4 秒,让系统报告你所在的 App;之后到达的空回调不会再抹掉已识别到的结果;跳转后几秒内才到达的识别结果也会被补记。
|
||||
- **Return target survives a cold launch and expires safely**: starting OSG after it had been force-quit no longer wipes the app to return to, and a recorded target older than two minutes is ignored so you are never sent to an app you left long ago. / **返回目标不再在冷启动时丢失,也不会过期误跳**:强杀后冷启动 OSG 不再清掉要返回的 App;记录超过两分钟的返回目标会被忽略,不会把你送回早已离开的 App。
|
||||
- **Auto-return to the previous app in every build**: host-app identification used to compile only in Debug, so TestFlight and App Store builds never knew which app you were typing in and could not jump back after starting a voice session. It now runs in Debug, TestFlight, and Release; if the system does not expose the host (iOS earlier than 26.4), OSG simply stays open as before. / **所有版本都能自动跳回原 App**:此前识别宿主 App 的逻辑只在 Debug 构建中编译,TestFlight 和 App Store 版本不知道你正在哪个 App 里输入,启动语音会话后无法自动跳回。现在 Debug、TestFlight、正式版都会识别;系统不提供宿主信息时(低于 iOS 26.4)与之前一样停留在 OSG。
|
||||
|
||||
## [2.1.6] - 2026-10-27
|
||||
|
||||
@@ -259,6 +259,10 @@ final class FlowSessionManager: ObservableObject {
|
||||
/// Throttles AI-mode LLM draft writes into App Group.
|
||||
private var aiAnswerStreamThrottle = AIAnswerStreamThrottle()
|
||||
/// Published so Home / debug UI can show "recording" instead of a false "ready".
|
||||
/// True while the "tap the ‹ at the top-left to go back" tip should show: the
|
||||
/// session is ready after a keyboard handoff but we could not return to the
|
||||
/// previous app ourselves. Cleared when the user leaves the app or taps it.
|
||||
@Published private(set) var isReturnTipVisible = false
|
||||
@Published private(set) var isUtteranceRecording = false
|
||||
/// True from `stopped` until the result/error is written back to App Group.
|
||||
@Published private(set) var isUtteranceProcessing = false
|
||||
@@ -286,6 +290,8 @@ final class FlowSessionManager: ObservableObject {
|
||||
private var isAppForeground = false
|
||||
/// True while handling a keyboard-initiated `startflow` cold start.
|
||||
private var isColdStartHandoff = false
|
||||
/// Drives the retrying "return to the previous app" attempts after a handoff.
|
||||
private var autoReturnTask: Task<Void, Never>?
|
||||
/// User intent outlives a transient PiP loss; only explicit teardown clears it.
|
||||
private var wantsActiveSession = false
|
||||
/// Invalidates late async completions after retry, stop, or process teardown.
|
||||
@@ -487,7 +493,15 @@ final class FlowSessionManager: ObservableObject {
|
||||
if isActive {
|
||||
endSession()
|
||||
} else {
|
||||
// `clearFlowState()` also wipes the pending host id. On a cold
|
||||
// `startflow` launch after a force-quit the keyboard has already
|
||||
// recorded where to return to; keep it or auto-return has nothing
|
||||
// to return to.
|
||||
let pendingHost = FlowSessionBridge.pendingHostBundleId()
|
||||
FlowSessionBridge.clearFlowState()
|
||||
if let pendingHost {
|
||||
FlowSessionBridge.setPendingHostBundleId(pendingHost)
|
||||
}
|
||||
}
|
||||
debug("reconciled zombie persisted Flow state")
|
||||
case .clearOrphanedRecording(let orphaned):
|
||||
@@ -607,6 +621,8 @@ final class FlowSessionManager: ObservableObject {
|
||||
pipRecoveryOperation &+= 1
|
||||
pipLifecycleState = .inactive
|
||||
isColdStartHandoff = false
|
||||
autoReturnTask?.cancel()
|
||||
autoReturnTask = nil
|
||||
startTask?.cancel()
|
||||
startTask = nil
|
||||
startupAudioHealthTask?.cancel()
|
||||
@@ -693,6 +709,8 @@ final class FlowSessionManager: ObservableObject {
|
||||
pipRecoveryOperation &+= 1
|
||||
pipLifecycleState = .inactive
|
||||
isColdStartHandoff = false
|
||||
autoReturnTask?.cancel()
|
||||
autoReturnTask = nil
|
||||
startTask?.cancel()
|
||||
startTask = nil
|
||||
startupAudioHealthTask?.cancel()
|
||||
@@ -785,6 +803,11 @@ final class FlowSessionManager: ObservableObject {
|
||||
} else if wantsActiveSession {
|
||||
pausePiPRecoveryUntilForeground()
|
||||
}
|
||||
// The user left (by our auto-return or on their own): stop retrying
|
||||
// and drop the tip so it is not still showing on their next visit.
|
||||
autoReturnTask?.cancel()
|
||||
autoReturnTask = nil
|
||||
dismissReturnTip()
|
||||
if isColdStartHandoff {
|
||||
completeColdStartHandoff()
|
||||
}
|
||||
@@ -1511,25 +1534,67 @@ final class FlowSessionManager: ObservableObject {
|
||||
return
|
||||
}
|
||||
|
||||
let hostEntry = HostReturnService.pendingHostEntry()
|
||||
guard hostEntry != nil else {
|
||||
// Honor the user's "return to the previous app" preference. When it is
|
||||
// off we never jump back on our own, but the tip still tells the user
|
||||
// how to go back themselves.
|
||||
guard FlowSessionPolicy.skipAppSwitch() else {
|
||||
presentReturnTip(reason: "autoReturnDisabled")
|
||||
completeColdStartHandoff()
|
||||
return
|
||||
}
|
||||
scheduleAutoReturnToHostIfNeeded(hostEntry: hostEntry)
|
||||
scheduleAutoReturnToHost()
|
||||
}
|
||||
|
||||
private func scheduleAutoReturnToHostIfNeeded(hostEntry: HostAppEntry?) {
|
||||
let skipSwitch = true
|
||||
guard skipSwitch, hostEntry != nil else { return }
|
||||
Task { @MainActor [weak self] in
|
||||
try? await Task.sleep(nanoseconds: 450_000_000)
|
||||
guard let self, self.isColdStartHandoff, self.isActive,
|
||||
FlowSessionBridge.isHostReady() else { return }
|
||||
if HostReturnService.openPendingHostIfPossible() {
|
||||
/// Tries to return to the previous app, retrying a few times. The host id the
|
||||
/// keyboard recorded can arrive a moment after the session is ready, and the
|
||||
/// first `open` can lose a race with PiP settling — one shot was brittle.
|
||||
///
|
||||
/// The tip goes up as soon as we know there is nowhere to return to, and also
|
||||
/// once every attempt has failed. If a later attempt does succeed, the app
|
||||
/// moves to the background and the tip is cleared with it.
|
||||
private func scheduleAutoReturnToHost() {
|
||||
autoReturnTask?.cancel()
|
||||
autoReturnTask = Task { @MainActor [weak self] in
|
||||
for delay in FlowHandoffPolicy.hostReturnRetryDelays {
|
||||
try? await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000))
|
||||
guard let self, !Task.isCancelled, self.isActive, self.isAppForeground else {
|
||||
return
|
||||
}
|
||||
// Not ready (yet): no point leaving, and "use it now" would be a lie.
|
||||
guard FlowSessionBridge.isHostReady() else { continue }
|
||||
|
||||
switch await HostReturnService.openPendingHost() {
|
||||
case .opened:
|
||||
self.traceState("autoReturn.opened")
|
||||
self.completeColdStartHandoff()
|
||||
return
|
||||
case .noTarget:
|
||||
// Nothing to return to — guide the user now, keep retrying
|
||||
// in case the keyboard's late binding lands.
|
||||
self.presentReturnTip(reason: "noTarget")
|
||||
case .openFailed:
|
||||
self.traceState("autoReturn.openFailed")
|
||||
}
|
||||
}
|
||||
guard let self, !Task.isCancelled, self.isActive, self.isAppForeground else {
|
||||
return
|
||||
}
|
||||
self.presentReturnTip(reason: "retriesExhausted")
|
||||
self.completeColdStartHandoff()
|
||||
}
|
||||
}
|
||||
|
||||
private func presentReturnTip(reason: String) {
|
||||
guard !isReturnTipVisible else { return }
|
||||
isReturnTipVisible = true
|
||||
traceState("returnTip.shown", extra: "reason=\(reason)")
|
||||
}
|
||||
|
||||
/// Hides the tip (user tapped it, or the app left the foreground).
|
||||
func dismissReturnTip() {
|
||||
guard isReturnTipVisible else { return }
|
||||
isReturnTipVisible = false
|
||||
traceState("returnTip.dismissed")
|
||||
}
|
||||
|
||||
/// Actively rebuilds the audio pipeline after a failed cold start instead
|
||||
|
||||
@@ -11,35 +11,42 @@ import OSGKeyboardShared
|
||||
import UIKit
|
||||
|
||||
enum HostReturnService {
|
||||
/// Attempts to return to the pending host app. Clears the pending bundle id on success.
|
||||
/// Result of one return attempt. Callers retry on everything except `.opened`.
|
||||
enum Outcome: String {
|
||||
/// iOS reported the host app was actually opened.
|
||||
case opened
|
||||
/// No usable return target (host not recorded, or not in the whitelist).
|
||||
case noTarget
|
||||
/// A target exists but iOS refused / failed to open it.
|
||||
case openFailed
|
||||
}
|
||||
|
||||
/// Attempts to return to the pending host app and waits for iOS's *real*
|
||||
/// answer. Clears the pending bundle id only when the open succeeded, so a
|
||||
/// failed attempt can be retried with the same target.
|
||||
///
|
||||
/// Re-reads the App Group first: the keyboard process wrote the target (and
|
||||
/// may still be late-binding it), so a cached value here can be stale.
|
||||
@MainActor
|
||||
static func openPendingHostIfPossible() -> Bool {
|
||||
static func openPendingHost() async -> Outcome {
|
||||
FlowSessionBridge.reloadFromDisk()
|
||||
let bundleId = FlowSessionBridge.pendingHostBundleId()
|
||||
guard let entry = HostAppURLRegistry.lookup(bundleId: bundleId),
|
||||
let url = entry.returnURL else {
|
||||
logReturn(bundleId: bundleId, outcome: "no-entry")
|
||||
return false
|
||||
logReturn(bundleId: bundleId, outcome: Outcome.noTarget.rawValue)
|
||||
return .noTarget
|
||||
}
|
||||
UIApplication.shared.open(url, options: [:]) { success in
|
||||
logReturn(bundleId: bundleId, outcome: success ? "opened" : "open-failed")
|
||||
let success = await withCheckedContinuation { (continuation: CheckedContinuation<Bool, Never>) in
|
||||
UIApplication.shared.open(url, options: [:]) { opened in
|
||||
continuation.resume(returning: opened)
|
||||
}
|
||||
}
|
||||
let outcome: Outcome = success ? .opened : .openFailed
|
||||
logReturn(bundleId: bundleId, outcome: outcome.rawValue)
|
||||
if success {
|
||||
FlowSessionBridge.clearPendingHostBundleId()
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
@MainActor
|
||||
static func openHost(entry: HostAppEntry) -> Bool {
|
||||
guard let url = entry.returnURL else {
|
||||
return false
|
||||
}
|
||||
UIApplication.shared.open(url, options: [:]) { success in
|
||||
if success {
|
||||
FlowSessionBridge.clearPendingHostBundleId()
|
||||
}
|
||||
}
|
||||
return true
|
||||
return outcome
|
||||
}
|
||||
|
||||
@MainActor
|
||||
|
||||
@@ -88,7 +88,16 @@ struct MainAppRoot: View {
|
||||
palette.background
|
||||
.ignoresSafeArea()
|
||||
mainContent
|
||||
|
||||
// After a keyboard handoff we could not auto-return from: point the
|
||||
// user at the system's back control.
|
||||
if flowManager.isReturnTipVisible {
|
||||
ReturnTipOverlay { flowManager.dismissReturnTip() }
|
||||
.transition(.opacity)
|
||||
.zIndex(10)
|
||||
}
|
||||
}
|
||||
.animation(.easeOut(duration: 0.2), value: flowManager.isReturnTipVisible)
|
||||
.environment(\.locale, config.uiLanguage.swiftUILocale)
|
||||
.environmentObject(flowManager)
|
||||
.environmentObject(accountSession)
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
// ReturnTipOverlay.swift
|
||||
// OSGKeyboard · Main App
|
||||
//
|
||||
// Shown after a keyboard → app handoff when we could not return to the previous
|
||||
// app on our own. It does one thing: point at the system's back control.
|
||||
//
|
||||
// iOS draws a small "◀ <previous app>" at the top-left whenever this app was
|
||||
// opened from another one. We cannot move or restyle it, so the tip is just a
|
||||
// light scrim, an arrow aimed at it, and a single sentence. It names no app —
|
||||
// the system control already does, and it works even when we could not tell
|
||||
// which app the user came from.
|
||||
|
||||
import OSGKeyboardShared
|
||||
import SwiftUI
|
||||
|
||||
struct ReturnTipOverlay: View {
|
||||
@Environment(\.themePalette) private var palette: ThemePalette
|
||||
@Environment(\.accessibilityReduceMotion) private var reduceMotion
|
||||
|
||||
/// Called when the user taps anywhere on the tip.
|
||||
let onDismiss: () -> Void
|
||||
|
||||
/// Drives the gentle up-and-down nudge of the arrow.
|
||||
@State private var nudge = false
|
||||
|
||||
/// Veil over the page: light enough that the app stays visible underneath,
|
||||
/// opaque enough that the centred sentence remains easy to read.
|
||||
private let scrimOpacity = 0.88
|
||||
|
||||
var body: some View {
|
||||
ZStack(alignment: .topLeading) {
|
||||
palette.background
|
||||
.opacity(scrimOpacity)
|
||||
.ignoresSafeArea()
|
||||
|
||||
arrowGuide
|
||||
.padding(.leading, 14)
|
||||
// This stack starts at the top safe-area edge (the status bar's
|
||||
// bottom), where the system back control sits just above. A small
|
||||
// negative inset tucks the arrow's tip right under it on every device.
|
||||
.padding(.top, -6)
|
||||
.offset(y: nudge ? 4 : -2)
|
||||
|
||||
Text("flow.returnTip.message")
|
||||
.font(.system(size: 22, weight: .semibold))
|
||||
.foregroundStyle(palette.textPrimary)
|
||||
.multilineTextAlignment(.center)
|
||||
.padding(.horizontal, Spacing.xl)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
}
|
||||
.contentShape(Rectangle())
|
||||
.onTapGesture(perform: onDismiss)
|
||||
.onAppear {
|
||||
guard !reduceMotion else { return }
|
||||
withAnimation(.easeInOut(duration: 0.9).repeatForever(autoreverses: true)) {
|
||||
nudge = true
|
||||
}
|
||||
}
|
||||
.accessibilityElement(children: .ignore)
|
||||
.accessibilityLabel(Text("flow.returnTip.message"))
|
||||
.accessibilityHint(Text("flow.returnTip.a11y.hint"))
|
||||
.accessibilityAddTraits(.isButton)
|
||||
}
|
||||
|
||||
// MARK: - Pieces
|
||||
|
||||
/// Curved arrow plus its one-line caption, pointing up-left at the system control.
|
||||
private var arrowGuide: some View {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
ReturnTipArrow()
|
||||
.stroke(
|
||||
palette.accent,
|
||||
style: StrokeStyle(lineWidth: 2.6, lineCap: .round, lineJoin: .round)
|
||||
)
|
||||
.frame(width: 60, height: 50)
|
||||
Text("flow.returnTip.arrowHint")
|
||||
.font(.system(size: 14, weight: .semibold))
|
||||
.foregroundStyle(palette.accent)
|
||||
}
|
||||
.accessibilityHidden(true)
|
||||
}
|
||||
}
|
||||
|
||||
/// Hand-drawn style curved arrow whose head points up-left.
|
||||
private struct ReturnTipArrow: Shape {
|
||||
func path(in rect: CGRect) -> Path {
|
||||
var path = Path()
|
||||
let start = CGPoint(x: rect.minX + 0.50 * rect.width, y: rect.maxY)
|
||||
let end = CGPoint(x: rect.minX + 0.12 * rect.width, y: rect.minY + 3)
|
||||
let control = CGPoint(x: rect.minX + 0.55 * rect.width, y: rect.minY + 0.25 * rect.height)
|
||||
path.move(to: start)
|
||||
path.addQuadCurve(to: end, control: control)
|
||||
|
||||
// Arrow head: two short strokes fanned ±30° off the curve's end tangent.
|
||||
let dx = end.x - control.x
|
||||
let dy = end.y - control.y
|
||||
let length = max(hypot(dx, dy), 0.001)
|
||||
let ux = dx / length
|
||||
let uy = dy / length
|
||||
let headLength: CGFloat = 11
|
||||
for sign in [CGFloat(1), CGFloat(-1)] {
|
||||
let angle = sign * .pi / 6
|
||||
let rx = -(ux * cos(angle) - uy * sin(angle))
|
||||
let ry = -(ux * sin(angle) + uy * cos(angle))
|
||||
path.move(to: end)
|
||||
path.addLine(to: CGPoint(x: end.x + rx * headLength, y: end.y + ry * headLength))
|
||||
}
|
||||
return path
|
||||
}
|
||||
}
|
||||
@@ -855,6 +855,11 @@
|
||||
|
||||
/* Cold-start / session errors (never say Picture in Picture to the user) */
|
||||
"flow.coldStart.error.audioTimeout" = "The microphone did not become ready in time. It may be busy in another app. Please try again.";
|
||||
|
||||
/* Shown after a keyboard handoff when we cannot return automatically: points at iOS's top-left back control. */
|
||||
"flow.returnTip.message" = "Tap back, start right away";
|
||||
"flow.returnTip.arrowHint" = "Tap here to go back";
|
||||
"flow.returnTip.a11y.hint" = "Double-tap to dismiss the tip.";
|
||||
"flow.session.error.unsupported" = "This device cannot keep a voice session alive.";
|
||||
"flow.session.error.hostNotReady" = "The voice session is not ready yet. Stay in the app and try again.";
|
||||
"flow.session.error.notPossible" = "The system cannot start the voice session right now. Keep the app in the foreground and try again.";
|
||||
|
||||
@@ -855,6 +855,11 @@
|
||||
|
||||
/* 冷启动 / 会话错误(用户可见文案不出现「画中画」) */
|
||||
"flow.coldStart.error.audioTimeout" = "麦克风未能及时就绪,可能正被通话或其他 App 占用。请稍后重试。";
|
||||
|
||||
/* 键盘跳转后无法自动返回时的提示:指向 iOS 左上角的系统返回按钮 */
|
||||
"flow.returnTip.message" = "点击返回,立即使用";
|
||||
"flow.returnTip.arrowHint" = "点这里返回";
|
||||
"flow.returnTip.a11y.hint" = "轻点关闭提示。";
|
||||
"flow.session.error.unsupported" = "此设备无法维持语音会话。";
|
||||
"flow.session.error.hostNotReady" = "语音会话尚未就绪,请留在 App 内稍后重试。";
|
||||
"flow.session.error.notPossible" = "系统暂时无法启动语音会话,请保持 App 在前台后重试。";
|
||||
|
||||
@@ -176,9 +176,9 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
|
||||
public override func viewDidLoad() {
|
||||
super.viewDidLoad()
|
||||
#if DEBUG
|
||||
HostBundleProbe.installLogging()
|
||||
#endif
|
||||
// Host identification (return-to-app after a cold-start handoff) runs in
|
||||
// every build; the handler also writes the DEBUG probe log.
|
||||
HostAppIdentity.installCaptureHandler()
|
||||
// Voice-first keyboard — hide the misleading "English" subtitle in Settings.
|
||||
primaryLanguage = "mis"
|
||||
let preferred = TypingInputConfiguration.preferredSurfaceOnOpen()
|
||||
@@ -373,10 +373,12 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
lockPresentedKeyboardHeight()
|
||||
refreshReturnKeyRole()
|
||||
scheduleTypingDocumentSynchronization()
|
||||
// Presentation math is locked before arming the PiP handoff.
|
||||
// Presentation math is locked before arming the PiP handoff. Only a
|
||||
// keyboard that opens straight onto the voice surface arms the host here;
|
||||
// typing opens never leave the host app.
|
||||
DispatchQueue.main.async { [weak self] in
|
||||
guard let self, self.heightPhase == .presented else { return }
|
||||
self.flowCoordinator.ensurePiPReadyOnKeyboardOpen()
|
||||
self.flowCoordinator.armHostForVoiceSurfaceIfNeeded(reason: "keyboardOpened")
|
||||
}
|
||||
#if DEBUG
|
||||
HostBundleProbe.recordPresentation()
|
||||
@@ -798,7 +800,13 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
state.copySelection = { [weak self] in self?.textInserter.copySelection() }
|
||||
state.cutSelection = { [weak self] in self?.textInserter.cutSelection() }
|
||||
state.setSurface = { [weak self] surface in
|
||||
self?.applySurface(surface)
|
||||
guard let self else { return }
|
||||
self.applySurface(surface)
|
||||
// Landing on the voice surface (tab tap, or an auto-fired clipboard
|
||||
// skill taking over the assistant surface) is the demand signal for
|
||||
// the host app. The coordinator ignores any other surface, and its
|
||||
// cooldown absorbs repeated flips.
|
||||
self.flowCoordinator.armHostForVoiceSurfaceIfNeeded(reason: "surfaceSelected")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -162,18 +162,29 @@ final class KeyboardFlowCoordinator {
|
||||
stopHostReadyWait()
|
||||
}
|
||||
|
||||
/// Ensure the containing app has armed its low-profile PiP even when the
|
||||
/// keyboard opens directly into Pinyin/English typing mode. The mic stays
|
||||
/// visually ready; if the host contract is missing, one automatic handoff
|
||||
/// prepares PiP so the next press does not need another app switch.
|
||||
/// Demand-driven host arm: jump to the containing app only when the user is
|
||||
/// actually on a surface that needs it (the voice / assistant surface).
|
||||
///
|
||||
/// Typing (拼音 / English) and clipboard history work without the host, so the
|
||||
/// keyboard no longer hops to the app just because it opened. Every other
|
||||
/// host-dependent entry — mic press, AI question, clipboard skill (manual or
|
||||
/// auto-fired) — already funnels through `startUtterance`, which cold-starts
|
||||
/// the host on demand. This covers the one remaining case: the user lands on
|
||||
/// the voice surface (keyboard opened there, or the tab was tapped) while the
|
||||
/// host's ready contract is missing, so the first press doesn't need a hop.
|
||||
///
|
||||
/// Must **not** re-jump when a session is already alive/warming (common after
|
||||
/// the user returns from startflow and taps the Voice tab — height/appear
|
||||
/// noise used to call this again while `ready` briefly lagged).
|
||||
func ensurePiPReadyOnKeyboardOpen() {
|
||||
/// noise used to call this again while `ready` briefly lagged). The shared
|
||||
/// arm cooldown also keeps repeated tab flips from re-launching the app.
|
||||
func armHostForVoiceSurfaceIfNeeded(reason: String) {
|
||||
#if DEBUG
|
||||
MemoryDeviceProbe.log("pip.ensure.entry")
|
||||
#endif
|
||||
guard state.surface == .voice else {
|
||||
traceState("hostArm.skipped", extra: "reason=\(reason) surface=\(state.surface.rawValue)")
|
||||
return
|
||||
}
|
||||
guard FlowHandoffPolicy.allowsProactiveHostAutoLaunch,
|
||||
voiceSetupReady,
|
||||
hasFullAccess(),
|
||||
@@ -196,7 +207,7 @@ final class KeyboardFlowCoordinator {
|
||||
heartbeatStaleness: FlowSessionBridge.heartbeatStaleness()
|
||||
)
|
||||
guard shouldArm else {
|
||||
traceState("keyboardOpen.autoArmPiP.skipped", extra: "gate=0")
|
||||
traceState("hostArm.skipped", extra: "reason=\(reason) gate=0")
|
||||
return
|
||||
}
|
||||
|
||||
@@ -206,7 +217,7 @@ final class KeyboardFlowCoordinator {
|
||||
MemoryDeviceProbe.log("pip.arm.beginFlowStart")
|
||||
#endif
|
||||
beginFlowStart(recordAfterHandoff: false)
|
||||
traceState("keyboardOpen.autoArmPiP")
|
||||
traceState("hostArm.started", extra: "reason=\(reason)")
|
||||
}
|
||||
|
||||
func refreshSessionState() {
|
||||
@@ -1138,10 +1149,34 @@ final class KeyboardFlowCoordinator {
|
||||
state.lastTranscript = ""
|
||||
recomputeMicVoiceAvailability()
|
||||
FlowSessionBridge.markPiPArmAttempt()
|
||||
|
||||
// The return target must be known *before* we leave — the host app reads
|
||||
// it as soon as its session is ready. The arbiter callback that carries
|
||||
// the host id can lag a few ms behind the keyboard appearing, so when the
|
||||
// probe is installed but has not delivered yet, wait briefly (bounded)
|
||||
// instead of racing it. Already-known ids and uninstalled probes skip the
|
||||
// wait and keep the jump synchronous.
|
||||
if HostAppIdentity.currentBundleId != nil || !HostAppIdentity.isProbeInstalled {
|
||||
jumpToHostForFlowStart()
|
||||
return
|
||||
}
|
||||
Task { @MainActor [weak self] in
|
||||
_ = await HostAppIdentity.resolveForHandoff()
|
||||
// The user may have released the mic / cancelled while we waited.
|
||||
guard let self, self.isPendingFlowStart else {
|
||||
self?.traceState("beginFlowStart.cancelledDuringIdentityWait")
|
||||
return
|
||||
}
|
||||
self.jumpToHostForFlowStart()
|
||||
}
|
||||
}
|
||||
|
||||
/// Publishes the return target and opens `osgkeyboard://startflow`.
|
||||
private func jumpToHostForFlowStart() {
|
||||
HostAppIdentity.publishReturnTarget()
|
||||
OSGDiag.log(
|
||||
"beginFlowStart → openHostApp(startflow) recordAfterHandoff=\(recordAfterHandoff) "
|
||||
+ "\(OSGDiag.memoryTag())",
|
||||
+ "hostKnown=\(HostAppIdentity.currentBundleId != nil) \(OSGDiag.memoryTag())",
|
||||
category: "boot"
|
||||
)
|
||||
openHostApp("startflow")
|
||||
|
||||
@@ -3,13 +3,26 @@
|
||||
//
|
||||
// 识别当前宿主 App 的 bundle ID,供冷启动跳转主 App 后自动跳回。
|
||||
// 依赖私有 arbiter 探针(`OSGHostArbiterProbe`),Debug / TestFlight / Release 均生效;
|
||||
// 探针装不上(系统 < 26.4、私有符号改名)时识别结果为 nil,退化为“不自动跳回”。
|
||||
// 探针装不上(系统 < 26.4、私有符号改名)时识别结果为 nil,退化为“不自动跳回”,
|
||||
// 主 App 随后会显示“点击左上角返回”的提示作为兜底。
|
||||
// 调试日志与首页测试卡片仍只在 DEBUG 编译。
|
||||
|
||||
import Foundation
|
||||
import os
|
||||
import OSGKeyboardShared
|
||||
|
||||
enum HostAppIdentity {
|
||||
/// 跳转前最多等多久让 arbiter 回调送达。回调通常在毫秒级到达,
|
||||
/// 0.4 秒既覆盖慢回调,又不会让用户感觉“点了没反应”。
|
||||
static let handoffWaitTimeout: TimeInterval = 0.4
|
||||
|
||||
/// 跳转之后,迟到的回调仍可补写返回目标的窗口。
|
||||
/// 覆盖主 App 冷启动到“就绪”的这段时间;就绪之后才读目标,所以迟到也来得及。
|
||||
static let lateBindWindow: TimeInterval = 6
|
||||
|
||||
/// 晚绑定窗口的截止时间;回调可能在任意线程,用锁保护。
|
||||
private static let lateBindDeadline = OSAllocatedUnfairLock<Date?>(initialState: nil)
|
||||
|
||||
/// 最近一次识别到的宿主;识别不到、或宿主是 OSG 自己时为 nil。
|
||||
static var currentBundleId: String? {
|
||||
guard let bundleId = OSGHostArbiterProbe.lastCapturedBundleId,
|
||||
@@ -19,16 +32,63 @@ enum HostAppIdentity {
|
||||
return bundleId
|
||||
}
|
||||
|
||||
/// 探针钩子是否装上了。没装上时等待没有意义,直接跳过。
|
||||
static var isProbeInstalled: Bool {
|
||||
OSGHostArbiterProbe.isInstalled
|
||||
}
|
||||
|
||||
/// 安装回调处理器(所有构建)。每次 arbiter 回调:
|
||||
/// 1. DEBUG 下写日志;
|
||||
/// 2. 若正处于“刚发起跳转”的晚绑定窗口内,把新识别到的宿主补写进 App Group。
|
||||
/// 探针只有一个处理器槽,所以日志与晚绑定必须合并在这里。
|
||||
static func installCaptureHandler() {
|
||||
OSGHostArbiterProbe.setCaptureHandler { @Sendable bundleId, detail in
|
||||
#if DEBUG
|
||||
HostBundleProbeLog.append(event: "arbiter", bundleId: bundleId, detail: detail)
|
||||
#endif
|
||||
guard let bundleId, !bundleId.isEmpty,
|
||||
!HostAppURLRegistry.isOwnBundleId(bundleId),
|
||||
isLateBindWindowOpen() else {
|
||||
return
|
||||
}
|
||||
// App Group 写入统一回到主线程(`FlowSessionBridge` 只在主线程同步)。
|
||||
DispatchQueue.main.async {
|
||||
FlowSessionBridge.setPendingHostBundleId(bundleId)
|
||||
#if DEBUG
|
||||
HostBundleProbeLog.append(event: "late-bind", bundleId: bundleId, detail: "published")
|
||||
#endif
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 键盘弹出时主动让 arbiter 回调一次,避免宿主没变时拿不到最新 ID。
|
||||
static func refreshOnPresentation() {
|
||||
OSGHostArbiterProbe.pingArbiter()
|
||||
}
|
||||
|
||||
/// 跳转主 App(`startflow`)前调用:把当前宿主写进 App Group,主 App 就绪后据此跳回。
|
||||
/// 跳转前调用:尽量拿到宿主 ID,但最多等 `timeout`。
|
||||
/// 已经有值就立刻返回;探针没装上也立刻返回 nil;其余情况 ping 一次后短轮询。
|
||||
@MainActor
|
||||
static func resolveForHandoff(timeout: TimeInterval = handoffWaitTimeout) async -> String? {
|
||||
if let bundleId = currentBundleId { return bundleId }
|
||||
guard isProbeInstalled else { return nil }
|
||||
|
||||
OSGHostArbiterProbe.pingArbiter()
|
||||
let deadline = Date().addingTimeInterval(timeout)
|
||||
while Date() < deadline {
|
||||
try? await Task.sleep(nanoseconds: 40_000_000)
|
||||
if let bundleId = currentBundleId { return bundleId }
|
||||
}
|
||||
return currentBundleId
|
||||
}
|
||||
|
||||
/// 跳转主 App(`startflow`)前调用:把当前宿主写进 App Group,主 App 就绪后据此跳回,
|
||||
/// 并打开晚绑定窗口,让迟到的回调也能补上。
|
||||
/// 识别不到时写 nil,清掉上一次残留,避免跳回错误的 App。
|
||||
static func publishReturnTarget() {
|
||||
let bundleId = currentBundleId
|
||||
FlowSessionBridge.setPendingHostBundleId(bundleId)
|
||||
lateBindDeadline.withLock { $0 = Date().addingTimeInterval(lateBindWindow) }
|
||||
#if DEBUG
|
||||
let returnable = HostAppURLRegistry.lookup(bundleId: bundleId) != nil
|
||||
HostBundleProbeLog.append(
|
||||
@@ -42,19 +102,20 @@ enum HostAppIdentity {
|
||||
/// 键盘收起时作废识别结果,下次弹出由新回调重新填充。
|
||||
static func invalidate() {
|
||||
OSGHostArbiterProbe.invalidateCapturedBundleId()
|
||||
lateBindDeadline.withLock { $0 = nil }
|
||||
}
|
||||
|
||||
private static func isLateBindWindowOpen() -> Bool {
|
||||
lateBindDeadline.withLock { deadline in
|
||||
guard let deadline else { return false }
|
||||
return Date() < deadline
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#if DEBUG
|
||||
/// 调试:把探针回调写进 App Group,主 App 首页的测试卡片读取展示。
|
||||
/// 调试:键盘弹出时记一条标记,主 App 首页的测试卡片读取展示。
|
||||
enum HostBundleProbe {
|
||||
/// 只需安装一次;回调可能来自非主线程,闭包必须是 nonisolated。
|
||||
static func installLogging() {
|
||||
OSGHostArbiterProbe.setCaptureHandler { @Sendable bundleId, detail in
|
||||
HostBundleProbeLog.append(event: "arbiter", bundleId: bundleId, detail: detail)
|
||||
}
|
||||
}
|
||||
|
||||
/// 键盘弹出时记一条标记,便于把记录和每次弹出对应起来(ping 由 `HostAppIdentity` 负责)。
|
||||
static func recordPresentation() {
|
||||
HostBundleProbeLog.append(
|
||||
|
||||
@@ -19,7 +19,8 @@ typedef void (^OSGHostArbiterCaptureHandler)(NSString *_Nullable bundleId, NSStr
|
||||
/// 钩子是否已成功安装(私有类或方法不存在时为 NO)。
|
||||
@property (class, nonatomic, readonly) BOOL isInstalled;
|
||||
|
||||
/// 最近一次回调读到的宿主 bundle ID;回调读不到或已作废时为 nil。线程安全。
|
||||
/// 最近一次回调读到的有效宿主 bundle ID;从未读到或已作废时为 nil。线程安全。
|
||||
/// 读不到 ID 的空回调不会清掉它,只有 invalidateCapturedBundleId 会。
|
||||
@property (class, nonatomic, readonly, nullable) NSString *lastCapturedBundleId;
|
||||
|
||||
/// 作废上一次的识别结果(键盘收起时调用,避免把上一个宿主当成当前宿主)。
|
||||
|
||||
@@ -44,8 +44,12 @@ static void OSGSwizzledKeyboardChanged(id self, SEL _cmd, id change, id completi
|
||||
NSString *detail = change ? NSStringFromClass([change class]) : @"change=nil";
|
||||
|
||||
os_unfair_lock_lock(&sLock);
|
||||
// 读不到 ID 的回调同样覆盖旧值:宿主已经变了,旧 ID 不能再用。
|
||||
sLastCapturedBundleId = bundleId.length > 0 ? bundleId : nil;
|
||||
// 只在读到有效 ID 时更新。读不到 ID 的回调(焦点切换、空事件)不再清掉已拿到的值:
|
||||
// 之前“最后一个事件为准”会让一次空回调把正确的宿主抹掉,是识别忽有忽无的来源之一。
|
||||
// 宿主真的变了,由键盘收起时的 invalidateCapturedBundleId 作废旧值。
|
||||
if (bundleId.length > 0) {
|
||||
sLastCapturedBundleId = bundleId;
|
||||
}
|
||||
OSGHostArbiterCaptureHandler handler = sHandler;
|
||||
os_unfair_lock_unlock(&sLock);
|
||||
if (handler) {
|
||||
|
||||
@@ -30,6 +30,13 @@ public enum FlowHandoffPolicy {
|
||||
/// capture, ASR and model warm-up remain lazy until an actual mic press.
|
||||
public static let allowsProactiveHostAutoLaunch = true
|
||||
|
||||
/// Delays (seconds) before each attempt to return to the previous app once
|
||||
/// the host session is ready after a keyboard handoff. A single try was
|
||||
/// brittle: the first `open` can lose a race with PiP settling, and a late
|
||||
/// host-id binding from the keyboard may land a moment after ready. The tip
|
||||
/// is the fallback once every attempt has failed.
|
||||
public static let hostReturnRetryDelays: [TimeInterval] = [0.45, 0.8, 1.2, 1.8]
|
||||
|
||||
/// Samples of "host truly dead" required before a cold-start jump is allowed
|
||||
/// from a mic-driven ready-wait recovery. Idle opens must never cold-start.
|
||||
/// Mic press uses `shouldOpenHostColdStart` directly.
|
||||
|
||||
@@ -144,15 +144,32 @@ extension FlowSessionBridge {
|
||||
let store = FlowSessionBridgeStorage.resolvedDefaults(defaults)
|
||||
if let bundleId, !bundleId.isEmpty {
|
||||
store.set(bundleId, forKey: FlowSessionKeys.pendingHostBundleId)
|
||||
store.set(Date().timeIntervalSince1970, forKey: FlowSessionKeys.pendingHostBundleIdAt)
|
||||
} else {
|
||||
store.removeObject(forKey: FlowSessionKeys.pendingHostBundleId)
|
||||
store.removeObject(forKey: FlowSessionKeys.pendingHostBundleIdAt)
|
||||
}
|
||||
FlowSessionBridgeStorage.flush(store)
|
||||
}
|
||||
|
||||
public static func pendingHostBundleId(defaults: UserDefaults? = nil) -> String? {
|
||||
/// The host the keyboard recorded for the current `startflow` handoff, or nil
|
||||
/// when none was recorded or it is older than `pendingHostBundleIdTTL`.
|
||||
/// A value without a timestamp (written by an older build) is still honoured.
|
||||
public static func pendingHostBundleId(
|
||||
defaults: UserDefaults? = nil,
|
||||
now: Date = Date()
|
||||
) -> String? {
|
||||
let store = FlowSessionBridgeStorage.resolvedDefaults(defaults)
|
||||
return store.string(forKey: FlowSessionKeys.pendingHostBundleId)
|
||||
guard let bundleId = store.string(forKey: FlowSessionKeys.pendingHostBundleId),
|
||||
!bundleId.isEmpty else {
|
||||
return nil
|
||||
}
|
||||
let writtenAt = store.double(forKey: FlowSessionKeys.pendingHostBundleIdAt)
|
||||
if writtenAt > 0,
|
||||
now.timeIntervalSince1970 - writtenAt > FlowSessionKeys.pendingHostBundleIdTTL {
|
||||
return nil
|
||||
}
|
||||
return bundleId
|
||||
}
|
||||
|
||||
public static func clearPendingHostBundleId(defaults: UserDefaults? = nil) {
|
||||
@@ -426,6 +443,7 @@ extension FlowSessionBridge {
|
||||
clearTranscription(defaults: store)
|
||||
store.removeObject(forKey: FlowSessionKeys.audioLevels)
|
||||
store.removeObject(forKey: FlowSessionKeys.pendingHostBundleId)
|
||||
store.removeObject(forKey: FlowSessionKeys.pendingHostBundleIdAt)
|
||||
store.removeObject(forKey: FlowSessionKeys.lastActivityAt)
|
||||
clearHostReady(defaults: store, notify: false)
|
||||
clearHostHeavy(defaults: store)
|
||||
|
||||
@@ -34,6 +34,11 @@ public enum FlowSessionKeys {
|
||||
public static let audioLevels = "flow.audioLevels"
|
||||
/// Bundle id of the app that opened `osgkeyboard://startflow` (scheme D).
|
||||
public static let pendingHostBundleId = "flow.pendingHostBundleId"
|
||||
/// Wall-clock of the last `pendingHostBundleId` write; drives the TTL below.
|
||||
public static let pendingHostBundleIdAt = "flow.pendingHostBundleIdAt.v1"
|
||||
/// A recorded host older than this is never a valid return target — the user
|
||||
/// has long since moved on, and jumping there would be a wrong-app surprise.
|
||||
public static let pendingHostBundleIdTTL: TimeInterval = 120
|
||||
/// Wall-clock of the last keyboard→`startflow` PiP arm attempt (debounce re-jumps).
|
||||
public static let lastPiPArmAttemptAt = "flow.lastPiPArmAttemptAt.v1"
|
||||
/// Minimum gap between repeated proactive `startflow` jumps.
|
||||
|
||||
@@ -263,6 +263,16 @@ final class FlowHandoffPolicyTests: XCTestCase {
|
||||
XCTAssertTrue(FlowHandoffPolicy.allowsProactiveHostAutoLaunch)
|
||||
}
|
||||
|
||||
/// Return-to-host retries: more than one chance (the first open can lose a
|
||||
/// race), spaced forward in time, and bounded so the tip appears promptly.
|
||||
func testHostReturnRetryDelaysAreMultipleIncreasingAndBounded() {
|
||||
let delays = FlowHandoffPolicy.hostReturnRetryDelays
|
||||
XCTAssertGreaterThan(delays.count, 1)
|
||||
XCTAssertTrue(delays.allSatisfy { $0 > 0 })
|
||||
XCTAssertEqual(delays, delays.sorted())
|
||||
XCTAssertLessThanOrEqual(delays.reduce(0, +), 6, "tip must not wait out the late-bind window")
|
||||
}
|
||||
|
||||
func testDebouncerIgnoresSingleDeadSample() {
|
||||
var debouncer = FlowColdStartDebouncer()
|
||||
XCTAssertFalse(debouncer.observe(hostTrulyDead: true))
|
||||
|
||||
@@ -52,4 +52,36 @@ final class FlowSessionPolicyTests: XCTestCase {
|
||||
FlowSessionBridge.clearPendingHostBundleId(defaults: defaults)
|
||||
XCTAssertNil(FlowSessionBridge.pendingHostBundleId(defaults: defaults))
|
||||
}
|
||||
|
||||
func testPendingHostBundleIdExpiresAfterTTL() {
|
||||
let defaults = makeDefaults()
|
||||
FlowSessionBridge.setPendingHostBundleId("com.tencent.xin", defaults: defaults)
|
||||
|
||||
let justInside = Date().addingTimeInterval(FlowSessionKeys.pendingHostBundleIdTTL - 5)
|
||||
XCTAssertEqual(
|
||||
FlowSessionBridge.pendingHostBundleId(defaults: defaults, now: justInside),
|
||||
"com.tencent.xin"
|
||||
)
|
||||
let pastTTL = Date().addingTimeInterval(FlowSessionKeys.pendingHostBundleIdTTL + 5)
|
||||
XCTAssertNil(FlowSessionBridge.pendingHostBundleId(defaults: defaults, now: pastTTL))
|
||||
}
|
||||
|
||||
func testPendingHostBundleIdWithoutTimestampStaysValid() {
|
||||
// Written by an older build: no timestamp key. Must keep working.
|
||||
let defaults = makeDefaults()
|
||||
defaults.set("com.tencent.xin", forKey: FlowSessionKeys.pendingHostBundleId)
|
||||
|
||||
let farFuture = Date().addingTimeInterval(FlowSessionKeys.pendingHostBundleIdTTL * 10)
|
||||
XCTAssertEqual(
|
||||
FlowSessionBridge.pendingHostBundleId(defaults: defaults, now: farFuture),
|
||||
"com.tencent.xin"
|
||||
)
|
||||
}
|
||||
|
||||
func testClearingPendingHostBundleIdAlsoClearsTimestamp() {
|
||||
let defaults = makeDefaults()
|
||||
FlowSessionBridge.setPendingHostBundleId("com.tencent.xin", defaults: defaults)
|
||||
FlowSessionBridge.clearPendingHostBundleId(defaults: defaults)
|
||||
XCTAssertNil(defaults.object(forKey: FlowSessionKeys.pendingHostBundleIdAt))
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user