chore(release): bump version to 1.0.1 (build 27)

Release the current PiP reliability, polish style, macOS dictionary, and UI updates.
This commit is contained in:
Rocky
2026-07-27 00:09:30 +08:00
parent 5b1283c3ed
commit 956331a2af
41 changed files with 1241 additions and 295 deletions
+15 -1
View File
@@ -7,16 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [1.0.1] - 2026-07-27
### Fixed ### Fixed
- **Onboarding Done stuck**: finishing step 6 no longer wraps the route change in an animation transaction; `MainAppRoot` force-swaps view identity so Settings replay cannot freeze on the last page. / **引导完成卡住**:第六步完成不再包进动画事务;`MainAppRoot` 强制切换视图身份,避免从设置重走引导后停在最后一页。
- **PiP closed loop**: activate a playback audio session before creating `AVPictureInPictureController`, never treat “armed but inactive” as success, and restore playback audio after releasing the mic between utterances so Home `hostReady` matches a real PiP window. / **PiP 闭环**:在创建画中画控制器前先激活 playback 音频会话,不再把「已武装但未激活」当成功,并在句间释麦后恢复 playback,使首页就绪状态与真实 PiP 窗口一致。
- **Onboarding after reinstall**: fresh installs clear the durable Keychain onboarding flag via a container install identity so delete-and-reinstall shows the welcome flow again. / **重装后引导**:通过容器安装身份在全新安装时清除 Keychain 引导标记,删除重装后会再次显示欢迎流程。
- **Cloud engine footer**: Home / preview status lines name the configured ASR provider and model instead of the polish LLM. / **云端引擎文案**:首页与预览状态行显示已配置的 ASR 服务商与模型,而不再显示润色 LLM。
- **PiP Flow handoff**: wait for the PiP host view, distinguish real start failures, and stop pointing users at a non-existent Settings toggle; cold-start / keyboard copy now match Picture in Picture keep-alive. / **PiP Flow 交接**:等待 PiP Host 就绪、区分真实启动失败,并不再引导不存在的系统设置开关;冷启动与键盘文案对齐画中画保活。
- **PiP start reliability**: restore unconditional `startPictureInPicture` retries, live `positiveInfinity` time range, DisplayImmediately sample buffers, and arm auto-inline PiP for background; Home status footer uses a bottom inset so the adaptive preview field shrinks instead of sitting under the tab dock. / **PiP 启动可靠性**:恢复无条件 `startPictureInPicture` 重试、直播 `positiveInfinity` 时间范围、DisplayImmediately 帧,并武装后台自动画中画;首页状态行改为 bottom inset,由自适应输入框让位,避免被 Tab 遮挡。
- **PiP / Live Activity exclusivity**: Picture in Picture sessions no longer start or refresh Live Activities. / **PiP / 灵动岛互斥**:画中画会话不再启动或刷新灵动岛 Live Activity。
- **Flow tail ASR drop**: after mic stop, iOS Flow now uses a longer silence drain (350 ms), a fixed 150 ms post-roll, expanded final-chunk ASR recovery, and a partial transcript guard so weak trailing syllables are less likely to disappear from the result. / **Flow 尾音识别丢失**:松手后 iOS Flow 采用更长的静音排空(350 ms)、固定 150 ms 尾音保留、增强末块 ASR 恢复与 partial 兜底,降低弱尾音从结果中消失的概率。 - **Flow tail ASR drop**: after mic stop, iOS Flow now uses a longer silence drain (350 ms), a fixed 150 ms post-roll, expanded final-chunk ASR recovery, and a partial transcript guard so weak trailing syllables are less likely to disappear from the result. / **Flow 尾音识别丢失**:松手后 iOS Flow 采用更长的静音排空(350 ms)、固定 150 ms 尾音保留、增强末块 ASR 恢复与 partial 兜底,降低弱尾音从结果中消失的概率。
- **Flow batch ASR fallback**: when pipelined chunk output is clearly shorter than the live partial, the host re-transcribes the full utterance PCM captured during recording (Mac-style safety net). / **Flow 整句 ASR 兜底**:流水线拼接结果明显短于实时 partial 时,主 App 对录音期间累积的整段 PCM 重新识别(对齐 Mac 双保险)。 - **Flow batch ASR fallback**: when pipelined chunk output is clearly shorter than the live partial, the host re-transcribes the full utterance PCM captured during recording (Mac-style safety net). / **Flow 整句 ASR 兜底**:流水线拼接结果明显短于实时 partial 时,主 App 对录音期间累积的整段 PCM 重新识别(对齐 Mac 双保险)。
### Changed ### Changed
- **Unified navigation icons**: iPhone History and Personal Dictionary tabs now use the same SF Symbols as the Mac and iPad sidebars. / **统一导航图标**:iPhone 的历史记录与个性词库 Tab 现使用与 Mac、iPad 侧边栏一致的 SF Symbols。
- **Mac header actions**: personal-word and polish-style add buttons now match the adjacent search field height and use a consistent capsule shape. / **Mac 页头操作**:添加个性词与添加润色风格按钮现与相邻搜索框等高,并统一使用胶囊外形。
- **Responsive Mac polish styles**: replace fixed full-width style rows with iOS-aligned adaptive cards that reflow with the window width. / **Mac 润色风格响应式布局**:将固定通栏列表改为对齐 iOS 的自适应卡片,并随窗口宽度自动重排。
- **Mac MLX tail drain**: streaming capture now uses the shared `FlowUtteranceEndCoordinator` (silence drain + post-roll) instead of an inline poll loop. / **Mac MLX 尾音排空**:流式采集改用 Shared 层 `FlowUtteranceEndCoordinator`(静音排空 + post-roll),替代内联轮询循环。 - **Mac MLX tail drain**: streaming capture now uses the shared `FlowUtteranceEndCoordinator` (silence drain + post-roll) instead of an inline poll loop. / **Mac MLX 尾音排空**:流式采集改用 Shared 层 `FlowUtteranceEndCoordinator`(静音排空 + post-roll),替代内联轮询循环。
- **Default Flow keep-alive**: new installs default to Picture in Picture instead of Dynamic Island. / **默认 Flow 保活**:新安装默认使用画中画,不再默认灵动岛。
### Added ### Added
- **Mac personal words**: add custom dictionary terms on macOS with automatic recognition-alias generation and iCloud sync. / **Mac 个性词**:可在 macOS 添加自定义词条,自动生成识别别名并通过 iCloud 同步。
- **Polish style packs**: choose a complete writing personality from the new iOS tab or Mac sidebar, create custom prompts, and sync selections and custom styles through iCloud. / **润色风格包**:可在 iOS 新 Tab 或 Mac 侧栏选择完整写作人格、创建自定义提示词,并通过 iCloud 同步选择与自定义风格。 - **Polish style packs**: choose a complete writing personality from the new iOS tab or Mac sidebar, create custom prompts, and sync selections and custom styles through iCloud. / **润色风格包**:可在 iOS 新 Tab 或 Mac 侧栏选择完整写作人格、创建自定义提示词,并通过 iCloud 同步选择与自定义风格。
- **PiP Flow keep-alive**: Settings → Voice session lets you choose **Dynamic Island** (default, unchanged behaviour) or **Picture in Picture** — a live waveform PiP keeps the host alive with the mic released between utterances; closing PiP ends the session. / **PiP Flow 保活**:设置 → 语音会话可选 **灵动岛**(默认,行为不变)或 **画中画** — 实时波形 PiP 保活、句间释麦;关闭 PiP 即结束会话。 - **PiP Flow keep-alive**: Settings → Voice session lets you choose **Picture in Picture** (default) or **Dynamic Island** — a live waveform PiP keeps the host alive with the mic released between utterances; closing PiP ends the session. / **PiP Flow 保活**:设置 → 语音会话可选 **画中画**(默认)或 **灵动岛** — 实时波形 PiP 保活、句间释麦;关闭 PiP 即结束会话。
- **Mac MLX streaming ASR**: local dictation uses Qwen3-ASR via mlx-audio-swift with overlay partial preview, tail drain, vocabulary prompt, and polish-before-insert. / **Mac MLX 流式 ASR**:本地听写改用 mlx-audio-swift 的 Qwen3-ASR,支持浮层 partial 预览、尾部截断、词库 prompt 与润色后再插入。 - **Mac MLX streaming ASR**: local dictation uses Qwen3-ASR via mlx-audio-swift with overlay partial preview, tail drain, vocabulary prompt, and polish-before-insert. / **Mac MLX 流式 ASR**:本地听写改用 mlx-audio-swift 的 Qwen3-ASR,支持浮层 partial 预览、尾部截断、词库 prompt 与润色后再插入。
### Changed ### Changed
+1
View File
@@ -107,6 +107,7 @@
<key>UIBackgroundModes</key> <key>UIBackgroundModes</key>
<array> <array>
<string>audio</string> <string>audio</string>
<string>picture-in-picture</string>
</array> </array>
<key>UILaunchScreen</key> <key>UILaunchScreen</key>
<dict> <dict>
@@ -1,55 +1,120 @@
// FlowPictureInPictureController.swift // FlowPictureInPictureController.swift
// OSGKeyboard · Main App // OSGKeyboard · Main App
// //
// PiP keep-alive for Flow sessions: enqueues live waveform sample buffers // PiP keep-alive for Flow sessions: enqueues a looping tuck to edge
// so the host process stays eligible for multitasking while the mic is off // teaching animation (OSG logo card) so the host stays eligible for
// between utterances. // multitasking while the mic is off between utterances.
import AVFoundation import AVFoundation
import AVKit import AVKit
import CoreMedia import CoreMedia
import UIKit import UIKit
/// Why `startAndWait` could not prove an active PiP window.
enum FlowPiPStartFailure: Equatable, Sendable {
case unsupported
case hostNotReady
case notPossible
case systemRejected
case timedOut
var localizationKey: String {
switch self {
case .unsupported: return "flow.pip.error.unsupported"
case .hostNotReady: return "flow.pip.error.hostNotReady"
case .notPossible: return "flow.pip.error.notPossible"
case .systemRejected: return "flow.pip.error.systemRejected"
case .timedOut: return "flow.pip.error.timedOut"
}
}
}
enum FlowPiPStartOutcome: Equatable, Sendable {
case started
case failed(FlowPiPStartFailure)
}
@MainActor @MainActor
final class FlowPictureInPictureController: NSObject { final class FlowPictureInPictureController: NSObject {
/// User closed the PiP window host should end the Flow session. /// User closed the PiP window host should end the Flow session.
var onUserDismissed: (() -> Void)? var onUserDismissed: (() -> Void)?
private(set) var isPictureInPictureActive = false private(set) var isPictureInPictureActive = false
/// True once a host UIView has been attached (may still be awaiting a window).
private(set) var hasHostView = false
let displayLayer = AVSampleBufferDisplayLayer() let displayLayer = AVSampleBufferDisplayLayer()
private var pipController: AVPictureInPictureController? private var pipController: AVPictureInPictureController?
private var displayLink: CADisplayLink? private var displayLink: CADisplayLink?
private weak var hostView: UIView? private weak var hostView: UIView?
private var waveformLevels: [Float] = Array(repeating: 0, count: 24)
private var isStoppingProgrammatically = false private var isStoppingProgrammatically = false
private var frameIndex: Int64 = 0 private var frameIndex: Int64 = 0
private var animationStartedAt: CFTimeInterval?
private var cachedLogo: CGImage?
/// Last system failure reported by the PiP delegate (cleared on each start).
private var lastSystemStartFailure: Error?
private enum Canvas {
static let width = 480
static let height = 270
static let fps: Int32 = 18
/// Full teaching loop length (seconds).
static let loopDuration: CFTimeInterval = 4.2
}
// MARK: - Host view // MARK: - Host view
func attachHostView(_ view: UIView) { func attachHostView(_ view: UIView) {
hostView = view hostView = view
displayLayer.frame = view.bounds hasHostView = true
let bounds = view.bounds
displayLayer.frame = (bounds.width >= 1 && bounds.height >= 1)
? bounds
: CGRect(x: 0, y: 0, width: 64, height: 36)
displayLayer.videoGravity = .resizeAspectFill displayLayer.videoGravity = .resizeAspectFill
displayLayer.removeFromSuperlayer() displayLayer.removeFromSuperlayer()
view.layer.addSublayer(displayLayer) view.layer.addSublayer(displayLayer)
configureControllerIfNeeded() // Do not create AVPictureInPictureController here it must be built
// only after an active AVAudioSession (see `start()`).
} }
func updateHostLayoutIfNeeded() { func updateHostLayoutIfNeeded() {
guard let hostView else { return } guard let hostView else { return }
displayLayer.frame = hostView.bounds let bounds = hostView.bounds
displayLayer.frame = (bounds.width >= 1 && bounds.height >= 1)
? bounds
: CGRect(x: 0, y: 0, width: 64, height: 36)
}
/// Host layer is in a UIWindow required before `startPictureInPicture()`.
var isHostInWindowHierarchy: Bool {
hostView?.window != nil
} }
// MARK: - Lifecycle // MARK: - Lifecycle
@discardableResult @discardableResult
func start() -> Bool { func start() -> Bool {
lastSystemStartFailure = nil
guard AVPictureInPictureController.isPictureInPictureSupported() else { guard AVPictureInPictureController.isPictureInPictureSupported() else {
return false return false
} }
guard hasHostView else { return false }
// Required before constructing the controller; without an active
// session, `isPictureInPicturePossible` stays false forever.
guard activateAudioSessionForPiP() else {
return false
}
// If a controller was somehow created before audio activation, rebuild.
if pipController != nil, !didActivateAudioSessionBeforeController {
pipController = nil
}
configureControllerIfNeeded() configureControllerIfNeeded()
warmLogoCacheIfNeeded()
animationStartedAt = CACurrentMediaTime()
startFramePump() startFramePump()
guard pipController != nil else { return false } guard pipController != nil else { return false }
@@ -58,43 +123,149 @@ final class FlowPictureInPictureController: NSObject {
return true return true
} }
enqueueWaveformFrame() // Prime a few frames before asking the system to start PiP.
enqueueGuideFrame()
enqueueGuideFrame()
pipController?.invalidatePlaybackState()
pipController?.startPictureInPicture() pipController?.startPictureInPicture()
return true return true
} }
func startAndWait(timeout: TimeInterval = 4) async -> Bool { /// Waits until the host is windowed and PiP is actually active.
if isPictureInPictureActive { return true } /// Does not treat "armed but inactive" as success that left sessions
guard start() else { return false } /// live while `hostReady` stayed false forever.
func startAndWait(
hostTimeout: TimeInterval = 3,
activeTimeout: TimeInterval = 8
) async -> FlowPiPStartOutcome {
if isPictureInPictureActive { return .started }
let deadline = Date().addingTimeInterval(timeout) guard AVPictureInPictureController.isPictureInPictureSupported() else {
return .failed(.unsupported)
}
let hostReady = await waitForHostInWindow(timeout: hostTimeout)
guard hostReady else {
return .failed(.hostNotReady)
}
lastSystemStartFailure = nil
guard start() else {
stopFramePump()
if lastSystemStartFailure != nil {
return .failed(.systemRejected)
}
return .failed(hasHostView ? .notPossible : .hostNotReady)
}
let deadline = Date().addingTimeInterval(activeTimeout)
while Date() < deadline { while Date() < deadline {
if isPictureInPictureActive { return true } if isPictureInPictureActive { return .started }
if pipController?.isPictureInPictureActive == true { if pipController?.isPictureInPictureActive == true {
isPictureInPictureActive = true isPictureInPictureActive = true
return true return .started
}
if let pipController, pipController.isPictureInPicturePossible {
pipController.startPictureInPicture()
} else {
pipController?.startPictureInPicture()
} }
try? await Task.sleep(nanoseconds: 50_000_000) try? await Task.sleep(nanoseconds: 50_000_000)
} }
return isPictureInPictureActive
if isPictureInPictureActive { return .started }
if pipController?.isPictureInPictureActive == true {
isPictureInPictureActive = true
return .started
}
// Real failure tear down so the next retry starts clean.
let failure: FlowPiPStartFailure
if lastSystemStartFailure != nil {
failure = .systemRejected
} else if pipController?.isPictureInPicturePossible != true {
failure = .notPossible
} else {
failure = .timedOut
}
FlowDiagnostics.log(
"PiP startAndWait failed: \(failure) possible=\(pipController?.isPictureInPicturePossible == true)"
)
stop()
return .failed(failure)
} }
func stop() { func stop() {
isStoppingProgrammatically = true isStoppingProgrammatically = true
stopFramePump() stopFramePump()
pipController?.stopPictureInPicture() pipController?.stopPictureInPicture()
displayLayer.flushAndRemoveImage() displayLayer.sampleBufferRenderer.flush(
removingDisplayedImage: true,
completionHandler: nil
)
isPictureInPictureActive = false isPictureInPictureActive = false
animationStartedAt = nil
lastSystemStartFailure = nil
isStoppingProgrammatically = false isStoppingProgrammatically = false
} }
/// Nudge the sample-buffer source right before resigning active so
/// `canStartPictureInPictureAutomaticallyFromInline` can take over.
func prepareForBackgroundAutoStart() {
guard isPictureInPictureActive || pipController != nil else { return }
_ = activateAudioSessionForPiP()
startFramePump()
enqueueGuideFrame()
pipController?.invalidatePlaybackState()
if !isPictureInPictureActive {
pipController?.startPictureInPicture()
}
}
/// Re-activate the playback session after utterance capture releases the
/// mic (`setActive(false)`). Without this, PiP can lose eligibility between
/// utterances even though the floating window is still visible.
@discardableResult
func reassertKeepAliveAudioSession() -> Bool {
activateAudioSessionForPiP()
}
/// Kept for FlowSessionManager call sites; guide animation ignores live levels.
func updateWaveformLevels(_ levels: [Float]) { func updateWaveformLevels(_ levels: [Float]) {
guard !levels.isEmpty else { return } _ = levels
waveformLevels = levels
} }
// MARK: - Private // MARK: - Private
/// Set once we successfully activate audio before building the controller.
private var didActivateAudioSessionBeforeController = false
@discardableResult
private func activateAudioSessionForPiP() -> Bool {
do {
let session = AVAudioSession.sharedInstance()
// Playback (not record) keeps PiP eligible between utterances without
// holding the mic. Utterance capture later switches to playAndRecord.
try session.setCategory(.playback, mode: .moviePlayback, options: [.mixWithOthers])
try session.setActive(true)
FlowDiagnostics.log("PiP audio session active category=playback")
return true
} catch {
FlowDiagnostics.log("PiP audio session failed: \(error.localizedDescription)")
return false
}
}
private func waitForHostInWindow(timeout: TimeInterval) async -> Bool {
if isHostInWindowHierarchy { return true }
let deadline = Date().addingTimeInterval(timeout)
while Date() < deadline {
if isHostInWindowHierarchy { return true }
try? await Task.sleep(nanoseconds: 50_000_000)
}
return isHostInWindowHierarchy
}
private func configureControllerIfNeeded() { private func configureControllerIfNeeded() {
guard pipController == nil else { return } guard pipController == nil else { return }
guard AVPictureInPictureController.isPictureInPictureSupported() else { return } guard AVPictureInPictureController.isPictureInPictureSupported() else { return }
@@ -106,13 +277,19 @@ final class FlowPictureInPictureController: NSObject {
let controller = AVPictureInPictureController(contentSource: contentSource) let controller = AVPictureInPictureController(contentSource: contentSource)
controller.delegate = self controller.delegate = self
controller.canStartPictureInPictureAutomaticallyFromInline = true controller.canStartPictureInPictureAutomaticallyFromInline = true
controller.requiresLinearPlayback = true
pipController = controller pipController = controller
didActivateAudioSessionBeforeController = true
} }
private func startFramePump() { private func startFramePump() {
guard displayLink == nil else { return } guard displayLink == nil else { return }
let link = CADisplayLink(target: self, selector: #selector(handleDisplayLink(_:))) let link = CADisplayLink(target: self, selector: #selector(handleDisplayLink(_:)))
link.preferredFrameRateRange = CAFrameRateRange(minimum: 20, maximum: 30, preferred: 24) link.preferredFrameRateRange = CAFrameRateRange(
minimum: 12,
maximum: 20,
preferred: Float(Canvas.fps)
)
link.add(to: .main, forMode: .common) link.add(to: .main, forMode: .common)
displayLink = link displayLink = link
} }
@@ -123,43 +300,52 @@ final class FlowPictureInPictureController: NSObject {
} }
@objc private func handleDisplayLink(_ link: CADisplayLink) { @objc private func handleDisplayLink(_ link: CADisplayLink) {
enqueueWaveformFrame() enqueueGuideFrame()
updateHostLayoutIfNeeded() updateHostLayoutIfNeeded()
if let pipController, !pipController.isPictureInPictureActive, pipController.isPictureInPicturePossible { guard let pipController, !pipController.isPictureInPictureActive else { return }
pipController.startPictureInPicture() if frameIndex % Int64(Canvas.fps) == 0 {
pipController.invalidatePlaybackState()
} }
// Retry regardless of `isPictureInPicturePossible` that flag often
// lags behind a warm sample-buffer source.
pipController.startPictureInPicture()
} }
private func enqueueWaveformFrame() { private func enqueueGuideFrame() {
guard let sampleBuffer = makeWaveformSampleBuffer(levels: resolvedLevels()) else { return } guard let sampleBuffer = makeGuideSampleBuffer() else { return }
if displayLayer.status == .failed { if displayLayer.sampleBufferRenderer.status == .failed {
displayLayer.flush() displayLayer.sampleBufferRenderer.flush()
} }
displayLayer.enqueue(sampleBuffer) displayLayer.sampleBufferRenderer.enqueue(sampleBuffer)
} }
private func resolvedLevels() -> [Float] { private func warmLogoCacheIfNeeded() {
if waveformLevels.contains(where: { $0 > 0.02 }) { guard cachedLogo == nil else { return }
return waveformLevels let logoColor = UIColor.white
} if let brand = UIImage(named: "OSGBrandMark")?
// Idle breathing animation between utterances. .withTintColor(logoColor, renderingMode: .alwaysOriginal)
let phase = Float(frameIndex) * 0.12 .cgImage {
return (0..<waveformLevels.count).map { index in cachedLogo = brand
let wave = sin(phase + Float(index) * 0.45) return
return max(0.04, 0.04 + wave * 0.03)
} }
cachedLogo = UIImage(named: "osglogo")?
.withTintColor(logoColor, renderingMode: .alwaysOriginal)
.cgImage
} }
private func makeWaveformSampleBuffer(levels: [Float]) -> CMSampleBuffer? { // MARK: - Frame rendering
let width = 320
let height = 180 private func makeGuideSampleBuffer() -> CMSampleBuffer? {
let width = Canvas.width
let height = Canvas.height
frameIndex += 1 frameIndex += 1
var pixelBuffer: CVPixelBuffer? var pixelBuffer: CVPixelBuffer?
let attrs: [String: Any] = [ let attrs: [String: Any] = [
kCVPixelBufferCGImageCompatibilityKey as String: true, kCVPixelBufferCGImageCompatibilityKey as String: true,
kCVPixelBufferCGBitmapContextCompatibilityKey as String: true, kCVPixelBufferCGBitmapContextCompatibilityKey as String: true,
kCVPixelBufferIOSurfacePropertiesKey as String: [:] as [String: Any],
] ]
let status = CVPixelBufferCreate( let status = CVPixelBufferCreate(
kCFAllocatorDefault, kCFAllocatorDefault,
@@ -184,32 +370,17 @@ final class FlowPictureInPictureController: NSObject {
bitsPerComponent: 8, bitsPerComponent: 8,
bytesPerRow: bytesPerRow, bytesPerRow: bytesPerRow,
space: colorSpace, space: colorSpace,
// BGRA pixel buffer requires little-endian byte order; without it
// R/B channels swap and greens render as purple.
bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue
| CGBitmapInfo.byteOrder32Little.rawValue
) else { return nil } ) else { return nil }
// Dark backdrop + accent waveform bars. // Flip to UIKit top-left coordinates for layout math.
context.setFillColor(UIColor(red: 0.07, green: 0.09, blue: 0.11, alpha: 1).cgColor) context.translateBy(x: 0, y: CGFloat(height))
context.fill(CGRect(x: 0, y: 0, width: width, height: height)) context.scaleBy(x: 1, y: -1)
let barCount = max(levels.count, 1) drawGuideFrame(in: context, width: width, height: height)
let barWidth = CGFloat(width) / CGFloat(barCount * 2)
let accent = UIColor(red: 0.20, green: 0.78, blue: 0.55, alpha: 1)
context.setFillColor(accent.cgColor)
for (index, level) in levels.enumerated() {
let clamped = CGFloat(min(max(level, 0), 1))
let barHeight = max(6, clamped * CGFloat(height) * 0.72)
let x = (CGFloat(index) * 2 + 0.5) * barWidth
let rect = CGRect(
x: x,
y: (CGFloat(height) - barHeight) / 2,
width: barWidth,
height: barHeight
)
let path = UIBezierPath(roundedRect: rect, cornerRadius: barWidth * 0.35)
context.addPath(path.cgPath)
context.fillPath()
}
var formatDescription: CMFormatDescription? var formatDescription: CMFormatDescription?
CMVideoFormatDescriptionCreateForImageBuffer( CMVideoFormatDescriptionCreateForImageBuffer(
@@ -220,8 +391,8 @@ final class FlowPictureInPictureController: NSObject {
guard let formatDescription else { return nil } guard let formatDescription else { return nil }
var timing = CMSampleTimingInfo( var timing = CMSampleTimingInfo(
duration: CMTime(value: 1, timescale: 24), duration: CMTime(value: 1, timescale: Canvas.fps),
presentationTimeStamp: CMTime(value: frameIndex, timescale: 24), presentationTimeStamp: CMTime(value: frameIndex, timescale: Canvas.fps),
decodeTimeStamp: .invalid decodeTimeStamp: .invalid
) )
@@ -236,17 +407,174 @@ final class FlowPictureInPictureController: NSObject {
sampleTiming: &timing, sampleTiming: &timing,
sampleBufferOut: &sampleBuffer sampleBufferOut: &sampleBuffer
) )
guard let sampleBuffer else { return nil }
// Required for sample-buffer PiP sources to present immediately.
CMSetAttachment(
sampleBuffer,
key: kCMSampleAttachmentKey_DisplayImmediately,
value: kCFBooleanTrue,
attachmentMode: kCMAttachmentMode_ShouldNotPropagate
)
return sampleBuffer return sampleBuffer
} }
private func drawGuideFrame(in context: CGContext, width: Int, height: Int) {
let canvas = CGRect(x: 0, y: 0, width: width, height: height)
// White PiP backdrop.
context.setFillColor(UIColor.white.cgColor)
context.fill(canvas)
// Soft phone silhouette gives the screen edge a visual anchor.
let phoneInset = CGFloat(22)
let phoneRect = canvas.insetBy(dx: phoneInset, dy: phoneInset)
let phonePath = UIBezierPath(roundedRect: phoneRect, cornerRadius: 28)
context.setStrokeColor(UIColor(red: 0.898, green: 0.906, blue: 0.922, alpha: 1).cgColor)
context.setLineWidth(2.5)
context.addPath(phonePath.cgPath)
context.strokePath()
let cardSize = CGSize(width: 148, height: 96)
let restOrigin = CGPoint(
x: phoneRect.midX - cardSize.width * 0.55,
y: phoneRect.midY - cardSize.height * 0.5
)
// Mostly off the right edge, leaving a peek strip (~28% visible).
let tuckedOrigin = CGPoint(
x: phoneRect.maxX - cardSize.width * 0.28,
y: restOrigin.y
)
let progress = cardTravelProgress()
let cardOrigin = CGPoint(
x: restOrigin.x + (tuckedOrigin.x - restOrigin.x) * progress,
y: restOrigin.y
)
let cardRect = CGRect(origin: cardOrigin, size: cardSize)
// Clip so the tucked card disappears past the phones right edge.
context.saveGState()
context.addPath(phonePath.cgPath)
context.clip()
drawLogoCard(in: context, rect: cardRect, tuckProgress: progress)
context.restoreGState()
}
private func drawLogoCard(in context: CGContext, rect: CGRect, tuckProgress: CGFloat) {
let cardPath = UIBezierPath(roundedRect: rect, cornerRadius: 16)
context.setFillColor(UIColor(red: 0.20, green: 0.78, blue: 0.55, alpha: 1).cgColor)
context.addPath(cardPath.cgPath)
context.fillPath()
context.setStrokeColor(UIColor(red: 0.20, green: 0.78, blue: 0.55, alpha: 1).cgColor)
context.setLineWidth(1.5)
context.addPath(cardPath.cgPath)
context.strokePath()
// Native PiP shows a left chevron on the peek strip when tucked right.
let arrowOpacity = max(0, min(1, (tuckProgress - 0.55) / 0.35))
if arrowOpacity > 0.01 {
drawEdgeChevron(in: context, cardRect: rect, opacity: arrowOpacity)
}
guard let logo = cachedLogo else { return }
let logoOpacity = 1 - arrowOpacity
guard logoOpacity > 0.01 else { return }
let maxLogoSide = min(rect.width, rect.height) * 0.52
let logoAspect = CGFloat(logo.width) / CGFloat(max(logo.height, 1))
let logoSize: CGSize
if logoAspect >= 1 {
logoSize = CGSize(width: maxLogoSide, height: maxLogoSide / logoAspect)
} else {
logoSize = CGSize(width: maxLogoSide * logoAspect, height: maxLogoSide)
}
let logoRect = CGRect(
x: rect.midX - logoSize.width / 2,
y: rect.midY - logoSize.height / 2,
width: logoSize.width,
height: logoSize.height
)
// Unflip locally so the CGImage is not drawn upside-down.
context.saveGState()
context.setAlpha(logoOpacity)
context.translateBy(x: logoRect.minX, y: logoRect.maxY)
context.scaleBy(x: 1, y: -1)
context.interpolationQuality = .high
context.draw(logo, in: CGRect(origin: .zero, size: logoSize))
context.restoreGState()
}
/// Left-pointing chevron on the visible peek strip (like system PiP).
private func drawEdgeChevron(
in context: CGContext,
cardRect: CGRect,
opacity: CGFloat
) {
// Anchor in the leftmost ~28% of the card that strip stays on-screen
// when tucked to the right edge.
let peekWidth = cardRect.width * 0.28
let center = CGPoint(
x: cardRect.minX + peekWidth * 0.5,
y: cardRect.midY
)
let halfH: CGFloat = 11
let halfW: CGFloat = 7
let path = UIBezierPath()
path.move(to: CGPoint(x: center.x + halfW, y: center.y - halfH))
path.addLine(to: CGPoint(x: center.x - halfW, y: center.y))
path.addLine(to: CGPoint(x: center.x + halfW, y: center.y + halfH))
context.saveGState()
context.setStrokeColor(UIColor.white.withAlphaComponent(opacity).cgColor)
context.setLineWidth(3)
context.setLineCap(.round)
context.setLineJoin(.round)
context.addPath(path.cgPath)
context.strokePath()
context.restoreGState()
}
/// 0 = rest (visible), 1 = tucked at right edge.
private func cardTravelProgress() -> CGFloat {
let started = animationStartedAt ?? CACurrentMediaTime()
if animationStartedAt == nil {
animationStartedAt = started
}
let t = (CACurrentMediaTime() - started)
.truncatingRemainder(dividingBy: Canvas.loopDuration)
// 0.00.6 rest 0.62.0 slide out 2.03.0 hold 3.04.2 return
if t < 0.6 {
return 0
}
if t < 2.0 {
return smoothstep((t - 0.6) / 1.4)
}
if t < 3.0 {
return 1
}
return 1 - smoothstep((t - 3.0) / 1.2)
}
private func smoothstep(_ x: CGFloat) -> CGFloat {
let c = min(max(x, 0), 1)
return c * c * (3 - 2 * c)
}
} }
// MARK: - AVPictureInPictureControllerDelegate // MARK: - AVPictureInPictureControllerDelegate
extension FlowPictureInPictureController: AVPictureInPictureControllerDelegate { extension FlowPictureInPictureController: @preconcurrency AVPictureInPictureControllerDelegate {
func pictureInPictureControllerDidStartPictureInPicture( func pictureInPictureControllerDidStartPictureInPicture(
_ pictureInPictureController: AVPictureInPictureController _ pictureInPictureController: AVPictureInPictureController
) { ) {
isPictureInPictureActive = true isPictureInPictureActive = true
lastSystemStartFailure = nil
} }
func pictureInPictureControllerDidStopPictureInPicture( func pictureInPictureControllerDidStopPictureInPicture(
@@ -258,6 +586,16 @@ extension FlowPictureInPictureController: AVPictureInPictureControllerDelegate {
onUserDismissed?() onUserDismissed?()
} }
func pictureInPictureController(
_ pictureInPictureController: AVPictureInPictureController,
failedToStartPictureInPictureWithError error: Error
) {
// First attempts often fail while the sample-buffer source is still
// warming; keep retrying via the display link / auto-inline path.
lastSystemStartFailure = error
FlowDiagnostics.log("PiP start attempt failed (will retry): \(error.localizedDescription)")
}
func pictureInPictureController( func pictureInPictureController(
_ pictureInPictureController: AVPictureInPictureController, _ pictureInPictureController: AVPictureInPictureController,
restoreUserInterfaceForPictureInPictureStopWithCompletionHandler completionHandler: @escaping (Bool) -> Void restoreUserInterfaceForPictureInPictureStopWithCompletionHandler completionHandler: @escaping (Bool) -> Void
@@ -268,22 +606,29 @@ extension FlowPictureInPictureController: AVPictureInPictureControllerDelegate {
// MARK: - AVPictureInPictureSampleBufferPlaybackDelegate // MARK: - AVPictureInPictureSampleBufferPlaybackDelegate
extension FlowPictureInPictureController: AVPictureInPictureSampleBufferPlaybackDelegate { extension FlowPictureInPictureController: @preconcurrency AVPictureInPictureSampleBufferPlaybackDelegate {
func pictureInPictureController( func pictureInPictureController(
_ pictureInPictureController: AVPictureInPictureController, _ pictureInPictureController: AVPictureInPictureController,
setPlaying playing: Bool setPlaying playing: Bool
) { ) {
if playing { if playing {
if animationStartedAt == nil {
animationStartedAt = CACurrentMediaTime()
}
startFramePump() startFramePump()
} else { } else {
stopFramePump() // Do not stop the frame pump on pause sample-buffer PiP keep-alive
// must keep feeding frames so auto-inline can resume.
animationStartedAt = CACurrentMediaTime()
startFramePump()
} }
} }
func pictureInPictureControllerTimeRangeForPlayback( func pictureInPictureControllerTimeRangeForPlayback(
_ pictureInPictureController: AVPictureInPictureController _ pictureInPictureController: AVPictureInPictureController
) -> CMTimeRange { ) -> CMTimeRange {
CMTimeRange(start: .zero, duration: CMTime(value: 3600, timescale: 1)) // Live / unbounded content finite durations make PiP stuck loading.
CMTimeRange(start: .zero, duration: .positiveInfinity)
} }
func pictureInPictureControllerIsPlaybackPaused( func pictureInPictureControllerIsPlaybackPaused(
+102 -30
View File
@@ -84,13 +84,33 @@ final class FlowSessionManager: ObservableObject {
/// Initial proof window cold mic sessions often need >2.5s after app switch. /// Initial proof window cold mic sessions often need >2.5s after app switch.
private static let coldStartAudioProofTimeout: TimeInterval = 6 private static let coldStartAudioProofTimeout: TimeInterval = 6
private var keepAliveMode: FlowKeepAliveMode {
FlowSessionPolicy.keepAliveMode()
}
private var usesPiPKeepAlive: Bool { private var usesPiPKeepAlive: Bool {
FlowSessionPolicy.keepAliveMode() == .pictureInPicture keepAliveMode == .pictureInPicture
} }
func attachPiPHostView(_ view: UIView) { func attachPiPHostView(_ view: UIView) {
pipController.attachHostView(view) pipController.attachHostView(view)
} }
/// Live Activity is mutually exclusive with PiP keep-alive.
private func updateLiveActivityPhase(_ phase: FlowActivityAttributes.ContentState.Phase) {
guard !usesPiPKeepAlive else { return }
FlowLiveActivityController.update(phase: phase)
}
private func startLiveActivityIfNeeded() {
guard !usesPiPKeepAlive else {
// Sweep any orphan island left from a previous Live Activity session.
FlowLiveActivityController.clearOrphanedActivities()
return
}
FlowLiveActivityController.startSession()
}
/// Guards the once-per-process launch reconciliation (scene reconnects /// Guards the once-per-process launch reconciliation (scene reconnects
/// recreate the `@StateObject`-owned manager within the same process). /// recreate the `@StateObject`-owned manager within the same process).
private static var didRunLaunchReconciliation = false private static var didRunLaunchReconciliation = false
@@ -463,8 +483,14 @@ final class FlowSessionManager: ObservableObject {
resumeAfterForeground() resumeAfterForeground()
case .inactive: case .inactive:
writeHeartbeatIfActive() writeHeartbeatIfActive()
if usesPiPKeepAlive, isActive {
pipController.prepareForBackgroundAutoStart()
}
case .background: case .background:
setAppForeground(false) setAppForeground(false)
if usesPiPKeepAlive, isActive {
pipController.prepareForBackgroundAutoStart()
}
if coldStartContext != nil { if coldStartContext != nil {
dismissColdStartOverlay() dismissColdStartOverlay()
} }
@@ -743,21 +769,22 @@ final class FlowSessionManager: ObservableObject {
} }
if usesPiPKeepAlive { if usesPiPKeepAlive {
let pipReady = await pipController.startAndWait() switch await pipController.startAndWait() {
guard pipReady else { case .started:
let message = AppL10n.string("flow.pip.error.unavailable") activateFlowSessionAfterPiPProof(duration: duration)
traceState("startSessionAsync.ready")
debug("Flow session started (PiP keep-alive), mic released between utterances")
case .failed(let failure):
let message = AppL10n.string(failure.localizationKey)
sessionWarning = message sessionWarning = message
traceState("startSessionAsync.failed", extra: "reason=pipUnavailable") traceState("startSessionAsync.failed", extra: "reason=pipUnavailable failure=\(failure)")
FlowSessionBridge.setHostReady(false) FlowSessionBridge.setHostReady(false)
if isColdStartHandoff { if isColdStartHandoff {
showColdStartAudioFailure(message: message) showColdStartPipFailure(message: message)
scheduleColdStartRecovery(duration: duration)
} }
debug("PiP keep-alive failed to start") debug("PiP keep-alive failed to start: \(failure)")
return
} }
activateFlowSessionAfterPiPProof(duration: duration)
traceState("startSessionAsync.ready")
debug("Flow session started (PiP keep-alive), mic released between utterances")
return return
} }
@@ -816,7 +843,7 @@ final class FlowSessionManager: ObservableObject {
bindSessionASRIfNeeded() bindSessionASRIfNeeded()
scheduleASRWarmup() scheduleASRWarmup()
FlowLiveActivityController.startSession() startLiveActivityIfNeeded()
refreshHostReady() refreshHostReady()
traceState("activateFlowSessionAfterPiPProof.done") traceState("activateFlowSessionAfterPiPProof.done")
@@ -841,7 +868,7 @@ final class FlowSessionManager: ObservableObject {
bindSessionASRIfNeeded() bindSessionASRIfNeeded()
scheduleASRWarmup() scheduleASRWarmup()
FlowLiveActivityController.startSession() startLiveActivityIfNeeded()
refreshHostReady() refreshHostReady()
traceState("activateFlowSessionAfterAudioProof.done") traceState("activateFlowSessionAfterAudioProof.done")
@@ -851,6 +878,20 @@ final class FlowSessionManager: ObservableObject {
guard isColdStartHandoff, isActive else { return } guard isColdStartHandoff, isActive else { return }
if usesPiPKeepAlive { if usesPiPKeepAlive {
sessionWarning = nil sessionWarning = nil
if !pipController.isPictureInPictureActive {
switch await pipController.startAndWait() {
case .started:
break
case .failed(let failure):
let message = AppL10n.string(failure.localizationKey)
sessionWarning = message
FlowSessionBridge.setHostReady(false)
showColdStartPipFailure(message: message)
scheduleColdStartRecovery(duration: nil)
debug("existing PiP session failed cold-start restart: \(failure)")
return
}
}
refreshHostReady() refreshHostReady()
handleColdStartAfterSessionReady() handleColdStartAfterSessionReady()
return return
@@ -890,9 +931,16 @@ final class FlowSessionManager: ObservableObject {
debug("cold-start handoff ignored: session busy with an utterance") debug("cold-start handoff ignored: session busy with an utterance")
return return
} }
let message = AppL10n.string("flow.coldStart.error.audioTimeout") let message: String
sessionWarning = message if usesPiPKeepAlive {
showColdStartAudioFailure(message: message) message = AppL10n.string("flow.pip.error.notPossible")
sessionWarning = message
showColdStartPipFailure(message: message)
} else {
message = AppL10n.string("flow.coldStart.error.audioTimeout")
sessionWarning = message
showColdStartAudioFailure(message: message)
}
scheduleColdStartRecovery(duration: nil) scheduleColdStartRecovery(duration: nil)
debug("cold-start blocked: host ready contract not published") debug("cold-start blocked: host ready contract not published")
return return
@@ -903,7 +951,11 @@ final class FlowSessionManager: ObservableObject {
private func presentColdStartReadyOverlay() { private func presentColdStartReadyOverlay() {
let hostEntry = HostReturnService.pendingHostEntry() let hostEntry = HostReturnService.pendingHostEntry()
coldStartContext = FlowColdStartContext(hostEntry: hostEntry, state: .ready) coldStartContext = FlowColdStartContext(
hostEntry: hostEntry,
state: .ready,
keepAliveMode: keepAliveMode
)
scheduleAutoReturnToHostIfNeeded(hostEntry: hostEntry) scheduleAutoReturnToHostIfNeeded(hostEntry: hostEntry)
} }
@@ -929,17 +981,23 @@ final class FlowSessionManager: ObservableObject {
coldStartRecoveryTask = Task { @MainActor [weak self] in coldStartRecoveryTask = Task { @MainActor [weak self] in
guard let self else { return } guard let self else { return }
if self.usesPiPKeepAlive { if self.usesPiPKeepAlive {
let recovered = await self.pipController.startAndWait() let outcome = await self.pipController.startAndWait()
self.traceState("coldStartRecovery.pip", extra: "recovered=\(recovered)") self.traceState("coldStartRecovery.pip", extra: "outcome=\(outcome)")
guard !Task.isCancelled, self.isColdStartHandoff else { return } guard !Task.isCancelled, self.isColdStartHandoff else { return }
if recovered { switch outcome {
case .started:
if self.isActive { if self.isActive {
self.sessionWarning = nil
self.refreshHostReady() self.refreshHostReady()
self.handleColdStartAfterSessionReady() self.handleColdStartAfterSessionReady()
} else { } else {
self.activateFlowSessionAfterPiPProof(duration: duration) self.activateFlowSessionAfterPiPProof(duration: duration)
self.handleColdStartAfterSessionReady() self.handleColdStartAfterSessionReady()
} }
case .failed(let failure):
let message = AppL10n.string(failure.localizationKey)
self.sessionWarning = message
self.showColdStartPipFailure(message: message)
} }
return return
} }
@@ -995,7 +1053,8 @@ final class FlowSessionManager: ObservableObject {
private func showColdStartPreparing() { private func showColdStartPreparing() {
coldStartContext = FlowColdStartContext( coldStartContext = FlowColdStartContext(
hostEntry: HostReturnService.pendingHostEntry(), hostEntry: HostReturnService.pendingHostEntry(),
state: .preparing state: .preparing,
keepAliveMode: keepAliveMode
) )
} }
@@ -1003,7 +1062,8 @@ final class FlowSessionManager: ObservableObject {
FlowSessionBridge.setHostReady(false) FlowSessionBridge.setHostReady(false)
coldStartContext = FlowColdStartContext( coldStartContext = FlowColdStartContext(
hostEntry: HostReturnService.pendingHostEntry(), hostEntry: HostReturnService.pendingHostEntry(),
state: .failed(.permission(message: permissionWarningMessage())) state: .failed(.permission(message: permissionWarningMessage())),
keepAliveMode: keepAliveMode
) )
} }
@@ -1011,7 +1071,17 @@ final class FlowSessionManager: ObservableObject {
FlowSessionBridge.setHostReady(false) FlowSessionBridge.setHostReady(false)
coldStartContext = FlowColdStartContext( coldStartContext = FlowColdStartContext(
hostEntry: HostReturnService.pendingHostEntry(), hostEntry: HostReturnService.pendingHostEntry(),
state: .failed(.audio(message: message)) state: .failed(.audio(message: message)),
keepAliveMode: keepAliveMode
)
}
private func showColdStartPipFailure(message: String) {
FlowSessionBridge.setHostReady(false)
coldStartContext = FlowColdStartContext(
hostEntry: HostReturnService.pendingHostEntry(),
state: .failed(.pip(message: message)),
keepAliveMode: .pictureInPicture
) )
} }
@@ -1221,6 +1291,8 @@ final class FlowSessionManager: ObservableObject {
guard usesPiPKeepAlive, capture.running else { return } guard usesPiPKeepAlive, capture.running else { return }
guard !isUtteranceRecording, !isUtteranceProcessing else { return } guard !isUtteranceRecording, !isUtteranceProcessing else { return }
capture.stop() capture.stop()
// Capture deactivates AVAudioSession; restore playback so PiP stays eligible.
_ = pipController.reassertKeepAliveAudioSession()
pipController.updateWaveformLevels([]) pipController.updateWaveformLevels([])
refreshHostReady() refreshHostReady()
} }
@@ -1274,7 +1346,7 @@ final class FlowSessionManager: ObservableObject {
utteranceRecordingStartedAt = Date() utteranceRecordingStartedAt = Date()
startUtteranceSafetyTimer() startUtteranceSafetyTimer()
refreshHostReady() refreshHostReady()
FlowLiveActivityController.update(phase: .recording) updateLiveActivityPhase(.recording)
FlowDiagnostics.log( FlowDiagnostics.log(
"beginUtterance engine=\(store.engineMode) " + "beginUtterance engine=\(store.engineMode) " +
"asrType=\(type(of: asr)) pipelined=true " + "asrType=\(type(of: asr)) pipelined=true " +
@@ -1349,7 +1421,7 @@ final class FlowSessionManager: ObservableObject {
utteranceSafetyTask?.cancel() utteranceSafetyTask?.cancel()
utteranceSafetyTask = nil utteranceSafetyTask = nil
refreshHostReady() refreshHostReady()
FlowLiveActivityController.update(phase: .processing) updateLiveActivityPhase(.processing)
// Snapshot pipelined partial before drain fallback if the final chunk ASR drops tail text. // Snapshot pipelined partial before drain fallback if the final chunk ASR drops tail text.
bestPartialSnapshot = currentPartial.trimmingCharacters(in: .whitespacesAndNewlines) bestPartialSnapshot = currentPartial.trimmingCharacters(in: .whitespacesAndNewlines)
@@ -1402,7 +1474,7 @@ final class FlowSessionManager: ObservableObject {
chunkWarnings = [] chunkWarnings = []
currentUtteranceId = nil currentUtteranceId = nil
currentCommandSeq = 0 currentCommandSeq = 0
FlowLiveActivityController.update(phase: .idle) updateLiveActivityPhase(.idle)
refreshHostReady() refreshHostReady()
debug("utterance aborted") debug("utterance aborted")
} }
@@ -1432,7 +1504,7 @@ final class FlowSessionManager: ObservableObject {
storeCurrentError(message, kind: kind) storeCurrentError(message, kind: kind)
currentUtteranceId = nil currentUtteranceId = nil
currentCommandSeq = 0 currentCommandSeq = 0
FlowLiveActivityController.update(phase: .idle) updateLiveActivityPhase(.idle)
refreshHostReady() refreshHostReady()
debug("utterance failed: \(message)") debug("utterance failed: \(message)")
} }
@@ -1458,7 +1530,7 @@ final class FlowSessionManager: ObservableObject {
storeCurrentError(message, kind: kind) storeCurrentError(message, kind: kind)
currentUtteranceId = nil currentUtteranceId = nil
currentCommandSeq = 0 currentCommandSeq = 0
FlowLiveActivityController.update(phase: .idle) updateLiveActivityPhase(.idle)
refreshHostReady() refreshHostReady()
debug("utterance processing failed: \(message)") debug("utterance processing failed: \(message)")
} }
@@ -1638,7 +1710,7 @@ final class FlowSessionManager: ObservableObject {
let wasProcessing = isUtteranceProcessing let wasProcessing = isUtteranceProcessing
isUtteranceProcessing = false isUtteranceProcessing = false
FlowLiveActivityController.update(phase: .idle) updateLiveActivityPhase(.idle)
if isActive { if isActive {
touchSessionActivity() touchSessionActivity()
} }
@@ -1856,7 +1928,7 @@ final class FlowSessionManager: ObservableObject {
FlowSessionBridge.writeHeartbeat() FlowSessionBridge.writeHeartbeat()
self.refreshHostReady() self.refreshHostReady()
tick += 1 tick += 1
if tick % liveActivityKeepAliveEveryTicks == 0 { if !self.usesPiPKeepAlive, tick % liveActivityKeepAliveEveryTicks == 0 {
FlowLiveActivityController.keepAlive() FlowLiveActivityController.keepAlive()
} }
try? await Task.sleep(nanoseconds: 1_000_000_000) try? await Task.sleep(nanoseconds: 1_000_000_000)
@@ -18,17 +18,18 @@ enum AppTab: Int, CaseIterable {
var icon: MaterialIconName { var icon: MaterialIconName {
switch self { switch self {
case .keyboard: return .keyboard case .keyboard: return .keyboard
case .history: return .menuBook case .history: return .menuBook // unused history uses SF Symbol
case .dictionary: return .menuBook // unused dictionary uses SF Symbol case .dictionary: return .menuBook // unused dictionary uses SF Symbol
case .styles: return .menuBook // unused styles uses SF Symbol case .styles: return .menuBook // unused styles uses SF Symbol
case .settings: return .settings case .settings: return .settings
} }
} }
/// Filled SF Symbol override for the dictionary tab. /// SF Symbol overrides shared with the Mac and iPad sidebars.
var sfSymbol: String? { var sfSymbol: String? {
switch self { switch self {
case .dictionary: return "square.stack.3d.down.right.fill" case .history: return "clock.arrow.circlepath"
case .dictionary: return "character.book.closed"
case .styles: return "text.badge.star" case .styles: return "text.badge.star"
default: return nil default: return nil
} }
+25 -7
View File
@@ -13,6 +13,8 @@ import OSGKeyboardShared
struct FlowColdStartContext: Equatable { struct FlowColdStartContext: Equatable {
let hostEntry: HostAppEntry? let hostEntry: HostAppEntry?
var state: FlowColdStartState var state: FlowColdStartState
/// Drives preparing / PiP-specific copy (Live Activity vs picture-in-picture).
var keepAliveMode: FlowKeepAliveMode
} }
enum FlowColdStartState: Equatable { enum FlowColdStartState: Equatable {
@@ -24,6 +26,8 @@ enum FlowColdStartState: Equatable {
enum FlowColdStartFailure: Equatable { enum FlowColdStartFailure: Equatable {
case permission(message: String) case permission(message: String)
case audio(message: String) case audio(message: String)
/// Picture-in-picture keep-alive could not be proven active.
case pip(message: String)
} }
struct FlowColdStartOverlay: View { struct FlowColdStartOverlay: View {
@@ -119,7 +123,7 @@ struct FlowColdStartOverlay: View {
ProgressView() ProgressView()
.tint(palette.accent) .tint(palette.accent)
.scaleEffect(1.1) .scaleEffect(1.1)
.accessibilityLabel(AppL10n.string("flow.coldStart.preparing")) .accessibilityLabel(preparingTitle)
case .ready: case .ready:
Image(systemName: "checkmark.circle.fill") Image(systemName: "checkmark.circle.fill")
.font(.system(size: 26, weight: .semibold)) .font(.system(size: 26, weight: .semibold))
@@ -142,7 +146,7 @@ struct FlowColdStartOverlay: View {
switch failure { switch failure {
case .permission: case .permission:
linkButton(AppL10n.string("flow.coldStart.action.settings"), action: onOpenSettings) linkButton(AppL10n.string("flow.coldStart.action.settings"), action: onOpenSettings)
case .audio: case .audio, .pip:
linkButton(AppL10n.string("flow.coldStart.action.retry"), action: onRetry) linkButton(AppL10n.string("flow.coldStart.action.retry"), action: onRetry)
} }
} }
@@ -157,10 +161,19 @@ struct FlowColdStartOverlay: View {
.buttonStyle(.plain) .buttonStyle(.plain)
} }
private var preparingTitle: String {
switch context.keepAliveMode {
case .pictureInPicture:
return AppL10n.string("flow.coldStart.preparing.pip")
case .liveActivity:
return AppL10n.string("flow.coldStart.preparing")
}
}
private var title: String { private var title: String {
switch context.state { switch context.state {
case .preparing: case .preparing:
return AppL10n.string("flow.coldStart.preparing") return preparingTitle
case .ready: case .ready:
return AppL10n.string("flow.coldStart.title") return AppL10n.string("flow.coldStart.title")
case .failed(let failure): case .failed(let failure):
@@ -169,6 +182,8 @@ struct FlowColdStartOverlay: View {
return AppL10n.string("flow.coldStart.permission.title") return AppL10n.string("flow.coldStart.permission.title")
case .audio: case .audio:
return AppL10n.string("flow.coldStart.audio.title") return AppL10n.string("flow.coldStart.audio.title")
case .pip:
return AppL10n.string("flow.coldStart.pip.title")
} }
} }
} }
@@ -176,14 +191,17 @@ struct FlowColdStartOverlay: View {
private var message: String { private var message: String {
switch context.state { switch context.state {
case .preparing: case .preparing:
return AppL10n.string("flow.coldStart.preparingHint") switch context.keepAliveMode {
case .pictureInPicture:
return AppL10n.string("flow.coldStart.preparingHint.pip")
case .liveActivity:
return AppL10n.string("flow.coldStart.preparingHint")
}
case .ready: case .ready:
return AppL10n.string("flow.coldStart.swipeHint") return AppL10n.string("flow.coldStart.swipeHint")
case .failed(let failure): case .failed(let failure):
switch failure { switch failure {
case .permission(let message): case .permission(let message), .audio(let message), .pip(let message):
return message
case .audio(let message):
return message return message
} }
} }
+25 -3
View File
@@ -9,15 +9,37 @@ import UIKit
struct FlowPiPHostView: UIViewRepresentable { struct FlowPiPHostView: UIViewRepresentable {
let attach: (UIView) -> Void let attach: (UIView) -> Void
func makeUIView(context: Context) -> UIView { func makeUIView(context: Context) -> FlowPiPHostUIView {
let view = UIView(frame: CGRect(x: 0, y: 0, width: 2, height: 2)) // Non-trivial size: a 1×1 / fully invisible host often keeps
// `isPictureInPicturePossible` false for sample-buffer sources.
let view = FlowPiPHostUIView(frame: CGRect(x: 0, y: 0, width: 64, height: 36))
view.isUserInteractionEnabled = false view.isUserInteractionEnabled = false
view.backgroundColor = .clear view.backgroundColor = .clear
view.isOpaque = false
view.onMovedToWindow = { [weak view] in
guard let view else { return }
attach(view)
}
attach(view) attach(view)
return view return view
} }
func updateUIView(_ uiView: UIView, context: Context) { func updateUIView(_ uiView: FlowPiPHostUIView, context: Context) {
attach(uiView) attach(uiView)
} }
} }
/// Reports window membership so PiP start can wait for a real hierarchy.
final class FlowPiPHostUIView: UIView {
var onMovedToWindow: (() -> Void)?
override func didMoveToWindow() {
super.didMoveToWindow()
onMovedToWindow?()
}
override func layoutSubviews() {
super.layoutSubviews()
onMovedToWindow?()
}
}
+28 -14
View File
@@ -92,9 +92,13 @@ struct HomeView: View {
let logoTopPadding = isCompact ? Spacing.lg : Spacing.xxl let logoTopPadding = isCompact ? Spacing.lg : Spacing.xxl
let logoBottomPadding = isCompact ? Spacing.lg : Spacing.xxl let logoBottomPadding = isCompact ? Spacing.lg : Spacing.xxl
let extrasBottomPadding = isCompact ? Spacing.sm : Spacing.lg let extrasBottomPadding = isCompact ? Spacing.sm : Spacing.lg
let statusTopPadding = isCompact ? Spacing.sm : Spacing.xl // /
// tab let previewMinHeight: CGFloat = {
let previewMinHeight: CGFloat = isCompact ? 72 : 160 if showsFlowSessionExtras {
return isCompact ? 44 : 88
}
return isCompact ? 72 : 160
}()
ZStack(alignment: .top) { ZStack(alignment: .top) {
sessionHeaderGradient(height: gradientHeight) sessionHeaderGradient(height: gradientHeight)
@@ -116,22 +120,17 @@ struct HomeView: View {
.padding(.horizontal, Spacing.lg) .padding(.horizontal, Spacing.lg)
.padding(.bottom, Spacing.md) .padding(.bottom, Spacing.md)
// // safeAreaInset
// tab dock
previewField(minHeight: previewMinHeight) previewField(minHeight: previewMinHeight)
.padding(.horizontal, Spacing.lg) .padding(.horizontal, Spacing.lg)
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top) .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top)
.layoutPriority(-1) .layoutPriority(-1)
HStack(spacing: Spacing.sm) {
engineStatusLine
flowStatusFooter
}
.frame(maxWidth: .infinity, alignment: .center)
.padding(.horizontal, Spacing.lg)
.padding(.top, statusTopPadding)
.padding(.bottom, Spacing.sm)
} }
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top) .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top)
.safeAreaInset(edge: .bottom, spacing: 0) {
phoneStatusFooter
}
} }
.background(palette.background) .background(palette.background)
.contentShape(Rectangle()) .contentShape(Rectangle())
@@ -143,6 +142,19 @@ struct HomeView: View {
} }
} }
/// Engine + Flow bottom inset tab
private var phoneStatusFooter: some View {
HStack(spacing: Spacing.sm) {
engineStatusLine
flowStatusFooter
}
.frame(maxWidth: .infinity, alignment: .center)
.padding(.horizontal, Spacing.lg)
.padding(.top, Spacing.sm)
.padding(.bottom, Spacing.sm)
.background(palette.background.opacity(0.96))
}
// MARK: - Wide layout (iPad / regular width) // MARK: - Wide layout (iPad / regular width)
private var wideBody: some View { private var wideBody: some View {
@@ -486,7 +498,9 @@ struct HomeView: View {
EngineServiceLabel.summary( EngineServiceLabel.summary(
engineMode: config.engineMode, engineMode: config.engineMode,
providerId: config.providerId, providerId: config.providerId,
model: config.model model: config.model,
asrProviderId: config.asrProviderId,
asrModel: config.asrModel
) )
) )
.font(TypeStyle.caption2) .font(TypeStyle.caption2)
+3 -1
View File
@@ -143,7 +143,9 @@ struct KeyboardPreviewSheet: View {
EngineServiceLabel.summary( EngineServiceLabel.summary(
engineMode: config.engineMode, engineMode: config.engineMode,
providerId: config.providerId, providerId: config.providerId,
model: config.model model: config.model,
asrProviderId: config.asrProviderId,
asrModel: config.asrModel
) )
} }
+9 -3
View File
@@ -11,15 +11,19 @@ import OSGKeyboardShared
struct MainAppRoot: View { struct MainAppRoot: View {
@Environment(\.scenePhase) private var scenePhase @Environment(\.scenePhase) private var scenePhase
@StateObject private var config = ProviderConfig.shared // Singleton is owned by `ProviderConfig.shared`, not by this view
// `@ObservedObject` keeps subscriptions correct across Settings replay.
@ObservedObject private var config = ProviderConfig.shared
@StateObject private var flowManager = FlowSessionManager() @StateObject private var flowManager = FlowSessionManager()
var body: some View { var body: some View {
Group { Group {
if config.hasCompletedOnboarding { if config.hasCompletedOnboarding {
MainTabView() MainTabView()
.id("main")
} else { } else {
OnboardingView(config: config) OnboardingView(config: config)
.id("onboarding")
} }
} }
.environment(\.locale, config.uiLanguage.swiftUILocale) .environment(\.locale, config.uiLanguage.swiftUILocale)
@@ -41,9 +45,11 @@ struct MainAppRoot: View {
FlowPiPHostView { view in FlowPiPHostView { view in
flowManager.attachPiPHostView(view) flowManager.attachPiPHostView(view)
} }
.frame(width: 2, height: 2) // Keep a small but real layer in the window hierarchy for PiP.
.opacity(0.001) .frame(width: 64, height: 36)
.opacity(0.02)
.allowsHitTesting(false) .allowsHitTesting(false)
.accessibilityHidden(true)
} }
.onAppear { .onAppear {
flowManager.setAppForeground(scenePhase == .active) flowManager.setAppForeground(scenePhase == .active)
+3 -1
View File
@@ -39,7 +39,9 @@ struct MainTabView: View {
.environment(\.isTabBarVisible, !isTabBarHidden) .environment(\.isTabBarVisible, !isTabBarHidden)
.safeAreaInset(edge: .bottom, spacing: 0) { .safeAreaInset(edge: .bottom, spacing: 0) {
if !isTabBarHidden { if !isTabBarHidden {
Color.clear.frame(height: 88) // Match floating dock + home-indicator clearance so
// page footers / scroll ends sit above MinimalTabBar.
Color.clear.frame(height: 100)
} }
} }
.onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in .onPreferenceChange(TabBarHiddenPreferenceKey.self) { hidden in
+11 -5
View File
@@ -176,13 +176,19 @@ struct OnboardingView: View {
private func advancePage() { private func advancePage() {
refreshPermissionStatuses() refreshPermissionStatuses()
withAnimation(Motion.soft) { // Routing out of onboarding must NOT run inside an animation
if isLastPage { // transaction. Animating OnboardingView MainTabView (plus a nested
// onboardingPage reset and Flow activateOnForeground) can leave the
// last page frozen even when hasCompletedOnboarding is already true.
if isLastPage || nextVisiblePage(after: config.onboardingPage) == nil {
var transaction = Transaction()
transaction.disablesAnimations = true
withTransaction(transaction) {
config.hasCompletedOnboarding = true config.hasCompletedOnboarding = true
} else if let next = nextVisiblePage(after: config.onboardingPage) { }
} else if let next = nextVisiblePage(after: config.onboardingPage) {
withAnimation(Motion.soft) {
config.onboardingPage = next config.onboardingPage = next
} else {
config.hasCompletedOnboarding = true
} }
} }
} }
+7 -3
View File
@@ -401,8 +401,12 @@ struct SettingsView: View {
sectionHeader("settings.about.title") sectionHeader("settings.about.title")
VStack(spacing: 0) { VStack(spacing: 0) {
Button { Button {
config.hasCompletedOnboarding = false var transaction = Transaction()
config.onboardingPage = 0 transaction.disablesAnimations = true
withTransaction(transaction) {
config.hasCompletedOnboarding = false
config.onboardingPage = 0
}
} label: { } label: {
HStack(spacing: Spacing.sm) { HStack(spacing: Spacing.sm) {
Text("settings.onboarding.replay") Text("settings.onboarding.replay")
@@ -582,7 +586,7 @@ private struct FlowKeepAliveModePickerRow: View {
selection: Binding( selection: Binding(
get: { selection.rawValue }, get: { selection.rawValue },
set: { newValue in set: { newValue in
selection = FlowKeepAliveMode(rawValue: newValue) ?? .liveActivity selection = FlowKeepAliveMode(rawValue: newValue) ?? .default
} }
) )
) )
+9 -1
View File
@@ -473,11 +473,19 @@
/* Cold-start handoff (scheme B) */ /* Cold-start handoff (scheme B) */
"flow.coldStart.title" = "Voice is ready"; "flow.coldStart.title" = "Voice is ready";
"flow.coldStart.preparing" = "Getting voice ready"; "flow.coldStart.preparing" = "Getting voice ready";
"flow.coldStart.preparing.pip" = "Starting Picture in Picture";
"flow.coldStart.preparingHint" = "Keep OSGKeyboard open for a moment while we start the microphone session."; "flow.coldStart.preparingHint" = "Keep OSGKeyboard open for a moment while we start the microphone session.";
"flow.coldStart.preparingHint.pip" = "Keep OSGKeyboard open while Picture in Picture starts. Then return to the keyboard to speak.";
"flow.coldStart.permission.title" = "Permission required"; "flow.coldStart.permission.title" = "Permission required";
"flow.coldStart.audio.title" = "Voice could not start"; "flow.coldStart.audio.title" = "Voice could not start";
"flow.coldStart.pip.title" = "Picture in Picture could not start";
"flow.coldStart.error.audioTimeout" = "The microphone did not become ready in time. It may be busy in another app. Please try again."; "flow.coldStart.error.audioTimeout" = "The microphone did not become ready in time. It may be busy in another app. Please try again.";
"flow.pip.error.unavailable" = "Picture in Picture could not start. Check that PiP is allowed for OSGKeyboard in Settings."; "flow.pip.error.unavailable" = "Picture in Picture could not start. Stay in the app and try again.";
"flow.pip.error.unsupported" = "This device does not support Picture in Picture.";
"flow.pip.error.hostNotReady" = "The Picture in Picture surface is not ready yet. Stay in the app and try again.";
"flow.pip.error.notPossible" = "The system cannot start Picture in Picture right now. Keep the app in the foreground and try again.";
"flow.pip.error.systemRejected" = "Picture in Picture was rejected by the system. Please try again shortly.";
"flow.pip.error.timedOut" = "Picture in Picture did not appear in time. Stay in the app and try again.";
"flow.coldStart.action.settings" = "Open Settings"; "flow.coldStart.action.settings" = "Open Settings";
"flow.coldStart.action.retry" = "Try Again"; "flow.coldStart.action.retry" = "Try Again";
"flow.coldStart.swipeHint" = "Swipe right along the bottom bar\nto return to your previous app."; "flow.coldStart.swipeHint" = "Swipe right along the bottom bar\nto return to your previous app.";
@@ -472,11 +472,19 @@
/* 冷启动兜底(方案 B */ /* 冷启动兜底(方案 B */
"flow.coldStart.title" = "语音已就绪"; "flow.coldStart.title" = "语音已就绪";
"flow.coldStart.preparing" = "正在就绪"; "flow.coldStart.preparing" = "正在就绪";
"flow.coldStart.preparing.pip" = "正在启动画中画";
"flow.coldStart.preparingHint" = "请先停留片刻,我们正在启动麦克风会话。"; "flow.coldStart.preparingHint" = "请先停留片刻,我们正在启动麦克风会话。";
"flow.coldStart.preparingHint.pip" = "请先停留片刻,我们正在启动画中画保活。就绪后可返回键盘直接说话。";
"flow.coldStart.permission.title" = "需要权限"; "flow.coldStart.permission.title" = "需要权限";
"flow.coldStart.audio.title" = "语音暂时无法启动"; "flow.coldStart.audio.title" = "语音暂时无法启动";
"flow.coldStart.pip.title" = "画中画暂时无法启动";
"flow.coldStart.error.audioTimeout" = "麦克风未能及时就绪,可能正被通话或其他 App 占用。请稍后重试。"; "flow.coldStart.error.audioTimeout" = "麦克风未能及时就绪,可能正被通话或其他 App 占用。请稍后重试。";
"flow.pip.error.unavailable" = "无法启动画中画,请在系统设置中允许 OSGKeyboard 使用画中画。"; "flow.pip.error.unavailable" = "无法启动画中画,请留在 App 内重试。";
"flow.pip.error.unsupported" = "此设备不支持画中画。";
"flow.pip.error.hostNotReady" = "画中画界面尚未就绪,请留在 App 内稍后重试。";
"flow.pip.error.notPossible" = "系统暂时无法开启画中画,请保持 App 在前台后重试。";
"flow.pip.error.systemRejected" = "画中画启动被系统拒绝,请稍后重试。";
"flow.pip.error.timedOut" = "画中画未能及时出现,请留在 App 内重试。";
"flow.coldStart.action.settings" = "前往设置"; "flow.coldStart.action.settings" = "前往设置";
"flow.coldStart.action.retry" = "重试"; "flow.coldStart.action.retry" = "重试";
"flow.coldStart.swipeHint" = "沿屏幕底部的横条,从左向右滑动\n返回上一个 App。"; "flow.coldStart.swipeHint" = "沿屏幕底部的横条,从左向右滑动\n返回上一个 App。";
@@ -178,10 +178,18 @@ final class KeyboardFlowCoordinator {
// host utt.rec=1 ready=false keyboard forever "". // host utt.rec=1 ready=false keyboard forever "".
let hostBusy = readySnapshot?.reason == .recording let hostBusy = readySnapshot?.reason == .recording
|| readySnapshot?.reason == .processing || readySnapshot?.reason == .processing
// PiP sessions publish `reason=.starting` while the small window is
// coming up treat that as warming so the mic stays orange (wait)
// instead of jumping into another cold start.
let hostWarming = !hostReady let hostWarming = !hostReady
&& !hostBusy && !hostBusy
&& FlowSessionBridge.isSessionActive() && FlowSessionBridge.isSessionActive()
&& (FlowSessionBridge.isHostReachable() || isPendingFlowStart || withinReadyGrace) && (
FlowSessionBridge.isHostReachable()
|| isPendingFlowStart
|| withinReadyGrace
|| readySnapshot?.reason == .starting
)
state.flowSessionActive = FlowSessionBridge.isSessionActive() state.flowSessionActive = FlowSessionBridge.isSessionActive()
state.debugPendingFlowStart = isPendingFlowStart state.debugPendingFlowStart = isPendingFlowStart
state.debugFlowRecording = isFlowRecording state.debugFlowRecording = isFlowRecording
@@ -475,7 +483,11 @@ final class KeyboardFlowCoordinator {
isPendingFlowStart = true isPendingFlowStart = true
isFlowRecording = false isFlowRecording = false
flowStartDeadline = Date().timeIntervalSince1970 + FlowWatchdog.startTimeout flowStartDeadline = Date().timeIntervalSince1970 + FlowWatchdog.startTimeout
state.lastTranscript = ExtL10n.string("keyboard.flow.startingSession") state.lastTranscript = ExtL10n.string(
FlowSessionPolicy.keepAliveMode() == .pictureInPicture
? "keyboard.flow.startingSession.pip"
: "keyboard.flow.startingSession"
)
recomputeMicVoiceAvailability() recomputeMicVoiceAvailability()
openHostApp("startflow") openHostApp("startflow")
startFlowStartWatchdog() startFlowStartWatchdog()
+10 -2
View File
@@ -418,9 +418,17 @@ private struct TranscriptLine: View {
case .unavailable(.missingAPIKey): case .unavailable(.missingAPIKey):
Text(micDisabledHint) Text(micDisabledHint)
case .unavailable(.hostNotReady): case .unavailable(.hostNotReady):
ExtL10n.text("keyboard.flow.sessionInactive") if FlowSessionPolicy.keepAliveMode() == .pictureInPicture {
ExtL10n.text("keyboard.flow.sessionInactive.pip")
} else {
ExtL10n.text("keyboard.flow.sessionInactive")
}
case .unavailable(.preparingSession): case .unavailable(.preparingSession):
ExtL10n.text("keyboard.flow.startingSession") if FlowSessionPolicy.keepAliveMode() == .pictureInPicture {
ExtL10n.text("keyboard.flow.startingSession.pip")
} else {
ExtL10n.text("keyboard.flow.startingSession")
}
case .unavailable(.noFullAccess): case .unavailable(.noFullAccess):
ExtL10n.text("keyboard.error.fullAccessRequired") ExtL10n.text("keyboard.error.fullAccessRequired")
case .unavailable(.appGroupUnavailable): case .unavailable(.appGroupUnavailable):
+2
View File
@@ -134,10 +134,12 @@
/* Flow session (keyboard) */ /* Flow session (keyboard) */
"keyboard.flow.sessionInactive" = "Voice session off"; "keyboard.flow.sessionInactive" = "Voice session off";
"keyboard.flow.sessionInactive.pip" = "Picture in Picture off — tap mic to open OSGKeyboard";
"keyboard.flow.start" = "Start"; "keyboard.flow.start" = "Start";
"keyboard.flow.startA11y" = "Start voice session"; "keyboard.flow.startA11y" = "Start voice session";
"keyboard.flow.sessionExpired" = "Voice session ended. Open OSGKeyboard to restart."; "keyboard.flow.sessionExpired" = "Voice session ended. Open OSGKeyboard to restart.";
"keyboard.flow.startingSession" = "Starting voice session…"; "keyboard.flow.startingSession" = "Starting voice session…";
"keyboard.flow.startingSession.pip" = "Starting Picture in Picture…";
"keyboard.flow.transcribing" = "Transcribing…"; "keyboard.flow.transcribing" = "Transcribing…";
"keyboard.flow.resultTimeout" = "Timed out waiting for transcription. Try again."; "keyboard.flow.resultTimeout" = "Timed out waiting for transcription. Try again.";
"keyboard.flow.hostDisconnected" = "Voice session disconnected. Open OSGKeyboard to restart."; "keyboard.flow.hostDisconnected" = "Voice session disconnected. Open OSGKeyboard to restart.";
@@ -134,10 +134,12 @@
/* Flow session (keyboard) */ /* Flow session (keyboard) */
"keyboard.flow.sessionInactive" = "语音会话未启动"; "keyboard.flow.sessionInactive" = "语音会话未启动";
"keyboard.flow.sessionInactive.pip" = "画中画未启动,点麦克风打开 OSGKeyboard";
"keyboard.flow.start" = "启动"; "keyboard.flow.start" = "启动";
"keyboard.flow.startA11y" = "启动语音会话"; "keyboard.flow.startA11y" = "启动语音会话";
"keyboard.flow.sessionExpired" = "语音会话已结束,请打开 OSGKeyboard 重新启动"; "keyboard.flow.sessionExpired" = "语音会话已结束,请打开 OSGKeyboard 重新启动";
"keyboard.flow.startingSession" = "正在启动语音会话…"; "keyboard.flow.startingSession" = "正在启动语音会话…";
"keyboard.flow.startingSession.pip" = "正在启动画中画…";
"keyboard.flow.transcribing" = "识别中…"; "keyboard.flow.transcribing" = "识别中…";
"keyboard.flow.resultTimeout" = "等待识别结果超时,请重试"; "keyboard.flow.resultTimeout" = "等待识别结果超时,请重试";
"keyboard.flow.hostDisconnected" = "语音会话已断开,请打开 OSGKeyboard 重新启动"; "keyboard.flow.hostDisconnected" = "语音会话已断开,请打开 OSGKeyboard 重新启动";
+21
View File
@@ -14,6 +14,8 @@ import SwiftUI
/// Fixed metrics that keep every desktop surface on the same grid. /// Fixed metrics that keep every desktop surface on the same grid.
enum MacMetrics { enum MacMetrics {
/// Shared height for search fields and primary actions in page headers.
static let pageHeaderControlHeight: CGFloat = 28
/// Shared height for credential inputs and icon buttons matches the iOS /// Shared height for credential inputs and icon buttons matches the iOS
/// settings controls (38). /// settings controls (38).
static let settingsControlHeight: CGFloat = 38 static let settingsControlHeight: CGFloat = 38
@@ -443,6 +445,25 @@ struct MacSettingRow<Content: View>: View {
// MARK: - Page header // MARK: - Page header
/// Capsule-shaped primary action aligned with page-header search controls.
struct MacHeaderActionButtonStyle: ButtonStyle {
@Environment(\.themePalette) private var palette
@Environment(\.isEnabled) private var isEnabled
func makeBody(configuration: Configuration) -> some View {
configuration.label
.padding(.horizontal, Spacing.md)
.frame(height: MacMetrics.pageHeaderControlHeight)
.foregroundStyle(.white)
.background(
palette.accent.opacity(configuration.isPressed ? 0.82 : 1),
in: Capsule()
)
.contentShape(Capsule())
.opacity(isEnabled ? 1 : 0.45)
}
}
/// Page title for History / Dictionary / Settings. Applies the shared /// Page title for History / Dictionary / Settings. Applies the shared
/// `pageHorizontalInset` so its left edge matches inset card content below. /// `pageHorizontalInset` so its left edge matches inset card content below.
/// Type size matches Home's brand line (`TypeStyle.pageTitle`). /// Type size matches Home's brand line (`TypeStyle.pageTitle`).
+111 -4
View File
@@ -11,6 +11,10 @@ struct MacDictionaryView: View {
@Environment(\.themePalette) private var palette @Environment(\.themePalette) private var palette
@State private var query = "" @State private var query = ""
@State private var entryPendingDeletion: PersonalDictionary.Entry? @State private var entryPendingDeletion: PersonalDictionary.Entry?
@State private var showEntryEditor = false
@State private var generatingAliasEntryIDs: Set<UUID> = []
private let aliasGenerator = DictionaryAliasGenerator()
private var lang: AppUILanguage { viewModel.config.uiLanguage } private var lang: AppUILanguage { viewModel.config.uiLanguage }
@@ -49,8 +53,19 @@ struct MacDictionaryView: View {
title: MacL10n.string("mac.section.dictionary", language: lang), title: MacL10n.string("mac.section.dictionary", language: lang),
subtitle: MacL10n.string("mac.page.dictionary.subtitle", language: lang) subtitle: MacL10n.string("mac.page.dictionary.subtitle", language: lang)
) { ) {
if !entries.isEmpty { HStack(spacing: Spacing.sm) {
searchField if !entries.isEmpty {
searchField
}
Button {
showEntryEditor = true
} label: {
Label(
MacL10n.string("mac.dict.add", language: lang),
systemImage: "plus"
)
}
.buttonStyle(MacHeaderActionButtonStyle())
} }
} }
@@ -67,6 +82,11 @@ struct MacDictionaryView: View {
} }
.background(palette.background) .background(palette.background)
.animation(Motion.soft, value: entries.isEmpty) .animation(Motion.soft, value: entries.isEmpty)
.sheet(isPresented: $showEntryEditor) {
MacDictionaryEntryEditor(language: lang) { term in
saveManualEntry(term: term)
}
}
.task { .task {
await MacICloudSyncBootstrap.dictionarySync.pullAndMergeIfEnabled() await MacICloudSyncBootstrap.dictionarySync.pullAndMergeIfEnabled()
viewModel.refreshDictionaryFromCloud() viewModel.refreshDictionaryFromCloud()
@@ -154,8 +174,7 @@ struct MacDictionaryView: View {
.font(TypeStyle.footnote) .font(TypeStyle.footnote)
} }
.padding(.horizontal, Spacing.sm) .padding(.horizontal, Spacing.sm)
.padding(.vertical, 6) .frame(width: 220, height: MacMetrics.pageHeaderControlHeight)
.frame(width: 220)
.background(palette.surface, in: Capsule()) .background(palette.surface, in: Capsule())
.overlay(Capsule().stroke(palette.divider, lineWidth: 0.5)) .overlay(Capsule().stroke(palette.divider, lineWidth: 0.5))
} }
@@ -178,6 +197,8 @@ struct MacDictionaryView: View {
} }
if !entry.aliases.isEmpty { if !entry.aliases.isEmpty {
parts.append(entry.aliases.joined(separator: " / ")) parts.append(entry.aliases.joined(separator: " / "))
} else if generatingAliasEntryIDs.contains(entry.id) {
parts.append(MacL10n.string("mac.dict.aliasesGenerating", language: lang))
} }
return parts.isEmpty ? nil : parts.joined(separator: " · ") return parts.isEmpty ? nil : parts.joined(separator: " · ")
} }
@@ -205,11 +226,97 @@ struct MacDictionaryView: View {
private func delete(_ entry: PersonalDictionary.Entry) { private func delete(_ entry: PersonalDictionary.Entry) {
let store = AppGroupStore(defaults: viewModel.defaults) let store = AppGroupStore(defaults: viewModel.defaults)
store.deletePersonalDictionaryEntry(id: entry.id) store.deletePersonalDictionaryEntry(id: entry.id)
generatingAliasEntryIDs.remove(entry.id)
viewModel.refreshDictionaryFromCloud() viewModel.refreshDictionaryFromCloud()
Task { Task {
try? await MacICloudSyncBootstrap.dictionarySync.pushLocalIfEnabled(store.personalDictionary) try? await MacICloudSyncBootstrap.dictionarySync.pushLocalIfEnabled(store.personalDictionary)
} }
} }
private func saveManualEntry(term: String) {
let store = AppGroupStore(defaults: viewModel.defaults)
var dictionary = store.personalDictionary
guard let saved = dictionary.upsertManual(term: term) else { return }
dictionary.version += 1
store.setPersonalDictionary(dictionary)
viewModel.refreshDictionaryFromCloud()
generatingAliasEntryIDs.insert(saved.id)
Task {
try? await MacICloudSyncBootstrap.dictionarySync.pushLocalIfEnabled(dictionary)
let aliases = await aliasGenerator.generateAliases(for: saved.term)
generatingAliasEntryIDs.remove(saved.id)
guard !aliases.isEmpty else { return }
var latest = store.personalDictionary
guard latest.entries.contains(where: {
$0.id == saved.id && $0.term == saved.term
}) else { return }
latest.updateAliases(for: saved.id, aliases: aliases)
latest.version += 1
store.setPersonalDictionary(latest)
viewModel.refreshDictionaryFromCloud()
try? await MacICloudSyncBootstrap.dictionarySync.pushLocalIfEnabled(latest)
}
}
}
private struct MacDictionaryEntryEditor: View {
let language: AppUILanguage
let onSave: (String) -> Void
@Environment(\.dismiss) private var dismiss
@Environment(\.themePalette) private var palette
@State private var term = ""
@FocusState private var termFocused: Bool
private var trimmedTerm: String {
term.trimmingCharacters(in: .whitespacesAndNewlines)
}
var body: some View {
VStack(alignment: .leading, spacing: Spacing.md) {
Text(MacL10n.string("mac.dict.add", language: language))
.font(TypeStyle.title2)
TextField(
MacL10n.string("mac.dict.addField", language: language),
text: $term
)
.textFieldStyle(.roundedBorder)
.focused($termFocused)
.onSubmit(save)
Text(MacL10n.string("mac.dict.addFooter", language: language))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.fixedSize(horizontal: false, vertical: true)
HStack {
Spacer()
Button(MacL10n.string("mac.cancel", language: language)) {
dismiss()
}
Button(MacL10n.string("mac.save", language: language), action: save)
.buttonStyle(.borderedProminent)
.tint(palette.accent)
.disabled(trimmedTerm.isEmpty)
}
}
.padding(Spacing.xl)
.frame(width: 440)
.background(palette.background)
.onAppear {
termFocused = true
}
}
private func save() {
guard !trimmedTerm.isEmpty else { return }
onSave(trimmedTerm)
dismiss()
}
} }
private struct MacDictionaryRow: View { private struct MacDictionaryRow: View {
@@ -25,7 +25,7 @@ actor MacMLXStreamingASRProvider {
locale: Locale locale: Locale
) async throws -> MacMLXStreamingSession { ) async throws -> MacMLXStreamingSession {
let qwen = try await loadModel(model) let qwen = try await loadModel(model)
var config = StreamingConfig( let config = StreamingConfig(
decodeIntervalSeconds: 0.5, decodeIntervalSeconds: 0.5,
boundaryDecodeIntervalSeconds: 0.2, boundaryDecodeIntervalSeconds: 0.2,
boundaryBoostSeconds: 1.0, boundaryBoostSeconds: 1.0,
+162 -53
View File
@@ -24,6 +24,15 @@ struct MacPolishStylesView: View {
_ = viewModel.polishStylesRevision _ = viewModel.polishStylesRevision
return store.activePolishStyleId return store.activePolishStyleId
} }
private var columns: [GridItem] {
[
GridItem(
.adaptive(minimum: 240, maximum: 360),
spacing: Spacing.md,
alignment: .top
),
]
}
var body: some View { var body: some View {
VStack(spacing: 0) { VStack(spacing: 0) {
@@ -40,7 +49,7 @@ struct MacPolishStylesView: View {
systemImage: "plus" systemImage: "plus"
) )
} }
.buttonStyle(.borderedProminent) .buttonStyle(MacHeaderActionButtonStyle())
.disabled(catalog.entries.count >= PolishStyleLimits.maximumUserPacks) .disabled(catalog.entries.count >= PolishStyleLimits.maximumUserPacks)
} }
@@ -96,72 +105,52 @@ struct MacPolishStylesView: View {
.font(MacSettingsType.sectionTitle) .font(MacSettingsType.sectionTitle)
.foregroundStyle(palette.textSecondary) .foregroundStyle(palette.textSecondary)
.textCase(.uppercase) .textCase(.uppercase)
MacCard(padding: 0) {
VStack(spacing: 0) { LazyVGrid(columns: columns, alignment: .leading, spacing: Spacing.md) {
ForEach(packs) { pack in ForEach(packs) { pack in
styleRow(pack) styleCard(pack)
if pack.id != packs.last?.id {
Divider().background(palette.divider)
}
}
} }
} }
} }
} }
private func styleRow(_ pack: PolishStylePack) -> some View { private func styleCard(_ pack: PolishStylePack) -> some View {
HStack(spacing: Spacing.md) { MacPolishStyleCard(
Button { name: pack.displayName(language: lang),
subtitle: subtitle(for: pack),
iconName: iconName(for: pack),
isSelected: pack.id == activeID,
isUserStyle: pack.kind == .user,
language: lang,
activate: {
activate(pack) activate(pack)
} label: { },
HStack(spacing: Spacing.md) { duplicate: {
Image(systemName: pack.id == activeID ? "checkmark.circle.fill" : "circle")
.foregroundStyle(pack.id == activeID ? palette.accent : palette.textTertiary)
VStack(alignment: .leading, spacing: 2) {
Text(pack.displayName(language: lang))
.font(TypeStyle.body)
.foregroundStyle(palette.textPrimary)
Text(subtitle(for: pack))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.lineLimit(1)
}
Spacer()
}
.contentShape(Rectangle())
}
.buttonStyle(.plain)
Button {
editingPack = PolishStylePack( editingPack = PolishStylePack(
name: "\(pack.displayName(language: lang)) \(MacL10n.string("mac.styles.copy", language: lang))", name: "\(pack.displayName(language: lang)) \(MacL10n.string("mac.styles.copy", language: lang))",
prompt: pack.prompt prompt: pack.prompt
) )
showEditor = true showEditor = true
} label: { },
Image(systemName: "plus.square.on.square") edit: {
editingPack = pack
showEditor = true
},
delete: {
delete(pack)
} }
.buttonStyle(.borderless) )
}
if pack.kind == .user { private func iconName(for pack: PolishStylePack) -> String {
Button { switch pack.id {
editingPack = pack case "builtin.structured": return "list.bullet.rectangle"
showEditor = true case "builtin.formal": return "briefcase"
} label: { case "builtin.dating": return "heart.text.square"
Image(systemName: "pencil") case "builtin.chat": return "bubble.left.and.bubble.right"
} case "builtin.light": return "wand.and.sparkles"
.buttonStyle(.borderless) default: return "text.badge.star"
Button(role: .destructive) {
delete(pack)
} label: {
Image(systemName: "trash")
}
.buttonStyle(.borderless)
}
} }
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.sm)
} }
private func subtitle(for pack: PolishStylePack) -> String { private func subtitle(for pack: PolishStylePack) -> String {
@@ -210,6 +199,126 @@ struct MacPolishStylesView: View {
} }
} }
private struct MacPolishStyleCard: View {
let name: String
let subtitle: String
let iconName: String
let isSelected: Bool
let isUserStyle: Bool
let language: AppUILanguage
let activate: () -> Void
let duplicate: () -> Void
let edit: () -> Void
let delete: () -> Void
@Environment(\.themePalette) private var palette
@State private var isHovering = false
private let shape = RoundedRectangle(cornerRadius: Radius.xl, style: .continuous)
var body: some View {
ZStack(alignment: .topTrailing) {
Button(action: activate) {
VStack(alignment: .leading, spacing: Spacing.sm) {
Image(systemName: iconName)
.font(.system(size: 24, weight: .medium))
.foregroundStyle(isSelected ? palette.accent : palette.textSecondary)
Spacer(minLength: Spacing.xs)
Text(name)
.font(TypeStyle.bodyEmph)
.foregroundStyle(palette.textPrimary)
.lineLimit(1)
Text(subtitle)
.font(TypeStyle.caption2)
.foregroundStyle(palette.textTertiary)
.lineLimit(2)
.fixedSize(horizontal: false, vertical: true)
}
.frame(maxWidth: .infinity, minHeight: 132, alignment: .leading)
.padding(Spacing.md)
.contentShape(Rectangle())
}
.buttonStyle(.plain)
actionButtons
.padding(Spacing.sm)
if isSelected {
Image(systemName: "checkmark.circle.fill")
.font(.system(size: 21, weight: .semibold))
.foregroundStyle(palette.accent)
.background(palette.surface, in: Circle())
.padding(Spacing.sm)
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .bottomTrailing)
.allowsHitTesting(false)
}
}
.background(
isSelected ? palette.accentMuted : palette.surface,
in: shape
)
.overlay(
shape.stroke(
isSelected ? palette.accent : hoverBorder,
lineWidth: isSelected ? 1.5 : 0.5
)
)
.clipShape(shape)
.scaleEffect(isHovering ? 1.01 : 1)
.animation(Motion.quick, value: isHovering)
.animation(Motion.quick, value: isSelected)
.onHover { isHovering = $0 }
}
private var actionButtons: some View {
HStack(spacing: Spacing.xxs) {
cardActionButton(
systemImage: "plus.square.on.square",
accessibilityLabel: MacL10n.string("mac.styles.copy", language: language),
action: duplicate
)
if isUserStyle {
cardActionButton(
systemImage: "pencil",
accessibilityLabel: MacL10n.string("mac.styles.edit", language: language),
action: edit
)
cardActionButton(
systemImage: "trash",
accessibilityLabel: MacL10n.string("mac.delete", language: language),
foreground: palette.danger,
action: delete
)
}
}
}
private func cardActionButton(
systemImage: String,
accessibilityLabel: String,
foreground: Color? = nil,
action: @escaping () -> Void
) -> some View {
Button(action: action) {
Image(systemName: systemImage)
.font(.system(size: 12, weight: .semibold))
.foregroundStyle(foreground ?? palette.textSecondary)
.frame(width: 28, height: 28)
.background(palette.background.opacity(isHovering ? 0.9 : 0.72), in: Circle())
}
.buttonStyle(.plain)
.accessibilityLabel(accessibilityLabel)
}
private var hoverBorder: Color {
isHovering ? palette.dividerStrong : palette.divider
}
}
private struct MacPolishStyleEditor: View { private struct MacPolishStyleEditor: View {
let pack: PolishStylePack? let pack: PolishStylePack?
let language: AppUILanguage let language: AppUILanguage
@@ -176,7 +176,7 @@ public struct SonicParticleField: View {
let opacity = (1.0 - progress) * 0.28 let opacity = (1.0 - progress) * 0.28
let lineWidth = max(0.8, 2.4 - progress * 1.4) let lineWidth = max(0.8, 2.4 - progress * 1.4)
var ringContext = context let ringContext = context
ringContext.stroke( ringContext.stroke(
Path(ellipseIn: CGRect( Path(ellipseIn: CGRect(
x: ripple.origin.x - radius, x: ripple.origin.x - radius,
@@ -10,6 +10,8 @@ public enum EngineServiceLabel {
engineMode: String, engineMode: String,
providerId: String, providerId: String,
model: String, model: String,
asrProviderId: String? = nil,
asrModel: String? = nil,
language: AppUILanguage? = nil language: AppUILanguage? = nil
) -> String { ) -> String {
let lang = language ?? AppGroupStore().uiLanguage let lang = language ?? AppGroupStore().uiLanguage
@@ -17,8 +19,17 @@ public enum EngineServiceLabel {
let asrName = SharedL10n.string("engine.asr.appleSpeech", language: lang) let asrName = SharedL10n.string("engine.asr.appleSpeech", language: lang)
return SharedL10n.format("engine.summary.local", language: lang, asrName) return SharedL10n.format("engine.summary.local", language: lang, asrName)
} }
let providerName = ProviderDisplayName.name(for: providerId, language: lang) // Cloud status line should name the speech engine, not the polish LLM.
let trimmedModel = model.trimmingCharacters(in: .whitespacesAndNewlines) let resolvedASRProvider: String = {
if let asrProviderId, !asrProviderId.isEmpty { return asrProviderId }
return providerId
}()
let resolvedASRModel: String = {
if let asrModel, !asrModel.isEmpty { return asrModel }
return model
}()
let providerName = ProviderDisplayName.name(for: resolvedASRProvider, language: lang)
let trimmedModel = resolvedASRModel.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmedModel.isEmpty { if trimmedModel.isEmpty {
return SharedL10n.format("engine.summary.cloud", language: lang, providerName) return SharedL10n.format("engine.summary.cloud", language: lang, providerName)
} }
@@ -26,7 +26,9 @@ public enum FlowColdStartOverlayDecision: Equatable, Sendable {
public enum FlowHandoffPolicy { public enum FlowHandoffPolicy {
/// Proactive keyboard auto-launch of the host is intentionally disabled. /// Proactive keyboard auto-launch of the host is intentionally disabled.
/// Opening the host must be driven by an explicit mic press (or Live Activity). /// Opening the host must be driven by an explicit mic press (or a Live
/// Activity tap when that keep-alive mode is selected). PiP sessions
/// never auto-jump once `hostReady` is published.
public static let allowsProactiveHostAutoLaunch = false public static let allowsProactiveHostAutoLaunch = false
/// Samples of "host truly dead" required before a cold-start jump is allowed /// Samples of "host truly dead" required before a cold-start jump is allowed
@@ -6,15 +6,15 @@
import Foundation import Foundation
public enum FlowKeepAliveMode: String, CaseIterable, Identifiable, Sendable, Codable { public enum FlowKeepAliveMode: String, CaseIterable, Identifiable, Sendable, Codable {
/// Continuous audio capture + Live Activity (current default behaviour). /// Continuous audio capture + Live Activity.
case liveActivity = "liveActivity" case liveActivity = "liveActivity"
/// Picture-in-picture waveform keep-alive; mic released between utterances. /// Picture-in-picture waveform keep-alive; mic released between utterances.
case pictureInPicture = "pictureInPicture" case pictureInPicture = "pictureInPicture"
public var id: String { rawValue } public var id: String { rawValue }
/// Existing installs keep the Live Activity / continuous-capture path. /// Used when no valid keep-alive preference has been stored.
public static let `default`: FlowKeepAliveMode = .liveActivity public static let `default`: FlowKeepAliveMode = .pictureInPicture
public var labelKey: String { public var labelKey: String {
switch self { switch self {
+40 -20
View File
@@ -57,26 +57,46 @@ public enum PolishIntensity: String, Codable, Sendable, CaseIterable {
/// but must not override the style pack's length and format rules. /// but must not override the style pack's length and format rules.
public func promptGuideline(styleID: String?) -> String { public func promptGuideline(styleID: String?) -> String {
let base: String let base: String
switch self { if styleID == "builtin.dating" {
case .light: switch self {
base = """ case .light:
Light rewrite: remove isolated filler words (, , , , , , ok, um, uh) and obvious duplicated fragments only. \ base = """
Do not rephrase otherwise-clear wording. \ Dating Light: apply ASR cleanup, then soften interrogation, lecturing, commands, dismissal, blame, or pressure even when the original wording is otherwise clear. \
Still restore punctuation and sentence breaks per the global output contract and active style pack. Make the message warm and respectful without adding flirtation, teasing, romantic intent, or relationship escalation. Preserve the speaker's recognizable voice.
""" """
case .medium: case .medium:
base = """ base = """
Medium rewrite: fix obvious ASR errors (homophones, missing/extra characters), remove fillers and duplicated fragments, \ Dating Medium: apply Light corrections, then add at most one grounded observation, affiliative joke, natural self-disclosure, follow-up question, or restrained signal of interest when supported by the transcript or preceding context. \
adjust obviously-broken word order. Preserve the speaker's voice. \ Keep any flirtation light, interpretable at face value, and easy to decline. Do not invent relationship context.
Still restore punctuation and breaks per the global output contract and active style pack. \ """
Do not invent facts or change numbers/proper nouns. case .heavy:
""" base = """
case .heavy: Dating Heavy: apply Medium corrections and, only when the transcript already expresses romantic interest or invitation and the context shows no rejection, discomfort, vulnerability, or power imbalance, make appreciation, longing, expectation, or invitation more proactive and clear. \
base = """ Increase romantic tension and directness, not ambiguity, pressure, sexual escalation, or message length. Preserve the speaker's recognizable voice and every fact.
Heavy rewrite: apply medium corrections, then you may reorganize paragraphs, split long sentences, and listify enumerated content when the active style pack allows it. \ """
Punctuation is mandatory at every intensity. \ }
Preserve every fact, number, and proper noun. Do not add information. } else {
""" switch self {
case .light:
base = """
Light rewrite: remove isolated filler words (, , , , , , ok, um, uh) and obvious duplicated fragments only. \
Do not rephrase otherwise-clear wording. \
Still restore punctuation and sentence breaks per the global output contract and active style pack.
"""
case .medium:
base = """
Medium rewrite: fix obvious ASR errors (homophones, missing/extra characters), remove fillers and duplicated fragments, \
adjust obviously-broken word order. Preserve the speaker's voice. \
Still restore punctuation and breaks per the global output contract and active style pack. \
Do not invent facts or change numbers/proper nouns.
"""
case .heavy:
base = """
Heavy rewrite: apply medium corrections, then you may reorganize paragraphs, split long sentences, and listify enumerated content when the active style pack allows it. \
Punctuation is mandatory at every intensity. \
Preserve every fact, number, and proper noun. Do not add information.
"""
}
} }
guard self == .heavy, guard self == .heavy,
+55 -21
View File
@@ -292,46 +292,80 @@ public enum PolishStylePackCatalog {
name: "直男癌拯救器", name: "直男癌拯救器",
prompt: """ prompt: """
# #
\(dictionaryPlaceholder) \(dictionaryPlaceholder)
\(sharedASRRules) \(sharedASRRules)
# #
> > > > >
± 20%
# #
1. ****使 lightmediumheavy
2. **** - **Light**
3. ****使 - **Medium** Light 退
4. **** - **Heavy**
5. ****
6. ****
7. ****
# #
- 使 Heavy
-
-
-
- Light
#
1. ****
2. ****
3. ****
4. ****
5. ****
6. ****
7. ****
8. ****
9. ****
10. ****
#
-
- 15 退Medium Heavy
-
- -
- 使 - 使
-
- 使 emoji - 使 emoji
# #
PUA PUA使
# #
Light
Medium
Heavy便
Light便
Medium
Heavy
Light
Medium
Heavy
Light
Medium
Heavy
Light
Medium
Heavy
# #
+44 -19
View File
@@ -125,8 +125,19 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
OSGLog.config.info("[onboarding] didSet → \(newValue, privacy: .public), mirroring to Keychain") OSGLog.config.info("[onboarding] didSet → \(newValue, privacy: .public), mirroring to Keychain")
Keychain.setOnboardingCompleted(hasCompletedOnboarding) Keychain.setOnboardingCompleted(hasCompletedOnboarding)
if hasCompletedOnboarding { if hasCompletedOnboarding {
// Persist page reset immediately, but defer the @Published bump
// so MainAppRoot's OnboardingView MainTabView swap is not
// coalesced with an in-flow page update (can freeze step 6).
configuration.onboardingPage = 0 configuration.onboardingPage = 0
onboardingPage = 0 let needsPublishedPageReset = onboardingPage != 0
persistConfiguration()
if needsPublishedPageReset {
Task { @MainActor in
guard self.hasCompletedOnboarding else { return }
self.onboardingPage = 0
}
}
return
} }
persistConfiguration() persistConfiguration()
} }
@@ -319,24 +330,38 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
self.defaults = resolvedDefaults self.defaults = resolvedDefaults
self.configuration = AppGroupConfiguration.load(fromAvailable: resolvedDefaults) self.configuration = AppGroupConfiguration.load(fromAvailable: resolvedDefaults)
// Onboarding completion must survive a device reboot. App Group // Fresh app container (reinstall after delete): wipe stale Keychain
// UserDefaults can transiently read empty right after boot, which would // onboarding so the welcome flow shows again. Reboot races still use
// falsely re-show onboarding. Trust the durable Keychain marker when the // Keychain restore when the install identity already exists.
// App Group value looks unset, and backfill it once the App Group value let isFreshInstall = Keychain.beginInstallIdentityIfNeeded()
// is confirmed true (covers users onboarded before this safeguard). if isFreshInstall {
let appGroupOnboarding = configuration.hasCompletedOnboarding configuration.hasCompletedOnboarding = false
let keychainOnboarding = Keychain.hasCompletedOnboarding() resolvedDefaults.set(false, forKey: AppGroupConfiguration.Keys.hasCompletedOnboarding)
// Distinguish "key absent" (nil plist not loaded / data-protection race) OSGLog.config.info("[onboarding] init: fresh install → force hasCompletedOnboarding=false")
// from "key present == false" (something actually wrote false). } else {
let rawKeyPresent = resolvedDefaults.object(forKey: AppGroupConfiguration.Keys.hasCompletedOnboarding) != nil // Onboarding completion must survive a device reboot. App Group
OSGLog.config.info( // UserDefaults can transiently read empty right after boot, which would
"[onboarding] init: appGroup=\(appGroupOnboarding, privacy: .public) (keyPresent=\(rawKeyPresent, privacy: .public)), keychain=\(keychainOnboarding, privacy: .public)" // falsely re-show onboarding. Trust the durable Keychain marker when the
) // App Group value looks unset, and backfill it once the App Group value
if appGroupOnboarding { // is confirmed true (covers users onboarded before this safeguard).
Keychain.setOnboardingCompleted(true) let appGroupOnboarding = configuration.hasCompletedOnboarding
} else if keychainOnboarding { let keychainOnboarding = Keychain.hasCompletedOnboarding()
configuration.hasCompletedOnboarding = true // Distinguish "key absent" (nil plist not loaded / data-protection race)
OSGLog.config.info("[onboarding] init: App Group read false but Keychain true → restored to true") // from "key present == false" (something actually wrote false).
let rawKeyPresent = resolvedDefaults.object(
forKey: AppGroupConfiguration.Keys.hasCompletedOnboarding
) != nil
OSGLog.config.info(
"[onboarding] init: appGroup=\(appGroupOnboarding, privacy: .public) (keyPresent=\(rawKeyPresent, privacy: .public)), keychain=\(keychainOnboarding, privacy: .public)"
)
if appGroupOnboarding {
Keychain.setOnboardingCompleted(true)
} else if keychainOnboarding {
configuration.hasCompletedOnboarding = true
OSGLog.config.info(
"[onboarding] init: App Group read false but Keychain true → restored to true"
)
}
} }
let finalOnboarding = configuration.hasCompletedOnboarding let finalOnboarding = configuration.hasCompletedOnboarding
OSGLog.config.info("[onboarding] init: final=\(finalOnboarding, privacy: .public)") OSGLog.config.info("[onboarding] init: final=\(finalOnboarding, privacy: .public)")
@@ -1,24 +1,23 @@
// DictionaryAliasGenerator.swift // DictionaryAliasGenerator.swift
// OSGKeyboard · Main App // OSGKeyboard · Shared
// //
// After the user manually adds or edits a personal-dictionary term, // After the user manually adds or edits a personal-dictionary term,
// asks the built-in DeepSeek endpoint for common ASR misrecognitions. // asks the built-in DeepSeek endpoint for common ASR misrecognitions.
// Runs only in the main app (Settings) the keyboard extension reads // Shared by the iOS and macOS dictionary editors; persisted aliases are
// the persisted aliases on the next polish / correction call. // available to the keyboard extension on the next polish / correction call.
import Foundation import Foundation
import OSGKeyboardShared
struct DictionaryAliasGenerator: Sendable { public struct DictionaryAliasGenerator: Sendable {
private let client: LLMClient? private let client: LLMClient?
private let timeout: TimeInterval private let timeout: TimeInterval
init(client: LLMClient? = nil, timeout: TimeInterval = 12) { public init(client: LLMClient? = nil, timeout: TimeInterval = 12) {
self.client = client self.client = client
self.timeout = timeout self.timeout = timeout
} }
func generateAliases(for term: String) async -> [String] { public func generateAliases(for term: String) async -> [String] {
let trimmed = term.trimmingCharacters(in: .whitespacesAndNewlines) let trimmed = term.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return [] } guard !trimmed.isEmpty else { return [] }
@@ -70,7 +69,7 @@ struct DictionaryAliasGenerator: Sendable {
""" """
} }
static func parseAliases(from raw: String, excludingTerm term: String) -> [String] { public static func parseAliases(from raw: String, excludingTerm term: String) -> [String] {
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines) let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
let jsonSlice = extractJSONArray(from: trimmed) ?? trimmed let jsonSlice = extractJSONArray(from: trimmed) ?? trimmed
guard let data = jsonSlice.data(using: .utf8), guard let data = jsonSlice.data(using: .utf8),
@@ -218,14 +218,14 @@ private final class AdaptiveDownsampler: @unchecked Sendable {
// duplicate the audio ~6× (stuttering ASR input). After the // duplicate the audio ~6× (stuttering ASR input). After the
// single feed we report "ran dry", so the expected status is // single feed we report "ran dry", so the expected status is
// `.inputRanDry` (output not full), not `.haveData`. // `.inputRanDry` (output not full), not `.haveData`.
var provided = false let provided = OSAllocatedUnfairLock(initialState: false)
var error: NSError? var error: NSError?
let status = current.converter.convert(to: current.scratch, error: &error) { _, outStatus in let status = current.converter.convert(to: current.scratch, error: &error) { _, outStatus in
if provided { if provided.withLock({ $0 }) {
outStatus.pointee = .noDataNow outStatus.pointee = .noDataNow
return nil return nil
} }
provided = true provided.withLock { $0 = true }
outStatus.pointee = .haveData outStatus.pointee = .haveData
return buffer return buffer
} }
+21
View File
@@ -383,6 +383,27 @@ public enum Keychain: @unchecked Sendable {
private static let onboardingService = "com.osgkeyboard.onboarding" private static let onboardingService = "com.osgkeyboard.onboarding"
private static let onboardingAccount = "hasCompletedOnboarding" private static let onboardingAccount = "hasCompletedOnboarding"
/// Survives reboots but is wiped with the app container (unlike Keychain).
private static let installIdentityKey = "osgkeyboard.installIdentity"
/// Call once at config init. Returns `true` when this is a brand-new app
/// container (first launch or reinstall after delete). Clears a stale
/// Keychain onboarding flag so deleted installs show the welcome flow again.
@discardableResult
public static func beginInstallIdentityIfNeeded() -> Bool {
let standard = UserDefaults.standard
if standard.string(forKey: installIdentityKey) != nil {
return false
}
standard.set(UUID().uuidString, forKey: installIdentityKey)
if hasCompletedOnboarding() {
setOnboardingCompleted(false)
OSGLog.config.info("[onboarding] fresh install: cleared stale Keychain onboarding flag")
} else {
OSGLog.config.info("[onboarding] fresh install: install identity created")
}
return true
}
public static func hasCompletedOnboarding() -> Bool { public static func hasCompletedOnboarding() -> Bool {
var query: [String: Any] = [ var query: [String: Any] = [
@@ -9,9 +9,7 @@ import os
public enum FlowPipelineDiagnostics { public enum FlowPipelineDiagnostics {
public static func logDrain(_ report: FlowCaptureDrainReport) { public static func logDrain(_ report: FlowCaptureDrainReport) {
OSGLog.flow.info( OSGLog.flow.info(
"tailDrain duration=\(report.drainDurationSeconds, format: .fixed(precision: 2))s " + "tailDrain duration=\(report.drainDurationSeconds, format: .fixed(precision: 2))s postRoll=\(report.postRollDurationSeconds, format: .fixed(precision: 2))s silenceEnd=\(report.endedBySilence) tailSamples=\(report.tailSampleCount)"
"postRoll=\(report.postRollDurationSeconds, format: .fixed(precision: 2))s " +
"silenceEnd=\(report.endedBySilence) tailSamples=\(report.tailSampleCount)"
) )
} }
@@ -32,8 +30,7 @@ public enum FlowPipelineDiagnostics {
batchLength: Int batchLength: Int
) { ) {
OSGLog.flow.info( OSGLog.flow.info(
"batchFallback samples=\(sampleCount) stitchedLen=\(stitchedLength) " + "batchFallback samples=\(sampleCount) stitchedLen=\(stitchedLength) partialLen=\(partialLength) batchLen=\(batchLength)"
"partialLen=\(partialLength) batchLen=\(batchLength)"
) )
} }
@@ -5,6 +5,7 @@
// batch ASR fallback when pipelined chunking drops weak tail segments. // batch ASR fallback when pipelined chunking drops weak tail segments.
import Foundation import Foundation
import os
public final class FlowUtterancePCMStore: @unchecked Sendable { public final class FlowUtterancePCMStore: @unchecked Sendable {
private let lock = OSAllocatedUnfairLock() private let lock = OSAllocatedUnfairLock()
+6 -2
View File
@@ -136,7 +136,7 @@
"mac.section.dictionary" = "Dictionary"; "mac.section.dictionary" = "Dictionary";
"mac.section.styles" = "Polish Styles"; "mac.section.styles" = "Polish Styles";
"mac.styles.subtitle" = "Choose or create a complete writing personality for polished dictation."; "mac.styles.subtitle" = "Choose or create a complete writing personality for polished dictation.";
"mac.styles.add" = "Add Style"; "mac.styles.add" = "Add Polish Style";
"mac.styles.edit" = "Edit Style"; "mac.styles.edit" = "Edit Style";
"mac.styles.builtin" = "Built-in"; "mac.styles.builtin" = "Built-in";
"mac.styles.custom" = "My Styles"; "mac.styles.custom" = "My Styles";
@@ -206,9 +206,13 @@
"mac.history.clearConfirm" = "Clear all"; "mac.history.clearConfirm" = "Clear all";
"mac.dict.health" = "Vocabulary Health"; "mac.dict.health" = "Vocabulary Health";
"mac.dict.healthDesc" = "Custom terms that bias recognition and are never rewritten."; "mac.dict.healthDesc" = "Custom terms that bias recognition and are never rewritten.";
"mac.dict.add" = "Add Personal Word";
"mac.dict.addField" = "Word or phrase";
"mac.dict.addFooter" = "Common recognition mistakes will be generated automatically after saving.";
"mac.dict.aliasesGenerating" = "Generating aliases…";
"mac.dict.search" = "Search words"; "mac.dict.search" = "Search words";
"mac.dict.empty" = "No words yet"; "mac.dict.empty" = "No words yet";
"mac.dict.emptyBody" = "Add words on your iPhone or iPad to improve recognition accuracy. They sync here via iCloud."; "mac.dict.emptyBody" = "Add personal words to improve recognition accuracy. They sync across your devices via iCloud.";
"mac.dict.noMatch" = "No matches"; "mac.dict.noMatch" = "No matches";
"mac.cancel" = "Cancel"; "mac.cancel" = "Cancel";
"mac.save" = "Save"; "mac.save" = "Save";
@@ -136,7 +136,7 @@
"mac.section.dictionary" = "词库"; "mac.section.dictionary" = "词库";
"mac.section.styles" = "润色风格"; "mac.section.styles" = "润色风格";
"mac.styles.subtitle" = "为听写润色选择或创建完整写作人格。"; "mac.styles.subtitle" = "为听写润色选择或创建完整写作人格。";
"mac.styles.add" = "添加风格"; "mac.styles.add" = "添加润色风格";
"mac.styles.edit" = "编辑风格"; "mac.styles.edit" = "编辑风格";
"mac.styles.builtin" = "内置风格"; "mac.styles.builtin" = "内置风格";
"mac.styles.custom" = "我的风格"; "mac.styles.custom" = "我的风格";
@@ -206,9 +206,13 @@
"mac.history.clearConfirm" = "全部清空"; "mac.history.clearConfirm" = "全部清空";
"mac.dict.health" = "词库健康度"; "mac.dict.health" = "词库健康度";
"mac.dict.healthDesc" = "影响识别偏置且润色时不会被改写的自定义词条。"; "mac.dict.healthDesc" = "影响识别偏置且润色时不会被改写的自定义词条。";
"mac.dict.add" = "添加个性词";
"mac.dict.addField" = "词语或短语";
"mac.dict.addFooter" = "保存后将自动生成常见的语音误识别别名。";
"mac.dict.aliasesGenerating" = "正在生成别名…";
"mac.dict.search" = "搜索词条"; "mac.dict.search" = "搜索词条";
"mac.dict.empty" = "还没有词条"; "mac.dict.empty" = "还没有词条";
"mac.dict.emptyBody" = "在 iPhone 或 iPad 上添加词条以提升识别准确性,它们会通过 iCloud 同步到这里。"; "mac.dict.emptyBody" = "添加个性词以提升识别准确性,它们会通过 iCloud 在设备间同步。";
"mac.dict.noMatch" = "无匹配结果"; "mac.dict.noMatch" = "无匹配结果";
"mac.cancel" = "取消"; "mac.cancel" = "取消";
"mac.save" = "保存"; "mac.save" = "保存";
@@ -33,10 +33,22 @@ final class AppGroupConfigurationTests: XCTestCase {
XCTAssertEqual(config.polishIntensity, .default) XCTAssertEqual(config.polishIntensity, .default)
XCTAssertTrue(config.personalDictionary.entries.isEmpty) XCTAssertTrue(config.personalDictionary.entries.isEmpty)
XCTAssertTrue(config.flowSkipAppSwitch) XCTAssertTrue(config.flowSkipAppSwitch)
XCTAssertEqual(config.flowKeepAliveMode, .liveActivity) XCTAssertEqual(config.flowKeepAliveMode, .pictureInPicture)
XCTAssertEqual(config.flowInactivityDuration, .fiveMinutes) XCTAssertEqual(config.flowInactivityDuration, .fiveMinutes)
} }
func testLoadPreservesStoredLiveActivityKeepAliveMode() {
let defaults = makeDefaults()
defaults.set(
FlowKeepAliveMode.liveActivity.rawValue,
forKey: AppGroupConfiguration.Keys.flowKeepAliveMode
)
let config = AppGroupConfiguration.load(fromAvailable: defaults)
XCTAssertEqual(config.flowKeepAliveMode, .liveActivity)
}
func testSaveAndLoadRoundTrip() { func testSaveAndLoadRoundTrip() {
let defaults = makeDefaults() let defaults = makeDefaults()
var config = AppGroupConfiguration.load(fromAvailable: defaults) var config = AppGroupConfiguration.load(fromAvailable: defaults)
@@ -29,10 +29,10 @@ final class FlowSessionPolicyTests: XCTestCase {
XCTAssertEqual(FlowInactivityDuration.tenMinutes.timeInterval, 10 * 60) XCTAssertEqual(FlowInactivityDuration.tenMinutes.timeInterval, 10 * 60)
} }
func testKeepAliveModeDefaultsToLiveActivity() { func testKeepAliveModeDefaultsToPictureInPicture() {
let defaults = makeDefaults() let defaults = makeDefaults()
XCTAssertEqual(FlowSessionPolicy.keepAliveMode(defaults: defaults), .liveActivity) XCTAssertEqual(FlowSessionPolicy.keepAliveMode(defaults: defaults), .pictureInPicture)
XCTAssertTrue(FlowSessionPolicy.usesInactivityExpiry(defaults: defaults)) XCTAssertFalse(FlowSessionPolicy.usesInactivityExpiry(defaults: defaults))
} }
func testPiPSessionHasNoInactivityExpiry() { func testPiPSessionHasNoInactivityExpiry() {
@@ -50,6 +50,10 @@ final class FlowSessionPolicyTests: XCTestCase {
func testTouchLastActivityExtendsExpiry() { func testTouchLastActivityExtendsExpiry() {
let defaults = makeDefaults() let defaults = makeDefaults()
defaults.set(
FlowKeepAliveMode.liveActivity.rawValue,
forKey: AppGroupConfiguration.Keys.flowKeepAliveMode
)
defaults.set(FlowInactivityDuration.tenMinutes.rawValue, forKey: AppGroupConfiguration.Keys.flowInactivityDuration) defaults.set(FlowInactivityDuration.tenMinutes.rawValue, forKey: AppGroupConfiguration.Keys.flowInactivityDuration)
FlowSessionBridge.markSessionActive(defaults: defaults) FlowSessionBridge.markSessionActive(defaults: defaults)
@@ -30,6 +30,20 @@ final class PolishStylePackTests: XCTestCase {
} }
} }
func testDatingStyleDefinesRelationshipAwareIntensityAndSafety() throws {
let style = try XCTUnwrap(
PolishStylePackCatalog.builtins.first { $0.id == "builtin.dating" }
)
XCTAssertTrue(style.prompt.contains("# 本风格的力度解释"))
XCTAssertTrue(style.prompt.contains("# 关系许可闸"))
XCTAssertTrue(style.prompt.contains("Light(暖而不撩)"))
XCTAssertTrue(style.prompt.contains("Medium(温度与趣味)"))
XCTAssertTrue(style.prompt.contains("Heavy(主动而明确)"))
XCTAssertTrue(style.prompt.contains("不把冷淡解释成欲擒故纵"))
XCTAssertTrue(style.prompt.contains("暧昧不能代替明确同意"))
}
func testCatalogRejectsNinthUserPack() throws { func testCatalogRejectsNinthUserPack() throws {
var catalog = PolishStyleCatalog() var catalog = PolishStyleCatalog()
for index in 0..<PolishStyleLimits.maximumUserPacks { for index in 0..<PolishStyleLimits.maximumUserPacks {
@@ -123,6 +137,20 @@ final class PolishStylePackTests: XCTestCase {
XCTAssertTrue(guideline.contains("active style pack")) XCTAssertTrue(guideline.contains("active style pack"))
} }
func testDatingStyleUsesRelationshipSpecificIntensityGuidelines() {
let light = PolishIntensity.light.promptGuideline(styleID: "builtin.dating")
let medium = PolishIntensity.medium.promptGuideline(styleID: "builtin.dating")
let heavy = PolishIntensity.heavy.promptGuideline(styleID: "builtin.dating")
XCTAssertTrue(light.contains("Dating Light"))
XCTAssertTrue(light.contains("without adding flirtation"))
XCTAssertTrue(medium.contains("Dating Medium"))
XCTAssertTrue(medium.contains("at most one"))
XCTAssertTrue(heavy.contains("Dating Heavy"))
XCTAssertTrue(heavy.contains("Increase romantic tension and directness"))
XCTAssertTrue(heavy.contains("Style override"))
}
func testHeavyIntensityStillAllowsStructuredStyle() { func testHeavyIntensityStillAllowsStructuredStyle() {
let guideline = PolishIntensity.heavy.promptGuideline(styleID: "builtin.structured") let guideline = PolishIntensity.heavy.promptGuideline(styleID: "builtin.structured")
+2 -2
View File
@@ -48,8 +48,8 @@ settings:
ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS: YES
STRING_CATALOG_GENERATE_SYMBOLS: YES STRING_CATALOG_GENERATE_SYMBOLS: YES
CLANG_CXX_LANGUAGE_STANDARD: c++17 CLANG_CXX_LANGUAGE_STANDARD: c++17
MARKETING_VERSION: "1.0" MARKETING_VERSION: "1.0.1"
CURRENT_PROJECT_VERSION: "26" CURRENT_PROJECT_VERSION: "27"
# 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖) # 签名配置来自 Signing.local.xcconfiggitignored,不会被覆盖)
# 项目级签名 xcconfig,适用于所有 target # 项目级签名 xcconfig,适用于所有 target