fix(mac,asr): harden menu-bar delivery, chunk retry, and polish validator
Retain the external target app for menu-bar paste and polish context, retry failed middle ASR chunks once, and stop false-positive path/number violations.
This commit is contained in:
@@ -18,6 +18,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- **Mac Settings translation target**: polish-provider section includes “Polish then translate” with the same locale picker as Home / menu bar. / **Mac 设置翻译目标**:润色(LLM)分区新增「润色后翻译」,与首页 / 菜单栏同一套目标语言选择。
|
- **Mac Settings translation target**: polish-provider section includes “Polish then translate” with the same locale picker as Home / menu bar. / **Mac 设置翻译目标**:润色(LLM)分区新增「润色后翻译」,与首页 / 菜单栏同一套目标语言选择。
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
- **macOS menu-bar dictation delivery**: menu-bar sessions now retain the external target app before the popover takes focus, use that app for polish context and local bias, and reactivate it before pasting. / **macOS 菜单栏听写投递**:菜单栏会话会在弹窗抢焦点前保留外部目标应用,并将其用于润色上下文与本地偏置,粘贴前重新激活目标应用。
|
||||||
|
- **macOS recording and clipboard fallback**: cancelling microphone preparation no longer lets an untracked button task restart recording; clipboard restoration waits longer, preserves newer third-party writes, clears an originally empty clipboard correctly, and Accessibility failures explain that the transcript remains available for manual paste. / **macOS 录音与剪贴板降级**:取消麦克风准备后,未跟踪的按钮任务不会再次启动录音;剪贴板恢复延长等待时间、保留第三方较新的写入、正确还原原本为空的剪贴板,并在辅助功能权限失败时明确提示可手动粘贴识别结果。
|
||||||
|
- **Polish validator false positives**: slash-form dates, fractions, and words such as `and/or` are no longer treated as hard-protected file paths; confirmed ordinal ASR repairs also stop polluting missing-number telemetry. / **润色校验误报**:斜杠日期、分数及 `and/or` 等词不再被误判为文件路径硬违规;已确认的序号 ASR 修复也不再污染数字缺失遥测。
|
||||||
|
- **Transient chunk loss**: a failed middle ASR chunk now retries once with the same PCM before the serial worker advances, preventing brief network failures from silently removing several seconds of speech. / **瞬时分块丢字**:中段 ASR 分块失败后会使用同一份 PCM 原地重试一次,再继续串行处理,避免短暂网络抖动静默丢失数秒语音。
|
||||||
- **macOS Option release freeze**: finishing a live snapshot stream no longer calls `AsyncStream.Continuation.finish()` while holding the recorder lock — the termination handler re-entered the same `NSLock` on the main thread and wedged the app when the hold-to-talk key was released. / **macOS 松开 Option 卡死**:结束实时 snapshot 流时不再在持有 recorder 锁的情况下调用 `AsyncStream.Continuation.finish()`;终止回调会在主线程重入同一把 `NSLock`,松开听写键时导致整个 App 无响应。
|
- **macOS Option release freeze**: finishing a live snapshot stream no longer calls `AsyncStream.Continuation.finish()` while holding the recorder lock — the termination handler re-entered the same `NSLock` on the main thread and wedged the app when the hold-to-talk key was released. / **macOS 松开 Option 卡死**:结束实时 snapshot 流时不再在持有 recorder 锁的情况下调用 `AsyncStream.Continuation.finish()`;终止回调会在主线程重入同一把 `NSLock`,松开听写键时导致整个 App 无响应。
|
||||||
- **macOS dictation HUD layout storm**: the floating pill no longer reassigns its hosting view and forces a synchronous relayout on every view-model tick (~20×/s from the level timer); it uses a fixed panel size and lets SwiftUI refresh through `@ObservedObject` instead. / **macOS 听写浮层布局风暴**:悬浮胶囊不再在每次 view-model 更新时重建 hosting 视图并强制同步重排(音量定时器约 20 次/秒);改为固定面板尺寸,由 SwiftUI `@ObservedObject` 驱动刷新。
|
- **macOS dictation HUD layout storm**: the floating pill no longer reassigns its hosting view and forces a synchronous relayout on every view-model tick (~20×/s from the level timer); it uses a fixed panel size and lets SwiftUI refresh through `@ObservedObject` instead. / **macOS 听写浮层布局风暴**:悬浮胶囊不再在每次 view-model 更新时重建 hosting 视图并强制同步重排(音量定时器约 20 次/秒);改为固定面板尺寸,由 SwiftUI `@ObservedObject` 驱动刷新。
|
||||||
- **Polish never answers the transcript**: fun styles (dating / flex / corp) could turn “你觉得这个包怎么样” into a reply such as “还行,挺顺眼的”. A top-priority “polish only, never answer” rule now sits in the global contract, every built-in style pack, the prompt safety boundary, and a router question guard that keeps question drafts as questions. / **润色不再代答转写内容**:趣味风格(直男癌/装逼指南/大厂黑话)曾把「你觉得这个包怎么样」润色成「还行,挺顺眼的」。现已在全局契约、全部内置风格包、提示词安全边界与路由问句守卫四层加入最高优先级的「只润色、不作答」规则,问句必须仍是问句。
|
- **Polish never answers the transcript**: fun styles (dating / flex / corp) could turn “你觉得这个包怎么样” into a reply such as “还行,挺顺眼的”. A top-priority “polish only, never answer” rule now sits in the global contract, every built-in style pack, the prompt safety boundary, and a router question guard that keeps question drafts as questions. / **润色不再代答转写内容**:趣味风格(直男癌/装逼指南/大厂黑话)曾把「你觉得这个包怎么样」润色成「还行,挺顺眼的」。现已在全局契约、全部内置风格包、提示词安全边界与路由问句守卫四层加入最高优先级的「只润色、不作答」规则,问句必须仍是问句。
|
||||||
|
|||||||
@@ -72,7 +72,7 @@ final class ChunkedUtterancePipelineTests: XCTestCase {
|
|||||||
XCTAssertFalse(partials.isEmpty)
|
XCTAssertFalse(partials.isEmpty)
|
||||||
}
|
}
|
||||||
|
|
||||||
func testPipelineDeliversPartialSuccessWhenOneChunkFails() async {
|
func testPipelineRetriesTransientMiddleChunkFailure() async {
|
||||||
let pipeline = ChunkedUtterancePipeline(
|
let pipeline = ChunkedUtterancePipeline(
|
||||||
asr: FailingSecondChunkASR(),
|
asr: FailingSecondChunkASR(),
|
||||||
locale: Locale(identifier: "zh-Hans"),
|
locale: Locale(identifier: "zh-Hans"),
|
||||||
@@ -86,6 +86,30 @@ final class ChunkedUtterancePipelineTests: XCTestCase {
|
|||||||
|
|
||||||
let outcome = await pipeline.transcribe(stream: stream) { _ in }
|
let outcome = await pipeline.transcribe(stream: stream) { _ in }
|
||||||
|
|
||||||
|
guard case .success(let success) = outcome else {
|
||||||
|
return XCTFail("expected partial success, got \(outcome)")
|
||||||
|
}
|
||||||
|
XCTAssertTrue(success.text.contains("recovered-middle"), "got \(success.text)")
|
||||||
|
XCTAssertTrue(success.chunkWarnings.isEmpty)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testPipelineWarnsAfterMiddleChunkRetryAlsoFails() async {
|
||||||
|
let pipeline = ChunkedUtterancePipeline(
|
||||||
|
asr: PermanentlyFailingMiddleChunkASR(),
|
||||||
|
locale: Locale(identifier: "zh-Hans"),
|
||||||
|
config: config(overlapSeconds: 0)
|
||||||
|
)
|
||||||
|
|
||||||
|
let (stream, continuation) = AsyncStream<AudioBufferSnapshot>.makeStream()
|
||||||
|
continuation.yield(
|
||||||
|
AudioBufferSnapshot(
|
||||||
|
samples: [Float](repeating: 0.1, count: 160),
|
||||||
|
sampleRate: 1_000
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continuation.finish()
|
||||||
|
|
||||||
|
let outcome = await pipeline.transcribe(stream: stream) { _ in }
|
||||||
guard case .success(let success) = outcome else {
|
guard case .success(let success) = outcome else {
|
||||||
return XCTFail("expected partial success, got \(outcome)")
|
return XCTFail("expected partial success, got \(outcome)")
|
||||||
}
|
}
|
||||||
@@ -224,6 +248,35 @@ private struct FailingSecondChunkASR: ASRService, @unchecked Sendable {
|
|||||||
if current == 1 {
|
if current == 1 {
|
||||||
return .failure("simulated chunk error")
|
return .failure("simulated chunk error")
|
||||||
}
|
}
|
||||||
|
if current == 2 {
|
||||||
|
return .success("recovered-middle")
|
||||||
|
}
|
||||||
|
return .success("seg\(samples.count)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private struct PermanentlyFailingMiddleChunkASR: ASRService, @unchecked Sendable {
|
||||||
|
private let callIndex = OSAllocatedUnfairLock(initialState: 0)
|
||||||
|
|
||||||
|
func transcribe(
|
||||||
|
stream: AsyncStream<AudioBufferSnapshot>,
|
||||||
|
locale: Locale
|
||||||
|
) -> AsyncStream<ASREvent> {
|
||||||
|
AsyncStream { $0.finish() }
|
||||||
|
}
|
||||||
|
|
||||||
|
func cancel() {}
|
||||||
|
|
||||||
|
func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult {
|
||||||
|
_ = locale
|
||||||
|
let current = callIndex.withLock { state in
|
||||||
|
let value = state
|
||||||
|
state += 1
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
if current == 1 || current == 2 {
|
||||||
|
return .failure("persistent simulated chunk error")
|
||||||
|
}
|
||||||
return .success("seg\(samples.count)")
|
return .success("seg\(samples.count)")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -68,7 +68,14 @@ enum MacAppContextService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
static func detectContext() -> AppContext {
|
static func detectContext() -> AppContext {
|
||||||
guard let bundleId = frontmostBundleIdentifier() else { return .unknown }
|
detectContext(bundleIdentifier: frontmostBundleIdentifier())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolve polish context from the application captured for this dictation
|
||||||
|
/// session. This avoids reading OSGKeyboard itself after a popover steals
|
||||||
|
/// focus.
|
||||||
|
static func detectContext(bundleIdentifier bundleId: String?) -> AppContext {
|
||||||
|
guard let bundleId else { return .unknown }
|
||||||
if let mapped = contextByBundleId[bundleId] { return mapped }
|
if let mapped = contextByBundleId[bundleId] { return mapped }
|
||||||
if chatBundleIdsFromRegistry.contains(bundleId) { return .chat }
|
if chatBundleIdsFromRegistry.contains(bundleId) { return .chat }
|
||||||
if bundleId.hasPrefix("com.apple.Safari") || bundleId.contains("chrome") {
|
if bundleId.hasPrefix("com.apple.Safari") || bundleId.contains("chrome") {
|
||||||
@@ -83,4 +90,12 @@ enum MacAppContextService {
|
|||||||
let context = detectContext()
|
let context = detectContext()
|
||||||
store.setDetectedAppContext(context)
|
store.setDetectedAppContext(context)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static func captureAndPersist(
|
||||||
|
application: NSRunningApplication?,
|
||||||
|
to store: AppGroupStore
|
||||||
|
) {
|
||||||
|
let context = detectContext(bundleIdentifier: application?.bundleIdentifier)
|
||||||
|
store.setDetectedAppContext(context)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,7 +8,14 @@
|
|||||||
|
|
||||||
@preconcurrency import AVFoundation
|
@preconcurrency import AVFoundation
|
||||||
|
|
||||||
final class MacAudioRecorder: @unchecked Sendable {
|
protocol MacAudioRecording: Sendable {
|
||||||
|
func level() -> Float
|
||||||
|
func start() async throws
|
||||||
|
func makeSnapshotStream() -> AsyncStream<AudioBufferSnapshot>
|
||||||
|
func stop() -> [Float]
|
||||||
|
}
|
||||||
|
|
||||||
|
final class MacAudioRecorder: MacAudioRecording, @unchecked Sendable {
|
||||||
enum RecorderError: Error, LocalizedError {
|
enum RecorderError: Error, LocalizedError {
|
||||||
case converterUnavailable
|
case converterUnavailable
|
||||||
case microphoneAccessDenied
|
case microphoneAccessDenied
|
||||||
|
|||||||
@@ -60,6 +60,7 @@ enum MacDictationPipeline {
|
|||||||
static func run(
|
static func run(
|
||||||
samples: [Float],
|
samples: [Float],
|
||||||
store: AppGroupStore,
|
store: AppGroupStore,
|
||||||
|
targetAppBundleIdentifier: String? = nil,
|
||||||
onPartial: (@Sendable (String) -> Void)? = nil
|
onPartial: (@Sendable (String) -> Void)? = nil
|
||||||
) async throws -> MacDictationResult {
|
) async throws -> MacDictationResult {
|
||||||
guard !samples.isEmpty else { throw MacDictationError.noAudio }
|
guard !samples.isEmpty else { throw MacDictationError.noAudio }
|
||||||
@@ -69,7 +70,11 @@ enum MacDictationPipeline {
|
|||||||
var localBias: LocalASRBiasPayload?
|
var localBias: LocalASRBiasPayload?
|
||||||
|
|
||||||
if store.engineMode == "local" {
|
if store.engineMode == "local" {
|
||||||
localBias = resolveLocalBias(store: store, locale: locale)
|
localBias = resolveLocalBias(
|
||||||
|
store: store,
|
||||||
|
locale: locale,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier
|
||||||
|
)
|
||||||
raw = try await MacLocalASRService.transcribe(
|
raw = try await MacLocalASRService.transcribe(
|
||||||
samples: samples,
|
samples: samples,
|
||||||
locale: locale,
|
locale: locale,
|
||||||
@@ -103,6 +108,7 @@ enum MacDictationPipeline {
|
|||||||
stream: AsyncStream<AudioBufferSnapshot>,
|
stream: AsyncStream<AudioBufferSnapshot>,
|
||||||
finishSignal: AsyncStream<Void>,
|
finishSignal: AsyncStream<Void>,
|
||||||
store: AppGroupStore,
|
store: AppGroupStore,
|
||||||
|
targetAppBundleIdentifier: String?,
|
||||||
onPartial: @escaping @Sendable (String) -> Void
|
onPartial: @escaping @Sendable (String) -> Void
|
||||||
) async -> MacLiveASRCaptureResult {
|
) async -> MacLiveASRCaptureResult {
|
||||||
if store.engineMode == "local", MacLocalASRService.usesMLXLiveStreaming() {
|
if store.engineMode == "local", MacLocalASRService.usesMLXLiveStreaming() {
|
||||||
@@ -110,6 +116,7 @@ enum MacDictationPipeline {
|
|||||||
audioStream: stream,
|
audioStream: stream,
|
||||||
finishSignal: finishSignal,
|
finishSignal: finishSignal,
|
||||||
store: store,
|
store: store,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier,
|
||||||
onPartial: onPartial
|
onPartial: onPartial
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -117,7 +124,11 @@ enum MacDictationPipeline {
|
|||||||
let locale = resolvedLocale(store: store)
|
let locale = resolvedLocale(store: store)
|
||||||
let localBias: LocalASRBiasPayload?
|
let localBias: LocalASRBiasPayload?
|
||||||
if store.engineMode == "local" {
|
if store.engineMode == "local" {
|
||||||
localBias = resolveLocalBias(store: store, locale: locale)
|
localBias = resolveLocalBias(
|
||||||
|
store: store,
|
||||||
|
locale: locale,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier
|
||||||
|
)
|
||||||
} else {
|
} else {
|
||||||
localBias = nil
|
localBias = nil
|
||||||
}
|
}
|
||||||
@@ -280,15 +291,15 @@ enum MacDictationPipeline {
|
|||||||
|
|
||||||
private static func resolveLocalBias(
|
private static func resolveLocalBias(
|
||||||
store: AppGroupStore,
|
store: AppGroupStore,
|
||||||
locale: Locale
|
locale: Locale,
|
||||||
|
targetAppBundleIdentifier: String?
|
||||||
) -> LocalASRBiasPayload? {
|
) -> LocalASRBiasPayload? {
|
||||||
MacAppContextService.captureAndPersist(to: store)
|
|
||||||
let capabilities = MacLocalASRService.currentCapabilities()
|
let capabilities = MacLocalASRService.currentCapabilities()
|
||||||
let bias = LocalASRBiasAdapter.adapt(
|
let bias = LocalASRBiasAdapter.adapt(
|
||||||
LocalASRBiasRequest(
|
LocalASRBiasRequest(
|
||||||
dictionary: store.personalDictionary,
|
dictionary: store.personalDictionary,
|
||||||
locale: locale,
|
locale: locale,
|
||||||
frontAppBundleId: MacAppContextService.frontmostBundleIdentifier(),
|
frontAppBundleId: targetAppBundleIdentifier,
|
||||||
capabilities: capabilities
|
capabilities: capabilities
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -75,14 +75,21 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
@Published var config: ProviderConfig
|
@Published var config: ProviderConfig
|
||||||
|
|
||||||
let defaults: UserDefaults
|
let defaults: UserDefaults
|
||||||
private let recorder = MacAudioRecorder()
|
private let recorder: any MacAudioRecording
|
||||||
private let hotkeyService = MacHotkeyService()
|
private let hotkeyService: MacHotkeyService
|
||||||
private var levelTimer: Timer?
|
private var levelTimer: Timer?
|
||||||
private var sessionTimer: Timer?
|
private var sessionTimer: Timer?
|
||||||
private var cancellables = Set<AnyCancellable>()
|
private var cancellables = Set<AnyCancellable>()
|
||||||
/// In-flight `beginRecording` started by the hotkey — cancelled if the
|
/// In-flight `beginRecording` started by the hotkey — cancelled if the
|
||||||
/// key is released before the engine is ready (avoids a stuck session).
|
/// key is released before the engine is ready (avoids a stuck session).
|
||||||
private var hotkeyBeginTask: Task<Void, Never>?
|
private var hotkeyBeginTask: Task<Void, Never>?
|
||||||
|
/// Button-triggered preparation needs the same cancellation semantics as
|
||||||
|
/// the hotkey path when the user clicks Stop before the engine is ready.
|
||||||
|
private var buttonBeginTask: Task<Void, Never>?
|
||||||
|
/// Captured before the menu-bar popover activates OSGKeyboard.
|
||||||
|
private var preparedPopoverTargetApplication: NSRunningApplication?
|
||||||
|
/// Frozen for one take so app switches during ASR cannot redirect delivery.
|
||||||
|
private var sessionTargetApplication: NSRunningApplication?
|
||||||
/// Live chunked / streaming ASR while recording (cloud or MLX local).
|
/// Live chunked / streaming ASR while recording (cloud or MLX local).
|
||||||
/// Finished in `finishRecording` so partials can become the final draft.
|
/// Finished in `finishRecording` so partials can become the final draft.
|
||||||
private var liveCaptureTask: Task<MacLiveASRCaptureResult, Never>?
|
private var liveCaptureTask: Task<MacLiveASRCaptureResult, Never>?
|
||||||
@@ -97,8 +104,15 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
static let hotkeyTrigger = MacHotkeyTrigger.storageKey
|
static let hotkeyTrigger = MacHotkeyTrigger.storageKey
|
||||||
}
|
}
|
||||||
|
|
||||||
init(defaults: UserDefaults = .standard) {
|
init(
|
||||||
|
defaults: UserDefaults = .standard,
|
||||||
|
recorder: any MacAudioRecording = MacAudioRecorder(),
|
||||||
|
hotkeyService: MacHotkeyService = MacHotkeyService(),
|
||||||
|
startHotkeyService: Bool = true
|
||||||
|
) {
|
||||||
self.defaults = defaults
|
self.defaults = defaults
|
||||||
|
self.recorder = recorder
|
||||||
|
self.hotkeyService = hotkeyService
|
||||||
self.config = ProviderConfig(defaults: defaults)
|
self.config = ProviderConfig(defaults: defaults)
|
||||||
self.usageStatistics = UsageStatisticsStore(defaults: defaults)
|
self.usageStatistics = UsageStatisticsStore(defaults: defaults)
|
||||||
self.autoPasteEnabled = defaults.object(forKey: StoredKeys.autoPaste) as? Bool ?? true
|
self.autoPasteEnabled = defaults.object(forKey: StoredKeys.autoPaste) as? Bool ?? true
|
||||||
@@ -111,7 +125,9 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
|
|
||||||
MacICloudSyncBootstrap.configure(defaults: defaults)
|
MacICloudSyncBootstrap.configure(defaults: defaults)
|
||||||
statusMessage = MacL10n.string("mac.status.ready", language: config.uiLanguage)
|
statusMessage = MacL10n.string("mac.status.ready", language: config.uiLanguage)
|
||||||
|
if startHotkeyService {
|
||||||
wireHotkeyService()
|
wireHotkeyService()
|
||||||
|
}
|
||||||
forwardNestedObjectChanges()
|
forwardNestedObjectChanges()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -132,6 +148,17 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
refreshForegroundAppName()
|
refreshForegroundAppName()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Called immediately before the menu-bar popover activates the app.
|
||||||
|
func prepareForPopoverPresentation() {
|
||||||
|
let target = MacTextInsertionService.captureTargetApplication()
|
||||||
|
preparedPopoverTargetApplication = target
|
||||||
|
foregroundAppName = target?.localizedName
|
||||||
|
}
|
||||||
|
|
||||||
|
func clearPreparedPopoverTarget() {
|
||||||
|
preparedPopoverTargetApplication = nil
|
||||||
|
}
|
||||||
|
|
||||||
func reloadConfigFromCloud() {
|
func reloadConfigFromCloud() {
|
||||||
config.reloadFromPersistedStorage()
|
config.reloadFromPersistedStorage()
|
||||||
statusMessage = MacL10n.string("mac.status.ready", language: config.uiLanguage)
|
statusMessage = MacL10n.string("mac.status.ready", language: config.uiLanguage)
|
||||||
@@ -256,7 +283,10 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
if isRecording || isPreparingToRecord {
|
if isRecording || isPreparingToRecord {
|
||||||
cancelOrFinishRecording()
|
cancelOrFinishRecording()
|
||||||
} else {
|
} else {
|
||||||
Task { await beginRecording() }
|
buttonBeginTask?.cancel()
|
||||||
|
buttonBeginTask = Task { [weak self] in
|
||||||
|
await self?.beginRecording()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -264,8 +294,12 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
guard !isProcessing, !isRecording, !isPreparingToRecord else { return }
|
guard !isProcessing, !isRecording, !isPreparingToRecord else { return }
|
||||||
isPreparingToRecord = true
|
isPreparingToRecord = true
|
||||||
let store = AppGroupStore(defaults: defaults)
|
let store = AppGroupStore(defaults: defaults)
|
||||||
MacAppContextService.captureAndPersist(to: store)
|
let targetApplication = preparedPopoverTargetApplication
|
||||||
refreshForegroundAppName()
|
?? MacTextInsertionService.captureTargetApplication()
|
||||||
|
preparedPopoverTargetApplication = nil
|
||||||
|
sessionTargetApplication = targetApplication
|
||||||
|
MacAppContextService.captureAndPersist(application: targetApplication, to: store)
|
||||||
|
foregroundAppName = targetApplication?.localizedName
|
||||||
|
|
||||||
do {
|
do {
|
||||||
try await recorder.start()
|
try await recorder.start()
|
||||||
@@ -274,14 +308,20 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
isPreparingToRecord = false
|
isPreparingToRecord = false
|
||||||
if Task.isCancelled {
|
if Task.isCancelled {
|
||||||
_ = recorder.stop()
|
_ = recorder.stop()
|
||||||
|
sessionTargetApplication = nil
|
||||||
|
buttonBeginTask = nil
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
buttonBeginTask = nil
|
||||||
isRecording = true
|
isRecording = true
|
||||||
transcript = ""
|
transcript = ""
|
||||||
isStreamingPartial = false
|
isStreamingPartial = false
|
||||||
statusMessage = MacL10n.string("mac.status.listening", language: config.uiLanguage)
|
statusMessage = MacL10n.string("mac.status.listening", language: config.uiLanguage)
|
||||||
startTimers()
|
startTimers()
|
||||||
startLiveCaptureIfSupported(store: store)
|
startLiveCaptureIfSupported(
|
||||||
|
store: store,
|
||||||
|
targetAppBundleIdentifier: targetApplication?.bundleIdentifier
|
||||||
|
)
|
||||||
// Tiny race: Option released between the cancel check and
|
// Tiny race: Option released between the cancel check and
|
||||||
// `isRecording = true`. Treat it as end-of-hold and finish.
|
// `isRecording = true`. Treat it as end-of-hold and finish.
|
||||||
if Task.isCancelled {
|
if Task.isCancelled {
|
||||||
@@ -289,6 +329,8 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
}
|
}
|
||||||
} catch {
|
} catch {
|
||||||
isPreparingToRecord = false
|
isPreparingToRecord = false
|
||||||
|
sessionTargetApplication = nil
|
||||||
|
buttonBeginTask = nil
|
||||||
if !Task.isCancelled {
|
if !Task.isCancelled {
|
||||||
statusMessage = error.localizedDescription
|
statusMessage = error.localizedDescription
|
||||||
}
|
}
|
||||||
@@ -307,6 +349,9 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
stopTimers()
|
stopTimers()
|
||||||
audioLevel = 0
|
audioLevel = 0
|
||||||
let store = AppGroupStore(defaults: defaults)
|
let store = AppGroupStore(defaults: defaults)
|
||||||
|
let targetApplication = sessionTargetApplication
|
||||||
|
let targetAppBundleIdentifier = targetApplication?.bundleIdentifier
|
||||||
|
sessionTargetApplication = nil
|
||||||
let usesDeferredStop = MacDictationPipeline.supportsLivePartials(store: store)
|
let usesDeferredStop = MacDictationPipeline.supportsLivePartials(store: store)
|
||||||
&& store.engineMode == "local"
|
&& store.engineMode == "local"
|
||||||
&& MacLocalASRService.usesMLXLiveStreaming()
|
&& MacLocalASRService.usesMLXLiveStreaming()
|
||||||
@@ -350,6 +395,7 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
result = try await MacDictationPipeline.run(
|
result = try await MacDictationPipeline.run(
|
||||||
samples: capturedSamples,
|
samples: capturedSamples,
|
||||||
store: store,
|
store: store,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier,
|
||||||
onPartial: { [weak self] partial in
|
onPartial: { [weak self] partial in
|
||||||
Task { @MainActor in
|
Task { @MainActor in
|
||||||
self?.transcript = partial
|
self?.transcript = partial
|
||||||
@@ -362,6 +408,7 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
result = try await MacDictationPipeline.run(
|
result = try await MacDictationPipeline.run(
|
||||||
samples: capturedSamples,
|
samples: capturedSamples,
|
||||||
store: store,
|
store: store,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier,
|
||||||
onPartial: { [weak self] partial in
|
onPartial: { [weak self] partial in
|
||||||
Task { @MainActor in
|
Task { @MainActor in
|
||||||
self?.transcript = partial
|
self?.transcript = partial
|
||||||
@@ -370,7 +417,10 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
self.transcript = result.text
|
self.transcript = result.text
|
||||||
let pasted = try await self.deliver(result.text)
|
let pasted = try await self.deliver(
|
||||||
|
result.text,
|
||||||
|
targetApplication: targetApplication
|
||||||
|
)
|
||||||
self.recordUsage(for: result.text)
|
self.recordUsage(for: result.text)
|
||||||
self.speechHistory.append(text: result.text)
|
self.speechHistory.append(text: result.text)
|
||||||
self.appendToOverview(result.text)
|
self.appendToOverview(result.text)
|
||||||
@@ -393,7 +443,10 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private func startLiveCaptureIfSupported(store: AppGroupStore) {
|
private func startLiveCaptureIfSupported(
|
||||||
|
store: AppGroupStore,
|
||||||
|
targetAppBundleIdentifier: String?
|
||||||
|
) {
|
||||||
guard MacDictationPipeline.supportsLivePartials(store: store) else { return }
|
guard MacDictationPipeline.supportsLivePartials(store: store) else { return }
|
||||||
let stream = recorder.makeSnapshotStream()
|
let stream = recorder.makeSnapshotStream()
|
||||||
let (finishStream, finishContinuation) = AsyncStream<Void>.makeStream(
|
let (finishStream, finishContinuation) = AsyncStream<Void>.makeStream(
|
||||||
@@ -405,6 +458,7 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
stream: stream,
|
stream: stream,
|
||||||
finishSignal: finishStream,
|
finishSignal: finishStream,
|
||||||
store: store,
|
store: store,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier,
|
||||||
onPartial: { [weak self] partial in
|
onPartial: { [weak self] partial in
|
||||||
Task { @MainActor in
|
Task { @MainActor in
|
||||||
guard let self else { return }
|
guard let self else { return }
|
||||||
@@ -449,22 +503,32 @@ final class MacDictationViewModel: ObservableObject {
|
|||||||
/// Stops an in-flight prepare, or finishes an active recording.
|
/// Stops an in-flight prepare, or finishes an active recording.
|
||||||
private func cancelOrFinishRecording() {
|
private func cancelOrFinishRecording() {
|
||||||
if isRecording {
|
if isRecording {
|
||||||
|
buttonBeginTask = nil
|
||||||
finishRecording()
|
finishRecording()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if isPreparingToRecord {
|
if isPreparingToRecord {
|
||||||
hotkeyBeginTask?.cancel()
|
hotkeyBeginTask?.cancel()
|
||||||
hotkeyBeginTask = nil
|
hotkeyBeginTask = nil
|
||||||
// If the button-triggered prepare wasn't tracked by hotkeyBeginTask,
|
buttonBeginTask?.cancel()
|
||||||
// still clear the preparing flag and stop any engine that raced in.
|
buttonBeginTask = nil
|
||||||
isPreparingToRecord = false
|
// Keep the preparation gate closed until the cancelled start call
|
||||||
|
// actually returns; otherwise a rapid third click can start a
|
||||||
|
// second recorder task while the first one is still unwinding.
|
||||||
cancelLiveCapture()
|
cancelLiveCapture()
|
||||||
_ = recorder.stop()
|
_ = recorder.stop()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private func deliver(_ text: String) async throws -> Bool {
|
private func deliver(
|
||||||
try await MacTextInsertionService.insert(text, autoPaste: autoPasteEnabled)
|
_ text: String,
|
||||||
|
targetApplication: NSRunningApplication?
|
||||||
|
) async throws -> Bool {
|
||||||
|
try await MacTextInsertionService.insert(
|
||||||
|
text,
|
||||||
|
autoPaste: autoPasteEnabled,
|
||||||
|
targetApp: targetApplication
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
private func statusAfterDelivery(
|
private func statusAfterDelivery(
|
||||||
|
|||||||
@@ -14,10 +14,15 @@ enum MacMLXLiveCapture {
|
|||||||
audioStream: AsyncStream<AudioBufferSnapshot>,
|
audioStream: AsyncStream<AudioBufferSnapshot>,
|
||||||
finishSignal: AsyncStream<Void>,
|
finishSignal: AsyncStream<Void>,
|
||||||
store: AppGroupStore,
|
store: AppGroupStore,
|
||||||
|
targetAppBundleIdentifier: String?,
|
||||||
onPartial: @escaping @Sendable (String) -> Void
|
onPartial: @escaping @Sendable (String) -> Void
|
||||||
) async -> MacLiveASRCaptureResult {
|
) async -> MacLiveASRCaptureResult {
|
||||||
let locale = Locale(identifier: store.localeId.isEmpty ? "zh-CN" : store.localeId)
|
let locale = Locale(identifier: store.localeId.isEmpty ? "zh-CN" : store.localeId)
|
||||||
let bias = resolveBias(store: store, locale: locale)
|
let bias = resolveBias(
|
||||||
|
store: store,
|
||||||
|
locale: locale,
|
||||||
|
targetAppBundleIdentifier: targetAppBundleIdentifier
|
||||||
|
)
|
||||||
|
|
||||||
guard let model = MacLocalASRService.selectedModelDefinition(),
|
guard let model = MacLocalASRService.selectedModelDefinition(),
|
||||||
model.backend == .mlx,
|
model.backend == .mlx,
|
||||||
@@ -133,14 +138,17 @@ enum MacMLXLiveCapture {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private static func resolveBias(store: AppGroupStore, locale: Locale) -> LocalASRBiasPayload? {
|
private static func resolveBias(
|
||||||
MacAppContextService.captureAndPersist(to: store)
|
store: AppGroupStore,
|
||||||
|
locale: Locale,
|
||||||
|
targetAppBundleIdentifier: String?
|
||||||
|
) -> LocalASRBiasPayload? {
|
||||||
let capabilities = MacLocalASRService.currentCapabilities()
|
let capabilities = MacLocalASRService.currentCapabilities()
|
||||||
let bias = LocalASRBiasAdapter.adapt(
|
let bias = LocalASRBiasAdapter.adapt(
|
||||||
LocalASRBiasRequest(
|
LocalASRBiasRequest(
|
||||||
dictionary: store.personalDictionary,
|
dictionary: store.personalDictionary,
|
||||||
locale: locale,
|
locale: locale,
|
||||||
frontAppBundleId: MacAppContextService.frontmostBundleIdentifier(),
|
frontAppBundleId: targetAppBundleIdentifier,
|
||||||
capabilities: capabilities
|
capabilities: capabilities
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -12,6 +12,10 @@ import Carbon
|
|||||||
import Foundation
|
import Foundation
|
||||||
|
|
||||||
enum MacTextInsertionService {
|
enum MacTextInsertionService {
|
||||||
|
/// Paste has no completion callback. Keep the transcript available long
|
||||||
|
/// enough for slower apps to consume the event before restoring clipboard.
|
||||||
|
static let pasteboardRestoreDelayNanoseconds: UInt64 = 500_000_000
|
||||||
|
|
||||||
enum InsertionError: Error, LocalizedError {
|
enum InsertionError: Error, LocalizedError {
|
||||||
case accessibilityNotGranted
|
case accessibilityNotGranted
|
||||||
|
|
||||||
@@ -72,6 +76,7 @@ enum MacTextInsertionService {
|
|||||||
let snapshot = snapshotItems(of: pasteboard)
|
let snapshot = snapshotItems(of: pasteboard)
|
||||||
pasteboard.clearContents()
|
pasteboard.clearContents()
|
||||||
pasteboard.setString(text, forType: .string)
|
pasteboard.setString(text, forType: .string)
|
||||||
|
let transcriptChangeCount = pasteboard.changeCount
|
||||||
|
|
||||||
guard autoPaste else { return false }
|
guard autoPaste else { return false }
|
||||||
guard AXIsProcessTrusted() else { throw InsertionError.accessibilityNotGranted }
|
guard AXIsProcessTrusted() else { throw InsertionError.accessibilityNotGranted }
|
||||||
@@ -84,11 +89,25 @@ enum MacTextInsertionService {
|
|||||||
|
|
||||||
// Give the target app time to read the transcript off the
|
// Give the target app time to read the transcript off the
|
||||||
// pasteboard, then restore whatever the user had on it.
|
// pasteboard, then restore whatever the user had on it.
|
||||||
try? await Task.sleep(nanoseconds: 300_000_000)
|
try? await Task.sleep(nanoseconds: pasteboardRestoreDelayNanoseconds)
|
||||||
|
if shouldRestorePasteboard(
|
||||||
|
transcriptChangeCount: transcriptChangeCount,
|
||||||
|
currentChangeCount: pasteboard.changeCount
|
||||||
|
) {
|
||||||
restoreItems(snapshot, to: pasteboard)
|
restoreItems(snapshot, to: pasteboard)
|
||||||
|
}
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Do not overwrite clipboard content written by the user, target app, or
|
||||||
|
/// a clipboard manager while the synthesized paste was in flight.
|
||||||
|
static func shouldRestorePasteboard(
|
||||||
|
transcriptChangeCount: Int,
|
||||||
|
currentChangeCount: Int
|
||||||
|
) -> Bool {
|
||||||
|
transcriptChangeCount == currentChangeCount
|
||||||
|
}
|
||||||
|
|
||||||
/// Brings `app` forward and waits (up to ~1 s) until it is frontmost so
|
/// Brings `app` forward and waits (up to ~1 s) until it is frontmost so
|
||||||
/// the synthesized keystroke isn't swallowed mid-switch.
|
/// the synthesized keystroke isn't swallowed mid-switch.
|
||||||
@MainActor
|
@MainActor
|
||||||
@@ -119,12 +138,12 @@ enum MacTextInsertionService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private static func restoreItems(
|
static func restoreItems(
|
||||||
_ items: [[NSPasteboard.PasteboardType: Data]],
|
_ items: [[NSPasteboard.PasteboardType: Data]],
|
||||||
to pasteboard: NSPasteboard
|
to pasteboard: NSPasteboard
|
||||||
) {
|
) {
|
||||||
guard !items.isEmpty else { return }
|
|
||||||
pasteboard.clearContents()
|
pasteboard.clearContents()
|
||||||
|
guard !items.isEmpty else { return }
|
||||||
pasteboard.writeObjects(items.map { flavours in
|
pasteboard.writeObjects(items.map { flavours in
|
||||||
let item = NSPasteboardItem()
|
let item = NSPasteboardItem()
|
||||||
for (type, data) in flavours { item.setData(data, forType: type) }
|
for (type, data) in flavours { item.setData(data, forType: type) }
|
||||||
|
|||||||
@@ -104,7 +104,7 @@ enum MacMainWindow {
|
|||||||
/// AppKit rather than SwiftUI's `MenuBarExtra` because the latter is flaky
|
/// AppKit rather than SwiftUI's `MenuBarExtra` because the latter is flaky
|
||||||
/// when combined with a primary `Window` scene (the icon can silently vanish).
|
/// when combined with a primary `Window` scene (the icon can silently vanish).
|
||||||
@MainActor
|
@MainActor
|
||||||
final class MacAppDelegate: NSObject, NSApplicationDelegate {
|
final class MacAppDelegate: NSObject, NSApplicationDelegate, NSPopoverDelegate {
|
||||||
private var statusItem: NSStatusItem?
|
private var statusItem: NSStatusItem?
|
||||||
private let popover = NSPopover()
|
private let popover = NSPopover()
|
||||||
|
|
||||||
@@ -183,6 +183,7 @@ final class MacAppDelegate: NSObject, NSApplicationDelegate {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private func configurePopover() {
|
private func configurePopover() {
|
||||||
|
popover.delegate = self
|
||||||
popover.behavior = .transient
|
popover.behavior = .transient
|
||||||
popover.animates = true
|
popover.animates = true
|
||||||
popover.contentSize = NSSize(width: 340, height: 420)
|
popover.contentSize = NSSize(width: 340, height: 420)
|
||||||
@@ -194,11 +195,18 @@ final class MacAppDelegate: NSObject, NSApplicationDelegate {
|
|||||||
if popover.isShown {
|
if popover.isShown {
|
||||||
popover.performClose(sender)
|
popover.performClose(sender)
|
||||||
} else {
|
} else {
|
||||||
|
// Capture before activation: once the popover becomes key,
|
||||||
|
// NSWorkspace reports OSGKeyboard instead of the user's target.
|
||||||
|
MacDictationViewModel.shared.prepareForPopoverPresentation()
|
||||||
NSApp.activate(ignoringOtherApps: true)
|
NSApp.activate(ignoringOtherApps: true)
|
||||||
popover.show(relativeTo: button.bounds, of: button, preferredEdge: .minY)
|
popover.show(relativeTo: button.bounds, of: button, preferredEdge: .minY)
|
||||||
popover.contentViewController?.view.window?.makeKey()
|
popover.contentViewController?.view.window?.makeKey()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func popoverDidClose(_ notification: Notification) {
|
||||||
|
MacDictationViewModel.shared.clearPreparedPopoverTarget()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// SwiftUI content hosted inside the status-bar popover. Shares the single
|
/// SwiftUI content hosted inside the status-bar popover. Shares the single
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
// MacDictationViewModelTests.swift
|
||||||
|
// OSGKeyboard · Mac tests
|
||||||
|
//
|
||||||
|
// Regression coverage for cancelling an asynchronous recorder start.
|
||||||
|
|
||||||
|
import Foundation
|
||||||
|
import XCTest
|
||||||
|
@testable import OSGKeyboard
|
||||||
|
|
||||||
|
@MainActor
|
||||||
|
final class MacDictationViewModelTests: XCTestCase {
|
||||||
|
|
||||||
|
func testCancellingButtonPreparationKeepsGateClosedUntilStartUnwinds() async {
|
||||||
|
let suiteName = "com.osgkeyboard.mac.tests.prepare.\(UUID().uuidString)"
|
||||||
|
let defaults = UserDefaults(suiteName: suiteName)!
|
||||||
|
defaults.removePersistentDomain(forName: suiteName)
|
||||||
|
defer { defaults.removePersistentDomain(forName: suiteName) }
|
||||||
|
|
||||||
|
let recorder = SuspendedMacAudioRecorder()
|
||||||
|
let viewModel = MacDictationViewModel(
|
||||||
|
defaults: defaults,
|
||||||
|
recorder: recorder,
|
||||||
|
startHotkeyService: false
|
||||||
|
)
|
||||||
|
|
||||||
|
viewModel.toggleRecording()
|
||||||
|
let didStartPreparing = await waitUntil { recorder.isStartPending }
|
||||||
|
XCTAssertTrue(didStartPreparing)
|
||||||
|
XCTAssertTrue(viewModel.isPreparingToRecord)
|
||||||
|
|
||||||
|
viewModel.toggleRecording()
|
||||||
|
|
||||||
|
XCTAssertTrue(
|
||||||
|
viewModel.isPreparingToRecord,
|
||||||
|
"Cancellation must not reopen the start gate while recorder.start() is still unwinding."
|
||||||
|
)
|
||||||
|
recorder.completeStart()
|
||||||
|
let didFinishCancelling = await waitUntil { !viewModel.isPreparingToRecord }
|
||||||
|
XCTAssertTrue(didFinishCancelling)
|
||||||
|
XCTAssertFalse(viewModel.isRecording)
|
||||||
|
XCTAssertFalse(viewModel.isProcessing)
|
||||||
|
XCTAssertGreaterThanOrEqual(recorder.stopCallCount, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
private func waitUntil(
|
||||||
|
_ predicate: @escaping @MainActor () -> Bool
|
||||||
|
) async -> Bool {
|
||||||
|
for _ in 0..<100 {
|
||||||
|
if predicate() { return true }
|
||||||
|
try? await Task.sleep(for: .milliseconds(5))
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private final class SuspendedMacAudioRecorder: MacAudioRecording, @unchecked Sendable {
|
||||||
|
private let lock = NSLock()
|
||||||
|
private var startContinuation: CheckedContinuation<Void, any Error>?
|
||||||
|
private var stops = 0
|
||||||
|
|
||||||
|
var isStartPending: Bool {
|
||||||
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
|
return startContinuation != nil
|
||||||
|
}
|
||||||
|
|
||||||
|
var stopCallCount: Int {
|
||||||
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
|
return stops
|
||||||
|
}
|
||||||
|
|
||||||
|
func level() -> Float { 0 }
|
||||||
|
|
||||||
|
func start() async throws {
|
||||||
|
try await withCheckedThrowingContinuation { continuation in
|
||||||
|
lock.lock()
|
||||||
|
startContinuation = continuation
|
||||||
|
lock.unlock()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func completeStart() {
|
||||||
|
lock.lock()
|
||||||
|
let continuation = startContinuation
|
||||||
|
startContinuation = nil
|
||||||
|
lock.unlock()
|
||||||
|
continuation?.resume()
|
||||||
|
}
|
||||||
|
|
||||||
|
func makeSnapshotStream() -> AsyncStream<AudioBufferSnapshot> {
|
||||||
|
AsyncStream { $0.finish() }
|
||||||
|
}
|
||||||
|
|
||||||
|
func stop() -> [Float] {
|
||||||
|
lock.lock()
|
||||||
|
stops += 1
|
||||||
|
lock.unlock()
|
||||||
|
return []
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
// MacTextInsertionServiceTests.swift
|
||||||
|
// OSGKeyboard · Mac tests
|
||||||
|
//
|
||||||
|
// Regression coverage for clipboard preservation and captured-app context.
|
||||||
|
|
||||||
|
import AppKit
|
||||||
|
import XCTest
|
||||||
|
@testable import OSGKeyboard
|
||||||
|
|
||||||
|
final class MacTextInsertionServiceTests: XCTestCase {
|
||||||
|
|
||||||
|
func testRestoreRequiresTranscriptToStillOwnPasteboard() {
|
||||||
|
XCTAssertTrue(
|
||||||
|
MacTextInsertionService.shouldRestorePasteboard(
|
||||||
|
transcriptChangeCount: 12,
|
||||||
|
currentChangeCount: 12
|
||||||
|
)
|
||||||
|
)
|
||||||
|
XCTAssertFalse(
|
||||||
|
MacTextInsertionService.shouldRestorePasteboard(
|
||||||
|
transcriptChangeCount: 12,
|
||||||
|
currentChangeCount: 13
|
||||||
|
),
|
||||||
|
"A newer clipboard write must not be overwritten by restoration."
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testRestoringOriginallyEmptyPasteboardClearsTranscript() {
|
||||||
|
let pasteboard = NSPasteboard(
|
||||||
|
name: NSPasteboard.Name("MacTextInsertionServiceTests.\(UUID().uuidString)")
|
||||||
|
)
|
||||||
|
pasteboard.clearContents()
|
||||||
|
pasteboard.setString("transcript", forType: .string)
|
||||||
|
|
||||||
|
MacTextInsertionService.restoreItems([], to: pasteboard)
|
||||||
|
|
||||||
|
XCTAssertNil(pasteboard.string(forType: .string))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testCapturedBundleIdentifierDrivesPolishContext() {
|
||||||
|
XCTAssertEqual(
|
||||||
|
MacAppContextService.detectContext(bundleIdentifier: "com.apple.dt.Xcode"),
|
||||||
|
.code
|
||||||
|
)
|
||||||
|
XCTAssertEqual(
|
||||||
|
MacAppContextService.detectContext(bundleIdentifier: "com.tencent.xinWeChat"),
|
||||||
|
.chat
|
||||||
|
)
|
||||||
|
XCTAssertEqual(
|
||||||
|
MacAppContextService.detectContext(bundleIdentifier: "com.osgkeyboard.mac"),
|
||||||
|
.unknown
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -143,7 +143,10 @@ public actor ChunkedUtterancePipeline {
|
|||||||
action: "preMerge",
|
action: "preMerge",
|
||||||
chunkIndex: chunk.index
|
chunkIndex: chunk.index
|
||||||
)
|
)
|
||||||
let mergedResult = await transcribeChunk(samples: preMerge.samples)
|
let mergedResult = await transcribeChunkWithRetry(
|
||||||
|
samples: preMerge.samples,
|
||||||
|
chunkIndex: chunk.index
|
||||||
|
)
|
||||||
switch mergedResult {
|
switch mergedResult {
|
||||||
case .success(let text):
|
case .success(let text):
|
||||||
// Empty / whitespace merge must NOT wipe a prior good segment
|
// Empty / whitespace merge must NOT wipe a prior good segment
|
||||||
@@ -182,7 +185,10 @@ public actor ChunkedUtterancePipeline {
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
let result = await transcribeChunk(samples: chunk.samples)
|
let result = await transcribeChunkWithRetry(
|
||||||
|
samples: chunk.samples,
|
||||||
|
chunkIndex: chunk.index
|
||||||
|
)
|
||||||
logChunkOutcome(chunk: chunk, result: result)
|
logChunkOutcome(chunk: chunk, result: result)
|
||||||
switch result {
|
switch result {
|
||||||
case .success(let text):
|
case .success(let text):
|
||||||
@@ -199,7 +205,10 @@ public actor ChunkedUtterancePipeline {
|
|||||||
action: "emptyRetry",
|
action: "emptyRetry",
|
||||||
chunkIndex: chunk.index
|
chunkIndex: chunk.index
|
||||||
)
|
)
|
||||||
let retryResult = await transcribeChunk(samples: retry.samples)
|
let retryResult = await transcribeChunkWithRetry(
|
||||||
|
samples: retry.samples,
|
||||||
|
chunkIndex: chunk.index
|
||||||
|
)
|
||||||
switch retryResult {
|
switch retryResult {
|
||||||
case .success(let retryText):
|
case .success(let retryText):
|
||||||
let trimmed = retryText.trimmingCharacters(in: .whitespacesAndNewlines)
|
let trimmed = retryText.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
@@ -311,6 +320,30 @@ public actor ChunkedUtterancePipeline {
|
|||||||
}.value
|
}.value
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Retry one failed chunk before advancing the serial worker. Keeping the
|
||||||
|
/// same PCM samples prevents a transient request failure from creating an
|
||||||
|
/// undetectable hole in an otherwise fluent stitched transcript.
|
||||||
|
private func transcribeChunkWithRetry(
|
||||||
|
samples: [Float],
|
||||||
|
chunkIndex: Int
|
||||||
|
) async -> ASRChunkResult {
|
||||||
|
let first = await transcribeChunk(samples: samples)
|
||||||
|
guard case .failure(let message) = first else { return first }
|
||||||
|
guard !cancelled, !Task.isCancelled else { return .cancelled }
|
||||||
|
|
||||||
|
FlowTrace.warn(
|
||||||
|
"pipeline.chunk.retry",
|
||||||
|
"chunk=\(chunkIndex) samples=\(samples.count) error=\(message)"
|
||||||
|
)
|
||||||
|
do {
|
||||||
|
try await Task.sleep(nanoseconds: 150_000_000)
|
||||||
|
} catch {
|
||||||
|
return .cancelled
|
||||||
|
}
|
||||||
|
guard !cancelled, !Task.isCancelled else { return .cancelled }
|
||||||
|
return await transcribeChunk(samples: samples)
|
||||||
|
}
|
||||||
|
|
||||||
/// Pairs each chunk's audio with the text it produced, so an empty
|
/// Pairs each chunk's audio with the text it produced, so an empty
|
||||||
/// transcript can be attributed to either silent audio or a mute engine.
|
/// transcript can be attributed to either silent audio or a mute engine.
|
||||||
private func logChunkOutcome(chunk: UtteranceAudioChunk, result: ASRChunkResult) {
|
private func logChunkOutcome(chunk: UtteranceAudioChunk, result: ASRChunkResult) {
|
||||||
|
|||||||
@@ -62,7 +62,10 @@ public enum PolishOutputValidator {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let inputNumbers = matches(#"\d+(?:[.,]\d+)*"#, in: input)
|
let inputNumbers = matches(#"\d+(?:[.,]\d+)*"#, in: input)
|
||||||
let missingNumbers = Array(Set(inputNumbers.filter { !output.contains($0) })).sorted()
|
let allowedOrdinalNumbers = allowedOrdinalRepairNumbers(input: input, output: output)
|
||||||
|
let missingNumbers = Array(Set(inputNumbers.filter {
|
||||||
|
!output.contains($0) && !allowedOrdinalNumbers.contains($0)
|
||||||
|
})).sorted()
|
||||||
if !missingNumbers.isEmpty {
|
if !missingNumbers.isEmpty {
|
||||||
violations.append(.missingNumbers(missingNumbers))
|
violations.append(.missingNumbers(missingNumbers))
|
||||||
}
|
}
|
||||||
@@ -106,7 +109,6 @@ public enum PolishOutputValidator {
|
|||||||
let patterns = [
|
let patterns = [
|
||||||
#"https?://[^\s<>"']+"#,
|
#"https?://[^\s<>"']+"#,
|
||||||
#"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"#,
|
#"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"#,
|
||||||
#"(?:^|[\s(])(?:[A-Za-z0-9_.-]+/)+[A-Za-z0-9_.-]+"#,
|
|
||||||
#"\b[A-Za-z][A-Za-z0-9]*_[A-Za-z0-9_]+\b"#,
|
#"\b[A-Za-z][A-Za-z0-9]*_[A-Za-z0-9_]+\b"#,
|
||||||
#"\b[A-Za-z]+[a-z0-9][A-Z][A-Za-z0-9]*\b"#,
|
#"\b[A-Za-z]+[a-z0-9][A-Z][A-Za-z0-9]*\b"#,
|
||||||
]
|
]
|
||||||
@@ -118,9 +120,99 @@ public enum PolishOutputValidator {
|
|||||||
)))
|
)))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
let pathPattern = #"(?:^|[\s(])(?:~?/|\.\.?/)?(?:[A-Za-z0-9_.-]+/)+[A-Za-z0-9_.-]+"#
|
||||||
|
for rawValue in matches(pathPattern, in: text) {
|
||||||
|
let value = rawValue.trimmingCharacters(in: .whitespacesAndNewlines.union(
|
||||||
|
CharacterSet(charactersIn: "(")
|
||||||
|
))
|
||||||
|
if isProtectedPath(value) {
|
||||||
|
result.insert(value)
|
||||||
|
}
|
||||||
|
}
|
||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static func isProtectedPath(_ value: String) -> Bool {
|
||||||
|
let explicitPrefix = value.hasPrefix("/")
|
||||||
|
|| value.hasPrefix("./")
|
||||||
|
|| value.hasPrefix("../")
|
||||||
|
|| value.hasPrefix("~/")
|
||||||
|
let normalized = value.trimmingCharacters(in: CharacterSet(charactersIn: "/"))
|
||||||
|
let segments = normalized.split(separator: "/", omittingEmptySubsequences: true)
|
||||||
|
guard segments.count >= 2 else { return false }
|
||||||
|
|
||||||
|
// Dates and fractions such as 2025/03/01, 3/4, and 3/5 are numeric
|
||||||
|
// values, not file paths. They remain covered by soft number telemetry.
|
||||||
|
if segments.allSatisfy({ $0.allSatisfy(\.isNumber) }) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if explicitPrefix { return true }
|
||||||
|
if segments.count >= 3 { return true }
|
||||||
|
return segments.contains { $0.contains(".") || $0.contains("_") }
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func allowedOrdinalRepairNumbers(
|
||||||
|
input: String,
|
||||||
|
output: String
|
||||||
|
) -> Set<String> {
|
||||||
|
let pattern = #"第\s*(\d+)\s*[::]\s*00"#
|
||||||
|
guard let regex = try? NSRegularExpression(pattern: pattern) else { return [] }
|
||||||
|
let fullRange = NSRange(input.startIndex..<input.endIndex, in: input)
|
||||||
|
var allowed = Set<String>()
|
||||||
|
|
||||||
|
for match in regex.matches(in: input, range: fullRange) {
|
||||||
|
guard match.numberOfRanges > 1,
|
||||||
|
let ordinalRange = Range(match.range(at: 1), in: input),
|
||||||
|
let matchRange = Range(match.range, in: input) else {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
let ordinal = String(input[ordinalRange])
|
||||||
|
let prefixRange = input.startIndex..<matchRange.lowerBound
|
||||||
|
let prefix = String(input[prefixRange])
|
||||||
|
guard hasEstablishedEnumeration(prefix) else { continue }
|
||||||
|
|
||||||
|
let escaped = NSRegularExpression.escapedPattern(for: ordinal)
|
||||||
|
let arabicListPattern = #"(?m)(?:^|\n)\s*"# + escaped + #"\s*[.、)]"#
|
||||||
|
let chineseOrdinal = Int(ordinal).flatMap(chineseNumeral)
|
||||||
|
let hasArabicOrdinal = output.range(
|
||||||
|
of: arabicListPattern,
|
||||||
|
options: .regularExpression
|
||||||
|
) != nil
|
||||||
|
let hasChineseOrdinal = chineseOrdinal.map {
|
||||||
|
output.contains("第\($0)点")
|
||||||
|
} ?? false
|
||||||
|
if hasArabicOrdinal || hasChineseOrdinal {
|
||||||
|
allowed.insert(ordinal)
|
||||||
|
allowed.insert("00")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return allowed
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func hasEstablishedEnumeration(_ prefix: String) -> Bool {
|
||||||
|
prefix.range(
|
||||||
|
of: #"(?:第一点|第[一二三四五六七八九十]+点|首先)"#,
|
||||||
|
options: .regularExpression
|
||||||
|
) != nil
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func chineseNumeral(_ value: Int) -> String? {
|
||||||
|
let digits = ["零", "一", "二", "三", "四", "五", "六", "七", "八", "九"]
|
||||||
|
switch value {
|
||||||
|
case 0...9:
|
||||||
|
return digits[value]
|
||||||
|
case 10:
|
||||||
|
return "十"
|
||||||
|
case 11...19:
|
||||||
|
return "十" + digits[value % 10]
|
||||||
|
case 20...99:
|
||||||
|
let tens = digits[value / 10] + "十"
|
||||||
|
return value % 10 == 0 ? tens : tens + digits[value % 10]
|
||||||
|
default:
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private static func matches(_ pattern: String, in text: String) -> [String] {
|
private static func matches(_ pattern: String, in text: String) -> [String] {
|
||||||
guard let regex = try? NSRegularExpression(pattern: pattern) else { return [] }
|
guard let regex = try? NSRegularExpression(pattern: pattern) else { return [] }
|
||||||
let range = NSRange(text.startIndex..<text.endIndex, in: text)
|
let range = NSRange(text.startIndex..<text.endIndex, in: text)
|
||||||
|
|||||||
@@ -396,7 +396,7 @@
|
|||||||
"mac.localASR.phase.finalizing" = "Finalizing";
|
"mac.localASR.phase.finalizing" = "Finalizing";
|
||||||
"mac.localASR.phase.failed" = "Failed";
|
"mac.localASR.phase.failed" = "Failed";
|
||||||
"mac.localASR.phase.completed" = "Completed";
|
"mac.localASR.phase.completed" = "Completed";
|
||||||
"mac.error.accessibilityRequired" = "Enable Accessibility for OSGKeyboard in System Settings";
|
"mac.error.accessibilityRequired" = "Dictation copied to the clipboard. Press ⌘V to paste, then enable Accessibility for OSGKeyboard in System Settings.";
|
||||||
"mac.foregroundApp" = "Front app: %@";
|
"mac.foregroundApp" = "Front app: %@";
|
||||||
"mac.sync.settingsTitle" = "Cross-Device iCloud Sync";
|
"mac.sync.settingsTitle" = "Cross-Device iCloud Sync";
|
||||||
"mac.sync.settingsSubtitle" = "Sync settings, history, and API keys across devices via iCloud.";
|
"mac.sync.settingsSubtitle" = "Sync settings, history, and API keys across devices via iCloud.";
|
||||||
|
|||||||
@@ -396,7 +396,7 @@
|
|||||||
"mac.localASR.phase.finalizing" = "完成安装";
|
"mac.localASR.phase.finalizing" = "完成安装";
|
||||||
"mac.localASR.phase.failed" = "失败";
|
"mac.localASR.phase.failed" = "失败";
|
||||||
"mac.localASR.phase.completed" = "已完成";
|
"mac.localASR.phase.completed" = "已完成";
|
||||||
"mac.error.accessibilityRequired" = "请在系统设置中为 OSGKeyboard 启用辅助功能";
|
"mac.error.accessibilityRequired" = "识别结果已复制到剪贴板,请按 ⌘V 粘贴,然后在系统设置中为 OSGKeyboard 启用辅助功能。";
|
||||||
"mac.foregroundApp" = "前台应用:%@";
|
"mac.foregroundApp" = "前台应用:%@";
|
||||||
"mac.sync.settingsTitle" = "跨设备iCloud 同步";
|
"mac.sync.settingsTitle" = "跨设备iCloud 同步";
|
||||||
"mac.sync.settingsSubtitle" = "通过 iCloud 跨设备同步设置、历史记录、API Key";
|
"mac.sync.settingsSubtitle" = "通过 iCloud 跨设备同步设置、历史记录、API Key";
|
||||||
|
|||||||
@@ -38,6 +38,85 @@ final class PolishOutputValidatorTests: XCTestCase {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func testDatesFractionsAndSlashWordsAreNotProtectedPaths() {
|
||||||
|
let cases = [
|
||||||
|
("在 2025/03/01 之前完成", "在2025年3月1日之前完成"),
|
||||||
|
("价格是 3/4 杯面粉", "价格是四分之三杯面粉"),
|
||||||
|
("我给 3/5 分", "我给五分之三"),
|
||||||
|
("读一下 and/or 的用法", "读一下 and or 的用法"),
|
||||||
|
]
|
||||||
|
|
||||||
|
for (input, output) in cases {
|
||||||
|
let violations = PolishOutputValidator.validate(
|
||||||
|
input: input,
|
||||||
|
output: output,
|
||||||
|
dictionary: .empty,
|
||||||
|
lengthRatio: 0.2...3
|
||||||
|
)
|
||||||
|
XCTAssertFalse(
|
||||||
|
violations.contains {
|
||||||
|
if case .missingIdentifiers = $0 { return true }
|
||||||
|
return false
|
||||||
|
},
|
||||||
|
"Must not classify slash value as a protected path: \(input)"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testStrongPathSignalsRemainHardProtectedIdentifiers() {
|
||||||
|
let inputs = [
|
||||||
|
"/usr/local/bin",
|
||||||
|
"../Sources/App.swift",
|
||||||
|
"Sources/Features/Auth",
|
||||||
|
"src/user_id",
|
||||||
|
]
|
||||||
|
for input in inputs {
|
||||||
|
let violations = PolishOutputValidator.validate(
|
||||||
|
input: "打开 \(input)",
|
||||||
|
output: "打开对应文件",
|
||||||
|
dictionary: .empty,
|
||||||
|
lengthRatio: 0.2...3
|
||||||
|
)
|
||||||
|
XCTAssertTrue(
|
||||||
|
violations.contains {
|
||||||
|
if case .missingIdentifiers(let values) = $0 {
|
||||||
|
return values.contains(input)
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
},
|
||||||
|
"Expected hard path protection for \(input)"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testOrdinalASRRepairDoesNotReportMissingZeroes() {
|
||||||
|
let violations = PolishOutputValidator.validate(
|
||||||
|
input: "第一点是A第2:00是B",
|
||||||
|
output: "第一点是 A\n2. B",
|
||||||
|
dictionary: .empty,
|
||||||
|
lengthRatio: 0.2...3
|
||||||
|
)
|
||||||
|
XCTAssertFalse(violations.contains {
|
||||||
|
if case .missingNumbers = $0 { return true }
|
||||||
|
return false
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func testRealTimeStillReportsMissingZeroes() {
|
||||||
|
let violations = PolishOutputValidator.validate(
|
||||||
|
input: "第一点是坐第2:00班车",
|
||||||
|
output: "第一点是坐第二班车",
|
||||||
|
dictionary: .empty,
|
||||||
|
lengthRatio: 0.2...3
|
||||||
|
)
|
||||||
|
XCTAssertTrue(violations.contains {
|
||||||
|
if case .missingNumbers(let values) = $0 {
|
||||||
|
return values.contains("00")
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func testNumbersLengthAndLanguageAreObservationOnly() {
|
func testNumbersLengthAndLanguageAreObservationOnly() {
|
||||||
let violations = PolishOutputValidator.validate(
|
let violations = PolishOutputValidator.validate(
|
||||||
input: "项目 123 明天下午交付并通知全部相关成员",
|
input: "项目 123 明天下午交付并通知全部相关成员",
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
// SpeechHistoryDayDeletionTests.swift
|
||||||
|
// OSGKeyboardTests
|
||||||
|
//
|
||||||
|
// Day-boundary and tombstone coverage for History's delete-day action.
|
||||||
|
|
||||||
|
import XCTest
|
||||||
|
@testable import OSGKeyboardShared
|
||||||
|
|
||||||
|
@MainActor
|
||||||
|
final class SpeechHistoryDayDeletionTests: XCTestCase {
|
||||||
|
|
||||||
|
func testDeleteEntriesRemovesOnlySelectedLocalDayAndRecordsTombstones() {
|
||||||
|
let suiteName = "group.com.osgkeyboard.shared.tests.delete-day.\(UUID().uuidString)"
|
||||||
|
let defaults = UserDefaults(suiteName: suiteName)!
|
||||||
|
defaults.removePersistentDomain(forName: suiteName)
|
||||||
|
defer { defaults.removePersistentDomain(forName: suiteName) }
|
||||||
|
|
||||||
|
let calendar = Calendar.current
|
||||||
|
let selectedDay = Date(timeIntervalSince1970: 1_752_163_200)
|
||||||
|
let start = calendar.startOfDay(for: selectedDay)
|
||||||
|
let end = calendar.date(byAdding: .day, value: 1, to: start)!
|
||||||
|
|
||||||
|
let previous = SpeechHistoryEntry(
|
||||||
|
text: "previous",
|
||||||
|
createdAt: start.addingTimeInterval(-1)
|
||||||
|
)
|
||||||
|
let firstSelected = SpeechHistoryEntry(
|
||||||
|
text: "first selected",
|
||||||
|
createdAt: start.addingTimeInterval(1)
|
||||||
|
)
|
||||||
|
let lastSelected = SpeechHistoryEntry(
|
||||||
|
text: "last selected",
|
||||||
|
createdAt: end.addingTimeInterval(-1)
|
||||||
|
)
|
||||||
|
let next = SpeechHistoryEntry(
|
||||||
|
text: "next",
|
||||||
|
createdAt: end
|
||||||
|
)
|
||||||
|
SpeechHistoryStorage.save(
|
||||||
|
SyncedSpeechHistory(
|
||||||
|
entries: [next, lastSelected, firstSelected, previous]
|
||||||
|
),
|
||||||
|
to: defaults
|
||||||
|
)
|
||||||
|
let store = SpeechHistoryStore(defaults: defaults)
|
||||||
|
|
||||||
|
store.deleteEntries(on: selectedDay)
|
||||||
|
|
||||||
|
let persisted = SpeechHistoryStorage.load(from: defaults)
|
||||||
|
XCTAssertEqual(
|
||||||
|
Set(persisted.entries.map(\.id)),
|
||||||
|
Set([previous.id, next.id])
|
||||||
|
)
|
||||||
|
XCTAssertNotNil(persisted.deletedEntryIDs[firstSelected.id])
|
||||||
|
XCTAssertNotNil(persisted.deletedEntryIDs[lastSelected.id])
|
||||||
|
XCTAssertNil(persisted.deletedEntryIDs[previous.id])
|
||||||
|
XCTAssertNil(persisted.deletedEntryIDs[next.id])
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user