From d22154e119938a7455a3a441c5e67315226eba04 Mon Sep 17 00:00:00 2001 From: Rocky <72559939+hkgood@users.noreply.github.com> Date: Thu, 18 Jun 2026 21:41:56 +0800 Subject: [PATCH] fix: preview installTap crash on simulator (0 Hz AURemoteIO output) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous build crashed on the first record-tap with: *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'Failed to create tap due to format mismatch, ' The crash surfaced at `[AVAudioNode installTap:...]` but the *root* cause was one frame deeper, in the underlying `AURemoteIO::enable` call: AURemoteIO.cpp:1135 failed: -10851 (enable 1, outf< 2 ch, 0 Hz, Float32, deinterleaved> inf< 1 ch, 48000 Hz, Float32>) `AURemoteIO` is a two-direction Audio Unit (input + output). `installTap` triggers `AURemoteIO::enable(1)` which tries to initialize *both* directions at once. On the iOS Simulator, the output direction reports a `0 Hz` "speaker" because the simulator has no real speaker, and `enable` fails with `kAudioUnitErr_FormatNotSupported` (-10851). The error then bubbles up through `installTap` as the misleading "format mismatch" — the input format we passed was correct, but the output side broke the whole `enable` call. The previous code asked the audio session for `.playAndRecord` mode, which requires the output direction to be enabled. On a real device, the speaker is 44100/48000 Hz and `.playAndRecord` works. On the simulator, it traps. Fix (two parts): 1) `start()` audio session now uses `.record` instead of `.playAndRecord`. The preview never plays back audio — it just records from the mic and hands the buffers to `SpeechAnalyzer`. `.record` skips the output direction entirely, so the simulator's 0 Hz speaker isn't a problem. The real keyboard extension (`OSGKeyboardExt`) keeps `.playAndRecord` because it runs on real devices and may want to play click sounds / haptic feedback — only the preview needs the simulator-friendly category. 2) `startEngineAndASR` adds a pre-flight check: refuse to call `installTap` when the input bus reports `sampleRate == 0` or `channelCount == 0`. The `installTap` failure mode is an `NSException` (Objective-C), not a Swift `Error` — `try`/`catch` can't intercept it, so the only safe defence is to never call it with a placeholder/unconfigured bus. We saw `0 Hz` inputs on the simulator when the host mic permission wasn't granted to CoreSimulator, and on devices with an unexpected audio-session state. The check sets a clear `.error` phase ("Microphone unavailable (hw format X Hz / Y ch)") instead of trapping the app. Why this also unblocks voice-to-text: the previous "no transcription" symptom was just the crash at step 7 of the start pipeline — the ASR never saw a single audio frame because the tap never installed. With the crash fixed, the `SpeechAnalyzer` pipeline can now actually receive buffers and emit `.partial` / `.final` events. Whether the *content* of those events is meaningful depends on the simulator's mic quality and the chosen locale (both separate concerns from this fix). Tests: 27/27 pass (no behavioural test for the pre-flight check — the check is 4 lines of obvious defence, and testing it would require either mocking the audio engine or installing CoreSimulator's host-mic-bridge, neither of which is worth the cost). Build: BUILD SUCCEEDED. 🤖 Generated with Claude Code --- OSGKeyboard/Views/PreviewASRController.swift | 41 +++++++++++++++++++- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/OSGKeyboard/Views/PreviewASRController.swift b/OSGKeyboard/Views/PreviewASRController.swift index a237ed3..470a80e 100644 --- a/OSGKeyboard/Views/PreviewASRController.swift +++ b/OSGKeyboard/Views/PreviewASRController.swift @@ -106,12 +106,30 @@ final class PreviewASRController: ObservableObject { } // 3. Audio session — only configure once per process. + // + // Category is `.record` (not `.playAndRecord`) because the + // preview never plays back audio — it just records from the + // mic and hands the buffers to `SpeechAnalyzer`. On the + // iOS Simulator, `.playAndRecord` requires the + // `AURemoteIO` Audio Unit's *output* side to also be + // enabled, but the simulator's "speaker" reports a 0 Hz + // hardware format, so `AURemoteIO::enable` fails with + // `kAudioUnitErr_FormatNotSupported` (-10851) and any + // subsequent `installTap` traps with "Failed to create tap + // due to format mismatch". `.record` skips the output + // side entirely, so the simulator can record. + // + // The real keyboard extension (`OSGKeyboardExt`) keeps + // `.playAndRecord` because it runs on a real device where + // the output side has a real hardware format, and may want + // to play click sounds / haptic feedback. Only the preview + // needs the simulator-friendly category. if !didConfigureAudioSession { do { let session = AVAudioSession.sharedInstance() - try session.setCategory(.playAndRecord, + try session.setCategory(.record, mode: .measurement, - options: [.defaultToSpeaker, .allowBluetoothHFP]) + options: []) try session.setActive(true, options: .notifyOthersOnDeactivation) didConfigureAudioSession = true } catch { @@ -187,6 +205,25 @@ final class PreviewASRController: ObservableObject { private func startEngineAndASR(locale: Locale) { let inputNode = audioEngine.inputNode let hwFormat = inputNode.outputFormat(forBus: 0) + + // Pre-flight check: a placeholder / unconfigured input bus + // reports `sampleRate == 0` (or `channelCount == 0`). + // `installTap` on such a bus traps with "Failed to create + // tap due to format mismatch" (an NSException, not a Swift + // `Error`, so we can't `try`/`catch` it). The safest fix + // is to refuse the tap up front and surface a clear + // `.error` phase instead of crashing the app. We've seen + // this on the iOS Simulator when the host's microphone + // permission isn't granted to CoreSimulator, and on + // devices where the audio session is in an unexpected + // state from a previous foreground/background transition. + guard hwFormat.sampleRate > 0, hwFormat.channelCount > 0 else { + phase = .error( + "麦克风不可用 · Microphone unavailable (hw format \(hwFormat.sampleRate) Hz / \(hwFormat.channelCount) ch)" + ) + return + } + let targetSampleRate: Double = 16_000 guard let targetFormat = AVAudioFormat( commonFormat: .pcmFormatFloat32,