fix: preview installTap crash on simulator (0 Hz AURemoteIO output)
The previous build crashed on the first record-tap with:
*** Terminating app due to uncaught exception
'com.apple.coreaudio.avfaudio', reason:
'Failed to create tap due to format mismatch,
<AVAudioFormat: 1 ch, 48000 Hz, Float32>'
The crash surfaced at `[AVAudioNode installTap:...]` but the
*root* cause was one frame deeper, in the underlying
`AURemoteIO::enable` call:
AURemoteIO.cpp:1135 failed: -10851
(enable 1, outf< 2 ch, 0 Hz, Float32, deinterleaved>
inf< 1 ch, 48000 Hz, Float32>)
`AURemoteIO` is a two-direction Audio Unit (input + output).
`installTap` triggers `AURemoteIO::enable(1)` which tries to
initialize *both* directions at once. On the iOS Simulator, the
output direction reports a `0 Hz` "speaker" because the
simulator has no real speaker, and `enable` fails with
`kAudioUnitErr_FormatNotSupported` (-10851). The error then
bubbles up through `installTap` as the misleading "format
mismatch" — the input format we passed was correct, but the
output side broke the whole `enable` call.
The previous code asked the audio session for `.playAndRecord`
mode, which requires the output direction to be enabled. On a
real device, the speaker is 44100/48000 Hz and `.playAndRecord`
works. On the simulator, it traps.
Fix (two parts):
1) `start()` audio session now uses `.record` instead of
`.playAndRecord`. The preview never plays back audio — it
just records from the mic and hands the buffers to
`SpeechAnalyzer`. `.record` skips the output direction
entirely, so the simulator's 0 Hz speaker isn't a problem.
The real keyboard extension (`OSGKeyboardExt`) keeps
`.playAndRecord` because it runs on real devices and may
want to play click sounds / haptic feedback — only the
preview needs the simulator-friendly category.
2) `startEngineAndASR` adds a pre-flight check: refuse to
call `installTap` when the input bus reports
`sampleRate == 0` or `channelCount == 0`. The `installTap`
failure mode is an `NSException` (Objective-C), not a Swift
`Error` — `try`/`catch` can't intercept it, so the only safe
defence is to never call it with a placeholder/unconfigured
bus. We saw `0 Hz` inputs on the simulator when the host
mic permission wasn't granted to CoreSimulator, and on
devices with an unexpected audio-session state. The check
sets a clear `.error` phase ("Microphone unavailable (hw
format X Hz / Y ch)") instead of trapping the app.
Why this also unblocks voice-to-text: the previous
"no transcription" symptom was just the crash at step 7 of
the start pipeline — the ASR never saw a single audio frame
because the tap never installed. With the crash fixed, the
`SpeechAnalyzer` pipeline can now actually receive buffers
and emit `.partial` / `.final` events. Whether the *content*
of those events is meaningful depends on the simulator's mic
quality and the chosen locale (both separate concerns from
this fix).
Tests: 27/27 pass (no behavioural test for the pre-flight
check — the check is 4 lines of obvious defence, and
testing it would require either mocking the audio engine
or installing CoreSimulator's host-mic-bridge, neither of
which is worth the cost).
Build: BUILD SUCCEEDED.
🤖 Generated with Claude Code
This commit is contained in:
@@ -106,12 +106,30 @@ final class PreviewASRController: ObservableObject {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 3. Audio session — only configure once per process.
|
// 3. Audio session — only configure once per process.
|
||||||
|
//
|
||||||
|
// Category is `.record` (not `.playAndRecord`) because the
|
||||||
|
// preview never plays back audio — it just records from the
|
||||||
|
// mic and hands the buffers to `SpeechAnalyzer`. On the
|
||||||
|
// iOS Simulator, `.playAndRecord` requires the
|
||||||
|
// `AURemoteIO` Audio Unit's *output* side to also be
|
||||||
|
// enabled, but the simulator's "speaker" reports a 0 Hz
|
||||||
|
// hardware format, so `AURemoteIO::enable` fails with
|
||||||
|
// `kAudioUnitErr_FormatNotSupported` (-10851) and any
|
||||||
|
// subsequent `installTap` traps with "Failed to create tap
|
||||||
|
// due to format mismatch". `.record` skips the output
|
||||||
|
// side entirely, so the simulator can record.
|
||||||
|
//
|
||||||
|
// The real keyboard extension (`OSGKeyboardExt`) keeps
|
||||||
|
// `.playAndRecord` because it runs on a real device where
|
||||||
|
// the output side has a real hardware format, and may want
|
||||||
|
// to play click sounds / haptic feedback. Only the preview
|
||||||
|
// needs the simulator-friendly category.
|
||||||
if !didConfigureAudioSession {
|
if !didConfigureAudioSession {
|
||||||
do {
|
do {
|
||||||
let session = AVAudioSession.sharedInstance()
|
let session = AVAudioSession.sharedInstance()
|
||||||
try session.setCategory(.playAndRecord,
|
try session.setCategory(.record,
|
||||||
mode: .measurement,
|
mode: .measurement,
|
||||||
options: [.defaultToSpeaker, .allowBluetoothHFP])
|
options: [])
|
||||||
try session.setActive(true, options: .notifyOthersOnDeactivation)
|
try session.setActive(true, options: .notifyOthersOnDeactivation)
|
||||||
didConfigureAudioSession = true
|
didConfigureAudioSession = true
|
||||||
} catch {
|
} catch {
|
||||||
@@ -187,6 +205,25 @@ final class PreviewASRController: ObservableObject {
|
|||||||
private func startEngineAndASR(locale: Locale) {
|
private func startEngineAndASR(locale: Locale) {
|
||||||
let inputNode = audioEngine.inputNode
|
let inputNode = audioEngine.inputNode
|
||||||
let hwFormat = inputNode.outputFormat(forBus: 0)
|
let hwFormat = inputNode.outputFormat(forBus: 0)
|
||||||
|
|
||||||
|
// Pre-flight check: a placeholder / unconfigured input bus
|
||||||
|
// reports `sampleRate == 0` (or `channelCount == 0`).
|
||||||
|
// `installTap` on such a bus traps with "Failed to create
|
||||||
|
// tap due to format mismatch" (an NSException, not a Swift
|
||||||
|
// `Error`, so we can't `try`/`catch` it). The safest fix
|
||||||
|
// is to refuse the tap up front and surface a clear
|
||||||
|
// `.error` phase instead of crashing the app. We've seen
|
||||||
|
// this on the iOS Simulator when the host's microphone
|
||||||
|
// permission isn't granted to CoreSimulator, and on
|
||||||
|
// devices where the audio session is in an unexpected
|
||||||
|
// state from a previous foreground/background transition.
|
||||||
|
guard hwFormat.sampleRate > 0, hwFormat.channelCount > 0 else {
|
||||||
|
phase = .error(
|
||||||
|
"麦克风不可用 · Microphone unavailable (hw format \(hwFormat.sampleRate) Hz / \(hwFormat.channelCount) ch)"
|
||||||
|
)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
let targetSampleRate: Double = 16_000
|
let targetSampleRate: Double = 16_000
|
||||||
guard let targetFormat = AVAudioFormat(
|
guard let targetFormat = AVAudioFormat(
|
||||||
commonFormat: .pcmFormatFloat32,
|
commonFormat: .pcmFormatFloat32,
|
||||||
|
|||||||
Reference in New Issue
Block a user