[P0-②] On-device ASR three-piece set

- Add NSSpeechRecognitionUsageDescription to both target Info.plists
- KeyboardViewController.requestSpeechPermission() wraps
  SFSpeechRecognizer.requestAuthorization
- pressBegan() now requests Speech permission right after mic permission
- ASRService emits a DEBUG warning when recognizer.supportsOnDeviceRecognition
  is false so cloud fallback is visible during dev
- CHANGELOG: move iOS 26 SpeechAnalyzer out of Unreleased into [0.2.0] Planned,
  so the iOS 18 SFSpeechRecognizer path is the only ASR shipped in v0.1

xcodebuild iOS Simulator: SUCCEEDED
This commit is contained in:
Zhongshu
2026-06-18 10:41:20 +08:00
parent e0b92ff35a
commit e93bab50b4
5 changed files with 68 additions and 15 deletions
+24 -13
View File
@@ -7,26 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Fixed
- **Theme follows system appearance**: main App now renders a true light palette in light mode via `ThemedRoot` + `EnvironmentKey<ThemePalette>`. The keyboard extension deliberately stays dark (Apple's default) and now uses a transparent `.background(Color.clear)` so the system UI chrome shows through.
- **Speech Recognition permission requested on first press**: added `NSSpeechRecognitionUsageDescription` to both targets' `Info.plist` and an explicit `SFSpeechRecognizer.requestAuthorization` call inside `pressBegan()`. Without these the iOS 18 ASR path silently returned `.denied` and the user heard nothing.
- **ASRService emits a DEBUG warning when on-device recognition isn't supported**, so it's obvious during dev that the request fell back to cloud.
- **App Group fallback behaviour**:
- In `DEBUG`, a missing App Group now `fatalError`s with a precise remediation message (was a soft print + `.standard` fallback, which desynced the keyboard extension from the main App).
- In release, the fallback is preserved but logged via `NSLog`.
- **`KeyboardViewController.loadPersistedLocale` self-check** (DEBUG only) prints the active provider / baseURL / masked API key / mode / locale, so the keyboard extension's view of the App Group is visible in the device console.
- **`KeyboardViewController.handleFinalTranscript` typed-error routing**: `LLMError.noAPIKey` (401) and `LLMError.http(429)` now surface as red, explicit error messages instead of silently inserting the raw transcript. Network / timeout errors still fall back to the raw transcript + error badge (no data loss).
- **`PolishingService` timeout** raised from 12 s to 15 s to align with `LLMClient`'s URL request timeout. Previously the polisher would race the network call and discard a successful response that arrived in the 1215 s window.
- **DEBUG `print` cleanup**: the four `🔥 [OSGKeyboardApp] …` instrumentation prints in `OSGKeyboardApp.init()` and root views are now wrapped in `#if DEBUG`.
### Added ### Added
- Initial open-source release. - "Test connection" button in `APISettingsCard`: fires a single `polish("ping")` round-trip and surfaces success or the typed LLM error inline. Helps the user confirm the App Group + key are working without leaving the main App.
- Custom Keyboard Extension with push-to-talk UI. - 5 new unit tests in `OSGKeyboardTests`: App Group cross-process persistence, 401 / 429 / timeout / noAPIKey catch paths, and `mode = .off` short-circuit.
- iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR.
- iOS 18 `SFSpeechRecognizer` fallback path (on-device only). ### Changed
- OpenAI-compatible LLM client (BaseURL / API Key / Model / System Prompt user-editable). - README + `README.zh.md`: replaced `<OWNER>` placeholder with `hkgood` and rephrased the iOS 26 `SpeechAnalyzer` line as "planned for the next release" (the iOS 18 `SFSpeechRecognizer` path remains the only working ASR for v0.1).
- Built-in presets: OpenAI, DeepSeek, Qwen DashScope, Custom.
- Three-page onboarding (welcome → enable keyboard → API config).
- `ProviderConfig` persisted in App Group `group.com.osgkeyboard.ios`.
- 8-second LLM call timeout with graceful fallback to raw transcript.
- App Store privacy manifest (`PrivacyInfo.xcprivacy`) for both targets.
- SwiftLint config, XcodeGen project definition, GitHub Actions CI.
- Unit tests for `ProviderConfig` and `OpenAICompatibleClient`.
### Known limitations ### Known limitations
- 2 failing tests stub state pollution was fixed in this release; regression coverage in place. - iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR is planned for **0.2.0** (moved out of Unreleased scope to keep the v0.1 release honest about what ships).
- The keyboard does not work in password fields (iOS limitation). - The keyboard does not work in password fields (iOS limitation).
- Microphone requires "Allow Full Access" to be enabled in iOS Settings. - Microphone requires "Allow Full Access" to be enabled in iOS Settings.
- Whisper.cpp / on-device LLM polish is intentionally out of scope for v1 (cloud-only). - Whisper.cpp / on-device LLM polish is intentionally out of scope for v1 (cloud-only).
## [0.2.0] - Planned
### Added
- iOS 26+ `SpeechAnalyzer` + `DictationTranscriber` on-device ASR (lower latency, more locales).
- Bilingual UI (中文 / English) driven by a real `Localizable.strings` table; will land alongside a Settings → Language picker.
## [0.1.0] - 2026-06-17 ## [0.1.0] - 2026-06-17
### Added ### Added
+2
View File
@@ -38,6 +38,8 @@
</dict> </dict>
<key>NSMicrophoneUsageDescription</key> <key>NSMicrophoneUsageDescription</key>
<string>OSGKeyboard needs microphone access to transcribe your voice into text.</string> <string>OSGKeyboard needs microphone access to transcribe your voice into text.</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>OSGKeyboard uses on-device speech recognition to transcribe your dictation. Audio never leaves your device.</string>
<key>UIApplicationSceneManifest</key> <key>UIApplicationSceneManifest</key>
<dict> <dict>
<key>UIApplicationSupportsMultipleScenes</key> <key>UIApplicationSupportsMultipleScenes</key>
+2
View File
@@ -20,6 +20,8 @@
<string>$(MARKETING_VERSION)</string> <string>$(MARKETING_VERSION)</string>
<key>CFBundleVersion</key> <key>CFBundleVersion</key>
<string>$(CURRENT_PROJECT_VERSION)</string> <string>$(CURRENT_PROJECT_VERSION)</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>OSGKeyboard uses on-device speech recognition to transcribe your dictation. Audio never leaves your device.</string>
<key>NSExtension</key> <key>NSExtension</key>
<dict> <dict>
<key>NSExtensionAttributes</key> <key>NSExtensionAttributes</key>
+35 -2
View File
@@ -20,6 +20,7 @@
import UIKit import UIKit
import SwiftUI import SwiftUI
import AVFoundation import AVFoundation
import Speech
import OSGKeyboardShared import OSGKeyboardShared
@objc(KeyboardViewController) @objc(KeyboardViewController)
@@ -199,12 +200,26 @@ public final class KeyboardViewController: UIInputViewController {
// user already feels the press registered. // user already feels the press registered.
Task { @MainActor [weak self] in Task { @MainActor [weak self] in
guard let self else { return } guard let self else { return }
let granted = await self.requestMicPermission() let micGranted = await self.requestMicPermission()
guard granted else { guard micGranted else {
self.state.phase = .error("麦克风被拒绝,请到「设置」中允许") self.state.phase = .error("麦克风被拒绝,请到「设置」中允许")
self.scheduleAutoClearError() self.scheduleAutoClearError()
return return
} }
// iOS 18 SFSpeechRecognizer path: we explicitly ask for Speech
// recognition permission. Without this call + the
// NSSpeechRecognitionUsageDescription key in Info.plist the
// recogniser silently returns .denied and the user hears
// nothing back.
// iOS 26 SpeechAnalyzer path (planned for the next release)
// does not expose an explicit request API the framework
// prompts via the same plist key on first use.
let speechGranted = await self.requestSpeechPermission()
guard speechGranted else {
self.state.phase = .error("语音识别被拒绝,请到「设置」中允许")
self.scheduleAutoClearError()
return
}
self.startPipeline() self.startPipeline()
} }
} }
@@ -348,6 +363,24 @@ public final class KeyboardViewController: UIInputViewController {
} }
} }
/// Request Speech Recognition permission. Returns true if granted
/// (or already authorised). For the iOS 18 SFSpeechRecognizer path
/// this is required before recognition can begin; for the iOS 26
/// SpeechAnalyzer path the framework prompts on first use.
private func requestSpeechPermission() async -> Bool {
await withCheckedContinuation { (cont: CheckedContinuation<Bool, Never>) in
SFSpeechRecognizer.requestAuthorization { status in
switch status {
case .authorized: cont.resume(returning: true)
case .denied, .restricted, .notDetermined:
cont.resume(returning: false)
@unknown default:
cont.resume(returning: false)
}
}
}
}
// MARK: - Open host app // MARK: - Open host app
private func openHostApp() { private func openHostApp() {
+5
View File
@@ -73,6 +73,11 @@ final class AppleSpeechASR: ASRService, @unchecked Sendable {
let request = SFSpeechAudioBufferRecognitionRequest() let request = SFSpeechAudioBufferRecognitionRequest()
request.shouldReportPartialResults = true request.shouldReportPartialResults = true
request.requiresOnDeviceRecognition = recognizer.supportsOnDeviceRecognition request.requiresOnDeviceRecognition = recognizer.supportsOnDeviceRecognition
if !recognizer.supportsOnDeviceRecognition {
#if DEBUG
print("⚠️ 设备不支持 \(locale.identifier) 端侧 ASR, 回退云端。")
#endif
}
let task = recognizer.recognitionTask(with: request) { result, error in let task = recognizer.recognitionTask(with: request) { result, error in
if let error { if let error {