[P0-②] On-device ASR three-piece set
- Add NSSpeechRecognitionUsageDescription to both target Info.plists - KeyboardViewController.requestSpeechPermission() wraps SFSpeechRecognizer.requestAuthorization - pressBegan() now requests Speech permission right after mic permission - ASRService emits a DEBUG warning when recognizer.supportsOnDeviceRecognition is false so cloud fallback is visible during dev - CHANGELOG: move iOS 26 SpeechAnalyzer out of Unreleased into [0.2.0] Planned, so the iOS 18 SFSpeechRecognizer path is the only ASR shipped in v0.1 xcodebuild iOS Simulator: SUCCEEDED
This commit is contained in:
Zhongshu
+24
-13
@@ -7,26 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **Theme follows system appearance**: main App now renders a true light palette in light mode via `ThemedRoot` + `EnvironmentKey<ThemePalette>`. The keyboard extension deliberately stays dark (Apple's default) and now uses a transparent `.background(Color.clear)` so the system UI chrome shows through.
|
||||
- **Speech Recognition permission requested on first press**: added `NSSpeechRecognitionUsageDescription` to both targets' `Info.plist` and an explicit `SFSpeechRecognizer.requestAuthorization` call inside `pressBegan()`. Without these the iOS 18 ASR path silently returned `.denied` and the user heard nothing.
|
||||
- **ASRService emits a DEBUG warning when on-device recognition isn't supported**, so it's obvious during dev that the request fell back to cloud.
|
||||
- **App Group fallback behaviour**:
|
||||
- In `DEBUG`, a missing App Group now `fatalError`s with a precise remediation message (was a soft print + `.standard` fallback, which desynced the keyboard extension from the main App).
|
||||
- In release, the fallback is preserved but logged via `NSLog`.
|
||||
- **`KeyboardViewController.loadPersistedLocale` self-check** (DEBUG only) prints the active provider / baseURL / masked API key / mode / locale, so the keyboard extension's view of the App Group is visible in the device console.
|
||||
- **`KeyboardViewController.handleFinalTranscript` typed-error routing**: `LLMError.noAPIKey` (401) and `LLMError.http(429)` now surface as red, explicit error messages instead of silently inserting the raw transcript. Network / timeout errors still fall back to the raw transcript + error badge (no data loss).
|
||||
- **`PolishingService` timeout** raised from 12 s to 15 s to align with `LLMClient`'s URL request timeout. Previously the polisher would race the network call and discard a successful response that arrived in the 12–15 s window.
|
||||
- **DEBUG `print` cleanup**: the four `🔥 [OSGKeyboardApp] …` instrumentation prints in `OSGKeyboardApp.init()` and root views are now wrapped in `#if DEBUG`.
|
||||
|
||||
### Added
|
||||
- Initial open-source release.
|
||||
- Custom Keyboard Extension with push-to-talk UI.
|
||||
- iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR.
|
||||
- iOS 18 `SFSpeechRecognizer` fallback path (on-device only).
|
||||
- OpenAI-compatible LLM client (BaseURL / API Key / Model / System Prompt user-editable).
|
||||
- Built-in presets: OpenAI, DeepSeek, Qwen DashScope, Custom.
|
||||
- Three-page onboarding (welcome → enable keyboard → API config).
|
||||
- `ProviderConfig` persisted in App Group `group.com.osgkeyboard.ios`.
|
||||
- 8-second LLM call timeout with graceful fallback to raw transcript.
|
||||
- App Store privacy manifest (`PrivacyInfo.xcprivacy`) for both targets.
|
||||
- SwiftLint config, XcodeGen project definition, GitHub Actions CI.
|
||||
- Unit tests for `ProviderConfig` and `OpenAICompatibleClient`.
|
||||
- "Test connection" button in `APISettingsCard`: fires a single `polish("ping")` round-trip and surfaces success or the typed LLM error inline. Helps the user confirm the App Group + key are working without leaving the main App.
|
||||
- 5 new unit tests in `OSGKeyboardTests`: App Group cross-process persistence, 401 / 429 / timeout / noAPIKey catch paths, and `mode = .off` short-circuit.
|
||||
|
||||
### Changed
|
||||
- README + `README.zh.md`: replaced `<OWNER>` placeholder with `hkgood` and rephrased the iOS 26 `SpeechAnalyzer` line as "planned for the next release" (the iOS 18 `SFSpeechRecognizer` path remains the only working ASR for v0.1).
|
||||
|
||||
### Known limitations
|
||||
- 2 failing tests stub state pollution was fixed in this release; regression coverage in place.
|
||||
- iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR is planned for **0.2.0** (moved out of Unreleased scope to keep the v0.1 release honest about what ships).
|
||||
- The keyboard does not work in password fields (iOS limitation).
|
||||
- Microphone requires "Allow Full Access" to be enabled in iOS Settings.
|
||||
- Whisper.cpp / on-device LLM polish is intentionally out of scope for v1 (cloud-only).
|
||||
|
||||
## [0.2.0] - Planned
|
||||
|
||||
### Added
|
||||
- iOS 26+ `SpeechAnalyzer` + `DictationTranscriber` on-device ASR (lower latency, more locales).
|
||||
- Bilingual UI (中文 / English) driven by a real `Localizable.strings` table; will land alongside a Settings → Language picker.
|
||||
|
||||
## [0.1.0] - 2026-06-17
|
||||
|
||||
### Added
|
||||
|
||||
@@ -38,6 +38,8 @@
|
||||
</dict>
|
||||
<key>NSMicrophoneUsageDescription</key>
|
||||
<string>OSGKeyboard needs microphone access to transcribe your voice into text.</string>
|
||||
<key>NSSpeechRecognitionUsageDescription</key>
|
||||
<string>OSGKeyboard uses on-device speech recognition to transcribe your dictation. Audio never leaves your device.</string>
|
||||
<key>UIApplicationSceneManifest</key>
|
||||
<dict>
|
||||
<key>UIApplicationSupportsMultipleScenes</key>
|
||||
|
||||
@@ -20,6 +20,8 @@
|
||||
<string>$(MARKETING_VERSION)</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>$(CURRENT_PROJECT_VERSION)</string>
|
||||
<key>NSSpeechRecognitionUsageDescription</key>
|
||||
<string>OSGKeyboard uses on-device speech recognition to transcribe your dictation. Audio never leaves your device.</string>
|
||||
<key>NSExtension</key>
|
||||
<dict>
|
||||
<key>NSExtensionAttributes</key>
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
import UIKit
|
||||
import SwiftUI
|
||||
import AVFoundation
|
||||
import Speech
|
||||
import OSGKeyboardShared
|
||||
|
||||
@objc(KeyboardViewController)
|
||||
@@ -199,12 +200,26 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
// user already feels the press registered.
|
||||
Task { @MainActor [weak self] in
|
||||
guard let self else { return }
|
||||
let granted = await self.requestMicPermission()
|
||||
guard granted else {
|
||||
let micGranted = await self.requestMicPermission()
|
||||
guard micGranted else {
|
||||
self.state.phase = .error("麦克风被拒绝,请到「设置」中允许")
|
||||
self.scheduleAutoClearError()
|
||||
return
|
||||
}
|
||||
// iOS 18 SFSpeechRecognizer path: we explicitly ask for Speech
|
||||
// recognition permission. Without this call + the
|
||||
// NSSpeechRecognitionUsageDescription key in Info.plist the
|
||||
// recogniser silently returns .denied and the user hears
|
||||
// nothing back.
|
||||
// iOS 26 SpeechAnalyzer path (planned for the next release)
|
||||
// does not expose an explicit request API — the framework
|
||||
// prompts via the same plist key on first use.
|
||||
let speechGranted = await self.requestSpeechPermission()
|
||||
guard speechGranted else {
|
||||
self.state.phase = .error("语音识别被拒绝,请到「设置」中允许")
|
||||
self.scheduleAutoClearError()
|
||||
return
|
||||
}
|
||||
self.startPipeline()
|
||||
}
|
||||
}
|
||||
@@ -348,6 +363,24 @@ public final class KeyboardViewController: UIInputViewController {
|
||||
}
|
||||
}
|
||||
|
||||
/// Request Speech Recognition permission. Returns true if granted
|
||||
/// (or already authorised). For the iOS 18 SFSpeechRecognizer path
|
||||
/// this is required before recognition can begin; for the iOS 26
|
||||
/// SpeechAnalyzer path the framework prompts on first use.
|
||||
private func requestSpeechPermission() async -> Bool {
|
||||
await withCheckedContinuation { (cont: CheckedContinuation<Bool, Never>) in
|
||||
SFSpeechRecognizer.requestAuthorization { status in
|
||||
switch status {
|
||||
case .authorized: cont.resume(returning: true)
|
||||
case .denied, .restricted, .notDetermined:
|
||||
cont.resume(returning: false)
|
||||
@unknown default:
|
||||
cont.resume(returning: false)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Open host app
|
||||
|
||||
private func openHostApp() {
|
||||
|
||||
@@ -73,6 +73,11 @@ final class AppleSpeechASR: ASRService, @unchecked Sendable {
|
||||
let request = SFSpeechAudioBufferRecognitionRequest()
|
||||
request.shouldReportPartialResults = true
|
||||
request.requiresOnDeviceRecognition = recognizer.supportsOnDeviceRecognition
|
||||
if !recognizer.supportsOnDeviceRecognition {
|
||||
#if DEBUG
|
||||
print("⚠️ 设备不支持 \(locale.identifier) 端侧 ASR, 回退云端。")
|
||||
#endif
|
||||
}
|
||||
|
||||
let task = recognizer.recognitionTask(with: request) { result, error in
|
||||
if let error {
|
||||
|
||||
Reference in New Issue
Block a user