[P0-②] On-device ASR three-piece set
- Add NSSpeechRecognitionUsageDescription to both target Info.plists - KeyboardViewController.requestSpeechPermission() wraps SFSpeechRecognizer.requestAuthorization - pressBegan() now requests Speech permission right after mic permission - ASRService emits a DEBUG warning when recognizer.supportsOnDeviceRecognition is false so cloud fallback is visible during dev - CHANGELOG: move iOS 26 SpeechAnalyzer out of Unreleased into [0.2.0] Planned, so the iOS 18 SFSpeechRecognizer path is the only ASR shipped in v0.1 xcodebuild iOS Simulator: SUCCEEDED
This commit is contained in:
+24
-13
@@ -7,26 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **Theme follows system appearance**: main App now renders a true light palette in light mode via `ThemedRoot` + `EnvironmentKey<ThemePalette>`. The keyboard extension deliberately stays dark (Apple's default) and now uses a transparent `.background(Color.clear)` so the system UI chrome shows through.
|
||||||
|
- **Speech Recognition permission requested on first press**: added `NSSpeechRecognitionUsageDescription` to both targets' `Info.plist` and an explicit `SFSpeechRecognizer.requestAuthorization` call inside `pressBegan()`. Without these the iOS 18 ASR path silently returned `.denied` and the user heard nothing.
|
||||||
|
- **ASRService emits a DEBUG warning when on-device recognition isn't supported**, so it's obvious during dev that the request fell back to cloud.
|
||||||
|
- **App Group fallback behaviour**:
|
||||||
|
- In `DEBUG`, a missing App Group now `fatalError`s with a precise remediation message (was a soft print + `.standard` fallback, which desynced the keyboard extension from the main App).
|
||||||
|
- In release, the fallback is preserved but logged via `NSLog`.
|
||||||
|
- **`KeyboardViewController.loadPersistedLocale` self-check** (DEBUG only) prints the active provider / baseURL / masked API key / mode / locale, so the keyboard extension's view of the App Group is visible in the device console.
|
||||||
|
- **`KeyboardViewController.handleFinalTranscript` typed-error routing**: `LLMError.noAPIKey` (401) and `LLMError.http(429)` now surface as red, explicit error messages instead of silently inserting the raw transcript. Network / timeout errors still fall back to the raw transcript + error badge (no data loss).
|
||||||
|
- **`PolishingService` timeout** raised from 12 s to 15 s to align with `LLMClient`'s URL request timeout. Previously the polisher would race the network call and discard a successful response that arrived in the 12–15 s window.
|
||||||
|
- **DEBUG `print` cleanup**: the four `🔥 [OSGKeyboardApp] …` instrumentation prints in `OSGKeyboardApp.init()` and root views are now wrapped in `#if DEBUG`.
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- Initial open-source release.
|
- "Test connection" button in `APISettingsCard`: fires a single `polish("ping")` round-trip and surfaces success or the typed LLM error inline. Helps the user confirm the App Group + key are working without leaving the main App.
|
||||||
- Custom Keyboard Extension with push-to-talk UI.
|
- 5 new unit tests in `OSGKeyboardTests`: App Group cross-process persistence, 401 / 429 / timeout / noAPIKey catch paths, and `mode = .off` short-circuit.
|
||||||
- iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR.
|
|
||||||
- iOS 18 `SFSpeechRecognizer` fallback path (on-device only).
|
### Changed
|
||||||
- OpenAI-compatible LLM client (BaseURL / API Key / Model / System Prompt user-editable).
|
- README + `README.zh.md`: replaced `<OWNER>` placeholder with `hkgood` and rephrased the iOS 26 `SpeechAnalyzer` line as "planned for the next release" (the iOS 18 `SFSpeechRecognizer` path remains the only working ASR for v0.1).
|
||||||
- Built-in presets: OpenAI, DeepSeek, Qwen DashScope, Custom.
|
|
||||||
- Three-page onboarding (welcome → enable keyboard → API config).
|
|
||||||
- `ProviderConfig` persisted in App Group `group.com.osgkeyboard.ios`.
|
|
||||||
- 8-second LLM call timeout with graceful fallback to raw transcript.
|
|
||||||
- App Store privacy manifest (`PrivacyInfo.xcprivacy`) for both targets.
|
|
||||||
- SwiftLint config, XcodeGen project definition, GitHub Actions CI.
|
|
||||||
- Unit tests for `ProviderConfig` and `OpenAICompatibleClient`.
|
|
||||||
|
|
||||||
### Known limitations
|
### Known limitations
|
||||||
- 2 failing tests stub state pollution was fixed in this release; regression coverage in place.
|
- iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR is planned for **0.2.0** (moved out of Unreleased scope to keep the v0.1 release honest about what ships).
|
||||||
- The keyboard does not work in password fields (iOS limitation).
|
- The keyboard does not work in password fields (iOS limitation).
|
||||||
- Microphone requires "Allow Full Access" to be enabled in iOS Settings.
|
- Microphone requires "Allow Full Access" to be enabled in iOS Settings.
|
||||||
- Whisper.cpp / on-device LLM polish is intentionally out of scope for v1 (cloud-only).
|
- Whisper.cpp / on-device LLM polish is intentionally out of scope for v1 (cloud-only).
|
||||||
|
|
||||||
|
## [0.2.0] - Planned
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- iOS 26+ `SpeechAnalyzer` + `DictationTranscriber` on-device ASR (lower latency, more locales).
|
||||||
|
- Bilingual UI (中文 / English) driven by a real `Localizable.strings` table; will land alongside a Settings → Language picker.
|
||||||
|
|
||||||
## [0.1.0] - 2026-06-17
|
## [0.1.0] - 2026-06-17
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
@@ -38,6 +38,8 @@
|
|||||||
</dict>
|
</dict>
|
||||||
<key>NSMicrophoneUsageDescription</key>
|
<key>NSMicrophoneUsageDescription</key>
|
||||||
<string>OSGKeyboard needs microphone access to transcribe your voice into text.</string>
|
<string>OSGKeyboard needs microphone access to transcribe your voice into text.</string>
|
||||||
|
<key>NSSpeechRecognitionUsageDescription</key>
|
||||||
|
<string>OSGKeyboard uses on-device speech recognition to transcribe your dictation. Audio never leaves your device.</string>
|
||||||
<key>UIApplicationSceneManifest</key>
|
<key>UIApplicationSceneManifest</key>
|
||||||
<dict>
|
<dict>
|
||||||
<key>UIApplicationSupportsMultipleScenes</key>
|
<key>UIApplicationSupportsMultipleScenes</key>
|
||||||
|
|||||||
@@ -20,6 +20,8 @@
|
|||||||
<string>$(MARKETING_VERSION)</string>
|
<string>$(MARKETING_VERSION)</string>
|
||||||
<key>CFBundleVersion</key>
|
<key>CFBundleVersion</key>
|
||||||
<string>$(CURRENT_PROJECT_VERSION)</string>
|
<string>$(CURRENT_PROJECT_VERSION)</string>
|
||||||
|
<key>NSSpeechRecognitionUsageDescription</key>
|
||||||
|
<string>OSGKeyboard uses on-device speech recognition to transcribe your dictation. Audio never leaves your device.</string>
|
||||||
<key>NSExtension</key>
|
<key>NSExtension</key>
|
||||||
<dict>
|
<dict>
|
||||||
<key>NSExtensionAttributes</key>
|
<key>NSExtensionAttributes</key>
|
||||||
|
|||||||
@@ -20,6 +20,7 @@
|
|||||||
import UIKit
|
import UIKit
|
||||||
import SwiftUI
|
import SwiftUI
|
||||||
import AVFoundation
|
import AVFoundation
|
||||||
|
import Speech
|
||||||
import OSGKeyboardShared
|
import OSGKeyboardShared
|
||||||
|
|
||||||
@objc(KeyboardViewController)
|
@objc(KeyboardViewController)
|
||||||
@@ -199,12 +200,26 @@ public final class KeyboardViewController: UIInputViewController {
|
|||||||
// user already feels the press registered.
|
// user already feels the press registered.
|
||||||
Task { @MainActor [weak self] in
|
Task { @MainActor [weak self] in
|
||||||
guard let self else { return }
|
guard let self else { return }
|
||||||
let granted = await self.requestMicPermission()
|
let micGranted = await self.requestMicPermission()
|
||||||
guard granted else {
|
guard micGranted else {
|
||||||
self.state.phase = .error("麦克风被拒绝,请到「设置」中允许")
|
self.state.phase = .error("麦克风被拒绝,请到「设置」中允许")
|
||||||
self.scheduleAutoClearError()
|
self.scheduleAutoClearError()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
// iOS 18 SFSpeechRecognizer path: we explicitly ask for Speech
|
||||||
|
// recognition permission. Without this call + the
|
||||||
|
// NSSpeechRecognitionUsageDescription key in Info.plist the
|
||||||
|
// recogniser silently returns .denied and the user hears
|
||||||
|
// nothing back.
|
||||||
|
// iOS 26 SpeechAnalyzer path (planned for the next release)
|
||||||
|
// does not expose an explicit request API — the framework
|
||||||
|
// prompts via the same plist key on first use.
|
||||||
|
let speechGranted = await self.requestSpeechPermission()
|
||||||
|
guard speechGranted else {
|
||||||
|
self.state.phase = .error("语音识别被拒绝,请到「设置」中允许")
|
||||||
|
self.scheduleAutoClearError()
|
||||||
|
return
|
||||||
|
}
|
||||||
self.startPipeline()
|
self.startPipeline()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -348,6 +363,24 @@ public final class KeyboardViewController: UIInputViewController {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Request Speech Recognition permission. Returns true if granted
|
||||||
|
/// (or already authorised). For the iOS 18 SFSpeechRecognizer path
|
||||||
|
/// this is required before recognition can begin; for the iOS 26
|
||||||
|
/// SpeechAnalyzer path the framework prompts on first use.
|
||||||
|
private func requestSpeechPermission() async -> Bool {
|
||||||
|
await withCheckedContinuation { (cont: CheckedContinuation<Bool, Never>) in
|
||||||
|
SFSpeechRecognizer.requestAuthorization { status in
|
||||||
|
switch status {
|
||||||
|
case .authorized: cont.resume(returning: true)
|
||||||
|
case .denied, .restricted, .notDetermined:
|
||||||
|
cont.resume(returning: false)
|
||||||
|
@unknown default:
|
||||||
|
cont.resume(returning: false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// MARK: - Open host app
|
// MARK: - Open host app
|
||||||
|
|
||||||
private func openHostApp() {
|
private func openHostApp() {
|
||||||
|
|||||||
@@ -73,6 +73,11 @@ final class AppleSpeechASR: ASRService, @unchecked Sendable {
|
|||||||
let request = SFSpeechAudioBufferRecognitionRequest()
|
let request = SFSpeechAudioBufferRecognitionRequest()
|
||||||
request.shouldReportPartialResults = true
|
request.shouldReportPartialResults = true
|
||||||
request.requiresOnDeviceRecognition = recognizer.supportsOnDeviceRecognition
|
request.requiresOnDeviceRecognition = recognizer.supportsOnDeviceRecognition
|
||||||
|
if !recognizer.supportsOnDeviceRecognition {
|
||||||
|
#if DEBUG
|
||||||
|
print("⚠️ 设备不支持 \(locale.identifier) 端侧 ASR, 回退云端。")
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
let task = recognizer.recognitionTask(with: request) { result, error in
|
let task = recognizer.recognitionTask(with: request) { result, error in
|
||||||
if let error {
|
if let error {
|
||||||
|
|||||||
Reference in New Issue
Block a user