rocky/OSGKeyboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: drop Qwen3 CoreML ASR, add local-engine cloud polish toggle

Browse Source 
Rolls back the v0.2.0 Qwen3 CoreML on-device ASR stack and replaces the
'local engine' UX with iOS 26 SpeechAnalyzer + DictationTranscriber only.

The 'Cloud polish after ASR' toggle (ProviderConfig.localModeCloudPolishEnabled)
lets users opt into a post-ASR DeepSeek round-trip from the local engine.
Defaults to off so the local engine stays genuinely local. New PolishError.missingAPIError
surfaces an inline 'fill in your key' warning when the toggle is on but the
Keychain is empty. DeepSeek preset default model bumped to deepseek-v4-flash.

Deleted:
  - OSGKeyboard/ThirdParty/Qwen3Speech/ (74 files, ~16k LoC)
  - OSGKeyboard/Services/ModelManager.swift (492)
  - OSGKeyboard/Services/OnDeviceModelWarmup.swift (197)
  - OSGKeyboard/Services/Qwen3ASRService.swift (257)
  - OSGKeyboard/Services/ModelDownloadSourcePicker.swift (126)
  - OSGKeyboard/Views/OnDeviceModelsView.swift (184)
  - OSGKeyboard/Views/DownloadConfirmSheet.swift (96)
  - OSGKeyboardShared/Models/OnDeviceModel.swift (140)
  - OSGKeyboardShared/Services/OnDeviceModelStatus.swift (104)
  - Qwen3ASRServiceProvider registration in OSGKeyboardApp
  - Qwen3Speech package declaration in project.yml
  - 5 .qwen3ASR enum / branch reference sites in HomeView, OnboardingView,
    LocalEngineSettingsRows, FlowSessionManager, ASRService, EngineServiceLabel
  - Two pre-existing Swift 6 strict-concurrency errors in
    LiveDictationController + FlowSessionManager (the weak [weak self] in
    detached-task MainActor.run blocks) that were blocking clean builds

Added:
  - LocalModelsGroup: 'Built-in iOS SpeechAnalyzer' badge + 'Cloud polish
    after ASR' Switch toggle
  - PolishingService: honour localModeCloudPolishEnabled; new .missingAPIKey
    error case with localised warning
  - AppGroupStore.localModeCloudPolishEnabled (mirrored into App Group
    so the keyboard extension honours the toggle during live dictation)
  - SettingsView: show provider/api sections when local-mode cloud polish
    is on so the user can paste a DeepSeek key
  - FlowSessionManager: route through PolishingService for local + polish-on
    flow; translate missingAPIKey into a polished warning
  - KeyboardViewController: handle PolishingService.PolishError.missingAPIKey
    in the keyboard-side live polish path
  - CHANGELOG v0.2.1: documents the rollback + new toggle
  - README.md / README.zh.md: engine matrix section, data flow note

Verified: xcodebuild -scheme OSGKeyboard -destination 'generic/platform=iOS Simulator'
build succeeds under SWIFT_STRICT_CONCURRENCY=complete.
This commit is contained in:
Rocky
2026-06-24 01:51:34 +08:00
parent 39690c0a93
commit c07cf4db9f
119 changed files with 401 additions and 18858 deletions
Download Patch File Download Diff File Expand all files Collapse all files
OSGKeyboardShared/Services/ASRService.swift
+10 -46
View File
@@ -116,68 +116,32 @@ public enum ASREvent: Sendable, Equatable {
// MARK: - Factory
public enum ASRServiceFactory {
/// Registry of backend-specific providers. The host app installs
/// a provider for the Qwen3-ASR backend at launch time (the
/// `Qwen3ASRProvider` lives in the app target because linking
/// `Qwen3ASR` pulls in mlx-swift, which the shared framework
/// deliberately stays off to keep `APPLICATION_EXTENSION_API_ONLY`
/// clean). The shared framework always provides a built-in
/// `SpeechAnalyzer` provider; custom providers override it.
/// Returns the on-device ASR backend. As of v0.2.0 the only
/// supported `LocalASRBackend` is iOS 26 `SpeechAnalyzer` +
/// `DictationTranscriber` (always on-device, no asset download),
/// so the factory collapses to a single concrete type. We keep the
/// `localBackend` parameter on the signature so the next non-iOS
/// backend can slot in without touching every call site.
///
/// `nonisolated(unsafe)` because the only writer is
/// `OSGKeyboardApp.init` (single-threaded, runs once at launch).
/// After launch, all callers read the dictionary from any
/// actor.
public nonisolated(unsafe) static var providers: [LocalASRBackend: any ASRServiceProvider] = [
.speechAnalyzer: SpeechAnalyzerProvider()
]
/// Returns the ASR backend chosen by the user. The cloud engine
/// always uses the iOS `SpeechAnalyzer` path — it has the lowest
/// latency and never hits the network, which matches the user's
/// expectation that "ASR" is the local half of the pipeline
/// The cloud engine also routes through `SpeechAnalyzerASR`: the
/// user expectation is that ASR is the local half of the pipeline
/// regardless of where the LLM polish happens.
///
/// For the local engine, we honour `LocalASRBackend`:
/// - `.speechAnalyzer` (default) → on-device iOS pipeline.
/// - `.qwen3ASR` → CoreML-backed Qwen3-ASR via `soniqo/speech-swift` (host app only)
/// (registered by the host app at launch).
public static func make(
engineMode: String,
localBackend: LocalASRBackend = .speechAnalyzer
) -> ASRService {
if engineMode == "local" {
if let provider = providers[localBackend] {
return provider.make()
}
}
return SpeechAnalyzerASR()
SpeechAnalyzerASR()
}
/// Back-compat overload for callers that only ever want the
/// SpeechAnalyzer path. The previous single-backend build used
/// this signature; new code should pass the engine mode explicitly
/// so the user's selection is honoured.
/// so any future non-iOS backend is honoured.
public static func make() -> ASRService {
SpeechAnalyzerASR()
}
}
/// Backend-specific ASR factory. The shared framework ships a default
/// `SpeechAnalyzerProvider`; the host app installs a `Qwen3ASRProvider`
/// at launch time so the Qwen3 backend is wired in only where its
/// large MLX dependency is also linked.
public protocol ASRServiceProvider: Sendable {
var backend: LocalASRBackend { get }
func make() -> ASRService
}
/// Built-in provider for the iOS SpeechAnalyzer path. Always present.
struct SpeechAnalyzerProvider: ASRServiceProvider {
let backend: LocalASRBackend = .speechAnalyzer
func make() -> ASRService { SpeechAnalyzerASR() }
}
// MARK: - PCM format conversion (testable helpers)
//
// Extracted from the audio-thread hot path so the scaling + clipping