rocky/OSGKeyboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: drop Qwen3 CoreML ASR, add local-engine cloud polish toggle

Browse Source 
Rolls back the v0.2.0 Qwen3 CoreML on-device ASR stack and replaces the
'local engine' UX with iOS 26 SpeechAnalyzer + DictationTranscriber only.

The 'Cloud polish after ASR' toggle (ProviderConfig.localModeCloudPolishEnabled)
lets users opt into a post-ASR DeepSeek round-trip from the local engine.
Defaults to off so the local engine stays genuinely local. New PolishError.missingAPIError
surfaces an inline 'fill in your key' warning when the toggle is on but the
Keychain is empty. DeepSeek preset default model bumped to deepseek-v4-flash.

Deleted:
  - OSGKeyboard/ThirdParty/Qwen3Speech/ (74 files, ~16k LoC)
  - OSGKeyboard/Services/ModelManager.swift (492)
  - OSGKeyboard/Services/OnDeviceModelWarmup.swift (197)
  - OSGKeyboard/Services/Qwen3ASRService.swift (257)
  - OSGKeyboard/Services/ModelDownloadSourcePicker.swift (126)
  - OSGKeyboard/Views/OnDeviceModelsView.swift (184)
  - OSGKeyboard/Views/DownloadConfirmSheet.swift (96)
  - OSGKeyboardShared/Models/OnDeviceModel.swift (140)
  - OSGKeyboardShared/Services/OnDeviceModelStatus.swift (104)
  - Qwen3ASRServiceProvider registration in OSGKeyboardApp
  - Qwen3Speech package declaration in project.yml
  - 5 .qwen3ASR enum / branch reference sites in HomeView, OnboardingView,
    LocalEngineSettingsRows, FlowSessionManager, ASRService, EngineServiceLabel
  - Two pre-existing Swift 6 strict-concurrency errors in
    LiveDictationController + FlowSessionManager (the weak [weak self] in
    detached-task MainActor.run blocks) that were blocking clean builds

Added:
  - LocalModelsGroup: 'Built-in iOS SpeechAnalyzer' badge + 'Cloud polish
    after ASR' Switch toggle
  - PolishingService: honour localModeCloudPolishEnabled; new .missingAPIKey
    error case with localised warning
  - AppGroupStore.localModeCloudPolishEnabled (mirrored into App Group
    so the keyboard extension honours the toggle during live dictation)
  - SettingsView: show provider/api sections when local-mode cloud polish
    is on so the user can paste a DeepSeek key
  - FlowSessionManager: route through PolishingService for local + polish-on
    flow; translate missingAPIKey into a polished warning
  - KeyboardViewController: handle PolishingService.PolishError.missingAPIKey
    in the keyboard-side live polish path
  - CHANGELOG v0.2.1: documents the rollback + new toggle
  - README.md / README.zh.md: engine matrix section, data flow note

Verified: xcodebuild -scheme OSGKeyboard -destination 'generic/platform=iOS Simulator'
build succeeds under SWIFT_STRICT_CONCURRENCY=complete.
This commit is contained in:
Rocky
2026-06-24 01:51:34 +08:00
parent 39690c0a93
commit c07cf4db9f
119 changed files with 401 additions and 18858 deletions
Download Patch File Download Diff File Expand all files Collapse all files
OSGKeyboardShared/Models/EngineServiceLabel.swift
+5 -7
View File
@@ -35,11 +35,9 @@ public enum EngineServiceLabel {
for backend: LocalASRBackend,
language: AppUILanguage
) -> String {
switch backend {
case .speechAnalyzer:
return SharedL10n.string("engine.asr.appleSpeech", language: language)
case .qwen3ASR:
return SharedL10n.string("model.qwen3asr.name", language: language)
}
// v0.2.0: only the iOS SpeechAnalyzer path remains. We keep the
// switch on `LocalASRBackend` so the next non-iOS backend can
// slot in without touching every call site.
return SharedL10n.string("engine.asr.appleSpeech", language: language)
}
}
}
OSGKeyboardShared/Models/LLMProvider.swift
+6 -2
View File
@@ -44,9 +44,13 @@ public struct LLMProvider: Identifiable, Codable, Hashable, Sendable {
id: "deepseek",
name: "DeepSeek",
defaultBaseURL: "https://api.deepseek.com/v1",
defaultModel: "deepseek-chat",
// v0.2.0: bumped default to `deepseek-v4-flash` for the
// local-mode cloud-polish toggle. `deepseek-chat` is
// retained as a valid user-overridable model name; only
// the default is updated.
defaultModel: "deepseek-v4-flash",
apiKeyURL: URL(string: "https://platform.deepseek.com/api_keys"),
blurb: "deepseek-chat · 中文友好 · Chinese-friendly"
blurb: "deepseek-v4-flash · 默认 · 快速且中文友好"
),
.init(
id: "qwen",
OSGKeyboardShared/Models/LocalASRBackend.swift
+22 -28
View File
@@ -6,51 +6,45 @@
// factory `ASRServiceFactory` dispatches on this enum; the settings UI
// renders it as a picker.
//
// Why an enum in `Shared` rather than living next to the concrete
// `ASRService` implementations: the value must be serialisable into
// the App Group store (so the keyboard extension can observe the
// selection), exposed via `ProviderConfig` (UI binding) and consumed
// by every layer that asks for an ASR backend.
// As of v0.2.0 the only on-device backend is iOS 26 `SpeechAnalyzer`
// + `DictationTranscriber`. The previous Qwen3-CoreML backend has
// been removed: that path required a ~1.6 GB CoreML bundle, a local
// SPM fork that pulled in mlx-swift, and significant app-side state
// (download manager, warm-up service, model registry). We now keep the
// local engine narrow — same iOS ASR the cloud engine already uses —
// and let users opt into a cloud polish step after the transcript is
// produced if they need stronger accuracy on noisy audio or dialectal
// Chinese. See `LocalPolishConfig` for the post-ASR polish toggle.
//
// Why an enum in `Shared` rather than a `Bool`: the value must remain
// serialisable into the App Group store (so the keyboard extension can
// observe the selection) and exposed via `ProviderConfig` (UI binding).
// Keeping the type stable even with a single case avoids a migration
// the next time someone adds a non-cloud backend (e.g. whisper.cpp).
import Foundation
public enum LocalASRBackend: String, CaseIterable, Identifiable, Sendable, Codable {
/// iOS 26 `SpeechAnalyzer` + `DictationTranscriber`. Always
/// on-device, no asset download, ships with iOS. Default for every
/// fresh install — anything else is opt-in.
/// on-device, no asset download, ships with iOS. The only local
/// backend in v0.2.0.
case speechAnalyzer
/// Qwen3-ASR-0.6B via CoreML (Neural Engine + CPU). Stronger on Chinese
/// dialects and noisy audio than `SpeechAnalyzer`, works in Flow while
/// the host app is backgrounded, but requires a ~1.6 GB download on first
/// use and iOS 18+.
case qwen3ASR
public var id: String { rawValue }
/// Localisation key for the human label in the settings picker.
public var labelKey: String {
switch self {
case .speechAnalyzer: return "asr.backend.speechAnalyzer.label"
case .qwen3ASR: return "asr.backend.qwen3.label"
}
"asr.backend.speechAnalyzer.label"
}
/// Localisation key for the one-line subtitle shown under the label.
public var blurbKey: String {
switch self {
case .speechAnalyzer: return "asr.backend.speechAnalyzer.blurb"
case .qwen3ASR: return "asr.backend.qwen3.blurb"
}
"asr.backend.speechAnalyzer.blurb"
}
/// Whether this backend needs the user to download a model file
/// before it can run. Used to gate the "Downloading Qwen3-ASR" UI
/// in a follow-up; for now we just expose the flag.
/// before it can run. Always `false` for iOS-bundled speech.
public var requiresModelDownload: Bool {
switch self {
case .speechAnalyzer: return false
case .qwen3ASR: return true
}
false
}
}
}
OSGKeyboardShared/Models/OnDeviceModel.swift
-54
View File
@@ -1,54 +0,0 @@
// OnDeviceModel.swift
// OSGKeyboard · Shared
//
// Identity of an on-device model the host app downloads and the
// keyboard extension observes via App Group flags (the extension
// cannot read the main app's Caches directory).
import Foundation
public enum OnDeviceModel: String, CaseIterable, Identifiable, Sendable {
case qwen3ASR
public var id: String { rawValue }
/// CoreML inference bundle (`aufklarer/Qwen3-ASR-CoreML`), derived from
/// official `Qwen/Qwen3-ASR-0.6B`.
public var repoId: String {
switch self {
case .qwen3ASR: return "aufklarer/Qwen3-ASR-CoreML"
}
}
/// Tokenizer files (vocab / merges) pulled from the upstream Qwen repo.
public var tokenizerRepoId: String {
switch self {
case .qwen3ASR: return "Qwen/Qwen3-ASR-0.6B"
}
}
public var displayName: String {
switch self {
case .qwen3ASR: return "Qwen3-ASR 0.6B (CoreML)"
}
}
public var approximateSizeMB: Int {
switch self {
case .qwen3ASR: return 1_600
}
}
public var compactSizeLabel: String {
"\(approximateSizeMB)M"
}
/// Settings list title: model name plus compact size.
public var listTitle: String {
"\(displayName) · \(compactSizeLabel)"
}
public var repoAndSizeLabel: String {
"\(repoId) · \(approximateSizeMB) MB"
}
}
OSGKeyboardShared/Models/ProviderConfig.swift
+39
View File
@@ -35,6 +35,10 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
// selection even though it never instantiates the backend itself.
static let localASRBackend = "config.localASRBackend"
static let uiLanguage = "config.uiLanguage"
// v0.2.0: optional cloud polish step after on-device ASR finishes
// in the local engine. Default `false` — keeps the local engine
// truly local unless the user explicitly opts in.
static let localModeCloudPolishEnabled = "config.localModeCloudPolishEnabled"
}
@Published public var providerId: String {
@@ -96,6 +100,17 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
@Published public var localASRBackend: LocalASRBackend {
didSet { defaults.set(localASRBackend.rawValue, forKey: Key.localASRBackend) }
}
/// When `engineMode == "local"`, optionally route the ASR transcript
/// through the user's configured LLM (DeepSeek by default) before
/// inserting at the cursor. The polish step runs through the same
/// `LLMClient` + `PolishingService` stack the cloud engine uses.
///
/// Defaults to `false` — the local engine is ASR-only out of the
/// box. Users opt in from Settings when the iOS ASR output isn't
/// strong enough (noisy far-field audio, dialectal Chinese, etc.).
@Published public var localModeCloudPolishEnabled: Bool {
didSet { defaults.set(localModeCloudPolishEnabled, forKey: Key.localModeCloudPolishEnabled) }
}
/// Host-app UI language. Also mirrored to the App Group for the keyboard extension.
@Published public var uiLanguage: AppUILanguage {
didSet { defaults.set(uiLanguage.rawValue, forKey: Key.uiLanguage) }
@@ -114,6 +129,21 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
/// On-device ASR only; no cloud API required.
public var isLocalEngine: Bool { engineMode == "local" }
/// Whether a transcript produced by the local engine should be
/// sent through the cloud LLM polish step before insertion.
///
/// v0.2.0: the local engine defaults to ASR-only. When the user
/// enables "Cloud polish after ASR" (`localModeCloudPolishEnabled`)
/// we route the transcript through the configured LLM (DeepSeek by
/// default in local mode) — same `PolishingService` code path the
/// cloud engine uses.
///
/// If the user hasn't entered an API key we can't run the polish
/// step; callers should check `Keychain.apiKey()` before invoking.
public var shouldPolishLocalTranscript: Bool {
isLocalEngine && localModeCloudPolishEnabled
}
/// The system prompt the user *sees* in the editor — fall back to the
/// provider-aware default from `AppGroupStore` when nothing is set.
public var defaultSystemPrompt: String {
@@ -151,6 +181,15 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
// rather than crashing inside `RawRepresentable.init`.
let rawBackend = resolvedDefaults.string(forKey: Key.localASRBackend) ?? LocalASRBackend.speechAnalyzer.rawValue
self.localASRBackend = LocalASRBackend(rawValue: rawBackend) ?? .speechAnalyzer
// v0.2.0: local-mode cloud polish toggle. Defaults off; users
// opt in from Settings when iOS ASR is too lossy for their
// environment. `object(forKey:) == nil` covers fresh installs
// and upgrades from builds that never wrote the key.
if resolvedDefaults.object(forKey: Key.localModeCloudPolishEnabled) == nil {
self.localModeCloudPolishEnabled = false
} else {
self.localModeCloudPolishEnabled = resolvedDefaults.bool(forKey: Key.localModeCloudPolishEnabled)
}
self.uiLanguage = AppUILanguage.fromStored(
resolvedDefaults.string(forKey: Key.uiLanguage)
)