Commit Graph

57 Commits

Author SHA1 Message Date
Rocky df1c5ff32c feat: migrate on-device Qwen3 ASR to CoreML for background Flow dictation
Replace MLX GPU inference with CoreML bundles so transcription continues
while the host app is backgrounded. Adds model download and warm-up,
vendored Qwen3Speech, and updates onboarding, settings, and copy for the
~1.6 GB CoreML package (iOS 18+).
2026-06-23 00:46:58 +08:00
Rocky 6148d05093 chore: align iOS 26 capability docs and UI localization
Keep repository messaging consistent with the implemented iOS 26 SpeechAnalyzer path, and remove mixed hardcoded copy by routing remaining UI/error text through localized string keys.
2026-06-18 22:27:22 +08:00
Rocky a803a27a88 fix: 3 review issues from the keyboard preview / onboarding flow
1) Preview chips weren't actually buttons.
   `modeChip` and `localeChip` in `KeyboardPreviewStub` were
   decorative HStacks — no `Button`, no action, no callback. The
   chevron-down glyph made them *look* like pickers, so a user
   tapping them got nothing. The screenshot the user shared
   ("润色 ▾" / "中文(简体) ▾") shows exactly that surface.

   Fix: wrap each chip in a `Button(action: ...)` with
   `.buttonStyle(.plain)`. The stub now takes `modeId`, `localeId`,
   `onModeCycle`, `onLocaleCycle` and the sheet's `cycleMode` /
   `cycleLocale` advance the config:
     - mode cycles [off → transcribe → polish] (mirrors Settings)
     - locale cycles [auto → zh-Hans → zh-Hant → en-US → ja-JP → ko-KR]
   Mid-recording locale switches call `asr.stop()` because ASR
   sessions are bound to the locale they were started with.
   `modeId == "off"` also stops any in-flight recording so the
   disc isn't recording into a mode that won't insert.

   The mode chip's icon also follows the mode (mic.slash /
   mic / wand) as a redundant visual cue, and both chips get
   accessibility labels (preview.modeChip.cycle /
   preview.localeChip.cycle) so VoiceOver users can use them.

2) Onboarding's "Next" stays enabled when local engine is picked
   but no API key is filled in. Root cause: `ProviderConfig.isConfigured`
   checks `!apiKey.isEmpty && !baseURL.isEmpty && !model.isEmpty` —
   it never asks whether the user *needs* a key. The local engine
   (on-device ASR) doesn't round-trip through the LLM, so an
   empty key on the local path is correct, not a configuration gap.

   Fix: short-circuit `isConfigured` to `true` when
   `engineMode == "local"`. The onboarding "Next" button is
   already disabled on the API page when `!isConfigured`; this
   just makes the gate respect the engine choice. New test
   `testIsConfiguredTrueForLocalEngineWithoutAPIKey` locks the
   behaviour in (local → true, cloud → false, flip back).

3) Add the two new accessibility keys to all four
   `Localizable.strings` files (en + zh-Hans, main app + ext)
   so VoiceOver and the cycle button labels resolve in both
   languages.

Build: BUILD SUCCEEDED.
Tests: 22/22 pass (1 new).

🤖 Generated with Claude Code
2026-06-18 20:17:30 +08:00
hkgood 79be7384dd [JJC-20260618-005-D] P1/P2 cleanup: structured errors, privacy audit, timeout SSOT, view-model tests
13 items, 555-line diff, build + 15/15 tests green.

ARCH-A3: Phase.error now carries ErrorKind (micDenied/speechDenied/asr/llm/
appGroupUnavailable/unknown) so the UI can pick icons/copy without parsing
free-form strings. Phase.ErrorKind, Phase, LLMError all Equatable.

ARCH-A4: Every TextField in APISettingsCard gets .keyboardType(.asciiCapable)
to defeat SwiftUI's iOS 18 system-keyboard hand-off that auto-suggests
Chinese/emoji and corrupts API keys / URLs / model names.

ARCH-A5 + DOC-3: PrivacyInfo.xcprivacy audited for honesty. Removed three
declared-but-unused APIs (FileTimestamp / DiskSpace / SystemBootTime) and
added ActiveKeyboards (DDA9.1) to the extension (it actually calls
advanceToNextInputMode in the tap path). Main App now declares only
UserDefaults (CA92.1). CHANGELOG updated.

ARCH-A6: Extracted PermissionManager (mic+speech permission flow, iOS 17
branching) and AppGroupPersistor (App Group load/persist) from the God
Object. KeyboardViewController drops 515 → 459 lines. KeyboardPipelineController
left in-place per risk plan — pressBegan state machine is too race-sensitive
to refactor in this pass.

RED-2: Deleted unused Theme enum (no call sites).
RED-3: Deleted unused cardStyle() alias (no call sites).
RED-7: Single source of truth for LLM timeout — LLMClient.requestTimeout +
LLMClientFactory.defaultRequestTimeout; PolishingService derives timeout from
defaultRequestTimeout+1 instead of hardcoding 15.
RED-8: ASRService.transcribe now emits .capability(onDeviceSupported:) as
first event per session; StatusBadge shows REC ⚠️ when the locale fell
back to cloud. New @Published var onDeviceSupported on State.

TEST-1: testPolishThrowsOnTransportTimeout now actually exercises
cancellation: StubURLProtocol delays response 5s, client.polish is
cancelled via Task.cancel(), test asserts the client throws .cancelled /
.transport / .decoding (was: silently passed).

TEST-2: New testPolisherSkipsNetworkWhenModeOff — PolishingService now
short-circuits when modeId == 'off' and returns trimmed input without
invoking LLMClient (proved via injected CountingLLMClient). Service was
moved to OSGKeyboardShared to be reachable from the test target.

TEST-3: KeyboardState (formerly KeyboardViewController.State) extracted
into OSGKeyboardShared so tests can @testable-import it. 5 phase/mode
tests in new KeyboardStateTests. Typealias preserves the old name.

TEST-4: New OSGKeyboardExtTests target with 6 tests covering State
initial values, phase transitions, structured-error round-trip, mode
switching, and InputMode rawValue round-trip.
2026-06-18 12:27:15 +08:00
rocky ed11a11329 [JJC-20260618-005-A] Review-driven P0 polish
6 review-driven small fixes to the P0 commit series (e0b92ff / e93bab5 /
2e2d8e3 / 013c498). No behavior changes, no new features, no push yet.

Code:
- RED-1: Remove empty init() from OSGKeyboardApp; @StateObject is initialized
  inline and the only thing init() did was print a DEBUG log.
- RED-4: Re-add .preferredColorScheme(.dark) to the 3 keyboard extension
  #Preview blocks. Custom keyboards always render dark; the preview should
  match.
- RED-6: ThemedRoot #Preview blocks now use the new EnvironmentKey
  (\.themePalette) via a private ThemedPreviewContent, instead of hard-coding
  Palette.dark / Palette.light.
- BUG-5: ThemedRoot body now also fills the background with the active
  palette's background color (repeats the colorScheme ternary — accepted
  because body must be a single expression).

Docs:
- DOC-1: CHANGELOG [Unreleased] top note: this is a v0.1.0 → v0.1.1 polish,
  nothing removed, iOS 26 SpeechAnalyzer still 0.2.0.
- DOC-2: README + README.zh.md Limitations: explicit note for iOS 26+ users
  in v0.1.1.

Verified: xcodebuild Debug-iphonesimulator BUILD SUCCEEDED, 8/8 unit
tests pass on iPhone 17 sim (iOS 26.3.1).
2026-06-18 11:29:14 +08:00
Zhongshu e93bab50b4 [P0-②] On-device ASR three-piece set
- Add NSSpeechRecognitionUsageDescription to both target Info.plists
- KeyboardViewController.requestSpeechPermission() wraps
  SFSpeechRecognizer.requestAuthorization
- pressBegan() now requests Speech permission right after mic permission
- ASRService emits a DEBUG warning when recognizer.supportsOnDeviceRecognition
  is false so cloud fallback is visible during dev
- CHANGELOG: move iOS 26 SpeechAnalyzer out of Unreleased into [0.2.0] Planned,
  so the iOS 18 SFSpeechRecognizer path is the only ASR shipped in v0.1

xcodebuild iOS Simulator: SUCCEEDED
2026-06-18 10:41:20 +08:00
rocky 07067ca6c5 feat: initial release v0.1.0
- Custom Keyboard Extension with push-to-talk UI
- iOS 26 SpeechAnalyzer + DictationTranscriber (iOS 18 SF fallback)
- OpenAI-compatible LLM client (4 built-in providers + custom)
- 3-page onboarding flow + provider config UI
- App Group shared storage for cross-process config
- 8s LLM timeout with raw-transcript fallback
- App Store privacy manifests for both targets
- SwiftLint + XcodeGen + GitHub Actions CI
- Unit tests (4/4 passing)
2026-06-17 23:08:58 +08:00