rocky/OSGKeyboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
26 Commits 2 Branches 0 Tags
7f059dbd45e3deb13d3e96b5ac5b8028666171ee
Commit Graph

8 Commits

Author SHA1 Message Date
Rocky 7f059dbd45 feat: TypeWhisper Flow sessions, Phase 4 UX, and GitHub Pages privacy site 
Migrate keyboard dictation to continuous Flow sessions with auto-start,
tap-to-toggle recording, 60s countdown, five-step onboarding, and App Group
IPC. Add docs/ GitHub Pages site with en/zh privacy policy for App Store compliance.
 2026-06-19 18:05:50 +08:00
Rocky 56d0da0a51 fix: preview disc stuck at .processing after stop 
The previous code path for the keyboard preview's ASR controller
cancelled the consumer task at the exact moment it closed the
audio stream:

    asrTask?.cancel()       // ← kills the .final consumer
    asrTask = nil
    ...
    bufferContinuation?.finish()   // tells ASR "no more audio"

The cancellation cascaded: the for-await on the events stream
exited → the AsyncStream's `continuation.onTermination` fired →
ASR.cancel() ran → producer task was marked cancelled → the
producer's `if !Task.isCancelled { yield(.final) }` guard
suppressed the .final event. Net result: nobody told the UI to
leave `.processing`, and the disc sat there forever.

Fix (4 changes):

1) `stop()` no longer cancels the consumer. The consumer task
   exits naturally when the events stream finishes, sees the
   `.final` event the producer still yields, and transitions
   the phase out of `.processing`. This is the primary fix.

2) `start()` cancels any leftover `asrTask` at the entry point
   as a safety net — covers the "user smashes the disc twice
   quickly" race where a previous consumer is still draining.

3) `stop()` schedules a 3-second safety-net Task: if the ASR
   pipeline never produces a `.final` (analyzer hang, system
   glitch), force the phase back to `.idle` so the user isn't
   stuck. Normal recordings complete well under 3 seconds, so
   the timeout is only hit on the unhappy path.

4) `KeyboardPreviewSheet` adds `.onDisappear { asr.stop() }`
   so closing the sheet mid-recording releases the
   AVAudioSession and mic. `stop()` is idempotent (no-op on
   non-recording phases), safe to call here.

State machine: `phase = .processing` now has TWO transition
paths out — the consumer receiving `.final` (fast path) and
the 3-second safety net (fallback). Both are required; the
fast path is the common case, the fallback is the
"guaranteed-progress" guarantee.

Testability: `asrTask` was `private`; relaxed to `internal` so
the regression test in
`OSGKeyboardTests/PreviewASRControllerStateTests.swift` can
install a known consumer task and assert `stop()` does not
cancel it. The class is `@MainActor` so Swift 6 isolation
rules still prevent production code outside the class from
racing on it.

Tests:
- `testStopDoesNotCancelConsumerTask` — primary fix regression.
- `testStopIsIdempotent` — `.onDisappear` after a manual stop
  doesn't misbehave.
- 27/27 tests pass (25 existing + 2 new).
- BUILD SUCCEEDED.

🤖 Generated with Claude Code
 2026-06-18 21:00:18 +08:00
Rocky a227309059 fix: feed DictationTranscriber Int16 PCM, not Float32 
The keyboard preview crashed on first record with a
`__abort_with_payload` deep inside Speech's
`DictationTranscriber`. The disassembly surfaced three
preconditions checked before a `brk #0x1`:

  +620  "Audio sample data must be 16-bit signed integers"
  +848  "Multi-channel audio is not supported"
  +1072 "Client info not fully initialized"

We hit the first one. `DictationTranscriber` (iOS 26's new
`SpeechAnalyzer`-backed engine) is strict about its input
format: only Int16 PCM, not the Float32 PCM that the iOS 18
`SFSpeechRecognizer` path accepted. Our audio-tap and
`AudioBufferSnapshot.samples: [Float]` are Float32 all the
way down — that was the SFSpeech shape, and the previous
`AppleSpeechASR` adapted internally. With iOS 26 as the
deployment target, the only ASR backend is
`SpeechAnalyzerASR`, and the conversion needed to happen at
the `AnalyzerInput` boundary.

Fix:
- `transcribe` builds the `AVAudioFormat` as
  `.pcmFormatInt16, 16 kHz, 1 ch, interleaved: true` (the
  canonical layout for Int16 Speech input).
- `makeInputStream` runs the per-sample conversion
  `Int16(round(clamp(s * 32767, -32768, 32767)))` into the
  `AVAudioPCMBuffer`'s `int16ChannelData[0]`. The explicit
  clip is required (a `s == 1.5` from a gain-overflow at the
  audio-engine boundary would otherwise wrap to a negative
  Int16 after the implicit truncation). `round()` (not
  truncate) preserves DC balance — `0.5` quantises to
  `+16384`, not `+16383`, matching what audio DAWs expect.
- The conversion helper is exposed as
  `ASRServiceFactory.convertFloat32ToInt16` so unit tests
  can lock the math without instantiating the full pipeline.

Why not change `AudioBufferSnapshot` to `[Int16]` instead
(see earlier first-principles discussion): the snapshot is a
transport format that both `AudioCaptureService` (in the
ext) and `PreviewASRController` (in the main app) produce.
Float32 is the natural shape coming out of `AVAudioEngine`,
and pushing the conversion to the ASR service keeps the
transport contract platform-agnostic — a future second
backend with different format needs can have its own
adaptation without dragging everyone else.

Tests:
- `testFloat32ToInt16EdgeCases` — 0, ±1, ±0.5, ±1.5
  (gain-overflow case).
- `testFloat32ToInt16RoundTrip` — quantisation step is
  1/32767 (so the asymmetric Int16 range is honoured: -32768
  has no exact Float source).
- `testFloat32ToInt16Empty` — `sourceCount == 0` with nil
  pointers is a no-op (function guards on count before
  dereferencing).
- All 25 tests pass (22 existing + 3 new).
- BUILD SUCCEEDED.

🤖 Generated with Claude Code
 2026-06-18 20:41:12 +08:00
Rocky a803a27a88 fix: 3 review issues from the keyboard preview / onboarding flow 
1) Preview chips weren't actually buttons.
   `modeChip` and `localeChip` in `KeyboardPreviewStub` were
   decorative HStacks — no `Button`, no action, no callback. The
   chevron-down glyph made them *look* like pickers, so a user
   tapping them got nothing. The screenshot the user shared
   ("润色 ▾" / "中文(简体) ▾") shows exactly that surface.

   Fix: wrap each chip in a `Button(action: ...)` with
   `.buttonStyle(.plain)`. The stub now takes `modeId`, `localeId`,
   `onModeCycle`, `onLocaleCycle` and the sheet's `cycleMode` /
   `cycleLocale` advance the config:
     - mode cycles [off → transcribe → polish] (mirrors Settings)
     - locale cycles [auto → zh-Hans → zh-Hant → en-US → ja-JP → ko-KR]
   Mid-recording locale switches call `asr.stop()` because ASR
   sessions are bound to the locale they were started with.
   `modeId == "off"` also stops any in-flight recording so the
   disc isn't recording into a mode that won't insert.

   The mode chip's icon also follows the mode (mic.slash /
   mic / wand) as a redundant visual cue, and both chips get
   accessibility labels (preview.modeChip.cycle /
   preview.localeChip.cycle) so VoiceOver users can use them.

2) Onboarding's "Next" stays enabled when local engine is picked
   but no API key is filled in. Root cause: `ProviderConfig.isConfigured`
   checks `!apiKey.isEmpty && !baseURL.isEmpty && !model.isEmpty` —
   it never asks whether the user *needs* a key. The local engine
   (on-device ASR) doesn't round-trip through the LLM, so an
   empty key on the local path is correct, not a configuration gap.

   Fix: short-circuit `isConfigured` to `true` when
   `engineMode == "local"`. The onboarding "Next" button is
   already disabled on the API page when `!isConfigured`; this
   just makes the gate respect the engine choice. New test
   `testIsConfiguredTrueForLocalEngineWithoutAPIKey` locks the
   behaviour in (local → true, cloud → false, flip back).

3) Add the two new accessibility keys to all four
   `Localizable.strings` files (en + zh-Hans, main app + ext)
   so VoiceOver and the cycle button labels resolve in both
   languages.

Build: BUILD SUCCEEDED.
Tests: 22/22 pass (1 new).

🤖 Generated with Claude Code
 2026-06-18 20:17:30 +08:00
Rocky 3c11ce2903 feat: API key in Keychain + actionable permission-denied UX 
Security: API key moves from App Group UserDefaults (plaintext on disk)
to the iOS Keychain. The host app writes in Settings; the keyboard
extension reads before each request. Cross-process sharing is via a new
shared keychain-access-group declared in both targets' entitlements.

UX: when the user denies microphone or speech-recognition permission,
the message becomes a tappable row that opens the host app's settings.
Previously the message said "请到「设置」中允许" but the only way to
actually get there was a top-bar ⚙ button that wasn't obviously
related. Auto-clear (2.4s) is now suppressed for .denied so the user
has time to read it. Re-pressing the mic from .denied re-checks
permission so the user can simply press again after granting.

API Keychain migration
----------------------
- New `OSGKeyboardShared/Services/Keychain.swift` — minimal
  `kSecClassGenericPassword` wrapper for one item
  (service "com.osgkeyboard.apikey", account "current"), backed by
  `kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly` (no iCloud sync).
  `setAPIKey("")` deletes the entry rather than storing an empty
  placeholder so "stored but empty" stays distinguishable from
  "not stored" for the noAPIKey error path.
- `ProviderConfig.apiKey` now reads/writes through Keychain instead
  of UserDefaults. `didSet` skips the round-trip when oldValue equals
  apiKey (init reads Keychain, then assigns — without this guard the
  init write would silently re-write the same value).
- One-shot migration: on first `ProviderConfig.init` after upgrade,
  a legacy `config.apiKey` UserDefaults entry is copied to Keychain
  and removed from UserDefaults. The legacy key is renamed in code to
  `apiKeyLegacy` so future reads of `config.apiKey` from UserDefaults
  would be a bug.
- `AppGroupStore.apiKey` reads from Keychain (was UserDefaults).
- Cross-process sharing: both targets' entitlements gain
  `com.apple.security.keychain-access-groups: ["com.osgkeyboard.shared"]`.
  `com.osgkeyboard.shared` is the first entry in both, so it becomes
  each process's default access group — Keychain queries don't need to
  specify `kSecAttrAccessGroup`.

Permission-denied UX
--------------------
- `KeyboardViewController.pressBegan` now accepts `.denied` and
  `.error` as starting states (previously only `.idle`), so pressing
  the mic after returning from Settings re-checks permission
  without waiting for an auto-clear.
- `scheduleAutoClearError` no longer clears `.denied` — only
  transient `.error` is timed. `.denied` is sticky until the user
  takes action (taps the row → settings, or presses mic → re-check).
- `TranscriptLine` `.denied` case now wraps the text in a Button
  that calls `state.openSettings`, with a `chevron.right` to make
  the affordance obvious. The text was shortened to
  "麦克风被拒绝" / "语音识别被拒绝" so the chevron has room and
  the action isn't implied twice (it was previously both in the
  text and via the top-bar ⚙ button).
- VoiceOver hint on the button: "Opens the OSGKeyboard settings
  page where you can grant microphone or speech recognition access."

Tests
-----
- New `OSGKeyboardTests/KeychainTests.swift` — 6 tests covering
  round-trip, empty-string-deletes, idempotent-delete,
  AppGroupStore-reads-from-Keychain, legacy UserDefaults → Keychain
  migration, and "Keychain wins when both are present".
- `LLMClientTests` setUp/tearDown now wipes the Keychain
  (`try? Keychain.deleteAPIKey()`) and clears `StubURLProtocolStorage`
  so tests are independent across runs in the same simulator process.
- All 21 tests pass (6 new + 15 existing).

🤖 Generated with Claude Code
 2026-06-18 12:41:07 +08:00
hkgood 79be7384dd [JJC-20260618-005-D] P1/P2 cleanup: structured errors, privacy audit, timeout SSOT, view-model tests 
13 items, 555-line diff, build + 15/15 tests green.

ARCH-A3: Phase.error now carries ErrorKind (micDenied/speechDenied/asr/llm/
appGroupUnavailable/unknown) so the UI can pick icons/copy without parsing
free-form strings. Phase.ErrorKind, Phase, LLMError all Equatable.

ARCH-A4: Every TextField in APISettingsCard gets .keyboardType(.asciiCapable)
to defeat SwiftUI's iOS 18 system-keyboard hand-off that auto-suggests
Chinese/emoji and corrupts API keys / URLs / model names.

ARCH-A5 + DOC-3: PrivacyInfo.xcprivacy audited for honesty. Removed three
declared-but-unused APIs (FileTimestamp / DiskSpace / SystemBootTime) and
added ActiveKeyboards (DDA9.1) to the extension (it actually calls
advanceToNextInputMode in the tap path). Main App now declares only
UserDefaults (CA92.1). CHANGELOG updated.

ARCH-A6: Extracted PermissionManager (mic+speech permission flow, iOS 17
branching) and AppGroupPersistor (App Group load/persist) from the God
Object. KeyboardViewController drops 515 → 459 lines. KeyboardPipelineController
left in-place per risk plan — pressBegan state machine is too race-sensitive
to refactor in this pass.

RED-2: Deleted unused Theme enum (no call sites).
RED-3: Deleted unused cardStyle() alias (no call sites).
RED-7: Single source of truth for LLM timeout — LLMClient.requestTimeout +
LLMClientFactory.defaultRequestTimeout; PolishingService derives timeout from
defaultRequestTimeout+1 instead of hardcoding 15.
RED-8: ASRService.transcribe now emits .capability(onDeviceSupported:) as
first event per session; StatusBadge shows REC ⚠️ when the locale fell
back to cloud. New @Published var onDeviceSupported on State.

TEST-1: testPolishThrowsOnTransportTimeout now actually exercises
cancellation: StubURLProtocol delays response 5s, client.polish is
cancelled via Task.cancel(), test asserts the client throws .cancelled /
.transport / .decoding (was: silently passed).

TEST-2: New testPolisherSkipsNetworkWhenModeOff — PolishingService now
short-circuits when modeId == 'off' and returns trimmed input without
invoking LLMClient (proved via injected CountingLLMClient). Service was
moved to OSGKeyboardShared to be reachable from the test target.

TEST-3: KeyboardState (formerly KeyboardViewController.State) extracted
into OSGKeyboardShared so tests can @testable-import it. 5 phase/mode
tests in new KeyboardStateTests. Typealias preserves the old name.

TEST-4: New OSGKeyboardExtTests target with 6 tests covering State
initial values, phase transitions, structured-error round-trip, mode
switching, and InputMode rawValue round-trip.
 2026-06-18 12:27:15 +08:00
Zhongshu 2e2d8e33b3 [P0-③] API Key data flow fix 
- AppGroup.defaults: in DEBUG, missing App Group is a hard fatalError
  with a precise remediation message (was a soft print + .standard
  fallback, which desynced the keyboard extension from the main App).
  Release keeps the fallback + NSLog so end-users still get a usable app.
- KeyboardViewController.loadPersistedLocale now prints a masked DEBUG
  view of the live App Group config (provider, baseURL, masked key,
  model, mode, locale) so the extension's view is visible in the
  device console.
- KeyboardViewController.handleFinalTranscript now routes by typed error:
    noAPIKey  → red error '未配置 API Key · 请在主 App 设置中填写'
    http 401  → red error 'API Key 无效 (401) · 请检查主 App 设置'
    http 429  → red error 'API 限流 (429) · 请稍后再试'
    other     → insert raw transcript + generic error badge
- APISettingsCard gains a 'Test connection' button that runs a single
  client.polish('ping') round-trip and surfaces the typed result inline.
- PolishingService.timeout raised 12s → 15s to match LLMClient.request
  timeout (was racing and discarding successful responses in 12–15s).
- Tests: 4 new cases (HTTP 429, transport timeout, App Group cross-process,
  AppGroupStore→LLMClient noAPIKey). All 8 tests pass on iPhone 16e sim.

xcodebuild iOS Simulator: SUCCEEDED
xcodebuild test: 8/8 passed
 2026-06-18 10:50:02 +08:00
Rocky bec36befa2 feat: comprehensive rewrite — push-to-talk pipeline, Typeless UI, Chinese 
This is a major rewrite of OpenLessKeyboard, renamed to OSGKeyboard
and rebuilt end-to-end. 59 files changed (+3205/-1550).

Architecture
------------
- Rename project, targets, directories from OpenLess* to OSGKeyboard*
  (OpenLess / OpenLessKeyboard / OpenLessShared / OpenLessTests).
- AudioCaptureService rewritten as @unchecked Sendable class with
  OSAllocatedUnfairLock instead of an actor, so it survives Swift 6
  strict-concurrency checks while still serialising engine + converter
  state correctly.
- Single design system (Palette / Spacing / Radius / TypeStyle /
  Motion) lifted into OSGKeyboardShared so the host app and the
  keyboard extension stay in lock-step.

Push-to-talk — first-principles fix
-----------------------------------
- App Group + audio-input entitlements were stripped by Xcode's
  Automatic Signing. They are now declared in project.yml so
  'xcodegen generate' re-emits them every time. iOS Developer
  Account is untouched; only the App Group capability was added.
- State machine uses a real stored `phase` (was a derived shim
  that locked out every press after the first because
  recordStream was never nilled after the pipeline finished).
- Microphone permission is requested inside pressBegan (async
  Task) so the press flow optimistically enters .recording;
  permission denial surfaces a short error and returns to idle.
- Replaced LongPressGesture(0.15s) with a DragGesture +
  TapGesture pair separated by pressArmed, so a single tap no
  longer fires both onPressBegan and onTap simultaneously.
- Real RMS / peak level meter from the AVAudioEngine tap (was a
  pseudo-random walk); the visible waveform is now driven by
  actual audio.
- SFSpeechRecognizer(locale:) with selectable ASR locales
  (auto / zh-Hans / zh-Hant / en-US / ja-JP / ko-KR) for
  first-class Chinese / English / Japanese / Korean dictation,
  with on-device recognition when supported.
- AVAudioSession now deactivates on stop so other apps' audio
  routing is restored.

Keyboard UI — Typeless-inspired layout
---------------------------------------
- Hero area is 280 pt with a 96 pt record disc, breathing outer
  ring, and a 12-bar waveform driven by the real RMS.
- inputView.allowsSelfSizing + a heightAnchor constraint so iOS
  no longer crops the keyboard under the Spotlight bar / home
  indicator.
- Top bar: mode chip (Off / 转写 / 润色) + locale chip
  (Auto / 简体 / 繁體 / EN / 日 / 한) + status badge + ⚙.
- Bottom bar: globe / delete / 空格 / return — all 40 pt and
  balanced.
- RecordButton onPressEnded is now safe to fire from a quick
  press; pressArmed prevents double-firing.

LLM / Polishing
---------------
- LLMClient: stopped leaking the server response body in errors
  (server body is now logged at debug, never surfaced to UI);
  added a dedicated .rateLimited case for 429.
- PolishingService timeout 8s → 12s to accommodate slower
  domestic LLM providers.
- AppGroupStore.defaultSystemPrompt is now provider-aware
  (Chinese for zhipu/moonshot/qwen/deepseek, English otherwise).

Onboarding & Settings
---------------------
- Re-themed OnboardingView / HomeView / SettingsView on the
  new design system.
- ProviderPickerSection now shows 6 providers (OpenAI, DeepSeek,
  Qwen DashScope, 智谱 GLM, 月之暗面 Moonshot, Custom) with
  blurb + selected accent.
- PickerRow for Mode and ASR locale; System Prompt editor with
  reset-to-default.
- API settings page "Get an API key" used SwiftUI Link, which
  has a hit-test bug on iOS 18 that ate gestures from adjacent
  TextFields (manifested as "typing jumps to a website"). It is
  now an explicit Button + contentShape + .submitLabel(.done) on
  the fields.

Polish & tests
--------------
- LLMClientTests: 4 unit tests passing (ProviderConfig
  persistence + OpenAI request/response + HTTP error + missing
  key); test App Group renamed to the correct identifier.
- ProviderConfig.apply now captures the previous provider id
  *before* mutating, so switching providers actually resets the
  system prompt to the new default.

Build
-----
- Swift 6 strict concurrency, iOS 18.0 deployment target.
- Tested on Xcode 26 + iPhone 17 Pro simulator. A real device on
  iOS 27 beta aborts with __abort_with_payload (dispatch
  library ABI mismatch); use an iOS 18 real device or the
  iOS 26 simulator for now.

🤖 Generated with Claude Code
 2026-06-18 01:22:12 +08:00