Custom polish styles can opt in to emotion-matched emoji (default off), with
prompt-level opt-in detection so paste-only styles keep model-added emoji.
Also include Volcengine API-Key ASR auth, voice-processing capture, PiP flash
fix, and related keyboard Shift/haptics reliability work.
Fix cold-start overlay recursion that overflowed the main-thread stack when
recording began while the ready overlay was still up; also remove temporary
on-screen Flow DEBUG panels after the orange-mic investigation, and land the
macOS overlay/catalog/layout polish plus related Flow recovery hardening.
Reduce perceived latency from key release to final text:
- Adaptive chunking: 2.5s first chunk + 5s follow-ups so short
utterances start on-device recognition while still recording.
- Session-level ASR warmup and audio-format cache reuse to remove
per-utterance cold-start of SpeechAnalyzer.
- Mirror live pipelined partials to the keyboard transcript line via
a new flow.transcriptionPartial App Group key + Darwin ping.
Also commits the accumulated custom language model, Flow session,
keyboard extension restructure, and Xiaomi MiMo provider work in
progress on this branch.
Batch of in-progress app work from the working tree.
- feat(keyboard): CursorNavigation + CursorDragPad for caret movement;
KeyboardSoundFeedback for system key click sounds
- feat(dictionary): DictionaryAliasGenerator + PersonalDictionaryEntrySheet;
TranscriptPostProcessor quality gate; retire DictionaryLearner
- feat(ui): TabBarVisibility handling; drop PageHeaderRow /
PageHeaderConfirmButton; refresh views and localizable strings
- fix(security): move the hardcoded DeepSeek key out of
PreconfiguredKeys.swift into a gitignored PreconfiguredKeys.local.swift
(seeded from .example by generate-xcodeproj.sh)
- docs(agents): add Conventional Commits versioning + bilingual changelog rules
- chore(gitignore): ignore PreconfiguredKeys.local.swift, .cache/, pycache
Custom language model / lexicon work stays on
feature/custom-language-model-asr. Changelog bullets added under
[Unreleased]; no version bump.
- Add LLMProvider.isUserSelectable (default true) and filter
ProviderPickerSection on it; next pass can hide non-user presets
(e.g. a future DeepSeek key-preset) without changing call sites.
- KeyboardRootView: hide TranslationChip when off (matches user's
mental model of an opt-in feature), drop the 'warming' branch
(Qwen3 download UX was removed with the backend in v0.2.0), unify
chip pill height to minHeight 28 + vertical 6 for visual rhythm
across all topbar chips.
- TranslationChip: drop isLocal warning path — both engines now
run the translate-and-polish step (local routes through DeepSeek
via ProviderConfig.localModeProviderId).
- OnboardingView: cloud engine branch now wraps the translation
row in the same surface card chrome as the local branch.
- Strings: drop keyboard.models.warming (no longer referenced).
DeepSeek key pre-fill deferred to a follow-up.
Rolls back the v0.2.0 Qwen3 CoreML on-device ASR stack and replaces the
'local engine' UX with iOS 26 SpeechAnalyzer + DictationTranscriber only.
The 'Cloud polish after ASR' toggle (ProviderConfig.localModeCloudPolishEnabled)
lets users opt into a post-ASR DeepSeek round-trip from the local engine.
Defaults to off so the local engine stays genuinely local. New PolishError.missingAPIError
surfaces an inline 'fill in your key' warning when the toggle is on but the
Keychain is empty. DeepSeek preset default model bumped to deepseek-v4-flash.
Deleted:
- OSGKeyboard/ThirdParty/Qwen3Speech/ (74 files, ~16k LoC)
- OSGKeyboard/Services/ModelManager.swift (492)
- OSGKeyboard/Services/OnDeviceModelWarmup.swift (197)
- OSGKeyboard/Services/Qwen3ASRService.swift (257)
- OSGKeyboard/Services/ModelDownloadSourcePicker.swift (126)
- OSGKeyboard/Views/OnDeviceModelsView.swift (184)
- OSGKeyboard/Views/DownloadConfirmSheet.swift (96)
- OSGKeyboardShared/Models/OnDeviceModel.swift (140)
- OSGKeyboardShared/Services/OnDeviceModelStatus.swift (104)
- Qwen3ASRServiceProvider registration in OSGKeyboardApp
- Qwen3Speech package declaration in project.yml
- 5 .qwen3ASR enum / branch reference sites in HomeView, OnboardingView,
LocalEngineSettingsRows, FlowSessionManager, ASRService, EngineServiceLabel
- Two pre-existing Swift 6 strict-concurrency errors in
LiveDictationController + FlowSessionManager (the weak [weak self] in
detached-task MainActor.run blocks) that were blocking clean builds
Added:
- LocalModelsGroup: 'Built-in iOS SpeechAnalyzer' badge + 'Cloud polish
after ASR' Switch toggle
- PolishingService: honour localModeCloudPolishEnabled; new .missingAPIKey
error case with localised warning
- AppGroupStore.localModeCloudPolishEnabled (mirrored into App Group
so the keyboard extension honours the toggle during live dictation)
- SettingsView: show provider/api sections when local-mode cloud polish
is on so the user can paste a DeepSeek key
- FlowSessionManager: route through PolishingService for local + polish-on
flow; translate missingAPIKey into a polished warning
- KeyboardViewController: handle PolishingService.PolishError.missingAPIKey
in the keyboard-side live polish path
- CHANGELOG v0.2.1: documents the rollback + new toggle
- README.md / README.zh.md: engine matrix section, data flow note
Verified: xcodebuild -scheme OSGKeyboard -destination 'generic/platform=iOS Simulator'
build succeeds under SWIFT_STRICT_CONCURRENCY=complete.
Three user-flagged fixes, scoped tightly to the files each affects.
The uncommitted Engine-mode wiring / locale picker / etc. from a
prior agent pass is intentionally not included in this commit.
1. Revert accent to brand green (#3AA05A).
Last commit flipped `AccentColor` + `Palette.light.accent` to
Apple system blue (#007AFF) on the assumption that "green CTA
on a near-white surface looks wrong." Review pushed back:
the brand *is* the green, and the system tint should match it
so that NavStack Done buttons, Toggles, and our custom
`primaryButton()` modifier all read as the same colour. Restored
`#3AA05A` in both `AccentColor.colorset/Contents.json` and
`Palette.light.accent` (plus the muted / glow variants).
2. Real iOS ASR in `KeyboardPreviewSheet`.
The previous fix only swapped the static placeholder for a
`TextField` and routed a hardcoded stub through it — review
asked, fairly, "are you actually calling `SFSpeechRecognizer`?"
Answer: no. This change makes the preview *run real ASR*:
- `ASRService` (+ the iOS 26 `SpeechAnalyzer` path) moves from
`OSGKeyboardExt/Services/` to `OSGKeyboardShared/Services/`,
so the host app can import the same `ASRServiceFactory.make()`
the keyboard extension uses.
- `OSGKeyboardShared` gains `Speech.framework` and
`AVFoundation.framework` as SDK dependencies in `project.yml`.
- New `OSGKeyboard/Views/PreviewASRController.swift` (the
extension's `AudioCaptureService` is app-extension-only, so
the preview owns its own `AVAudioEngine` + `AVAudioSession`
and downsamples to 16 kHz mono Float32 via `AVAudioConverter`).
- `KeyboardPreviewSheet.cyclePhase()` now calls
`asr.start(locale:)` / `asr.stop()` instead of toggling a
`StubPhase`. The disc's level meter is driven by RMS from the
actual audio tap; the transcript line under the chips shows
the live `SFSpeechRecognizer` partial; the top textbox
receives the `.final` transcript via `onChange(of: lastFinal)`.
3. Record disc in BOTH stubs now uses the green accent for idle.
Was `Color(white: 0.22)` dark-gray, which read as "inert
surface" rather than "tap me". The keyboard extension and the
in-app preview now share the same brand-green disc gradient so
the keyboard's primary CTA is the same colour in both modes.
4. Bilingual audit across every user-facing string.
Every Text() in the main-app and extension views is now
`中文 · English` or carries an English secondary line. Covered:
- OnboardingView (Back, Next, Done, Continue, step instructions,
PrivacyFootnote rows)
- HomeView (status header, hero label, accessibility)
- APISettingsCard (Base URL, API Key, Model, "Get an API key",
"Connection", test-connection states + error messages)
- KeyboardPreviewSheet (title, subtitle, TextField placeholder,
clear button accessibility)
- KeyboardPreviewStub (mode/locale chips, REC badge, space bar)
- KeyboardRootView (gear accessibility, space bar, requesting
state, denied messages, local-engine chip)
- RecordButton (accessibility label)
- SettingsView (Done, Reset dialog, language section subtitle,
engine section, local-engine subtitle, on-device label)
- AppGroupErrorView (title, body, three remediation steps)
- LLMProvider preset names and blurbs
Where the prior pattern was "Chinese headline + English footnote"
(e.g. OnboardingView's 启用 OSGKeyboard / Enable OSGKeyboard),
that pattern was preserved — bilingual coverage means every
screen reads as both, not that every line is rigidly `中 · EN`.
Build: BUILD SUCCEEDED on iPhone 17 Pro / iOS 26 simulator.
Tests: 21/21 pass (no test changes).
Visual: light-mode home shot at /tmp/osgk_light_green.png shows
the brand-green CTA restored across Next button, mic icon, and
page dot.
🤖 Generated with Claude Code
This is a major rewrite of OpenLessKeyboard, renamed to OSGKeyboard
and rebuilt end-to-end. 59 files changed (+3205/-1550).
Architecture
------------
- Rename project, targets, directories from OpenLess* to OSGKeyboard*
(OpenLess / OpenLessKeyboard / OpenLessShared / OpenLessTests).
- AudioCaptureService rewritten as @unchecked Sendable class with
OSAllocatedUnfairLock instead of an actor, so it survives Swift 6
strict-concurrency checks while still serialising engine + converter
state correctly.
- Single design system (Palette / Spacing / Radius / TypeStyle /
Motion) lifted into OSGKeyboardShared so the host app and the
keyboard extension stay in lock-step.
Push-to-talk — first-principles fix
-----------------------------------
- App Group + audio-input entitlements were stripped by Xcode's
Automatic Signing. They are now declared in project.yml so
'xcodegen generate' re-emits them every time. iOS Developer
Account is untouched; only the App Group capability was added.
- State machine uses a real stored `phase` (was a derived shim
that locked out every press after the first because
recordStream was never nilled after the pipeline finished).
- Microphone permission is requested inside pressBegan (async
Task) so the press flow optimistically enters .recording;
permission denial surfaces a short error and returns to idle.
- Replaced LongPressGesture(0.15s) with a DragGesture +
TapGesture pair separated by pressArmed, so a single tap no
longer fires both onPressBegan and onTap simultaneously.
- Real RMS / peak level meter from the AVAudioEngine tap (was a
pseudo-random walk); the visible waveform is now driven by
actual audio.
- SFSpeechRecognizer(locale:) with selectable ASR locales
(auto / zh-Hans / zh-Hant / en-US / ja-JP / ko-KR) for
first-class Chinese / English / Japanese / Korean dictation,
with on-device recognition when supported.
- AVAudioSession now deactivates on stop so other apps' audio
routing is restored.
Keyboard UI — Typeless-inspired layout
---------------------------------------
- Hero area is 280 pt with a 96 pt record disc, breathing outer
ring, and a 12-bar waveform driven by the real RMS.
- inputView.allowsSelfSizing + a heightAnchor constraint so iOS
no longer crops the keyboard under the Spotlight bar / home
indicator.
- Top bar: mode chip (Off / 转写 / 润色) + locale chip
(Auto / 简体 / 繁體 / EN / 日 / 한) + status badge + ⚙.
- Bottom bar: globe / delete / 空格 / return — all 40 pt and
balanced.
- RecordButton onPressEnded is now safe to fire from a quick
press; pressArmed prevents double-firing.
LLM / Polishing
---------------
- LLMClient: stopped leaking the server response body in errors
(server body is now logged at debug, never surfaced to UI);
added a dedicated .rateLimited case for 429.
- PolishingService timeout 8s → 12s to accommodate slower
domestic LLM providers.
- AppGroupStore.defaultSystemPrompt is now provider-aware
(Chinese for zhipu/moonshot/qwen/deepseek, English otherwise).
Onboarding & Settings
---------------------
- Re-themed OnboardingView / HomeView / SettingsView on the
new design system.
- ProviderPickerSection now shows 6 providers (OpenAI, DeepSeek,
Qwen DashScope, 智谱 GLM, 月之暗面 Moonshot, Custom) with
blurb + selected accent.
- PickerRow for Mode and ASR locale; System Prompt editor with
reset-to-default.
- API settings page "Get an API key" used SwiftUI Link, which
has a hit-test bug on iOS 18 that ate gestures from adjacent
TextFields (manifested as "typing jumps to a website"). It is
now an explicit Button + contentShape + .submitLabel(.done) on
the fields.
Polish & tests
--------------
- LLMClientTests: 4 unit tests passing (ProviderConfig
persistence + OpenAI request/response + HTTP error + missing
key); test App Group renamed to the correct identifier.
- ProviderConfig.apply now captures the previous provider id
*before* mutating, so switching providers actually resets the
system prompt to the new default.
Build
-----
- Swift 6 strict concurrency, iOS 18.0 deployment target.
- Tested on Xcode 26 + iPhone 17 Pro simulator. A real device on
iOS 27 beta aborts with __abort_with_payload (dispatch
library ABI mismatch); use an iOS 18 real device or the
iOS 26 simulator for now.
🤖 Generated with Claude Code
Rocky
rocky