Rotate AI idle suggestions with optional remote packs, move history/dictionary onto self-sizing Home preview cards, harden clipboard capture/prompting, and simplify keyboard chrome by dropping most liquid-glass shadows.
Ship optional keyboard clipboard history with settings, suggestion strip, and paste-permission guidance; let undo roll back clipboard inserts; bump build to 64.
Add the AI keyboard tab, Agent settings, and user-owned LLM key path for 1.7.0, including streaming answers and web-search transports without the built-in DeepSeek fallback.
Adapt typing/voice surfaces for iPad width and height, add the system globe
key and last-input editing flow, harden host-only Rime deployment, and remove
clipboard voice commands. Bump build to 61.
Keep marketing version at 1.6.6 and bump build to 59. Strengthen never-answer
polish safeguards, clipboard reply-intent continuity, voice undo UX, device
UITests harness, eval fixtures, and What's New assets.
Stabilize clipboard long-press prepare/resume across paste alerts and cold start, add an in-app release-notes sheet with remote bilingual HTML, localize typing input settings, and bump build to 55.
Long-press mic runs ASR as an instruction over eligible clipboard text,
with host-confirm recording UI, min-record gate, and light ASR prewarm.
Bump CURRENT_PROJECT_VERSION to 53.
Custom polish styles can opt in to emotion-matched emoji (default off), with
prompt-level opt-in detection so paste-only styles keep model-added emoji.
Also include Volcengine API-Key ASR auth, voice-processing capture, PiP flash
fix, and related keyboard Shift/haptics reliability work.
The hold-to-talk release path called AsyncStream.Continuation.finish()
while holding MacAudioRecorder's NSLock; onTermination re-entered the
same lock on the main thread and froze the app. Finish snapshot sinks
outside the lock and guard the audio tap once stop() begins.
The dictation HUD no longer reassigns its hosting view and forces a
synchronous relayout on every view-model tick; it uses a fixed panel
size and SwiftUI @ObservedObject refresh instead.
Adds OSGKeyboardMacTests with a regression test that fails under the
old lock re-entry pattern.
Unify Bailian/Volcengine/OpenAI realtime streaming, ABE polish routing with
fun styles, and a shared card-page Settings hierarchy; bump to 1.1 (build 32).
- Add FlowKeepAliveMode (Dynamic Island vs PiP) with iCloud sync
- Settings: keep-alive picker; inactivity timeout only for Live Activity
- PiP: waveform sample-buffer controller, mic released between utterances
- PiP sessions have no idle expiry; user closing PiP ends the session
- FlowSessionManager branches hostReady, session start, and utterance paths
- UIBackgroundModes picture-in-picture; bilingual strings and tests
Co-authored-by: Rocky <hkgood@users.noreply.github.com>
Use compliant permission prompts, align cloud ASR privacy disclosures, and default Flow sessions to a shorter inactivity window for safer resubmission.
Replace the Sherpa offline pipeline with native MLX streaming, resilient model downloads, live transcript previews, and supporting tests and documentation.
Add System/Light/Dark preference for iPhone and iPad, restore the iPad
sidebar brand mark, ship a DEBUG seed-demo URL for rich placeholder data,
and refresh GitHub Pages with the device composite, full-bleed pale-green
hero wash, and transparent Mac screenshot chrome.
Add optional StoreKit support tip (features stay free), redesign the
landing page with zh/en + light/dark and product shots, and ship
iPhone/iPad/Mac App Store screenshot sets.
Fix cold-start overlay recursion that overflowed the main-thread stack when
recording began while the ready overlay was still up; also remove temporary
on-screen Flow DEBUG panels after the orange-mic investigation, and land the
macOS overlay/catalog/layout polish plus related Flow recovery hardening.
End Live Activities and release the audio session synchronously on
applicationWillTerminate, and continue macOS local-model install,
onboarding, and settings polish on this branch.
Adds a bundled local ASR model catalog for the macOS app with one-click
Sherpa Qwen3 / SenseVoice downloads (pause/resume, inline actions) and a
shared model storage directory used by MLX Qwen3. Fixes the light-mode
sidebar material and makes the menu-bar icon follow the system appearance
with a refreshed status mark. Renames the built product to OSGKeyboard.app.
Bumps version to 0.5.2 (build 19).
Introduce a standalone macOS menu-bar app (OSGKeyboardMac) that reuses the
platform-agnostic OSGKeyboardShared core: record -> cloud/local ASR -> polish
-> insert. Local mode uses Qwen3-ASR via mlx-swift-asr (macOS 15+, Apple
Silicon); iOS targets stay zero-SPM.
Harden iCloud sync for multi-device correctness:
- Per-field settings merge (appSettings.v2) so concurrent edits no longer
clobber each other's unrelated fields.
- Per-device usage statistics (G-Counter) that sum instead of max().
- Tombstoned dictionary/history merge so deletes propagate and entries can't
resurrect.
- API keys replicate via iCloud Keychain, never iCloud KVS JSON; pulling a
legacy blob without key fields no longer wipes local Keychain entries.
- Add a low-risk "Sync Now" action in Settings.
Fix Flow keyboard mic state: stay orange until the host publishes a real ready
contract, share a single MicVoiceAvailability gate, and self-heal stale
cross-process heartbeat jitter instead of getting stuck.
Extract shared storage (SpeechHistoryStore/UsageStatisticsStore,
ConfigurationStore) into OSGKeyboardShared and add tests for the new
sync/merge logic.
Mirror the personal dictionary through iCloud Key-Value Store with
deterministic merge rules, main-app-only sync UI, and App Group as the
keyboard runtime cache. Add cloud-engine ASR with dictionary bias and
expand the bundled custom language model lexicon.
- fix(keyboard): drop unusable extensionContext.open; report the real
result of UIApplication.open and cancel the start watchdog immediately
on failure with clear "open OSGKeyboard" guidance instead of a 30s spin
- feat(capture): recover from audio route changes (AirPods/wired headset
plug-unplug) and phone-call interruptions by rebuilding the engine/tap
against the live hardware format
- feat(home): add a "Start" button in the Home footer when a session is
inactive and permissions are granted (no-jump manual restart)
- feat(hostapps): expand the return whitelist to 46 apps (Signal, Zoom,
Instagram, X, Douyin, Zhihu, Bilibili, Things, Firefox, ...) with
en/zh-Hans names and synced LSApplicationQueriesSchemes
- feat(island): use a "ready" checkmark instead of a mic glyph while idle
and set a staleDate so an orphaned Live Activity fades after a force-quit
- feat(coldstart): use the template OSG brand mark instead of a system
waveform glyph in the cold-start overlay
- perf(keyboard): rely on Darwin notifications as the primary session
signal and drop the fallback poll from 1s to 3s
- fix(keyboard): remove the redundant "Start" text button next to the
inactive-session hint on the keyboard
- docs: correct the per-take cap to 210s (3.5 min) and document that
force-quitting no longer resurrects the session (tracker + READMEs)
- Add OSGKeyboardLiveActivity widget extension (ActivityKit) with branded
compactLeading mark in the Dynamic Island during Flow sessions
- Wire FlowLiveActivityController into FlowSessionManager lifecycle
- Set keyboard PrimaryLanguage to mis in project.yml, Info.plist, and at runtime
- Enable NSSupportsLiveActivities on main app and widget extension targets
Co-authored-by: Rocky <hkgood@users.noreply.github.com>
Reduce perceived latency from key release to final text:
- Adaptive chunking: 2.5s first chunk + 5s follow-ups so short
utterances start on-device recognition while still recording.
- Session-level ASR warmup and audio-format cache reuse to remove
per-utterance cold-start of SpeechAnalyzer.
- Mirror live pipelined partials to the keyboard transcript line via
a new flow.transcriptionPartial App Group key + Darwin ping.
Also commits the accumulated custom language model, Flow session,
keyboard extension restructure, and Xiaomi MiMo provider work in
progress on this branch.
Batch of in-progress app work from the working tree.
- feat(keyboard): CursorNavigation + CursorDragPad for caret movement;
KeyboardSoundFeedback for system key click sounds
- feat(dictionary): DictionaryAliasGenerator + PersonalDictionaryEntrySheet;
TranscriptPostProcessor quality gate; retire DictionaryLearner
- feat(ui): TabBarVisibility handling; drop PageHeaderRow /
PageHeaderConfirmButton; refresh views and localizable strings
- fix(security): move the hardcoded DeepSeek key out of
PreconfiguredKeys.swift into a gitignored PreconfiguredKeys.local.swift
(seeded from .example by generate-xcodeproj.sh)
- docs(agents): add Conventional Commits versioning + bilingual changelog rules
- chore(gitignore): ignore PreconfiguredKeys.local.swift, .cache/, pycache
Custom language model / lexicon work stays on
feature/custom-language-model-asr. Changelog bullets added under
[Unreleased]; no version bump.
Remove fileTypes icon override and explicit type:file so XcodeGen 2.37+
emits folder.iconcomposer.icon directly. Skip patch when native output is
valid; otherwise run patch v3 with broader PBX discovery and validation.
Fixes persistent 'AppIcon.icon entry has unexpected shape' on Xcode 26.
Co-authored-by: Rocky <hkgood@users.noreply.github.com>
Rework the action cluster to a mic-above-bottom-row layout, add left/right
handedness setting that swaps delete and return, and keep the screen awake
during Flow recording sessions.
Add preset-driven polish scenarios (Settings, onboarding, ScenarioChip)
with ScenarioPrompt and style directives; drive keyboard height from
content (240pt) and use viewIsAppearing encapsulated-height offset for
smoother keyboard switches; remove redundant StatusBadge and hide system
dictation via hasDictationKey.
Adds an opt-in translation pipeline that reuses the existing
PolishingService + LLMClient + AppGroupStore chain. Translation is
implemented as a new PolishMode (.translate(targetLocaleId:)); all
existing call sites are unchanged.
Settings:
- New TranslationPickerRow in the language tab (Toggle + 10-locale
picker: en/zh-Hans/zh-Hant/ja/ko/fr/de/es/ru/pt), persisted to the
App Group so the keyboard extension can read it during live dictation.
- 5 new strings per language (en + zh-Hans).
Keyboard:
- New TranslationChip on the top bar to the right of LocaleChip;
same Menu pattern, lets users toggle or quickly switch target
language without leaving the keyboard.
- PolishingService dispatches .translate with a parameterised prompt
(en/zh variants selected by provider id); PolishingService.error
gains a translationNotAvailable case so local-engine users get a
clear inline warning when the toggle is on but cloud is off.
- 6 new strings per language (en + zh-Hans) for the chip + banner.
Local engine policy:
- Translation is cloud-only by design (local engine stays ASR-only
to honour the no-roundtrip promise). Chip shows a 'cloud required'
state and raw transcript still inserts on failure — no data loss.
Build:
- OSGKeyboardShared adds TranslationLanguage enum (10 locales) and
TranslationPrompt factory.
- 4 new files, 9 modified. xcodebuild scheme=OSGKeyboard
config=Debug destination=iPhone 17 Simulator: BUILD SUCCEEDED
(0 warning, 0 error).
Also pins DEVELOPMENT_TEAM in project.yml for TestFlight uploads
(3 targets; Team X329MZU23S).
Rolls back the v0.2.0 Qwen3 CoreML on-device ASR stack and replaces the
'local engine' UX with iOS 26 SpeechAnalyzer + DictationTranscriber only.
The 'Cloud polish after ASR' toggle (ProviderConfig.localModeCloudPolishEnabled)
lets users opt into a post-ASR DeepSeek round-trip from the local engine.
Defaults to off so the local engine stays genuinely local. New PolishError.missingAPIError
surfaces an inline 'fill in your key' warning when the toggle is on but the
Keychain is empty. DeepSeek preset default model bumped to deepseek-v4-flash.
Deleted:
- OSGKeyboard/ThirdParty/Qwen3Speech/ (74 files, ~16k LoC)
- OSGKeyboard/Services/ModelManager.swift (492)
- OSGKeyboard/Services/OnDeviceModelWarmup.swift (197)
- OSGKeyboard/Services/Qwen3ASRService.swift (257)
- OSGKeyboard/Services/ModelDownloadSourcePicker.swift (126)
- OSGKeyboard/Views/OnDeviceModelsView.swift (184)
- OSGKeyboard/Views/DownloadConfirmSheet.swift (96)
- OSGKeyboardShared/Models/OnDeviceModel.swift (140)
- OSGKeyboardShared/Services/OnDeviceModelStatus.swift (104)
- Qwen3ASRServiceProvider registration in OSGKeyboardApp
- Qwen3Speech package declaration in project.yml
- 5 .qwen3ASR enum / branch reference sites in HomeView, OnboardingView,
LocalEngineSettingsRows, FlowSessionManager, ASRService, EngineServiceLabel
- Two pre-existing Swift 6 strict-concurrency errors in
LiveDictationController + FlowSessionManager (the weak [weak self] in
detached-task MainActor.run blocks) that were blocking clean builds
Added:
- LocalModelsGroup: 'Built-in iOS SpeechAnalyzer' badge + 'Cloud polish
after ASR' Switch toggle
- PolishingService: honour localModeCloudPolishEnabled; new .missingAPIKey
error case with localised warning
- AppGroupStore.localModeCloudPolishEnabled (mirrored into App Group
so the keyboard extension honours the toggle during live dictation)
- SettingsView: show provider/api sections when local-mode cloud polish
is on so the user can paste a DeepSeek key
- FlowSessionManager: route through PolishingService for local + polish-on
flow; translate missingAPIKey into a polished warning
- KeyboardViewController: handle PolishingService.PolishError.missingAPIKey
in the keyboard-side live polish path
- CHANGELOG v0.2.1: documents the rollback + new toggle
- README.md / README.zh.md: engine matrix section, data flow note
Verified: xcodebuild -scheme OSGKeyboard -destination 'generic/platform=iOS Simulator'
build succeeds under SWIFT_STRICT_CONCURRENCY=complete.
Replace MLX GPU inference with CoreML bundles so transcription continues
while the host app is backgrounded. Adds model download and warm-up,
vendored Qwen3Speech, and updates onboarding, settings, and copy for the
~1.6 GB CoreML package (iOS 18+).
Replace MIT with a restrictive source-available license and align README/docs copy. Add cloud polish acknowledgment, privacy manifest updates, voice history disclosure, export compliance metadata, and support links for App Store readiness.
Apple's annual encryption self-classification questionnaire
pops up for every build that does not declare this key. All
network calls in OSGKeyboard are HTTPS (the LLM polish call
hits a user-configured OpenAI-compatible endpoint) and Apple
classifies standard HTTPS as exempt under EAR Category 5 Part
2 Note 4, so the app does not need an encryption registration.
The Info.plist properties live in project.yml (XcodeGen), which
is what xcodebuild actually consumes. The Info.plist stub on
disk is regenerated from project.yml by xcodegen and is
gitignored in spirit (it is checked in for tool compatibility
but XcodeGen treats it as the canonical source).
Verified by:
plutil -p .derivedData/.../OSGKeyboard.app/Info.plist | grep -i encrypt
-> "ITSAppUsesNonExemptEncryption" => 0
Refs: AUDIT_APPSTORE.md P0-4
Rocky
rocky