Long-press mic runs ASR as an instruction over eligible clipboard text,
with host-confirm recording UI, min-record gate, and light ASR prewarm.
Bump CURRENT_PROJECT_VERSION to 53.
Custom polish styles can opt in to emotion-matched emoji (default off), with
prompt-level opt-in detection so paste-only styles keep model-added emoji.
Also include Volcengine API-Key ASR auth, voice-processing capture, PiP flash
fix, and related keyboard Shift/haptics reliability work.
Redeploy an osg_personal import table on dictionary add/delete/sync so
Chinese, English typing, and ASR share one curated lexicon (next keyboard open).
Ship official store badges on READMEs and the site, expand discoverability
with FAQ/JSON-LD/llms.txt/content pages, and add the Search Console HTML
verification file so the live Pages site can be claimed.
Ship official store badges on READMEs and the site, expand discoverability
with FAQ/JSON-LD/llms.txt/content pages, and add the Search Console HTML
verification file so the live Pages site can be claimed.
Retain the external target app for menu-bar paste and polish context, retry
failed middle ASR chunks once, and stop false-positive path/number violations.
Use redacted cursor neighborhood and pause-aware chunks for more natural polish,
validate protected terms with retry/local fallback, and structure bilingual prompts
for consistency and provider prefix caching.
The hold-to-talk release path called AsyncStream.Continuation.finish()
while holding MacAudioRecorder's NSLock; onTermination re-entered the
same lock on the main thread and froze the app. Finish snapshot sinks
outside the lock and guard the audio tap once stop() begins.
The dictation HUD no longer reassigns its hosting view and forces a
synchronous relayout on every view-model tick; it uses a fixed panel
size and SwiftUI @ObservedObject refresh instead.
Adds OSGKeyboardMacTests with a regression test that fails under the
old lock re-entry pattern.
Harden polish so question drafts stay questions, add local density
routing with style-specific degrade, expand fun style packs, and add
end-to-end FlowTrace logging plus offline guard eval scripts.
Unify Bailian/Volcengine/OpenAI realtime streaming, ABE polish routing with
fun styles, and a shared card-page Settings hierarchy; bump to 1.1 (build 32).
Add shared prompt composition, custom style management, iCloud sync, and native iOS/macOS selection interfaces so users can keep a consistent writing voice across devices.
Use compliant permission prompts, align cloud ASR privacy disclosures, and default Flow sessions to a shorter inactivity window for safer resubmission.
Replace the Sherpa offline pipeline with native MLX streaming, resilient model downloads, live transcript previews, and supporting tests and documentation.
Replace the five-card stack with a pill tab bar and elevated detail
stage, use a white band for the competitor matrix, and a soft gray
band for the three-step setup section.
Replace the duplicate BYOK block with hover/tap detail panels under
the differentiators, drop the hero title period, center the matrix
footnote, and alternate section washes so vs / steps read as separate bands.
Retitle the differentiation, BYOK, and setup sections with richer
copy, and add a Typeless / Superwhisper / Openless / OSGKeyboard table
covering pricing, on-device ASR, BYOK, and platforms.
Add System/Light/Dark preference for iPhone and iPad, restore the iPad
sidebar brand mark, ship a DEBUG seed-demo URL for rich placeholder data,
and refresh GitHub Pages with the device composite, full-bleed pale-green
hero wash, and transparent Mac screenshot chrome.
Capture Mac and iPhone product shots for zh/en × light/dark, store under
assets/screenshots/{lang}/{theme}/, and swap images when the page
toggles language or appearance (including system theme).
Add optional StoreKit support tip (features stay free), redesign the
landing page with zh/en + light/dark and product shots, and ship
iPhone/iPad/Mac App Store screenshot sets.
Fix cold-start overlay recursion that overflowed the main-thread stack when
recording began while the ready overlay was still up; also remove temporary
on-screen Flow DEBUG panels after the orange-mic investigation, and land the
macOS overlay/catalog/layout polish plus related Flow recovery hardening.
End Live Activities and release the audio session synchronously on
applicationWillTerminate, and continue macOS local-model install,
onboarding, and settings polish on this branch.
Adds a bundled local ASR model catalog for the macOS app with one-click
Sherpa Qwen3 / SenseVoice downloads (pause/resume, inline actions) and a
shared model storage directory used by MLX Qwen3. Fixes the light-mode
sidebar material and makes the menu-bar icon follow the system appearance
with a refreshed status mark. Renames the built product to OSGKeyboard.app.
Bumps version to 0.5.2 (build 19).
Introduce a standalone macOS menu-bar app (OSGKeyboardMac) that reuses the
platform-agnostic OSGKeyboardShared core: record -> cloud/local ASR -> polish
-> insert. Local mode uses Qwen3-ASR via mlx-swift-asr (macOS 15+, Apple
Silicon); iOS targets stay zero-SPM.
Harden iCloud sync for multi-device correctness:
- Per-field settings merge (appSettings.v2) so concurrent edits no longer
clobber each other's unrelated fields.
- Per-device usage statistics (G-Counter) that sum instead of max().
- Tombstoned dictionary/history merge so deletes propagate and entries can't
resurrect.
- API keys replicate via iCloud Keychain, never iCloud KVS JSON; pulling a
legacy blob without key fields no longer wipes local Keychain entries.
- Add a low-risk "Sync Now" action in Settings.
Fix Flow keyboard mic state: stay orange until the host publishes a real ready
contract, share a single MicVoiceAvailability gate, and self-heal stale
cross-process heartbeat jitter instead of getting stuck.
Extract shared storage (SpeechHistoryStore/UsageStatisticsStore,
ConfigurationStore) into OSGKeyboardShared and add tests for the new
sync/merge logic.
Mirror the personal dictionary through iCloud Key-Value Store with
deterministic merge rules, main-app-only sync UI, and App Group as the
keyboard runtime cache. Add cloud-engine ASR with dictionary bias and
expand the bundled custom language model lexicon.
- fix(keyboard): drop unusable extensionContext.open; report the real
result of UIApplication.open and cancel the start watchdog immediately
on failure with clear "open OSGKeyboard" guidance instead of a 30s spin
- feat(capture): recover from audio route changes (AirPods/wired headset
plug-unplug) and phone-call interruptions by rebuilding the engine/tap
against the live hardware format
- feat(home): add a "Start" button in the Home footer when a session is
inactive and permissions are granted (no-jump manual restart)
- feat(hostapps): expand the return whitelist to 46 apps (Signal, Zoom,
Instagram, X, Douyin, Zhihu, Bilibili, Things, Firefox, ...) with
en/zh-Hans names and synced LSApplicationQueriesSchemes
- feat(island): use a "ready" checkmark instead of a mic glyph while idle
and set a staleDate so an orphaned Live Activity fades after a force-quit
- feat(coldstart): use the template OSG brand mark instead of a system
waveform glyph in the cold-start overlay
- perf(keyboard): rely on Darwin notifications as the primary session
signal and drop the fallback poll from 1s to 3s
- fix(keyboard): remove the redundant "Start" text button next to the
inactive-session hint on the keyboard
- docs: correct the per-take cap to 210s (3.5 min) and document that
force-quitting no longer resurrects the session (tracker + READMEs)
Reduce perceived latency from key release to final text:
- Adaptive chunking: 2.5s first chunk + 5s follow-ups so short
utterances start on-device recognition while still recording.
- Session-level ASR warmup and audio-format cache reuse to remove
per-utterance cold-start of SpeechAnalyzer.
- Mirror live pipelined partials to the keyboard transcript line via
a new flow.transcriptionPartial App Group key + Darwin ping.
Also commits the accumulated custom language model, Flow session,
keyboard extension restructure, and Xiaomi MiMo provider work in
progress on this branch.
Rework the marketing site with a cleaner, more restrained layout, replace emoji with Material Symbols monochrome icons, and add full SEO (Open Graph, Twitter cards, JSON-LD, canonical, robots.txt, sitemap.xml).
Rework the marketing site with a cleaner, more restrained layout, replace emoji with Material Symbols monochrome icons, and add full SEO (Open Graph, Twitter cards, JSON-LD, canonical, robots.txt, sitemap.xml).
Build a first-version SFCustomLanguageModelData for on-device ASR
customization, generated offline on macOS (no app code dependency).
- Scripts/lexicon: Python builders for Sogou-derived phrases (scel +
SogouPopularDict) and a curated bilingual AI/tech/brand seed lexicon
- export_clm.swift / prepare_clm.swift: macOS CLI tools that export the
.bin training asset and compile it into LM + Vocab via the Speech
framework
- Resources/CustomLanguageModel: generated phrases.tsv, manifests, the
129k-phrase .bin, and compiled LM/Vocab assets (zh_CN)
- .gitignore: ignore lexicon build cache and Python bytecode
Note: Sogou-derived data is for internal experimentation only; the
curated AI/tech seed is MIT and safe to ship.
Batch of in-progress app work from the working tree.
- feat(keyboard): CursorNavigation + CursorDragPad for caret movement;
KeyboardSoundFeedback for system key click sounds
- feat(dictionary): DictionaryAliasGenerator + PersonalDictionaryEntrySheet;
TranscriptPostProcessor quality gate; retire DictionaryLearner
- feat(ui): TabBarVisibility handling; drop PageHeaderRow /
PageHeaderConfirmButton; refresh views and localizable strings
- fix(security): move the hardcoded DeepSeek key out of
PreconfiguredKeys.swift into a gitignored PreconfiguredKeys.local.swift
(seeded from .example by generate-xcodeproj.sh)
- docs(agents): add Conventional Commits versioning + bilingual changelog rules
- chore(gitignore): ignore PreconfiguredKeys.local.swift, .cache/, pycache
Custom language model / lexicon work stays on
feature/custom-language-model-asr. Changelog bullets added under
[Unreleased]; no version bump.
Rework the action cluster to a mic-above-bottom-row layout, add left/right
handedness setting that swaps delete and return, and keep the screen awake
during Flow recording sessions.
Add preset-driven polish scenarios (Settings, onboarding, ScenarioChip)
with ScenarioPrompt and style directives; drive keyboard height from
content (240pt) and use viewIsAppearing encapsulated-height offset for
smoother keyboard switches; remove redundant StatusBadge and hide system
dictation via hasDictationKey.
Replace MLX GPU inference with CoreML bundles so transcription continues
while the host app is backgrounded. Adds model download and warm-up,
vendored Qwen3Speech, and updates onboarding, settings, and copy for the
~1.6 GB CoreML package (iOS 18+).
Replace MIT with a restrictive source-available license and align README/docs copy. Add cloud polish acknowledgment, privacy manifest updates, voice history disclosure, export compliance metadata, and support links for App Store readiness.
Streamline the five-step flow with smarter skip logic, a centered welcome intro, clearer zh copy, keyboard setup detection via the extension, and home tips when permissions are still missing after onboarding completes.
Resolve TestFlight ITMS-91056 by correcting privacy manifest keys and reason arrays. Add left-aligned page headers with circular confirm buttons, keep the keyboard surface transparent, and improve unselected dock icon contrast in light and dark mode.
Renew voice sessions while the host app stays foreground, auto-start flow from the keyboard with a Start action, and prevent leading audio loss via pre-roll buffering and faster signal polling.
Introduce MainTabView with history and Liquid Glass dock; refresh home and
onboarding layouts; unify accent green and provider localization; refine
settings/API rows; redesign keyboard mic and flanking controls; set Utility
iPhone-only targets and Flow session reliability fixes.
XcodeGen merged Localizable.strings into the main app only, so the keyboard
extension had no .lproj files and NSLocalizedString returned raw keys. Use a
dedicated Keyboard.strings table and explicit buildPhase: resources entries.
Icon Composer AppIcon.icon must not be compiled alongside AppIcon.appiconset;
actool crashes with a nil insert error. Use appiconset only and exclude .icon
bundles from the target.
Refresh mic/speech status when returning from TCC dialogs, persist onboarding
page across Settings trips, load extension strings from .lproj bundles via
ExtL10n, and keep flow result polling alive across host-app jumps.
Remove KeyboardL10n hard-coded fallbacks in favor of ExtL10n and extension
Localizable.strings. Add Flow session expiry hints, Darwin cross-process
notifications, session monitor on the keyboard, and app icon on GitHub Pages.
Migrate keyboard dictation to continuous Flow sessions with auto-start,
tap-to-toggle recording, 60s countdown, five-step onboarding, and App Group
IPC. Add docs/ GitHub Pages site with en/zh privacy policy for App Store compliance.
Keep repository messaging consistent with the implemented iOS 26 SpeechAnalyzer path, and remove mixed hardcoded copy by routing remaining UI/error text through localized string keys.
The previous build crashed on the first record-tap with:
*** Terminating app due to uncaught exception
'com.apple.coreaudio.avfaudio', reason:
'Failed to create tap due to format mismatch,
<AVAudioFormat: 1 ch, 48000 Hz, Float32>'
The crash surfaced at `[AVAudioNode installTap:...]` but the
*root* cause was one frame deeper, in the underlying
`AURemoteIO::enable` call:
AURemoteIO.cpp:1135 failed: -10851
(enable 1, outf< 2 ch, 0 Hz, Float32, deinterleaved>
inf< 1 ch, 48000 Hz, Float32>)
`AURemoteIO` is a two-direction Audio Unit (input + output).
`installTap` triggers `AURemoteIO::enable(1)` which tries to
initialize *both* directions at once. On the iOS Simulator, the
output direction reports a `0 Hz` "speaker" because the
simulator has no real speaker, and `enable` fails with
`kAudioUnitErr_FormatNotSupported` (-10851). The error then
bubbles up through `installTap` as the misleading "format
mismatch" — the input format we passed was correct, but the
output side broke the whole `enable` call.
The previous code asked the audio session for `.playAndRecord`
mode, which requires the output direction to be enabled. On a
real device, the speaker is 44100/48000 Hz and `.playAndRecord`
works. On the simulator, it traps.
Fix (two parts):
1) `start()` audio session now uses `.record` instead of
`.playAndRecord`. The preview never plays back audio — it
just records from the mic and hands the buffers to
`SpeechAnalyzer`. `.record` skips the output direction
entirely, so the simulator's 0 Hz speaker isn't a problem.
The real keyboard extension (`OSGKeyboardExt`) keeps
`.playAndRecord` because it runs on real devices and may
want to play click sounds / haptic feedback — only the
preview needs the simulator-friendly category.
2) `startEngineAndASR` adds a pre-flight check: refuse to
call `installTap` when the input bus reports
`sampleRate == 0` or `channelCount == 0`. The `installTap`
failure mode is an `NSException` (Objective-C), not a Swift
`Error` — `try`/`catch` can't intercept it, so the only safe
defence is to never call it with a placeholder/unconfigured
bus. We saw `0 Hz` inputs on the simulator when the host
mic permission wasn't granted to CoreSimulator, and on
devices with an unexpected audio-session state. The check
sets a clear `.error` phase ("Microphone unavailable (hw
format X Hz / Y ch)") instead of trapping the app.
Why this also unblocks voice-to-text: the previous
"no transcription" symptom was just the crash at step 7 of
the start pipeline — the ASR never saw a single audio frame
because the tap never installed. With the crash fixed, the
`SpeechAnalyzer` pipeline can now actually receive buffers
and emit `.partial` / `.final` events. Whether the *content*
of those events is meaningful depends on the simulator's mic
quality and the chosen locale (both separate concerns from
this fix).
Tests: 27/27 pass (no behavioural test for the pre-flight
check — the check is 4 lines of obvious defence, and
testing it would require either mocking the audio engine
or installing CoreSimulator's host-mic-bridge, neither of
which is worth the cost).
Build: BUILD SUCCEEDED.
🤖 Generated with Claude Code
Rocky