Rocky
bec36befa2
This is a major rewrite of OpenLessKeyboard, renamed to OSGKeyboard
and rebuilt end-to-end. 59 files changed (+3205/-1550).
Architecture
------------
- Rename project, targets, directories from OpenLess* to OSGKeyboard*
(OpenLess / OpenLessKeyboard / OpenLessShared / OpenLessTests).
- AudioCaptureService rewritten as @unchecked Sendable class with
OSAllocatedUnfairLock instead of an actor, so it survives Swift 6
strict-concurrency checks while still serialising engine + converter
state correctly.
- Single design system (Palette / Spacing / Radius / TypeStyle /
Motion) lifted into OSGKeyboardShared so the host app and the
keyboard extension stay in lock-step.
Push-to-talk — first-principles fix
-----------------------------------
- App Group + audio-input entitlements were stripped by Xcode's
Automatic Signing. They are now declared in project.yml so
'xcodegen generate' re-emits them every time. iOS Developer
Account is untouched; only the App Group capability was added.
- State machine uses a real stored `phase` (was a derived shim
that locked out every press after the first because
recordStream was never nilled after the pipeline finished).
- Microphone permission is requested inside pressBegan (async
Task) so the press flow optimistically enters .recording;
permission denial surfaces a short error and returns to idle.
- Replaced LongPressGesture(0.15s) with a DragGesture +
TapGesture pair separated by pressArmed, so a single tap no
longer fires both onPressBegan and onTap simultaneously.
- Real RMS / peak level meter from the AVAudioEngine tap (was a
pseudo-random walk); the visible waveform is now driven by
actual audio.
- SFSpeechRecognizer(locale:) with selectable ASR locales
(auto / zh-Hans / zh-Hant / en-US / ja-JP / ko-KR) for
first-class Chinese / English / Japanese / Korean dictation,
with on-device recognition when supported.
- AVAudioSession now deactivates on stop so other apps' audio
routing is restored.
Keyboard UI — Typeless-inspired layout
---------------------------------------
- Hero area is 280 pt with a 96 pt record disc, breathing outer
ring, and a 12-bar waveform driven by the real RMS.
- inputView.allowsSelfSizing + a heightAnchor constraint so iOS
no longer crops the keyboard under the Spotlight bar / home
indicator.
- Top bar: mode chip (Off / 转写 / 润色) + locale chip
(Auto / 简体 / 繁體 / EN / 日 / 한) + status badge + ⚙.
- Bottom bar: globe / delete / 空格 / return — all 40 pt and
balanced.
- RecordButton onPressEnded is now safe to fire from a quick
press; pressArmed prevents double-firing.
LLM / Polishing
---------------
- LLMClient: stopped leaking the server response body in errors
(server body is now logged at debug, never surfaced to UI);
added a dedicated .rateLimited case for 429.
- PolishingService timeout 8s → 12s to accommodate slower
domestic LLM providers.
- AppGroupStore.defaultSystemPrompt is now provider-aware
(Chinese for zhipu/moonshot/qwen/deepseek, English otherwise).
Onboarding & Settings
---------------------
- Re-themed OnboardingView / HomeView / SettingsView on the
new design system.
- ProviderPickerSection now shows 6 providers (OpenAI, DeepSeek,
Qwen DashScope, 智谱 GLM, 月之暗面 Moonshot, Custom) with
blurb + selected accent.
- PickerRow for Mode and ASR locale; System Prompt editor with
reset-to-default.
- API settings page "Get an API key" used SwiftUI Link, which
has a hit-test bug on iOS 18 that ate gestures from adjacent
TextFields (manifested as "typing jumps to a website"). It is
now an explicit Button + contentShape + .submitLabel(.done) on
the fields.
Polish & tests
--------------
- LLMClientTests: 4 unit tests passing (ProviderConfig
persistence + OpenAI request/response + HTTP error + missing
key); test App Group renamed to the correct identifier.
- ProviderConfig.apply now captures the previous provider id
*before* mutating, so switching providers actually resets the
system prompt to the new default.
Build
-----
- Swift 6 strict concurrency, iOS 18.0 deployment target.
- Tested on Xcode 26 + iPhone 17 Pro simulator. A real device on
iOS 27 beta aborts with __abort_with_payload (dispatch
library ABI mismatch); use an iOS 18 real device or the
iOS 26 simulator for now.
🤖 Generated with Claude Code
154 lines
6.5 KiB
Markdown
154 lines
6.5 KiB
Markdown
# OSGKeyboard
|
|
|
|
> Hold a key, speak, release — AI-polished text appears at your cursor in any app.
|
|
> An open-source, custom-keyboard-based voice input tool for iOS 18+, inspired by [Typeless](https://typeless.com) and [OpenLess](https://github.com/Open-Less/openless).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[中文 README](./README.zh.md)
|
|
|
|
---
|
|
|
|
## What is it?
|
|
|
|
OSGKeyboard is a free, open alternative to commercial voice-input tools. It runs as a **Custom Keyboard Extension** on iOS, so you can use it in **any app** — Messages, Notes, Mail, ChatGPT, Claude, Cursor, you name it.
|
|
|
|
1. Press and hold the mic key
|
|
2. Speak naturally
|
|
3. Release — the AI polishes your words into clean text and inserts it at the cursor
|
|
|
|
The audio stays on-device (transcribed by Apple's on-device `SpeechAnalyzer` on iOS 26+, or `SFSpeechRecognizer` on iOS 18/19). Only the **polished transcript** is sent to your chosen cloud LLM. **No audio ever leaves your phone.**
|
|
|
|
---
|
|
|
|
## Features
|
|
|
|
- 🎙 **Push-to-talk** with a Typeless-style circular mic button
|
|
- 🧠 **On-device ASR** (iOS 26 `SpeechAnalyzer` + `DictationTranscriber`; iOS 18 `SFSpeechRecognizer` fallback)
|
|
- ✍️ **AI polishing** — adds structure, punctuation, fixes grammar, optionally produces lists
|
|
- 🔌 **Bring-your-own API** — works with any OpenAI-compatible endpoint (OpenAI, DeepSeek, Qwen DashScope, your own self-hosted server, …)
|
|
- 🔒 **Privacy first** — audio never leaves your device; transcripts only sent to the LLM you choose
|
|
- 🎨 **Native SwiftUI** — dark theme, frosted glass, ~2000 lines of Swift
|
|
- 🪶 **Zero dependencies** — no SwiftPM packages, no CocoaPods, no Carthage
|
|
|
|
---
|
|
|
|
## Quick start
|
|
|
|
### Requirements
|
|
|
|
- macOS with **Xcode 16+** (Xcode 26 recommended)
|
|
- iPhone running **iOS 18.0+** (iOS 26+ for the best on-device ASR)
|
|
- [XcodeGen](https://github.com/yonaskolb/XcodeGen): `brew install xcodegen`
|
|
- An OpenAI-compatible API key (e.g. from [OpenAI](https://platform.openai.com/api-keys), [DeepSeek](https://platform.deepseek.com/api_keys), or [Qwen DashScope](https://dashscope.console.aliyun.com/apiKey))
|
|
|
|
### Build & run
|
|
|
|
```bash
|
|
git clone https://github.com/<OWNER>/OSGKeyboard.git
|
|
cd OSGKeyboard
|
|
xcodegen generate # produces OSGKeyboard.xcodeproj
|
|
open OSGKeyboard.xcodeproj # or build via CLI:
|
|
xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
|
|
-destination 'generic/platform=iOS Simulator' build
|
|
```
|
|
|
|
### Enable the keyboard in iOS
|
|
|
|
1. Run the app on your device or simulator.
|
|
2. Follow the 3-step onboarding: **enable the keyboard** in iOS Settings, then **allow Full Access** (required for the mic and LLM calls), then **paste your API key**.
|
|
3. In any text field, tap 🌐 to switch to **OSGKeyboard**.
|
|
4. Press and hold the mic, speak, release. ✨
|
|
|
|
> **"Allow Full Access" is required.** Without it, iOS blocks the keyboard from using the microphone and from making network requests. We never log, store, or transmit your keystrokes — see [`PrivacyInfo.xcprivacy`](./OSGKeyboard/PrivacyInfo.xcprivacy).
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
OSGKeyboard/
|
|
├── OSGKeyboard/ # Main iOS app (settings, onboarding)
|
|
│ ├── Views/ # SwiftUI screens
|
|
│ ├── OSGKeyboardApp.swift # @main entry
|
|
│ ├── PrivacyInfo.xcprivacy # Required privacy manifest
|
|
│ └── OSGKeyboard.entitlements # App Group declaration
|
|
├── OSGKeyboardExt/ # Custom Keyboard Extension
|
|
│ ├── KeyboardViewController.swift # Principal class
|
|
│ ├── Services/
|
|
│ │ ├── AudioCaptureService.swift # AVAudioEngine → 16 kHz PCM
|
|
│ │ ├── ASRService.swift # iOS 26 + iOS 18 ASR
|
|
│ │ └── PolishingService.swift # LLM call with timeout
|
|
│ └── Views/ # RecordButton, Waveform, KeyboardRootView
|
|
├── OSGKeyboardShared/ # Framework shared by app + extension
|
|
│ ├── Models/ # ProviderConfig, LLMRequest, LLMProvider
|
|
│ ├── Services/ # LLMClient (OpenAI-compatible)
|
|
│ └── Constants/ # AppGroup identifier
|
|
├── OSGKeyboardTests/ # XCTest unit tests
|
|
├── project.yml # XcodeGen project definition
|
|
└── .github/workflows/ci.yml # Lint + build CI
|
|
```
|
|
|
|
### Data flow
|
|
|
|
```
|
|
[Long-press mic] → AudioCaptureService → AudioBufferSnapshot (16 kHz mono)
|
|
↓
|
|
ASRService.transcribe()
|
|
↓
|
|
ASREvent.final(rawTranscript)
|
|
↓
|
|
PolishingService.polish()
|
|
↓
|
|
LLMClient (OpenAI-compatible)
|
|
↓
|
|
textDocumentProxy.insertText(polished)
|
|
```
|
|
|
|
---
|
|
|
|
## Adding a new LLM provider
|
|
|
|
Open `OSGKeyboardShared/Models/LLMProvider.swift` and append a new `LLMProvider` to the `presets` array. The default `OpenAICompatibleClient` handles any endpoint that speaks the `POST /chat/completions` protocol.
|
|
|
|
```swift
|
|
LLMProvider(
|
|
id: "groq",
|
|
name: "Groq",
|
|
defaultBaseURL: "https://api.groq.com/openai/v1",
|
|
defaultModel: "llama-3.1-70b-versatile",
|
|
apiKeyURL: URL(string: "https://console.groq.com/keys")
|
|
)
|
|
```
|
|
|
|
That's it. No other code changes required.
|
|
|
|
---
|
|
|
|
## Limitations
|
|
|
|
- iOS sandboxes keyboard extensions: ~60 MB memory cap, Full Access required.
|
|
- The keyboard does **not** work in password fields or some `WKWebView` textareas (iOS limitation).
|
|
- iOS 18 uses `SFSpeechRecognizer` instead of `SpeechAnalyzer`; on iOS 26, `SpeechAnalyzer` is significantly faster and supports more locales.
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
[MIT](./LICENSE) — use it, fork it, ship it. No warranty.
|
|
|
|
---
|
|
|
|
## Acknowledgements
|
|
|
|
- Inspired by [Typeless](https://typeless.com) and the desktop open-source [OpenLess](https://github.com/Open-Less/openless)
|
|
- Built with [XcodeGen](https://github.com/yonaskolb/XcodeGen)
|
|
- Powered by Apple's [SpeechAnalyzer](https://developer.apple.com/documentation/speech/speechanalyzer) and [SFSpeechRecognizer](https://developer.apple.com/documentation/speech/sfspeechrecognizer)
|
|
|
|
---
|
|
|
|
**Note:** replace `<OWNER>` in badges and the git clone URL with your GitHub username.
|