OSGKeyboard

Tap to talk, tap to stop — AI-polished text appears at your cursor in any app. A source-available, custom-keyboard-based voice input tool for iOS 26+, inspired by Typeless and OpenLess.

中文 README · Privacy Policy

---

What is it?

OSGKeyboard is a free, source-available alternative to commercial voice-input tools. It runs as a Custom Keyboard Extension on iOS, so you can use it in any app — Messages, Notes, Mail, WeChat, ChatGPT, Claude, Cursor, you name it.

1. Tap the mic to start recording
2. Speak naturally (up to 3.5 minutes / 210 seconds per take)
3. Tap again to stop — the AI polishes your words into clean text and inserts at the cursor

By default, audio is transcribed on-device by Apple's SpeechAnalyzer + DictationTranscriber (iOS 26+) — no audio leaves your phone unless you say so. If polish is enabled, only the transcript text goes to your chosen LLM. Optionally, you can switch to a cloud ASR engine (explicit opt-in with a confirmation): in that mode your recordings are uploaded to the ASR provider you configure.

Under the hood, OSGKeyboard uses a Flow session model: a long-lived audio session runs in the host app, the keyboard extension writes tiny "start / stop" signals to the App Group, and the polished text is delivered back to the keyboard for insertion. You do not need to jump back to the host app between recordings.

---

Features

• 🎙 Tap-to-toggle recording with a Typeless-style circular mic button, 3.5-minute (210s) per-take cap with live countdown
• 🧠 On-device ASR (SpeechAnalyzer + DictationTranscriber, iOS 26+)
• ✍️ AI polishing — adds structure, punctuation, fixes grammar, optionally produces lists
• 🧩 Local + cloud polish toggle — local engine is ASR-only by default; opt into a post-ASR cloud polish step (DeepSeek by default) when the iOS speech recognition isn't strong enough for your environment (noisy far-field audio, strong accents, etc.)
• 🔌 Bring-your-own API — works with any OpenAI-compatible endpoint (OpenAI, DeepSeek, Qwen DashScope, Moonshot, Zhipu, your own self-hosted server, …)
• 🔒 Privacy first — on-device ASR by default, so audio never leaves your device unless you explicitly opt into the cloud engine; polish sends only the transcript to the LLM you choose
• 🎨 Native SwiftUI — dark theme, frosted glass, pure Swift 6, ~3,600 lines of code
• 🪶 Zero dependencies — no SwiftPM packages, no CocoaPods, no Carthage
• 🔁 Flow session — keep recording across multiple takes without bouncing back to the host app

---

Quick start

Requirements

• macOS with Xcode 26 (matches project.yml deployment target iOS 26)
• iPhone or iPad running iOS 26.0+ (iPad fully supported: Split View / Stage Manager, adaptive layout)
• XcodeGen: brew install xcodegen
• An OpenAI-compatible API key (e.g. from OpenAI, DeepSeek, or Qwen DashScope). Not needed if you stay on the "local ASR only" engine.

Build & run

git clone https://github.com/hkgood/OSGKeyboard.git
cd OSGKeyboard
./Scripts/generate-xcodeproj.sh   # generates OSGKeyboard.xcodeproj via XcodeGen
open OSGKeyboard.xcodeproj        # or build via CLI:
xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
  -destination 'generic/platform=iOS Simulator' build

The OSGKeyboard.xcodeproj is not committed — it is regenerated from project.yml by Scripts/generate-xcodeproj.sh. Always re-run the script after git pull if project.yml has changed.

macOS distribution (decision)

The macOS menu-bar app ships via Developer ID direct distribution (notarized, non-sandboxed), NOT the Mac App Store. This is deliberate: its core features — global hold-to-talk hotkey, Accessibility-based text insertion into other apps, and synthesized ⌘V — are incompatible with the Mac App Store sandbox, and a sandboxed Accessibility grant also tends to reset after every app update. OSGKeyboardMac.entitlements therefore keeps com.apple.security.app-sandbox set to false; do not flip it back on without redesigning the insertion path. (The iOS app targets the iOS App Store as usual — see AUDIT_APPSTORE.md.)

Enable the keyboard in iOS

The host app walks you through a 5-step onboarding:

1. Welcome — intro to OSGKeyboard
2. Microphone — request mic access
3. Speech recognition — request on-device speech recognition access
4. Enable keyboard + Full Access — open iOS Settings to add OSGKeyboard and allow Full Access
5. Engine + API — pick the local or cloud engine, then paste your API key (cloud / cloud-polish only)

After onboarding, in any text field, tap 🌐 to switch to OSGKeyboard, then tap the circular mic to start, speak, and tap again to stop.

"Allow Full Access" is required. Without it, iOS blocks the keyboard from using the microphone and from making network requests. We never log, store, or transmit your keystrokes — see PrivacyInfo.xcprivacy and our Privacy Policy.

---

Architecture

OSGKeyboard/
├── OSGKeyboard/                 # Main iOS app (host of the Flow session)
│   ├── Services/                # FlowSessionManager, AppPermissions, SpeechHistoryStore, …
│   ├── Views/                   # SwiftUI: OnboardingView, HomeView, SettingsView, HistoryView, …
│   ├── OSGKeyboardApp.swift     # @main entry, owns the FlowSessionManager
│   ├── PrivacyInfo.xcprivacy    # Required privacy manifest
│   └── OSGKeyboard.entitlements # App Group + Keychain Group
├── OSGKeyboardExt/              # Custom Keyboard Extension
│   ├── KeyboardViewController.swift   # Principal class (drives SwiftUI)
│   ├── Services/                # AppGroupPersistor, HostAppLauncher, AudioCaptureService (legacy, unused)
│   ├── Views/                   # KeyboardRootView, RecordButton, WaveformView
│   └── PrivacyInfo.xcprivacy
├── OSGKeyboardShared/           # Framework shared by app + extension (APPLICATION_EXTENSION_API_ONLY=YES)
│   ├── Services/                # FlowSessionBridge, FlowSessionDarwin, LLMClient, PolishingService, ASRService, Keychain, AppGroupStore, …
│   ├── Models/                  # LLMProvider, ProviderConfig, TranscriptionDelivery, AudioBufferSnapshot, …
│   ├── DesignSystem/            # Theme, ThemedRoot
│   └── Constants/               # AppGroup identifier
├── OSGKeyboardTests/            # XCTest unit tests (LLM, Keychain, ASR, Flow bridge, …)
├── OSGKeyboardExtTests/         # Keyboard-extension-side unit tests
├── Scripts/                     # generate-xcodeproj.sh, patch-icon-composer.sh
├── docs/                        # GitHub Pages site (privacy policy + landing)
├── project.yml                  # XcodeGen project definition (source of truth)
└── .github/workflows/ci.yml     # Lint + build CI

Data flow — Flow session model

[Tap mic in keyboard]
  └─► KeyboardViewController.pressBegan
        └─► FlowSessionBridge.setRecordingState(.recording)  [App Group UserDefaults]
        └─► Darwin notification: "recordingState changed"
              └─► FlowSessionManager (host app) sees the signal
                    └─► FlowContinuousCapture feeds 16 kHz PCM into ChunkedUtterancePipeline
                          └─► ASRService.transcribe (iOS 26 SpeechAnalyzer)
                                └─► ASREvent.partial / .final
                                      └─► UtteranceTranscriptStitcher stitches the chunks
                                            └─► PolishingService (LLMClient)  [optional, configurable]
                                                  └─► FlowSessionBridge.storeTranscriptionResult
[Keyboard polls + Darwin notif]
  └─► KeyboardViewController sees the result
        └─► textDocumentProxy.insertText(polished)

Engine modes:

• local (default) — on-device ASR via SpeechAnalyzer; transcript is inserted as-is. No network round-trip.
• local + "Cloud polish after ASR" toggle (Settings → Engine) — same on-device ASR, but the transcript (text only) is routed through your configured LLM before insertion. Useful when iOS speech recognition isn't accurate enough in your environment.
• cloud (opt-in, requires an explicit confirmation) — your voice recordings are uploaded to the ASR provider you configure (e.g. OpenAI /audio/transcriptions, DashScope, Zhipu), and the resulting transcript is sent to your LLM for polish. Choose this only when you accept your provider's privacy terms.

Cross-process plumbing (host app ↔ keyboard extension):

• App Group group.com.osgkeyboard.shared — UserDefaults for the live Flow session state, recording state, audio levels, transcription delivery, and most preferences.
• Shared Keychain group com.osgkeyboard.shared — the LLM API key is written by the host app's Settings, read by both processes before every LLM call.
• Darwin notifications (CFNotificationCenter) — light-weight "something changed" pings; payloads still travel through the App Group.

---

Adding a new LLM provider

Open OSGKeyboardShared/Models/LLMProvider.swift and append a new LLMProvider to the presets array. The default OpenAICompatibleClient handles any endpoint that speaks the POST /chat/completions protocol.

LLMProvider(
    id: "groq",
    name: "Groq",
    defaultBaseURL: "https://api.groq.com/openai/v1",
    defaultModel: "llama-3.1-70b-versatile",
    apiKeyURL: URL(string: "https://console.groq.com/keys")
)

That's it — no other code changes required.

To set it as the new default for first-time users, also bump the defaultProviderId constant used by ProviderConfig.

---

Known limitations

• iOS 26+ only. Earlier iOS versions are not supported. We dropped the pre-26 SFSpeechRecognizer / AVAudioSession branching so the entire ASR path can use the iOS 26 SpeechAnalyzer API exclusively.
• ~60 MB memory cap for the keyboard extension (iOS sandbox). The Flow session is hosted in the main app, so audio buffers and ASR models live there, not in the extension.
• "Allow Full Access" required. Without it, the keyboard can't reach the microphone or make network requests for cloud polish.
• Password fields and some WKWebView textareas are blocked by iOS itself — not something we can work around.
• 3.5-minute (210s) per-take cap. A long take is automatically stopped and dispatched for transcription; a new take can be started immediately.
• Force-quitting the host app does not resurrect the old session. The Live Activity is cleared immediately; the next time you open the app (with permissions granted) a fresh voice session starts automatically.
• 3-minute per-utterance ASR cap. If you exceed it, the pipeline gracefully splits into multiple stitched chunks.
• No on-device LLM polish. The local engine is ASR-only; "AI polish" is always cloud-based and configurable. On-device model support was explored in v0.2.0 and rolled back in v0.2.1 to keep the dependency surface at zero SPM packages.
• URL scheme osgkeyboard:// can be opened by any app on the device. We don't trust it for anything beyond "wake the host app and (re)start the Flow session"; it never carries your API key or other secrets.

---

Development

• Build setup — see the Build Setup section at the top of this file. Run ./Scripts/generate-xcodeproj.sh after any project.yml change.
• Tests — xcodebuild test -project OSGKeyboard.xcodeproj -scheme OSGKeyboard -destination 'platform=iOS Simulator,name=iPhone 17' runs both OSGKeyboardTests and OSGKeyboardExtTests targets.
• CI — .github/workflows/ci.yml runs SwiftLint, a clean Debug build, and the test suite on every push to 0.1 / 0.2 and PRs.
• Logging — print is debug-only; release builds use NSLog for the few cross-process status messages.

---

Project status

• Current release: v0.2.1 (2026-06-24)
• Default branch: 0.2 (renamed from main on 2026-06-24; the previous main is preserved as 0.1).
• See CHANGELOG.md for the full release history and TYPEWHISPER_FLOW_MIGRATION_TRACKER.md for the architecture-decision log behind the Flow session model.

---

License

OSGKeyboard Source Available License — personal learning and non-commercial local use only. No commercial use, redistribution, or public forks without permission. Commercial licensing: rocky.hk@gmail.com.

---

Acknowledgements

• Inspired by Typeless and the desktop open-source OpenLess
• Built with XcodeGen
• Powered by Apple's SpeechAnalyzer and SFSpeechRecognizer