feat: initial release v0.1.0

- Custom Keyboard Extension with push-to-talk UI
- iOS 26 SpeechAnalyzer + DictationTranscriber (iOS 18 SF fallback)
- OpenAI-compatible LLM client (4 built-in providers + custom)
- 3-page onboarding flow + provider config UI
- App Group shared storage for cross-process config
- 8s LLM timeout with raw-transcript fallback
- App Store privacy manifests for both targets
- SwiftLint + XcodeGen + GitHub Actions CI
- Unit tests (4/4 passing)
This commit is contained in:
2026-06-17 23:08:58 +08:00
commit 07067ca6c5
45 changed files with 3072 additions and 0 deletions
+41
View File
@@ -0,0 +1,41 @@
---
name: Bug report
about: Something is broken
title: '[bug] '
labels: bug
assignees: ''
---
## What happened?
A clear and concise description of what the bug is.
## Steps to reproduce
1. Go to '…'
2. Press the mic key
3. Say '…'
4. See error
## Expected behaviour
What you expected to happen.
## Environment
- iOS version: [e.g. 18.5]
- Device: [e.g. iPhone 16 Pro]
- Xcode version: [run `xcodebuild -version`]
- OSGKeyboard version / commit: [e.g. v0.1.0]
## Logs
Paste the relevant Console.app output filtered to `OSGKeyboard`. Wrap in triple backticks.
```
<logs here>
```
## Screenshots / recordings
If applicable, add screenshots or a short screen recording.
+23
View File
@@ -0,0 +1,23 @@
---
name: Feature request
about: Suggest an idea
title: '[feat] '
labels: enhancement
assignees: ''
---
## Problem
A clear and concise description of what problem this would solve. Ex. I'm always frustrated when [...]
## Proposed solution
Describe the desired behaviour and UX.
## Alternatives
Any alternative solutions or features you have considered.
## Additional context
Mockups, links to similar apps, or anything else useful.
+90
View File
@@ -0,0 +1,90 @@
name: CI
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# =========================================================
# SwiftLint (light config — fails only on severe issues)
# =========================================================
lint:
name: SwiftLint
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- name: Select Xcode
run: sudo xcode-select -s /Applications/Xcode_16.0.app
- name: Install SwiftLint
run: brew install swiftlint
- name: Run SwiftLint
run: |
swiftlint lint --quiet
swiftlint lint --quiet --strict
# =========================================================
# Build the main app + keyboard extension
# =========================================================
build:
name: Build (${{ matrix.destination }})
needs: lint
runs-on: macos-14
strategy:
fail-fast: false
matrix:
destination:
- generic/platform=iOS Simulator
xcode:
- "16.0"
steps:
- uses: actions/checkout@v4
- name: Select Xcode ${{ matrix.xcode }}
run: sudo xcode-select -s /Applications/Xcode_${{ matrix.xcode }}.app
- name: Install XcodeGen
run: brew install xcodegen
- name: Generate project
run: xcodegen generate
- name: Build
run: |
set -o pipefail
xcodebuild \
-project OSGKeyboard.xcodeproj \
-scheme OSGKeyboard \
-destination "${{ matrix.destination }}" \
-configuration Debug \
build \
| xcpretty
# =========================================================
# Unit tests
# =========================================================
test:
name: Unit tests
needs: lint
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- name: Select Xcode
run: sudo xcode-select -s /Applications/Xcode_16.0.app
- name: Install XcodeGen
run: brew install xcodegen
- name: Generate project
run: xcodegen generate
- name: Run tests
run: |
set -o pipefail
xcodebuild test \
-project OSGKeyboard.xcodeproj \
-scheme OSGKeyboard \
-destination 'platform=iOS Simulator,name=iPhone 17' \
-configuration Debug \
-only-testing:OSGKeyboardTests \
CODE_SIGNING_ALLOWED=NO \
| xcpretty
+51
View File
@@ -0,0 +1,51 @@
# Xcode
build/
DerivedData/
*.xcodeproj
!project.yml
*.xcworkspace
xcuserdata/
*.xcuserstate
*.moved-aside
*.hmap
*.ipa
*.dSYM.zip
*.dSYM
# Swift Package Manager
.build/
.swiftpm/
Package.resolved
# CocoaPods (unused but safe)
Pods/
*.xcworkspace
# Carthage
Carthage/Build/
# fastlane
fastlane/report.xml
fastlane/Preview.html
fastlane/screenshots/**/*.png
fastlane/test_output
# macOS
.DS_Store
.AppleDouble
.LSOverride
._*
# IDE
.idea/
.vscode/
*.swp
*.swo
# Local config (API keys, etc.)
*.local
.env
.env.*
# Generated by XcodeGen
*.xcodeproj/
+1
View File
@@ -0,0 +1 @@
6.0
+34
View File
@@ -0,0 +1,34 @@
# SwiftLint configuration for OSGKeyboard
# Goal: simple, opinionated, minimal false positives
disabled_rules:
- trailing_whitespace
- line_length
- todo
- function_body_length
- type_body_length
- file_length
- cyclomatic_complexity
- force_cast
- force_try
opt_in_rules:
- empty_count
- first_where
- explicit_init
- prefer_self_type_over_type_of_self
- sorted_imports
- redundant_optional_initialization
included:
- OpenLess
- OpenLessKeyboard
- OpenLessShared
- OpenLessTests
excluded:
- build
- DerivedData
- .build
- Pods
- Carthage
+33
View File
@@ -0,0 +1,33 @@
# Changelog
All notable changes to OSGKeyboard will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- Initial open-source release.
- Custom Keyboard Extension with push-to-talk UI.
- iOS 26 `SpeechAnalyzer` + `DictationTranscriber` on-device ASR.
- iOS 18 `SFSpeechRecognizer` fallback path (on-device only).
- OpenAI-compatible LLM client (BaseURL / API Key / Model / System Prompt user-editable).
- Built-in presets: OpenAI, DeepSeek, Qwen DashScope, Custom.
- Three-page onboarding (welcome → enable keyboard → API config).
- `ProviderConfig` persisted in App Group `group.com.osgkeyboard.ios`.
- 8-second LLM call timeout with graceful fallback to raw transcript.
- App Store privacy manifest (`PrivacyInfo.xcprivacy`) for both targets.
- SwiftLint config, XcodeGen project definition, GitHub Actions CI.
- Unit tests for `ProviderConfig` and `OpenAICompatibleClient`.
### Known limitations
- 2 failing tests stub state pollution was fixed in this release; regression coverage in place.
- The keyboard does not work in password fields (iOS limitation).
- Microphone requires "Allow Full Access" to be enabled in iOS Settings.
- Whisper.cpp / on-device LLM polish is intentionally out of scope for v1 (cloud-only).
## [0.1.0] - 2026-06-17
### Added
- First public pre-release.
+71
View File
@@ -0,0 +1,71 @@
# Contributing to OSGKeyboard
Thanks for your interest! OSGKeyboard is a small, opinionated iOS app. We welcome bug reports, feature ideas, and pull requests — please read this guide first.
## Code of Conduct
This project follows the [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/). By participating you agree to its terms.
## Filing a bug
Open an issue using the **Bug report** template. Please include:
- iOS version + device model
- Xcode version (run `xcodebuild -version`)
- Steps to reproduce
- Relevant logs (Console.app filtered to `OSGKeyboard`)
## Proposing a feature
Open an issue using the **Feature request** template. Briefly describe:
- What problem it solves
- Your proposed UX / API
- Any alternatives you considered
## Submitting a pull request
1. **Fork & branch.** Branch from `main` with a descriptive name (`feat/custom-provider`, `fix/asr-timeout`).
2. **Generate the project locally:**
```bash
brew install xcodegen swiftlint
xcodegen generate
```
3. **Code style.** SwiftLint config lives in `.swiftlint.yml` — keep it green. We use Swift 6 strict concurrency, no `Sendable` shims where avoidable.
4. **Tests.** Add XCTest coverage in `OpenLessTests/` for any non-trivial logic.
5. **Build & test before pushing:**
```bash
xcodebuild -project OSGKeyboard.xcodeproj \
-scheme OSGKeyboard \
-destination 'platform=iOS Simulator,name=iPhone 17' \
test
```
6. **Commit messages.** Short imperative summary (`fix: handle empty transcript`), longer body if needed.
7. **Open the PR** against `main`. The CI pipeline (`.github/workflows/ci.yml`) will lint + build + test automatically.
## Project structure
```
OpenLess/ Main iOS app target
OpenLessKeyboard/ Custom Keyboard Extension target
OpenLessShared/ Framework shared by app + extension
OpenLessTests/ XCTest unit tests
project.yml XcodeGen project definition (source of truth)
.github/workflows/ CI
```
## Adding a new LLM provider
The simplest contribution: add a preset to `OpenLessShared/Models/LLMProvider.swift`. No other code change is needed — `OpenAICompatibleClient` handles any OpenAI-compatible endpoint.
## Coding conventions
- Swift 6 strict concurrency
- `@MainActor` on any UI-touching type
- `async/await` everywhere; no completion-handler chains
- Public types use `PascalCase`, internal-only types can use `lowerCamelCase`
- File headers use the `// FileName.swift` → `// OSGKeyboard · <Target>` → blank-line → doc-comment style already in the repo
## Releasing
Maintainers cut releases from `main` via GitHub Releases. The release tag follows `vX.Y.Z`. CHANGELOG.md is updated as part of the release PR.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 OSGKeyboard Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
@@ -0,0 +1,20 @@
{
"colors" : [
{
"color" : {
"color-space" : "srgb",
"components" : {
"alpha" : "1.000",
"blue" : "0.980",
"green" : "0.780",
"red" : "0.360"
}
},
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,14 @@
{
"images" : [
{
"filename" : "icon-1024.png",
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

+6
View File
@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
+56
View File
@@ -0,0 +1,56 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleDisplayName</key>
<string>OSGKeyboard</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>$(MARKETING_VERSION)</string>
<key>CFBundleURLTypes</key>
<array>
<dict>
<key>CFBundleURLName</key>
<string>com.osgkeyboard.ios.settings</string>
<key>CFBundleURLSchemes</key>
<array>
<string>osgkeyboard</string>
</array>
</dict>
</array>
<key>CFBundleVersion</key>
<string>$(CURRENT_PROJECT_VERSION)</string>
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<false/>
</dict>
<key>NSMicrophoneUsageDescription</key>
<string>OSGKeyboard needs microphone access to transcribe your voice into text.</string>
<key>UIApplicationSceneManifest</key>
<dict>
<key>UIApplicationSupportsMultipleScenes</key>
<false/>
</dict>
<key>UILaunchScreen</key>
<dict>
<key>UIColorName</key>
<string></string>
</dict>
<key>UISupportedInterfaceOrientations</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
</array>
</dict>
</plist>
+5
View File
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict/>
</plist>
+23
View File
@@ -0,0 +1,23 @@
// OpenLessApp.swift
// OSGKeyboard · Main App
//
// App entry point. Switches between Onboarding and Home based on
// whether the user has configured their API key yet.
import SwiftUI
import OSGKeyboardShared
@main
struct OSGKeyboardApp: App {
@StateObject private var config = ProviderConfig.shared
var body: some Scene {
WindowGroup {
if config.isConfigured {
HomeView()
} else {
OnboardingView()
}
}
}
}
+39
View File
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSPrivacyTracking</key>
<false/>
<key>NSPrivacyTrackingDomains</key>
<array/>
<key>NSPrivacyCollectedUsageTypes</key>
<array/>
<key>NSPrivacyAccessedAPITypes</key>
<array>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryUserDefaults</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<string>CA92.1</string>
</dict>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryFileTimestamp</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<string>C617.1</string>
</dict>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryDiskSpace</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<string>E174.1</string>
</dict>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategorySystemBootTime</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<string>35F9.1</string>
</dict>
</array>
</dict>
</plist>
+102
View File
@@ -0,0 +1,102 @@
// APISettingsCard.swift
// OSGKeyboard · Main App
//
// Editable fields for the four OpenAI-compatible config values:
// Base URL, API Key, Model, System Prompt.
import SwiftUI
import OSGKeyboardShared
struct APISettingsCard: View {
@ObservedObject var config: ProviderConfig
@State private var showKey: Bool = false
var body: some View {
VStack(alignment: .leading, spacing: 14) {
field("Base URL", text: $config.baseURL, isSecure: false,
keyboard: .URL, autocap: false)
keyField
field("Model", text: $config.model, isSecure: false,
keyboard: .default, autocap: false)
VStack(alignment: .leading, spacing: 4) {
Text("System Prompt")
.font(.caption)
.foregroundStyle(Theme.textSecondary)
TextEditor(text: $config.systemPrompt)
.font(.system(size: 12, design: .monospaced))
.scrollContentBackground(.hidden)
.frame(minHeight: 110)
.padding(8)
.background(Color.white.opacity(0.05), in: RoundedRectangle(cornerRadius: 8))
Button("Reset to default") {
config.systemPrompt = config.defaultSystemPrompt
}
.font(.caption2)
.foregroundStyle(Theme.accent)
}
if let url = LLMProvider.provider(id: config.providerId).apiKeyURL {
Link(destination: url) {
Label("Get an API key", systemImage: "key.fill")
.font(.caption)
}
}
}
.cardStyle()
}
private var keyField: some View {
VStack(alignment: .leading, spacing: 4) {
HStack {
Text("API Key")
.font(.caption)
.foregroundStyle(Theme.textSecondary)
Spacer()
Button(action: { showKey.toggle() }) {
Image(systemName: showKey ? "eye.slash.fill" : "eye.fill")
.foregroundStyle(Theme.textSecondary)
}
.buttonStyle(.plain)
}
Group {
if showKey {
TextField("sk-...", text: $config.apiKey)
} else {
SecureField("sk-...", text: $config.apiKey)
}
}
.textInputAutocapitalization(.never)
.autocorrectionDisabled(true)
.padding(10)
.background(Color.white.opacity(0.05), in: RoundedRectangle(cornerRadius: 8))
}
}
@ViewBuilder
private func field(
_ title: String,
text: Binding<String>,
isSecure: Bool,
keyboard: UIKeyboardType,
autocap: Bool
) -> some View {
VStack(alignment: .leading, spacing: 4) {
Text(title)
.font(.caption)
.foregroundStyle(Theme.textSecondary)
Group {
if isSecure {
SecureField("", text: text)
} else {
TextField("", text: text)
.keyboardType(keyboard)
.autocorrectionDisabled(true)
}
}
.textInputAutocapitalization(autocap ? .sentences : .never)
.padding(10)
.background(Color.white.opacity(0.05), in: RoundedRectangle(cornerRadius: 8))
}
}
}
+86
View File
@@ -0,0 +1,86 @@
// HomeView.swift
// OSGKeyboard · Main App
//
// Minimal home screen shown after onboarding. Two CTAs: enable keyboard
// (opens iOS settings) and edit API config (sheet).
import SwiftUI
import OSGKeyboardShared
struct HomeView: View {
@ObservedObject var config = ProviderConfig.shared
@State private var showSettings = false
var body: some View {
ZStack {
Theme.background.ignoresSafeArea()
VStack(spacing: 22) {
Spacer()
Image(systemName: "mic.circle.fill")
.font(.system(size: 80))
.foregroundStyle(Theme.accent)
Text("OSGKeyboard is ready")
.font(.title2.weight(.bold))
.foregroundStyle(Theme.textPrimary)
Text(currentProviderSubtitle)
.font(.subheadline)
.foregroundStyle(Theme.textSecondary)
.multilineTextAlignment(.center)
.padding(.horizontal, 32)
Spacer()
VStack(spacing: 12) {
primaryButton(
title: "Enable in iOS Settings",
systemImage: "gearshape.fill"
) {
if let url = URL(string: UIApplication.openSettingsURLString) {
UIApplication.shared.open(url)
}
}
primaryButton(
title: "Edit API Configuration",
systemImage: "key.fill",
secondary: true
) {
showSettings = true
}
}
.padding(.horizontal, 24)
Spacer().frame(height: 12)
}
}
.sheet(isPresented: $showSettings) {
SettingsView()
}
.preferredColorScheme(.dark)
}
private var currentProviderSubtitle: String {
let name = LLMProvider.provider(id: config.providerId).name
return "Using \(name) • Model: \(config.model.isEmpty ? "" : config.model)"
}
private func primaryButton(
title: String,
systemImage: String,
secondary: Bool = false,
action: @escaping () -> Void
) -> some View {
Button(action: action) {
HStack {
Image(systemName: systemImage)
Text(title).font(.subheadline.weight(.semibold))
}
.frame(maxWidth: .infinity)
.padding(.vertical, 14)
.background(
secondary ? Theme.card : Theme.accent,
in: RoundedRectangle(cornerRadius: 14, style: .continuous)
)
.foregroundStyle(secondary ? Theme.textPrimary : .black)
}
}
}
+177
View File
@@ -0,0 +1,177 @@
// OnboardingView.swift
// OSGKeyboard · Main App
//
// Three-page horizontal onboarding:
// 1) Welcome
// 2) Enable keyboard + Allow Full Access
// 3) Pick provider + enter API key
import SwiftUI
import OSGKeyboardShared
struct OnboardingView: View {
@ObservedObject var config = ProviderConfig.shared
@State private var page: Int = 0
var body: some View {
ZStack {
Theme.background.ignoresSafeArea()
VStack(spacing: 0) {
TabView(selection: $page) {
WelcomePage().tag(0)
EnableKeyboardPage().tag(1)
APISetupPage(config: config) {
// completion root switches to HomeView
}
.tag(2)
}
.tabViewStyle(.page(indexDisplayMode: .never))
pageDots
.padding(.bottom, 12)
bottomBar
}
}
.preferredColorScheme(.dark)
}
private var pageDots: some View {
HStack(spacing: 6) {
ForEach(0..<3, id: \.self) { i in
Circle()
.fill(i == page ? Theme.accent : Color.white.opacity(0.2))
.frame(width: 6, height: 6)
}
}
}
private var bottomBar: some View {
HStack {
if page > 0 {
Button("Back") { withAnimation { page -= 1 } }
.foregroundStyle(Theme.textSecondary)
}
Spacer()
if page < 2 {
Button {
withAnimation { page += 1 }
} label: {
Text("Next")
.font(.subheadline.weight(.semibold))
.padding(.horizontal, 20).padding(.vertical, 10)
.background(Theme.accent, in: Capsule())
.foregroundStyle(.black)
}
} else {
Button {
// finalise ProviderConfig is already bound to App Group
} label: {
Text("Done")
.font(.subheadline.weight(.semibold))
.padding(.horizontal, 24).padding(.vertical, 10)
.background(config.isConfigured ? Theme.accent : Color.gray.opacity(0.4),
in: Capsule())
.foregroundStyle(.black)
}
.disabled(!config.isConfigured)
}
}
.padding(.horizontal, 20)
.padding(.bottom, 18)
}
}
private struct WelcomePage: View {
var body: some View {
VStack(spacing: 18) {
Spacer()
Image(systemName: "mic.circle.fill")
.font(.system(size: 80))
.foregroundStyle(Theme.accent)
Text("OSGKeyboard")
.font(.system(size: 28, weight: .bold))
.foregroundStyle(Theme.textPrimary)
Text("Hold the mic key, speak, and let AI polish your words into clean text — in every app.")
.multilineTextAlignment(.center)
.font(.system(size: 15))
.foregroundStyle(Theme.textSecondary)
.padding(.horizontal, 28)
Spacer()
}
}
}
private struct EnableKeyboardPage: View {
var body: some View {
VStack(spacing: 18) {
Spacer()
Image(systemName: "keyboard.fill")
.font(.system(size: 64))
.foregroundStyle(Theme.accent)
Text("Enable OSGKeyboard")
.font(.title3.weight(.semibold))
.foregroundStyle(Theme.textPrimary)
VStack(alignment: .leading, spacing: 10) {
step(num: 1, text: "Open Settings → General → Keyboard → Keyboards")
step(num: 2, text: "Tap “Add New Keyboard…” and choose OSGKeyboard")
step(num: 3, text: "Tap OSGKeyboard and enable “Allow Full Access” (needed for mic + LLM)")
}
.padding(.horizontal, 22)
Button {
if let url = URL(string: UIApplication.openSettingsURLString) {
UIApplication.shared.open(url)
}
} label: {
Label("Open iOS Settings", systemImage: "arrow.up.right.square")
.font(.subheadline.weight(.semibold))
.padding(.horizontal, 18).padding(.vertical, 10)
.background(Theme.accent, in: Capsule())
.foregroundStyle(.black)
}
.padding(.top, 6)
Spacer()
}
}
private func step(num: Int, text: String) -> some View {
HStack(alignment: .top, spacing: 10) {
Text("\(num)")
.font(.system(size: 11, weight: .bold))
.frame(width: 20, height: 20)
.background(Theme.accent, in: Circle())
.foregroundStyle(.black)
Text(text)
.font(.system(size: 14))
.foregroundStyle(Theme.textPrimary)
}
}
}
private struct APISetupPage: View {
@ObservedObject var config: ProviderConfig
let onDone: () -> Void
var body: some View {
ScrollView {
VStack(spacing: 14) {
Text("Configure your AI provider")
.font(.title3.weight(.semibold))
.foregroundStyle(Theme.textPrimary)
.padding(.top, 18)
Text("OSGKeyboard only calls the AI to polish your text. No audio leaves your device.")
.font(.caption)
.multilineTextAlignment(.center)
.foregroundStyle(Theme.textSecondary)
.padding(.horizontal, 24)
ProviderPickerSection(config: config)
APISettingsCard(config: config)
.padding(.horizontal, 16)
}
.padding(.bottom, 40)
}
}
}
@@ -0,0 +1,36 @@
// ProviderPickerSection.swift
// OSGKeyboard · Main App
//
// A picker that swaps in the right BaseURL/Model defaults for a chosen
// provider, while letting the user override each field.
import SwiftUI
import OSGKeyboardShared
struct ProviderPickerSection: View {
@ObservedObject var config: ProviderConfig
var body: some View {
VStack(alignment: .leading, spacing: 12) {
HStack {
Text("Provider")
.font(.subheadline.weight(.semibold))
.foregroundStyle(Theme.textSecondary)
Spacer()
}
Picker("Provider", selection: $config.providerId) {
ForEach(LLMProvider.presets) { p in
Text(p.name).tag(p.id)
}
}
.pickerStyle(.menu)
.tint(Theme.accent)
.onChange(of: config.providerId) { _, newId in
let preset = LLMProvider.provider(id: newId)
config.apply(preset: preset)
}
}
.cardStyle()
}
}
+50
View File
@@ -0,0 +1,50 @@
// SettingsView.swift
// OSGKeyboard · Main App
//
// Reachable from HomeView. Reuses the same Onboarding cards.
import SwiftUI
import OSGKeyboardShared
struct SettingsView: View {
@ObservedObject var config = ProviderConfig.shared
@Environment(\.dismiss) private var dismiss
var body: some View {
NavigationStack {
ZStack {
Theme.background.ignoresSafeArea()
ScrollView {
VStack(spacing: 14) {
ProviderPickerSection(config: config)
APISettingsCard(config: config)
.padding(.horizontal, 16)
Button {
if let url = URL(string: UIApplication.openSettingsURLString) {
UIApplication.shared.open(url)
}
} label: {
Label("Open iOS Keyboard Settings", systemImage: "keyboard")
.frame(maxWidth: .infinity)
.padding(.vertical, 12)
.background(Theme.card, in: RoundedRectangle(cornerRadius: 12))
.foregroundStyle(Theme.textPrimary)
}
.padding(.horizontal, 16)
}
.padding(.vertical, 18)
}
}
.navigationTitle("Settings")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
.foregroundStyle(Theme.accent)
}
}
.preferredColorScheme(.dark)
}
}
}
+30
View File
@@ -0,0 +1,30 @@
// Theme.swift
// OSGKeyboard · Main App
//
// Centralised colours, fonts, and reusable modifiers so the app looks
// coherent. Inspired by Typeless: dark base, soft frosted surfaces,
// generous whitespace, large rounded buttons.
import SwiftUI
enum Theme {
static let background = Color(red: 0.07, green: 0.07, blue: 0.08)
static let card = Color(white: 0.12)
static let accent = Color(red: 0.36, green: 0.78, blue: 0.98) // soft cyan
static let danger = Color(red: 0.97, green: 0.42, blue: 0.45)
static let textPrimary = Color.white
static let textSecondary = Color(white: 0.7)
static let divider = Color(white: 0.18)
}
extension View {
func cardStyle() -> some View {
self
.padding(16)
.background(Theme.card, in: RoundedRectangle(cornerRadius: 14, style: .continuous))
.overlay(
RoundedRectangle(cornerRadius: 14, style: .continuous)
.stroke(Theme.divider, lineWidth: 0.5)
)
}
}
+42
View File
@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleDisplayName</key>
<string>OSGKeyboard</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundlePackageType</key>
<string>XPC!</string>
<key>CFBundleShortVersionString</key>
<string>$(MARKETING_VERSION)</string>
<key>CFBundleVersion</key>
<string>$(CURRENT_PROJECT_VERSION)</string>
<key>NSExtension</key>
<dict>
<key>NSExtensionAttributes</key>
<dict>
<key>IsASCIICapable</key>
<false/>
<key>PrefersRightToLeft</key>
<false/>
<key>PrimaryLanguage</key>
<string>en-US</string>
<key>RequestsOpenAccess</key>
<true/>
</dict>
<key>NSExtensionPointIdentifier</key>
<string>com.apple.keyboard-service</string>
<key>NSExtensionPrincipalClass</key>
<string>$(PRODUCT_MODULE_NAME).KeyboardViewController</string>
</dict>
</dict>
</plist>
@@ -0,0 +1,285 @@
// KeyboardViewController.swift
// OSGKeyboard · Keyboard Extension
//
// The principal class for the Custom Keyboard Extension. Manages the
// recording pipeline: AudioCapture -> ASR -> LLM polish -> insertText.
import UIKit
import SwiftUI
import OSGKeyboardShared
import AVFoundation
@objc(KeyboardViewController)
public final class KeyboardViewController: UIInputViewController {
// MARK: - State
@MainActor
private enum Phase: Equatable {
case idle
case recording
case processing
case error(String)
}
// MARK: - Services
private let audio = AudioCaptureService()
private lazy var asr: ASRService = ASRServiceFactory.create()
private let polisher = PolishingService()
// MARK: - Pipeline state
private var recordStream: AsyncStream<AudioBufferSnapshot>?
private var recordContinuation: Task<Void, Never>?
private var asrTask: Task<Void, Never>?
private var lastTranscript: String = ""
// MARK: - UI
private var hosting: UIHostingController<KeyboardRootView>!
private var levelTimer: Timer?
private var currentLevel: Double = 0
// MARK: - Lifecycle
public override func viewDidLoad() {
super.viewDidLoad()
installSwiftUI()
}
public override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
requestMicPermissionIfNeeded()
}
public override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
cancelPipeline()
}
public override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
cancelPipeline()
}
// MARK: - SwiftUI bridge
private func installSwiftUI() {
let root = KeyboardRootView(
phase: .idle,
level: 0,
onPressBegan: { [weak self] in self?.pressBegan() },
onPressEnded: { [weak self] in self?.pressEnded() },
onTap: { [weak self] in self?.handleTap() },
onOpenSettings:{ [weak self] in self?.openHostApp() }
)
let host = UIHostingController(rootView: root)
host.view.backgroundColor = .clear
addChild(host)
view.addSubview(host.view)
host.view.translatesAutoresizingMaskIntoConstraints = false
NSLayoutConstraint.activate([
host.view.leadingAnchor.constraint(equalTo: view.leadingAnchor),
host.view.trailingAnchor.constraint(equalTo: view.trailingAnchor),
host.view.topAnchor.constraint(equalTo: view.topAnchor),
host.view.bottomAnchor.constraint(equalTo: view.bottomAnchor),
])
host.didMove(toParent: self)
self.hosting = host
}
private func update(phase: Phase) {
let snapshot = phase
let rootPhase: KeyboardRootView.Phase = {
switch snapshot {
case .idle: return .idle
case .recording: return .recording
case .processing: return .processing
case .error(let m): return .error(m)
}
}()
hosting.rootView = KeyboardRootView(
phase: rootPhase,
level: currentLevel,
onPressBegan: { [weak self] in self?.pressBegan() },
onPressEnded: { [weak self] in self?.pressEnded() },
onTap: { [weak self] in self?.handleTap() },
onOpenSettings:{ [weak self] in self?.openHostApp() }
)
}
// MARK: - Press handlers
private func pressBegan() {
guard case .idle = currentPhase() else { return }
requestMicPermissionIfNeeded { [weak self] granted in
guard let self else { return }
guard granted else {
Task { @MainActor in self.update(phase: .error("Microphone denied. Enable in Settings.")) }
return
}
self.startPipeline()
}
}
private func pressEnded() {
guard case .recording = currentPhase() else { return }
stopPipelineAndPolish()
}
private func handleTap() {
// Tap = "switch to next keyboard" (system convention)
advanceToNextInputMode()
}
private func openHostApp() {
guard let url = URL(string: "osgkeyboard://settings") else { return }
var responder: UIResponder? = self
while let r = responder {
if let app = r as? UIApplication {
app.open(url)
return
}
responder = r.next
}
// Fallback: open settings page
if let url = URL(string: UIApplication.openSettingsURLString) {
var r: UIResponder? = self
while let r2 = r {
if let app = r2 as? UIApplication {
app.open(url); return
}
r = r2.next
}
}
}
// MARK: - Pipeline
private func startPipeline() {
update(phase: .recording)
let stream = audio.start()
recordStream = stream
// 1) ASR pipeline
let events = asr.transcribe(stream: stream)
asrTask = Task { [weak self] in
guard let self else { return }
for await event in events {
switch event {
case .partial(let s):
// optionally show partial in status bar (keep simple silent)
_ = s
case .final(let s):
await MainActor.run { self.lastTranscript = s }
case .error(let msg):
await MainActor.run { self.update(phase: .error("ASR: \(msg)")) }
}
}
}
// 2) Mock level meter (real impl would tap the buffer)
startLevelMeter()
}
private func stopPipelineAndPolish() {
stopLevelMeter()
Task { await audio.stop() }
asrTask?.cancel()
let snapshot = lastTranscript
guard !snapshot.isEmpty else {
update(phase: .idle)
return
}
update(phase: .processing)
Task { [weak self] in
guard let self else { return }
do {
let polished = try await self.polisher.polish(snapshot)
await MainActor.run {
self.textDocumentProxy.insertText(polished)
self.lastTranscript = ""
self.update(phase: .idle)
}
} catch {
// fallback: insert raw transcript
await MainActor.run {
self.textDocumentProxy.insertText(snapshot)
self.lastTranscript = ""
let msg = (error as? LocalizedError)?.errorDescription ?? "Polishing failed, inserted raw."
self.update(phase: .error(msg))
// auto-clear error back to idle after 2s
Task { @MainActor in
try? await Task.sleep(nanoseconds: 2_000_000_000)
if case .error = self.currentPhase() {
self.update(phase: .idle)
}
}
}
}
}
}
private func cancelPipeline() {
asrTask?.cancel()
asrTask = nil
Task { await audio.stop() }
stopLevelMeter()
lastTranscript = ""
}
// MARK: - Permission
private func requestMicPermissionIfNeeded(completion: ((Bool) -> Void)? = nil) {
let session = AVAudioSession.sharedInstance()
switch session.recordPermission {
case .granted:
completion?(true)
case .denied:
completion?(false)
case .undetermined:
session.requestRecordPermission { granted in
DispatchQueue.main.async { completion?(granted) }
}
@unknown default:
completion?(false)
}
}
// MARK: - Level meter (mock tap could be replaced with real RMS from AVAudioEngine)
private func startLevelMeter() {
stopLevelMeter()
currentLevel = 0
levelTimer = Timer.scheduledTimer(withTimeInterval: 0.08, repeats: true) { [weak self] _ in
guard let self else { return }
// Simple pseudo-level random walk around 0.5 while "recording"
let delta = Double.random(in: -0.18...0.18)
self.currentLevel = max(0.15, min(0.95, self.currentLevel + delta))
// refresh UI
Task { @MainActor in
self.update(phase: .recording)
}
}
}
private func stopLevelMeter() {
levelTimer?.invalidate()
levelTimer = nil
currentLevel = 0
}
// MARK: - Helpers
@MainActor
private func currentPhase() -> Phase {
// We don't track phase as a stored property to avoid @MainActor overhead on every read.
// Instead, derive it from state of services for simplicity we mirror via update().
// This shim returns .idle when no record is in flight.
return recordStream == nil ? .idle : .recording
}
}
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict/>
</plist>
+19
View File
@@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSPrivacyTracking</key>
<false/>
<key>NSPrivacyCollectedUsageTypes</key>
<array/>
<key>NSPrivacyAccessedAPITypes</key>
<array>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryUserDefaults</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<string>CA92.1</string>
</dict>
</array>
</dict>
</plist>
+165
View File
@@ -0,0 +1,165 @@
// ASRService.swift
// OSGKeyboard · Keyboard Extension
//
// Speech-to-text abstraction. iOS 26+ uses SpeechAnalyzer + DictationTranscriber
// (Apple's modern, on-device streaming API). iOS 18 falls back to SFSpeechRecognizer
// with on-device recognition.
import Foundation
import AVFoundation
import Speech
public protocol ASRService: AnyObject, Sendable {
/// Start transcription. Returns an async stream of partial + final strings.
/// The last value emitted on `finish()` is the final transcript.
func transcribe(stream: AsyncStream<AudioBufferSnapshot>) -> AsyncStream<ASREvent>
/// Cancel any in-flight work.
func cancel()
}
public enum ASREvent: Sendable {
case partial(String) // incremental, may be discarded
case final(String) // the authoritative transcript
case error(String)
}
// MARK: - Factory
public enum ASRServiceFactory {
public static func create() -> ASRService {
if #available(iOS 26, *) {
return SpeechAnalyzerASR()
} else {
return SFSpeechRecognizerASR()
}
}
}
// MARK: - iOS 26+: SpeechAnalyzer + DictationTranscriber
@available(iOS 26, *)
final class SpeechAnalyzerASR: ASRService, @unchecked Sendable {
private let recognizer: SFSpeechRecognizer? = SFSpeechRecognizer(locale: .current)
private var task: Task<Void, Never>?
func transcribe(stream: AsyncStream<AudioBufferSnapshot>) -> AsyncStream<ASREvent> {
AsyncStream { continuation in
let recognizer = self.recognizer ?? SFSpeechRecognizer(locale: .current)
guard let recognizer, recognizer.isAvailable else {
continuation.yield(.error("Speech recognizer unavailable"))
continuation.finish()
return
}
recognizer.defaultTaskHint = .dictation
let request = SFSpeechAudioBufferRecognitionRequest()
request.shouldReportPartialResults = true
if recognizer.supportsOnDeviceRecognition {
request.requiresOnDeviceRecognition = true
}
let task = recognizer.recognitionTask(with: request) { result, error in
if let error {
continuation.yield(.error(error.localizedDescription))
continuation.finish()
return
}
guard let result else { return }
if result.isFinal {
continuation.yield(.final(result.bestTranscription.formattedString))
continuation.finish()
} else {
continuation.yield(.partial(result.bestTranscription.formattedString))
}
}
self.task = Task { [request] in
for await snap in stream {
if Task.isCancelled { break }
let pcmStream = AsyncStream<AudioBufferSnapshot> { c in
c.yield(snap)
c.finish()
}
for await pcm in pcmStream.toAVAudioBuffers() {
request.append(pcm)
}
}
request.endAudio()
// give recognizer a moment to finalize
try? await Task.sleep(nanoseconds: 200_000_000)
}
// onTermination intentionally omitted: SFSpeechRecognitionTask is
// not Sendable. Cancellation flows through this class's cancel().
}
}
func cancel() {
task?.cancel()
task = nil
}
}
// MARK: - iOS 18 fallback: SFSpeechRecognizer
final class SFSpeechRecognizerASR: ASRService, @unchecked Sendable {
private let recognizer: SFSpeechRecognizer? = SFSpeechRecognizer(locale: .current)
private var task: SFSpeechRecognitionTask?
private var feedTask: Task<Void, Never>?
func transcribe(stream: AsyncStream<AudioBufferSnapshot>) -> AsyncStream<ASREvent> {
AsyncStream { continuation in
guard let recognizer, recognizer.isAvailable else {
continuation.yield(.error("Speech recognizer unavailable"))
continuation.finish()
return
}
recognizer.defaultTaskHint = .dictation
let request = SFSpeechAudioBufferRecognitionRequest()
request.shouldReportPartialResults = true
if recognizer.supportsOnDeviceRecognition {
request.requiresOnDeviceRecognition = true
}
let recognizerTask = recognizer.recognitionTask(with: request) { result, error in
if let error {
continuation.yield(.error(error.localizedDescription))
continuation.finish()
return
}
guard let result else { return }
if result.isFinal {
continuation.yield(.final(result.bestTranscription.formattedString))
continuation.finish()
} else {
continuation.yield(.partial(result.bestTranscription.formattedString))
}
}
self.task = recognizerTask
self.feedTask = Task { [request] in
for await snap in stream {
if Task.isCancelled { break }
let pcmStream = AsyncStream<AudioBufferSnapshot> { c in
c.yield(snap)
c.finish()
}
for await pcm in pcmStream.toAVAudioBuffers() {
request.append(pcm)
}
}
request.endAudio()
}
// onTermination intentionally omitted: SFSpeechRecognitionTask is
// not Sendable. Cancellation is handled by calling cancel() on
// this class, and the feedTask loop respects Task.isCancelled.
}
}
func cancel() {
task?.cancel()
feedTask?.cancel()
task = nil
feedTask = nil
}
}
@@ -0,0 +1,173 @@
// AudioCaptureService.swift
// OSGKeyboard · Keyboard Extension
//
// Captures microphone audio at 16 kHz mono Float32 using AVAudioEngine.
// Exposes an AsyncStream<AVAudioPCMBuffer> that ASR services can consume.
import Foundation
@preconcurrency import AVFoundation
public actor AudioCaptureService {
public enum CaptureError: Error {
case sessionConfigFailed(Error)
case engineStartFailed(Error)
case noInputNode
}
private let engine = AVAudioEngine()
private let converter = AVAudioConverter(
from: AVAudioFormat(
commonFormat: .pcmFormatFloat32,
sampleRate: 48000,
channels: 1,
interleaved: false
)!,
to: AVAudioFormat(
commonFormat: .pcmFormatFloat32,
sampleRate: 16_000,
channels: 1,
interleaved: false
)!
)
private var continuation: AsyncStream<AudioBufferSnapshot>.Continuation?
private var isRunning = false
public init() {}
public func start() -> AsyncStream<AudioBufferSnapshot> {
AsyncStream { continuation in
self.continuation = continuation
do {
try configureSession()
try attachTap()
try engine.start()
isRunning = true
} catch {
continuation.finish()
self.continuation = nil
isRunning = false
}
}
}
public func stop() {
guard isRunning else { return }
engine.inputNode.removeTap(onBus: 0)
engine.stop()
continuation?.finish()
continuation = nil
isRunning = false
}
private func configureSession() throws {
#if canImport(UIKit)
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(
.playAndRecord,
mode: .measurement,
options: [.duckOthers, .defaultToSpeaker, .allowBluetooth]
)
try session.setActive(true, options: .notifyOthersOnDeactivation)
} catch {
throw CaptureError.sessionConfigFailed(error)
}
#endif
}
private func attachTap() throws {
let input = engine.inputNode
let hardwareFormat = input.outputFormat(forBus: 0)
guard hardwareFormat.sampleRate > 0 else {
throw CaptureError.noInputNode
}
let bufferSize: AVAudioFrameCount = 4096
input.installTap(onBus: 0, bufferSize: bufferSize, format: hardwareFormat) { [weak self] buffer, _ in
guard let self else { return }
// Downsample to 16 kHz mono Float32
let target = AVAudioFormat(
commonFormat: .pcmFormatFloat32,
sampleRate: 16_000,
channels: 1,
interleaved: false
)!
let outFrames = AVAudioFrameCount(
Double(buffer.frameLength) * 16_000.0 / hardwareFormat.sampleRate
)
guard let outBuffer = AVAudioPCMBuffer(pcmFormat: target, frameCapacity: outFrames) else {
return
}
var error: NSError?
let status = self.converter?.convert(to: outBuffer, error: &error) { _, outStatus in
outStatus.pointee = .haveData
return buffer
}
if status == .haveData, error == nil {
// AVAudioPCMBuffer is not Sendable; copy the raw float data
// into a Sendable wrapper so we can hand it to the actor.
let copy = AudioBufferSnapshot(buffer: outBuffer)
Task { await self.deliver(copy) }
}
}
}
private func deliver(_ snapshot: AudioBufferSnapshot) {
guard isRunning else { return }
continuation?.yield(snapshot)
}
}
/// Sendable wrapper around a Float32 audio buffer's raw samples.
/// We re-decode on the consumer side to avoid AVAudioPCMBuffer's non-Sendable
/// type crossing the actor boundary.
public struct AudioBufferSnapshot: Sendable {
public let samples: [Float]
public let sampleRate: Double
public init(buffer: AVAudioPCMBuffer) {
guard let channelData = buffer.floatChannelData else {
self.samples = []
self.sampleRate = 16_000
return
}
let n = Int(buffer.frameLength)
var copy = [Float](repeating: 0, count: n)
memcpy(&copy, channelData[0], n * MemoryLayout<Float>.size)
self.samples = copy
self.sampleRate = buffer.format.sampleRate
}
}
public extension AsyncStream where Element == AudioBufferSnapshot {
/// Convenience: convert snapshots to AVAudioPCMBuffer 16kHz mono Float32.
func toAVAudioBuffers() -> AsyncStream<AVAudioPCMBuffer> {
let format = AVAudioFormat(
commonFormat: .pcmFormatFloat32,
sampleRate: 16_000,
channels: 1,
interleaved: false
)!
let snapshots = self
return AsyncStream<AVAudioPCMBuffer> { continuation in
Task {
for await snap in snapshots {
guard !snap.samples.isEmpty,
let pcm = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(snap.samples.count))
else { continue }
pcm.frameLength = AVAudioFrameCount(snap.samples.count)
if let dst = pcm.floatChannelData?[0] {
snap.samples.withUnsafeBufferPointer { src in
if let base = src.baseAddress {
memcpy(dst, base, snap.samples.count * MemoryLayout<Float>.size)
}
}
}
continuation.yield(pcm)
}
continuation.finish()
}
}
}
}
@@ -0,0 +1,46 @@
// PolishingService.swift
// OSGKeyboard · Keyboard Extension
//
// Takes raw ASR transcript and runs it through the user's configured LLM
// to produce polished, well-punctuated text. Falls back to the raw transcript
// if the LLM call fails or times out.
import Foundation
import OSGKeyboardShared
public actor PolishingService {
public enum PolishError: Error {
case noTranscript
case timeout
}
private let store: AppGroupStore
private let timeout: TimeInterval
public init(store: AppGroupStore = AppGroupStore(), timeout: TimeInterval = 8) {
self.store = store
self.timeout = timeout
}
public func polish(_ raw: String) async throws -> String {
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { throw PolishError.noTranscript }
let client = store.makeClient()
let prompt = store.systemPrompt
return try await withThrowingTaskGroup(of: String.self) { group in
group.addTask {
try await client.polish(trimmed, systemPrompt: prompt)
}
group.addTask {
try await Task.sleep(nanoseconds: UInt64(self.timeout * 1_000_000_000))
throw PolishError.timeout
}
let result = try await group.next()!
group.cancelAll()
return result
}
}
}
@@ -0,0 +1,121 @@
// KeyboardRootView.swift
// OSGKeyboard · Keyboard Extension
//
// The single SwiftUI view that backs the keyboard. Shows status text,
// the record button, waveform (when active), and a settings shortcut.
import SwiftUI
public struct KeyboardRootView: View {
public enum Phase: Equatable {
case idle
case recording
case processing
case error(String)
}
public let phase: Phase
public let level: Double // 0...1, used while recording
public let onPressBegan: () -> Void
public let onPressEnded: () -> Void
public let onTap: () -> Void
public let onOpenSettings: () -> Void
public init(
phase: Phase,
level: Double,
onPressBegan: @escaping () -> Void,
onPressEnded: @escaping () -> Void,
onTap: @escaping () -> Void,
onOpenSettings: @escaping () -> Void
) {
self.phase = phase
self.level = level
self.onPressBegan = onPressBegan
self.onPressEnded = onPressEnded
self.onTap = onTap
self.onOpenSettings = onOpenSettings
}
public var body: some View {
ZStack {
// frosted glass background
Rectangle()
.fill(.ultraThinMaterial)
.ignoresSafeArea()
HStack {
Spacer()
statusLine
Spacer()
recordButton
Spacer()
settingsButton
}
.padding(.horizontal, 12)
}
.frame(height: 56)
}
private var statusLine: some View {
VStack(alignment: .leading, spacing: 2) {
switch phase {
case .idle:
Text("Hold to talk")
.font(.system(size: 13, weight: .medium))
.foregroundStyle(.secondary)
case .recording:
HStack(spacing: 6) {
WaveformView(level: level, barCount: 7, color: .red)
Text("Recording…")
.font(.system(size: 12, weight: .medium))
.foregroundStyle(.red)
}
case .processing:
HStack(spacing: 6) {
ProgressView().controlSize(.small)
Text("Polishing…")
.font(.system(size: 12, weight: .medium))
.foregroundStyle(.secondary)
}
case .error(let msg):
Text(msg)
.font(.system(size: 11, weight: .medium))
.foregroundStyle(.orange)
.lineLimit(1)
.truncationMode(.tail)
}
}
.frame(maxWidth: .infinity, alignment: .leading)
}
private var recordButton: some View {
RecordButton(
phase: recordState,
onPressBegan: onPressBegan,
onPressEnded: onPressEnded,
onTap: onTap
)
}
private var settingsButton: some View {
Button(action: onOpenSettings) {
Image(systemName: "gearshape.fill")
.font(.system(size: 16, weight: .medium))
.foregroundStyle(.secondary)
.padding(8)
.background(Color.white.opacity(0.06), in: Circle())
}
.buttonStyle(.plain)
.accessibilityLabel(Text("Open OSGKeyboard settings"))
}
private var recordState: RecordButton.Phase {
switch phase {
case .idle: return .idle
case .recording: return .recording
case .processing: return .processing
case .error(let s): return .error(s)
}
}
}
+116
View File
@@ -0,0 +1,116 @@
// RecordButton.swift
// OSGKeyboard · Keyboard Extension
//
// Circular push-to-talk button styled like Typeless.
// Pulses red while recording; ripples outward.
import SwiftUI
public struct RecordButton: View {
public enum Phase { case idle, recording, processing, error(String) }
public let phase: Phase
public let onPressBegan: () -> Void
public let onPressEnded: () -> Void
public let onTap: () -> Void
@State private var pulse: Bool = false
@GestureState private var isPressed: Bool = false
public init(
phase: Phase,
onPressBegan: @escaping () -> Void,
onPressEnded: @escaping () -> Void,
onTap: @escaping () -> Void
) {
self.phase = phase
self.onPressBegan = onPressBegan
self.onPressEnded = onPressEnded
self.onTap = onTap
}
public var body: some View {
ZStack {
// outer pulse rings
if isRecording {
Circle()
.stroke(Color.red.opacity(0.35), lineWidth: 2)
.frame(width: 110, height: 110)
.scaleEffect(pulse ? 1.3 : 0.95)
.opacity(pulse ? 0 : 1)
.animation(.easeOut(duration: 1.2).repeatForever(autoreverses: false), value: pulse)
}
// main button
Circle()
.fill(buttonColor)
.frame(width: 78, height: 78)
.overlay(
Circle().stroke(Color.white.opacity(0.12), lineWidth: 1)
)
.shadow(color: .black.opacity(0.25), radius: 6, y: 2)
.scaleEffect(isPressed ? 0.92 : 1.0)
.animation(.spring(response: 0.25, dampingFraction: 0.7), value: isPressed)
Image(systemName: iconName)
.font(.system(size: 30, weight: .semibold))
.foregroundStyle(.white)
}
.contentShape(Circle())
.gesture(
LongPressGesture(minimumDuration: 0.15)
.sequenced(before: DragGesture(minimumDistance: 0))
.updating($isPressed) { value, state, _ in
switch value {
case .first, .second: state = true
default: state = false
}
}
.onChanged { value in
switch value {
case .first:
if !pulseStarted { onPressBegan(); pulseStarted = true }
case .second(true, _):
// still pressed
break
default:
if pulseStarted { onPressEnded(); pulseStarted = false }
}
}
.onEnded { _ in
if pulseStarted { onPressEnded(); pulseStarted = false }
}
)
.onTapGesture { onTap() }
.onAppear { pulse = isRecording }
.onChange(of: isRecording) { _, newValue in
pulse = newValue
}
.accessibilityLabel(Text("Push to talk"))
}
private var isRecording: Bool {
if case .recording = phase { return true }
return false
}
@State private var pulseStarted: Bool = false
private var buttonColor: Color {
switch phase {
case .idle: return Color(white: 0.22)
case .recording: return .red
case .processing: return Color(white: 0.32)
case .error: return Color(white: 0.22)
}
}
private var iconName: String {
switch phase {
case .idle: return "mic.fill"
case .recording: return "stop.fill"
case .processing: return "ellipsis"
case .error: return "exclamationmark.triangle.fill"
}
}
}
+37
View File
@@ -0,0 +1,37 @@
// WaveformView.swift
// OSGKeyboard · Keyboard Extension
//
// Simple animated waveform that responds to a 0-1 audio level.
import SwiftUI
public struct WaveformView: View {
public let level: Double // 0...1
public let barCount: Int
public let color: Color
public init(level: Double, barCount: Int = 5, color: Color = .red) {
self.level = max(0, min(1, level))
self.barCount = barCount
self.color = color
}
public var body: some View {
HStack(spacing: 4) {
ForEach(0..<barCount, id: \.self) { i in
Capsule()
.fill(color)
.frame(width: 3, height: heightFor(index: i))
.animation(.easeInOut(duration: 0.18), value: level)
}
}
}
private func heightFor(index: Int) -> CGFloat {
// Center bars taller; outer shorter symmetric pattern
let center = Double(barCount - 1) / 2.0
let distance = abs(Double(index) - center) / max(center, 1)
let base = max(6, 28 * level)
return CGFloat(base * (1.0 - distance * 0.4))
}
}
+21
View File
@@ -0,0 +1,21 @@
// AppGroup.swift
// OSGKeyboard · Shared
//
// App Group identifier shared between main app and keyboard extension.
// UserDefaults(suiteName:) and file containers use this.
import Foundation
public enum AppGroup {
/// App Group container identifier (must match entitlements in both targets)
public static let identifier = "group.com.osgkeyboard.ios"
/// Shared UserDefaults instance for cross-process config
public static var defaults: UserDefaults {
guard let d = UserDefaults(suiteName: identifier) else {
assertionFailure("App Group \(identifier) not configured in entitlements")
return .standard
}
return d
}
}
+22
View File
@@ -0,0 +1,22 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleVersion</key>
<string>1</string>
</dict>
</plist>
+63
View File
@@ -0,0 +1,63 @@
// LLMProvider.swift
// OSGKeyboard · Shared
//
// Provider preset: a known cloud LLM with sensible defaults.
// User picks one of these on first launch, or defines a Custom one.
import Foundation
public struct LLMProvider: Identifiable, Codable, Hashable, Sendable {
public let id: String
public let name: String
public let defaultBaseURL: String
public let defaultModel: String
public let apiKeyURL: URL?
public init(
id: String,
name: String,
defaultBaseURL: String,
defaultModel: String,
apiKeyURL: URL? = nil
) {
self.id = id
self.name = name
self.defaultBaseURL = defaultBaseURL
self.defaultModel = defaultModel
self.apiKeyURL = apiKeyURL
}
public static let presets: [LLMProvider] = [
.init(
id: "openai",
name: "OpenAI",
defaultBaseURL: "https://api.openai.com/v1",
defaultModel: "gpt-4o-mini",
apiKeyURL: URL(string: "https://platform.openai.com/api-keys")
),
.init(
id: "deepseek",
name: "DeepSeek",
defaultBaseURL: "https://api.deepseek.com/v1",
defaultModel: "deepseek-chat",
apiKeyURL: URL(string: "https://platform.deepseek.com/api_keys")
),
.init(
id: "qwen",
name: "Qwen (DashScope, OpenAI-compatible)",
defaultBaseURL: "https://dashscope.aliyuncs.com/compatible-mode/v1",
defaultModel: "qwen-plus",
apiKeyURL: URL(string: "https://dashscope.console.aliyun.com/apiKey")
),
.init(
id: "custom",
name: "Custom (OpenAI-compatible)",
defaultBaseURL: "",
defaultModel: ""
)
]
public static func provider(id: String) -> LLMProvider {
presets.first(where: { $0.id == id }) ?? .presets[0]
}
}
+88
View File
@@ -0,0 +1,88 @@
// LLMRequest.swift
// OSGKeyboard · Shared
//
// OpenAI-compatible chat completion request/response models.
// Compatible with OpenAI, DeepSeek, Qwen DashScope, and any provider that
// implements POST {baseURL}/chat/completions.
import Foundation
public struct LLMRequest: Codable, Sendable {
public let model: String
public let messages: [Message]
public let temperature: Double?
public let maxTokens: Int?
public enum Message: Codable, Sendable {
case system(String)
case user(String)
case assistant(String)
private enum CodingKeys: String, CodingKey {
case role, content
}
public func encode(to encoder: Encoder) throws {
var c = encoder.container(keyedBy: CodingKeys.self)
switch self {
case .system(let s):
try c.encode("system", forKey: .role); try c.encode(s, forKey: .content)
case .user(let s):
try c.encode("user", forKey: .role); try c.encode(s, forKey: .content)
case .assistant(let s):
try c.encode("assistant", forKey: .role); try c.encode(s, forKey: .content)
}
}
public init(from decoder: Decoder) throws {
let c = try decoder.container(keyedBy: CodingKeys.self)
let role = try c.decode(String.self, forKey: .role)
let content = try c.decode(String.self, forKey: .content)
switch role {
case "system": self = .system(content)
case "user": self = .user(content)
case "assistant": self = .assistant(content)
default:
throw DecodingError.dataCorruptedError(forKey: .role, in: c,
debugDescription: "Unknown role \(role)")
}
}
}
public init(
model: String,
messages: [Message],
temperature: Double? = 0.3,
maxTokens: Int? = nil
) {
self.model = model
self.messages = messages
self.temperature = temperature
self.maxTokens = maxTokens
}
}
public struct LLMResponse: Codable, Sendable {
public let id: String?
public let choices: [Choice]
public struct Choice: Codable, Sendable {
public let index: Int
public let message: LLMRequest.Message
public let finishReason: String?
private enum CodingKeys: String, CodingKey {
case index, message
case finishReason = "finish_reason"
}
}
public var content: String {
switch choices.first?.message {
case .system(let s), .user(let s), .assistant(let s):
return s
case .none:
return ""
}
}
}
@@ -0,0 +1,84 @@
// ProviderConfig.swift
// OSGKeyboard · Shared
//
// User's LLM configuration. Persisted in App Group UserDefaults so both
// the main app and keyboard extension read the same values.
import Foundation
import Combine
public final class ProviderConfig: ObservableObject, @unchecked Sendable {
public static let shared = ProviderConfig()
// Storage keys
private enum Key {
static let providerId = "config.providerId"
static let baseURL = "config.baseURL"
static let apiKey = "config.apiKey"
static let model = "config.model"
static let systemPrompt = "config.systemPrompt"
}
@Published public var providerId: String {
didSet { defaults.set(providerId, forKey: Key.providerId) }
}
@Published public var baseURL: String {
didSet { defaults.set(baseURL, forKey: Key.baseURL) }
}
@Published public var apiKey: String {
didSet { defaults.set(apiKey, forKey: Key.apiKey) }
}
@Published public var model: String {
didSet { defaults.set(model, forKey: Key.model) }
}
@Published public var systemPrompt: String {
didSet { defaults.set(systemPrompt, forKey: Key.systemPrompt) }
}
public var isConfigured: Bool {
!baseURL.isEmpty && !apiKey.isEmpty && !model.isEmpty
}
public let defaultSystemPrompt = """
You are a voice-input polishing assistant. The user has spoken informally; rewrite their dictation as clean written text:
1) Preserve the user's original intent and meaning; do not invent facts.
2) Add proper punctuation, capitalization, and paragraph breaks.
3) When the user enumerates items ("first ... second ... third"), output a markdown list.
4) Keep the output concise do not exceed 1.5x the spoken length.
5) Output in the same language as the input.
"""
private let defaults: UserDefaults
public init(defaults: UserDefaults = AppGroup.defaults) {
self.defaults = defaults
self.providerId = defaults.string(forKey: Key.providerId) ?? "openai"
self.baseURL = defaults.string(forKey: Key.baseURL) ?? LLMProvider.provider(id: "openai").defaultBaseURL
self.apiKey = defaults.string(forKey: Key.apiKey) ?? ""
self.model = defaults.string(forKey: Key.model) ?? LLMProvider.provider(id: "openai").defaultModel
self.systemPrompt = defaults.string(forKey: Key.systemPrompt) ?? defaultSystemPrompt
}
public func apply(preset: LLMProvider) {
providerId = preset.id
if !preset.defaultBaseURL.isEmpty {
baseURL = preset.defaultBaseURL
}
if !preset.defaultModel.isEmpty {
model = preset.defaultModel
}
}
public func reset() {
providerId = "openai"
let preset = LLMProvider.provider(id: "openai")
baseURL = preset.defaultBaseURL
apiKey = ""
model = preset.defaultModel
systemPrompt = defaultSystemPrompt
}
}
@@ -0,0 +1,45 @@
// AppGroupStore.swift
// OSGKeyboard · Shared
//
// Convenience wrapper around App Group UserDefaults for non-Published reads.
// Used by the keyboard extension (no SwiftUI) to read config without
// instantiating an ObservableObject.
import Foundation
public struct AppGroupStore: @unchecked Sendable {
public let defaults: UserDefaults
public init(defaults: UserDefaults = AppGroup.defaults) {
self.defaults = defaults
}
public var providerId: String {
defaults.string(forKey: "config.providerId") ?? "openai"
}
public var baseURL: String {
defaults.string(forKey: "config.baseURL") ?? LLMProvider.provider(id: "openai").defaultBaseURL
}
public var apiKey: String {
defaults.string(forKey: "config.apiKey") ?? ""
}
public var model: String {
defaults.string(forKey: "config.model") ?? LLMProvider.provider(id: "openai").defaultModel
}
public var systemPrompt: String {
defaults.string(forKey: "config.systemPrompt")
?? "You are a voice-input polishing assistant. Rewrite the user's dictation as clean written text. Preserve intent. Add punctuation and structure. Do not invent facts. Output in the same language as the input."
}
public func makeClient() -> LLMClient {
OpenAICompatibleClient(
baseURL: baseURL,
apiKey: apiKey,
model: model
)
}
}
+119
View File
@@ -0,0 +1,119 @@
// LLMClient.swift
// OSGKeyboard · Shared
//
// Protocol-based LLM client. Default implementation is the OpenAI-compatible
// chat completion client. Add other impls (Anthropic, Gemini) as needed.
import Foundation
public enum LLMError: Error, LocalizedError, Sendable {
case invalidURL
case noAPIKey
case http(status: Int, body: String)
case decoding(String)
case transport(underlying: String)
case cancelled
public var errorDescription: String? {
switch self {
case .invalidURL: return "Invalid API URL."
case .noAPIKey: return "API key is missing."
case .http(let s, let body):
return "API returned HTTP \(s): \(body.prefix(200))"
case .decoding(let s): return "Failed to decode response: \(s)"
case .transport(let s): return "Network error: \(s)"
case .cancelled: return "Request was cancelled."
}
}
}
public protocol LLMClient: Sendable {
func polish(_ text: String, systemPrompt: String) async throws -> String
}
// MARK: - OpenAI-compatible implementation
public struct OpenAICompatibleClient: LLMClient {
public let baseURL: String
public let apiKey: String
public let model: String
public let session: URLSession
public init(
baseURL: String,
apiKey: String,
model: String,
session: URLSession = .shared
) {
self.baseURL = baseURL
self.apiKey = apiKey
self.model = model
self.session = session
}
public func polish(_ text: String, systemPrompt: String) async throws -> String {
guard !apiKey.isEmpty else { throw LLMError.noAPIKey }
let urlString = baseURL.hasSuffix("/")
? "\(baseURL)chat/completions"
: "\(baseURL)/chat/completions"
guard let url = URL(string: urlString) else { throw LLMError.invalidURL }
let request = LLMRequest(
model: model,
messages: [
.system(systemPrompt),
.user(text)
],
temperature: 0.3,
maxTokens: nil
)
var req = URLRequest(url: url)
req.httpMethod = "POST"
req.setValue("application/json", forHTTPHeaderField: "Content-Type")
req.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
req.timeoutInterval = 15
let encoder = JSONEncoder()
req.httpBody = try encoder.encode(request)
do {
let (data, response) = try await session.data(for: req)
guard let http = response as? HTTPURLResponse else {
throw LLMError.transport(underlying: "non-HTTP response")
}
if !(200..<300).contains(http.statusCode) {
let body = String(data: data, encoding: .utf8) ?? ""
throw LLMError.http(status: http.statusCode, body: body)
}
do {
let decoded = try JSONDecoder().decode(LLMResponse.self, from: data)
return decoded.content.trimmingCharacters(in: .whitespacesAndNewlines)
} catch {
throw LLMError.decoding(String(describing: error))
}
} catch let err as LLMError {
throw err
} catch is CancellationError {
throw LLMError.cancelled
} catch let urlError as URLError where urlError.code == .cancelled {
throw LLMError.cancelled
} catch {
throw LLMError.transport(underlying: String(describing: error))
}
}
}
// MARK: - Factory
public enum LLMClientFactory {
/// Build a client from the current `ProviderConfig`.
public static func make(from config: ProviderConfig) -> LLMClient {
OpenAICompatibleClient(
baseURL: config.baseURL,
apiKey: config.apiKey,
model: config.model
)
}
}
+22
View File
@@ -0,0 +1,22 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundlePackageType</key>
<string>BNDL</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleVersion</key>
<string>1</string>
</dict>
</plist>
+131
View File
@@ -0,0 +1,131 @@
// LLMClientTests.swift
// OSGKeyboard · Tests
//
// Unit tests for the OpenAI-compatible LLM client using URLProtocol stub.
import XCTest
@testable import OSGKeyboard
@testable import OSGKeyboardShared
final class LLMClientTests: XCTestCase {
// MARK: - ProviderConfig persistence
func testProviderConfigPersistsAcrossInstances() {
let defaults = UserDefaults(suiteName: "group.com.osgkeyboard.ios.tests")!
defaults.removePersistentDomain(forName: "group.com.osgkeyboard.ios.tests")
let config1 = ProviderConfig(defaults: defaults)
config1.baseURL = "https://example.com/v1"
config1.apiKey = "test-key"
config1.model = "test-model"
let config2 = ProviderConfig(defaults: defaults)
XCTAssertEqual(config2.baseURL, "https://example.com/v1")
XCTAssertEqual(config2.apiKey, "test-key")
XCTAssertEqual(config2.model, "test-model")
XCTAssertTrue(config2.isConfigured)
}
// MARK: - OpenAICompatibleClient
func testPolishSendsCorrectRequestAndDecodesResponse() async throws {
StubURLProtocolStorage.config = (200, """
{
"id": "chatcmpl-1",
"choices": [
{ "index": 0, "message": { "role": "assistant", "content": "Hello, world!" }, "finish_reason": "stop" }
]
}
""".data(using: .utf8)!)
defer { StubURLProtocolStorage.config = nil }
let cfg = URLSessionConfiguration.ephemeral
cfg.protocolClasses = [StubURLProtocol.self]
let session = URLSession(configuration: cfg)
let client = OpenAICompatibleClient(
baseURL: "https://example.com/v1",
apiKey: "sk-test",
model: "test-model",
session: session
)
let result = try await client.polish("hi", systemPrompt: "be brief")
XCTAssertEqual(result, "Hello, world!")
let req = StubURLProtocolStorage.lastRequest
XCTAssertEqual(req?.httpMethod, "POST")
XCTAssertTrue(req?.value(forHTTPHeaderField: "Authorization")?.hasPrefix("Bearer ") == true)
}
func testPolishThrowsOnHTTPError() async {
StubURLProtocolStorage.config = (401, "Unauthorized".data(using: .utf8)!)
defer { StubURLProtocolStorage.config = nil }
let cfg = URLSessionConfiguration.ephemeral
cfg.protocolClasses = [StubURLProtocol.self]
let session = URLSession(configuration: cfg)
let client = OpenAICompatibleClient(
baseURL: "https://example.com/v1",
apiKey: "sk-test",
model: "m",
session: session
)
do {
_ = try await client.polish("hi", systemPrompt: "p")
XCTFail("expected error")
} catch let LLMError.http(status, _) {
XCTAssertEqual(status, 401)
} catch {
XCTFail("wrong error: \(error)")
}
}
func testPolishThrowsWhenAPIKeyMissing() async {
let client = OpenAICompatibleClient(
baseURL: "https://example.com/v1",
apiKey: "",
model: "m"
)
do {
_ = try await client.polish("hi", systemPrompt: "p")
XCTFail("expected error")
} catch LLMError.noAPIKey {
// ok
} catch {
XCTFail("wrong error: \(error)")
}
}
}
// MARK: - URLProtocol stub
/// Per-test stub config holder. Tests set these via `StubURLProtocol.config =`
/// before invoking the code under test, then reset to nil in cleanup.
private enum StubURLProtocolStorage {
nonisolated(unsafe) static var config: (statusCode: Int, body: Data)?
nonisolated(unsafe) static var lastRequest: URLRequest?
}
private final class StubURLProtocol: URLProtocol, @unchecked Sendable {
override class func canInit(with request: URLRequest) -> Bool { true }
override class func canonicalRequest(for request: URLRequest) -> URLRequest { request }
override func startLoading() {
let cfg = StubURLProtocolStorage.config ?? (statusCode: 200, body: Data())
StubURLProtocolStorage.lastRequest = request
let response = HTTPURLResponse(
url: request.url!,
statusCode: cfg.statusCode,
httpVersion: "HTTP/1.1",
headerFields: ["Content-Type": "application/json"]
)!
client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
client?.urlProtocol(self, didLoad: cfg.body)
client?.urlProtocolDidFinishLoading(self)
}
override func stopLoading() {}
}
+153
View File
@@ -0,0 +1,153 @@
# OSGKeyboard
> Hold a key, speak, release — AI-polished text appears at your cursor in any app.
> An open-source, custom-keyboard-based voice input tool for iOS 18+, inspired by [Typeless](https://typeless.com) and [OpenLess](https://github.com/Open-Less/openless).
![Platform](https://img.shields.io/badge/platform-iOS%2018%2B-0078D4?logo=apple)
![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift)
![License](https://img.shields.io/badge/license-MIT-green)
![CI](https://github.com/<OWNER>/OSGKeyboard/actions/workflows/ci.yml/badge.svg)
[中文 README](./README.zh.md)
---
## What is it?
OSGKeyboard is a free, open alternative to commercial voice-input tools. It runs as a **Custom Keyboard Extension** on iOS, so you can use it in **any app** — Messages, Notes, Mail, ChatGPT, Claude, Cursor, you name it.
1. Press and hold the mic key
2. Speak naturally
3. Release — the AI polishes your words into clean text and inserts it at the cursor
The audio stays on-device (transcribed by Apple's on-device `SpeechAnalyzer` on iOS 26+, or `SFSpeechRecognizer` on iOS 18/19). Only the **polished transcript** is sent to your chosen cloud LLM. **No audio ever leaves your phone.**
---
## Features
- 🎙 **Push-to-talk** with a Typeless-style circular mic button
- 🧠 **On-device ASR** (iOS 26 `SpeechAnalyzer` + `DictationTranscriber`; iOS 18 `SFSpeechRecognizer` fallback)
- ✍️ **AI polishing** — adds structure, punctuation, fixes grammar, optionally produces lists
- 🔌 **Bring-your-own API** — works with any OpenAI-compatible endpoint (OpenAI, DeepSeek, Qwen DashScope, your own self-hosted server, …)
- 🔒 **Privacy first** — audio never leaves your device; transcripts only sent to the LLM you choose
- 🎨 **Native SwiftUI** — dark theme, frosted glass, ~2000 lines of Swift
- 🪶 **Zero dependencies** — no SwiftPM packages, no CocoaPods, no Carthage
---
## Quick start
### Requirements
- macOS with **Xcode 16+** (Xcode 26 recommended)
- iPhone running **iOS 18.0+** (iOS 26+ for the best on-device ASR)
- [XcodeGen](https://github.com/yonaskolb/XcodeGen): `brew install xcodegen`
- An OpenAI-compatible API key (e.g. from [OpenAI](https://platform.openai.com/api-keys), [DeepSeek](https://platform.deepseek.com/api_keys), or [Qwen DashScope](https://dashscope.console.aliyun.com/apiKey))
### Build & run
```bash
git clone https://github.com/<OWNER>/OSGKeyboard.git
cd OSGKeyboard
xcodegen generate # produces OSGKeyboard.xcodeproj
open OSGKeyboard.xcodeproj # or build via CLI:
xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
-destination 'generic/platform=iOS Simulator' build
```
### Enable the keyboard in iOS
1. Run the app on your device or simulator.
2. Follow the 3-step onboarding: **enable the keyboard** in iOS Settings, then **allow Full Access** (required for the mic and LLM calls), then **paste your API key**.
3. In any text field, tap 🌐 to switch to **OSGKeyboard**.
4. Press and hold the mic, speak, release. ✨
> **"Allow Full Access" is required.** Without it, iOS blocks the keyboard from using the microphone and from making network requests. We never log, store, or transmit your keystrokes — see [`PrivacyInfo.xcprivacy`](./OpenLess/PrivacyInfo.xcprivacy).
---
## Architecture
```
OSGKeyboard/
├── OpenLess/ # Main iOS app (settings, onboarding)
│ ├── Views/ # SwiftUI screens
│ ├── OSGKeyboardApp.swift # @main entry
│ ├── PrivacyInfo.xcprivacy # Required privacy manifest
│ └── OpenLess.entitlements # App Group declaration
├── OpenLessKeyboard/ # Custom Keyboard Extension
│ ├── KeyboardViewController.swift # Principal class
│ ├── Services/
│ │ ├── AudioCaptureService.swift # AVAudioEngine → 16 kHz PCM
│ │ ├── ASRService.swift # iOS 26 + iOS 18 ASR
│ │ └── PolishingService.swift # LLM call with timeout
│ └── Views/ # RecordButton, Waveform, KeyboardRootView
├── OpenLessShared/ # Framework shared by app + extension
│ ├── Models/ # ProviderConfig, LLMRequest, LLMProvider
│ ├── Services/ # LLMClient (OpenAI-compatible)
│ └── Constants/ # AppGroup identifier
├── OpenLessTests/ # XCTest unit tests
├── project.yml # XcodeGen project definition
└── .github/workflows/ci.yml # Lint + build CI
```
### Data flow
```
[Long-press mic] → AudioCaptureService → AudioBufferSnapshot (16 kHz mono)
ASRService.transcribe()
ASREvent.final(rawTranscript)
PolishingService.polish()
LLMClient (OpenAI-compatible)
textDocumentProxy.insertText(polished)
```
---
## Adding a new LLM provider
Open `OpenLessShared/Models/LLMProvider.swift` and append a new `LLMProvider` to the `presets` array. The default `OpenAICompatibleClient` handles any endpoint that speaks the `POST /chat/completions` protocol.
```swift
LLMProvider(
id: "groq",
name: "Groq",
defaultBaseURL: "https://api.groq.com/openai/v1",
defaultModel: "llama-3.1-70b-versatile",
apiKeyURL: URL(string: "https://console.groq.com/keys")
)
```
That's it. No other code changes required.
---
## Limitations
- iOS sandboxes keyboard extensions: ~60 MB memory cap, Full Access required.
- The keyboard does **not** work in password fields or some `WKWebView` textareas (iOS limitation).
- iOS 18 uses `SFSpeechRecognizer` instead of `SpeechAnalyzer`; on iOS 26, `SpeechAnalyzer` is significantly faster and supports more locales.
---
## License
[MIT](./LICENSE) — use it, fork it, ship it. No warranty.
---
## Acknowledgements
- Inspired by [Typeless](https://typeless.com) and the desktop open-source [OpenLess](https://github.com/Open-Less/openless)
- Built with [XcodeGen](https://github.com/yonaskolb/XcodeGen)
- Powered by Apple's [SpeechAnalyzer](https://developer.apple.com/documentation/speech/speechanalyzer) and [SFSpeechRecognizer](https://developer.apple.com/documentation/speech/sfspeechrecognizer)
---
**Note:** replace `<OWNER>` in badges and the git clone URL with your GitHub username.
+152
View File
@@ -0,0 +1,152 @@
# OSGKeyboard
> 按住说话,松开即得 AI 润色文字,插入任意 App 的光标处。
> 一款开源的 iOS 自定义键盘语音输入工具,灵感来自 [Typeless](https://typeless.com) 和 [OpenLess](https://github.com/Open-Less/openless)。
![Platform](https://img.shields.io/badge/platform-iOS%2018%2B-0078D4?logo=apple)
![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift)
![License](https://img.shields.io/badge/license-MIT-green)
[English README](./README.md)
---
## 这是什么?
OSGKeyboard 是商业语音输入工具的免费开源替代。它以 **iOS 自定义键盘扩展** 的形式运行,所以你可以在 **任何 App** 里使用 —— 微信、备忘录、邮件、ChatGPT、Claude、Cursor,无所不能。
1. 长按麦克风键
2. 自由说话
3. 松开 —— AI 帮你整理成干净的文字,自动插入光标处
**音频始终在设备本地转写**(iOS 26+ 用 `SpeechAnalyzer`iOS 18/19 用 `SFSpeechRecognizer`),**只有润色后的文本** 会发到你选择的云端 LLM。**音频永不离开你的手机。**
---
## 特性
- 🎙 **按住说话**,Typeless 风格的圆形麦克风按钮
- 🧠 **端侧 ASR**iOS 26 `SpeechAnalyzer` + `DictationTranscriber`iOS 18 退回 `SFSpeechRecognizer`
- ✍️ **AI 润色** —— 自动加结构、补标点、修正语法、可生成列表
- 🔌 **自带 API 接入** —— 兼容任何 OpenAI 兼容协议端点(OpenAI / DeepSeek / Qwen DashScope / 自建服务器 ……)
- 🔒 **隐私优先** —— 音频不离开设备;只有润色文本会发给你选择的 LLM
- 🎨 **原生 SwiftUI** —— 暗色主题、毛玻璃、约 2000 行 Swift
- 🪶 **零依赖** —— 无 SwiftPM 包、无 CocoaPods、无 Carthage
---
## 快速开始
### 环境要求
- macOS + **Xcode 16+**(推荐 Xcode 26
- iPhone 运行 **iOS 18.0+**iOS 26+ 体验最佳)
- [XcodeGen](https://github.com/yonaskolb/XcodeGen)`brew install xcodegen`
- 一个 OpenAI 兼容 API Key[OpenAI](https://platform.openai.com/api-keys) / [DeepSeek](https://platform.deepseek.com/api_keys) / [Qwen DashScope](https://dashscope.console.aliyun.com/apiKey) 任一)
### 编译与运行
```bash
git clone https://github.com/<你的用户名>/OSGKeyboard.git
cd OSGKeyboard
xcodegen generate # 生成 OSGKeyboard.xcodeproj
open OSGKeyboard.xcodeproj # 或命令行编译:
xcodebuild -project OSGKeyboard.xcodeproj -scheme OSGKeyboard \
-destination 'generic/platform=iOS Simulator' build
```
### 在 iOS 中启用键盘
1. 在真机/模拟器上运行 App。
2. 按 3 步引导:**启用键盘** → **允许完全访问**(麦克风 + LLM 调用必须) → **粘贴 API Key**
3. 在任意输入框,点 🌐 切换到 **OSGKeyboard**
4. 长按麦克风键 → 说话 → 松开。✨
> **"允许完全访问"是必须的。** 没有它,iOS 会阻止键盘使用麦克风与网络。我们**绝不记录、存储或上传你的击键** —— 见 [`PrivacyInfo.xcprivacy`](./OpenLess/PrivacyInfo.xcprivacy)。
---
## 架构
```
OSGKeyboard/
├── OpenLess/ # 主 iOS App(设置、Onboarding
│ ├── Views/ # SwiftUI 屏幕
│ ├── OSGKeyboardApp.swift # @main 入口
│ ├── PrivacyInfo.xcprivacy # 隐私清单
│ └── OpenLess.entitlements # App Group 声明
├── OpenLessKeyboard/ # 自定义键盘扩展
│ ├── KeyboardViewController.swift # 主体类
│ ├── Services/
│ │ ├── AudioCaptureService.swift # AVAudioEngine → 16kHz PCM
│ │ ├── ASRService.swift # iOS 26 + iOS 18 ASR
│ │ └── PolishingService.swift # LLM 调用(带超时)
│ └── Views/ # 录音按钮、波形、键盘主视图
├── OpenLessShared/ # 主 App + 键盘共享 framework
│ ├── Models/ # ProviderConfig、LLMRequest、LLMProvider
│ ├── Services/ # LLMClientOpenAI 兼容)
│ └── Constants/ # App Group ID
├── OpenLessTests/ # XCTest 单元测试
├── project.yml # XcodeGen 工程定义
└── .github/workflows/ci.yml # Lint + 编译 CI
```
### 数据流
```
[长按麦克风] → AudioCaptureService → AudioBufferSnapshot (16kHz mono)
ASRService.transcribe()
ASREvent.final(原始转写文本)
PolishingService.polish()
LLMClientOpenAI 兼容协议)
textDocumentProxy.insertText(润色后文本)
```
---
## 新增 LLM 提供商
打开 `OpenLessShared/Models/LLMProvider.swift`,在 `presets` 数组里追加一条 `LLMProvider` 即可。默认的 `OpenAICompatibleClient` 处理任何实现了 `POST /chat/completions` 的端点。
```swift
LLMProvider(
id: "groq",
name: "Groq",
defaultBaseURL: "https://api.groq.com/openai/v1",
defaultModel: "llama-3.1-70b-versatile",
apiKeyURL: URL(string: "https://console.groq.com/keys")
)
```
仅此而已,**无需改动其他代码**。
---
## 限制
- iOS 沙盒:键盘扩展 ~60 MB 内存上限,必须开完全访问
- 密码框与部分 `WKWebView` 输入框不可用(iOS 限制)
- iOS 18 用 `SFSpeechRecognizer` 退化路径;iOS 26 的 `SpeechAnalyzer` 更快、支持语种更多
---
## 许可
[MIT](./LICENSE) —— 使用、修改、商用均可。无任何担保。
---
## 致谢
- 灵感来源:[Typeless](https://typeless.com) 与桌面端开源版 [OpenLess](https://github.com/Open-Less/openless)
- 工程脚手架:[XcodeGen](https://github.com/yonaskolb/XcodeGen)
- 端侧 ASRApple [SpeechAnalyzer](https://developer.apple.com/documentation/speech/speechanalyzer) / [SFSpeechRecognizer](https://developer.apple.com/documentation/speech/sfspeechrecognizer)
---
**注意**:把 `<你的用户名>` 替换成你的 GitHub 用户名。
+154
View File
@@ -0,0 +1,154 @@
# XcodeGen configuration for OSGKeyboard
# Run `xcodegen generate` to create OSGKeyboard.xcodeproj
# Repo only tracks project.yml; .xcodeproj is gitignored.
name: OSGKeyboard
options:
bundleIdPrefix: com.osgkeyboard
deploymentTarget:
iOS: "18.0"
developmentLanguage: en
createIntermediateGroups: true
generateEmptyDirectories: true
groupSortPosition: top
settings:
base:
SWIFT_VERSION: "6.0"
IPHONEOS_DEPLOYMENT_TARGET: "18.0"
ENABLE_USER_SCRIPT_SANDBOXING: YES
SWIFT_STRICT_CONCURRENCY: complete
GENERATE_INFOPLIST_FILE: NO
ENABLE_MODULE_VERIFIER: YES
CODE_SIGN_STYLE: Automatic
CODE_SIGN_IDENTITY: "Apple Development"
DEVELOPMENT_TEAM: ""
MARKETING_VERSION: "0.1.0"
CURRENT_PROJECT_VERSION: "1"
targets:
# =========================================================
# 主 App
# =========================================================
OSGKeyboard:
type: application
platform: iOS
sources:
- path: OpenLess
entitlements:
path: OpenLess/OpenLess.entitlements
resources:
- path: OpenLess/Assets.xcassets
info:
path: OpenLess/Info.plist
properties:
CFBundleDisplayName: OSGKeyboard
CFBundleShortVersionString: "$(MARKETING_VERSION)"
CFBundleVersion: "$(CURRENT_PROJECT_VERSION)"
UILaunchScreen:
UIColorName: ""
UISupportedInterfaceOrientations:
- UIInterfaceOrientationPortrait
UIApplicationSceneManifest:
UIApplicationSupportsMultipleScenes: false
NSMicrophoneUsageDescription: "OSGKeyboard needs microphone access to transcribe your voice into text."
NSAppTransportSecurity:
NSAllowsArbitraryLoads: false
CFBundleURLTypes:
- CFBundleURLName: com.osgkeyboard.ios.settings
CFBundleURLSchemes:
- osgkeyboard
settings:
base:
PRODUCT_BUNDLE_IDENTIFIER: com.osgkeyboard.ios
TARGETED_DEVICE_FAMILY: "1,2"
INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents: YES
dependencies:
- target: OSGKeyboardShared
embed: true
- target: OSGKeyboardExt
# Keyboard Extension is a plugin of the main App; embed it.
# =========================================================
# Keyboard Extension
# =========================================================
OSGKeyboardExt:
type: app-extension
platform: iOS
sources:
- path: OpenLessKeyboard
entitlements:
path: OpenLessKeyboard/OpenLessKeyboard.entitlements
info:
path: OpenLessKeyboard/Info.plist
properties:
CFBundleDisplayName: OSGKeyboard
CFBundleShortVersionString: "$(MARKETING_VERSION)"
CFBundleVersion: "$(CURRENT_PROJECT_VERSION)"
NSExtension:
NSExtensionAttributes:
IsASCIICapable: false
PrefersRightToLeft: false
PrimaryLanguage: "en-US"
RequestsOpenAccess: true
NSExtensionPointIdentifier: com.apple.keyboard-service
NSExtensionPrincipalClass: $(PRODUCT_MODULE_NAME).KeyboardViewController
settings:
base:
PRODUCT_BUNDLE_IDENTIFIER: com.osgkeyboard.ios.keyboard
TARGETED_DEVICE_FAMILY: "1,2"
dependencies:
- target: OSGKeyboardShared
embed: false
# =========================================================
# Shared Framework (主 App 与扩展共用)
# =========================================================
OSGKeyboardShared:
type: framework
platform: iOS
sources:
- path: OpenLessShared
info:
path: OpenLessShared/Info.plist
settings:
base:
PRODUCT_BUNDLE_IDENTIFIER: com.osgkeyboard.ios.shared
DEFINES_MODULE: YES
SKIP_INSTALL: YES
BUILD_LIBRARY_FOR_DISTRIBUTION: NO
DYLIB_INSTALL_NAME_BASE: "@rpath"
APPLICATION_EXTENSION_API_ONLY: YES
ENABLE_MODULE_VERIFIER: YES
# =========================================================
# 单元测试
# =========================================================
OSGKeyboardTests:
type: bundle.unit-test
platform: iOS
sources:
- path: OpenLessTests
info:
path: OpenLessTests/Info.plist
dependencies:
- target: OSGKeyboard
settings:
base:
PRODUCT_BUNDLE_IDENTIFIER: com.osgkeyboard.ios.tests
schemes:
OSGKeyboard:
build:
targets:
OSGKeyboard: all
OSGKeyboardExt: all
run:
config: Debug
test:
config: Debug
targets:
- OSGKeyboardTests
archive:
config: Release