feat(keyboard): in-keyboard onboarding overlay + AppContext chip

Two long-standing UX papercuts, fixed without leaving the keyboard:

1. **First-launch onboarding inside the keyboard** — iOS keyboard
   extensions *cannot* programmatically switch back to the previous
   app after jumping out, so the old flow (jump to host app → user
   has to manually navigate back) was 5+ taps of friction. The new
   `KeyboardOnboardingOverlay` keeps the user inside the keyboard
   for steps 1 (welcome), 2 (mic permission), 3 (speech permission),
   and 5 (API key hint). The only step that *must* leave is step 4
   ("Enable Keyboard"), which jumps to `Settings.app` via
   `UIApplication.openSettingsURLString` — on return,
   `viewWillAppear` calls `autoAdvancePastKeyboardSetupStepIfNeeded`
   which silently advances past that step if the keyboard is now
   enabled. Net UX: user types in their app, keyboard walks them
   through setup, normal UI appears as soon as setup is done.

2. **Per-app context chip on the keyboard top bar** — the v0.3.0
   intelligent-prompt pipeline already adapted tone to detected
   context (code/email/chat/document), but without a UI cue the user
   had no way to know which mode was active or override the heuristic
   when it guessed wrong. The new `AppContextChip` surfaces the
   detected context; tap-to-override writes back to
   `AppGroupStore.setDetectedAppContext` so the next LLM call
   picks up the new tone immediately. Wired into `pressBegan` so
   the chip updates in real time as the user types into different
   fields.

### Files added
- `OSGKeyboardExt/Views/AppContextChip.swift` — chip + Menu override
- `OSGKeyboardExt/Views/KeyboardOnboardingOverlay.swift` — 5-step overlay
- `OSGKeyboardTests/KeyboardOnboardingOverlayTests.swift` — round-trip + enum surface tests

### Files modified
- `OSGKeyboardShared/Services/KeyboardState.swift`
  + `hasCompletedOnboarding`, `onboardingPage`, `appContext`
  + `setAppContext`, `advanceOnboarding`, `completeOnboarding`
  + `requestMicPermission`, `requestSpeechPermission`, `openSystemSettings`
- `OSGKeyboardShared/Services/AppGroupStore.swift`
  + `hasCompletedOnboarding` / `onboardingPage` accessors (mirror of
    `ProviderConfig` keys, so the keyboard extension never has to
    instantiate the host-app config)
- `OSGKeyboardExt/KeyboardViewController.swift`
  + action hooks wired (`installStateActions`)
  + `syncOnboardingStateFromAppGroup` / `syncAppContextFromAppGroup`
    called on `viewWillAppear` and `loadPersistedConfig`
  + `autoAdvancePastKeyboardSetupStepIfNeeded` for the silent
    "jump out → come back" flow
  + `openSystemSettingsFromExtension` opens `Settings.app` via
    `UIApplication.openSettingsURLString` (the only system URL
    the extension is allowed to open)
  + `detectAndStoreAppContext` mirrors to `state.appContext` so
    the chip updates without waiting for `viewWillAppear`
- `OSGKeyboardExt/Views/KeyboardRootView.swift`
  + overlay mounted in `ZStack` over normal UI (animated)
  + AppContextChip in top bar (hidden during onboarding)
- `OSGKeyboardExt/{en,zh-Hans}.lproj/Keyboard.strings`
  + onboarding copy + chip labels (35 keys per language)

### iOS sandbox notes (kept here for posterity)
- Keyboard extensions **cannot** present AVAudioSession /
  SFSpeechRecognizer permission dialogs directly. The overlay's
  step 2/3 buttons optimistically advance; the actual permission
  is granted when the user first opens the host app (which the
  step-5 "Open OSGKeyboard" button triggers). This is the same
  pattern the previous "jump to host app" flow used — just
  without the broken return trip.
- `UIApplication.openSettingsURLString` is the only system URL
  reachable from `extensionContext.open`. Both step 4 and the
  "Open Settings" button route through `HostAppLauncher` so the
  responder-chain fallback also kicks in if needed.

Co-authored-by: Mavis <Mavis@hkgood.dev>
This commit is contained in:
Mavis
2026-07-03 07:55:12 +00:00
parent 7dc37e4272
commit 0401f67a89
9 changed files with 760 additions and 16 deletions
+76
View File
@@ -0,0 +1,76 @@
// AppContextChip.swift
// OSGKeyboard · Keyboard Extension
//
// Surfaces the v0.3.0 per-app polish context on the keyboard top
// bar. The LLM prompt already adapts to the detected context (see
// `PolishingService.buildPrompt(for:context:)`), but without a UI
// cue the user has no way to know "I'm currently in code mode"
// and no way to override the heuristic when it guesses wrong.
//
// Tap the chip cycle through the five `AppContext` cases. The new
// value is written to `AppGroupStore.setDetectedAppContext(_:at:)`,
// so the next `PolishingService` call picks it up immediately.
import SwiftUI
import OSGKeyboardShared
struct AppContextChip: View {
@Environment(\.themePalette) private var palette: ThemePalette
@ObservedObject var state: KeyboardViewController.State
var body: some View {
Menu {
ForEach(AppContext.allCases, id: \.self) { context in
Button {
state.setAppContext(context)
} label: {
if context == state.appContext {
Label(menuLabel(for: context), systemImage: "checkmark")
} else {
Text(menuLabel(for: context))
}
}
}
} label: {
label
}
.menuStyle(.button)
.accessibilityLabel(ExtL10n.text("keyboard.appContext.a11y"))
.accessibilityHint(ExtL10n.text("keyboard.appContext.a11yHint"))
}
private var label: some View {
HStack(spacing: 4) {
Image(systemName: iconName(for: state.appContext))
Text(chipText)
Image(systemName: "chevron.down")
.font(.system(size: 8, weight: .bold))
}
.font(TypeStyle.caption2)
.foregroundStyle(palette.textPrimary)
.padding(.horizontal, Spacing.xs + 2)
.padding(.vertical, 6)
.frame(minHeight: 28)
.background(palette.surfaceElevated, in: Capsule())
.overlay(Capsule().stroke(palette.divider, lineWidth: 0.5))
}
private var chipText: String {
ExtL10n.text("keyboard.appContext.chip.\(state.appContext.rawValue)")
}
private func menuLabel(for context: AppContext) -> String {
ExtL10n.text("keyboard.appContext.menu.\(context.rawValue)")
}
private func iconName(for context: AppContext) -> String {
switch context {
case .code: return "chevron.left.forwardslash.chevron.right"
case .email: return "envelope"
case .chat: return "bubble.left"
case .document: return "doc.text"
case .unknown: return "questionmark.circle"
}
}
}
@@ -0,0 +1,293 @@
// KeyboardOnboardingOverlay.swift
// OSGKeyboard · Keyboard Extension
//
// v0.3.0 in-keyboard onboarding. Replaces the previous "jump out to
// the host app" flow with a five-step overlay that lives on top of
// the normal keyboard UI.
//
// Why in-keyboard instead of jumping to the host app?
//
// - iOS keyboard extensions **cannot programmatically switch back
// to the previous app** after a host-app jump. The user has to
// re-find their app, re-tap a text field, and re-select OSGKeyboard
// from the globe menu. That's a 5+ tap friction.
//
// - Steps 1, 2, 4 (welcome, mic permission, speech permission,
// API key) need nothing the host app owns. They can all live in
// the keyboard.
//
// - The only step that *must* leave the keyboard is step 3
// ("Enable Keyboard") iOS requires the user to flip a toggle
// in `Settings.app`, which is reachable from the extension via
// `UIApplication.openSettingsURLString`. After the user comes
// back, `viewWillAppear` reads `KeyboardSetupBridge.isReadyForOnboardingSkip`
// and the overlay auto-advances past step 3.
//
// The overlay mounts only when `state.hasCompletedOnboarding == false`.
// All inputs route through `KeyboardState` action hooks, so the
// controller can mirror them into the App Group without the view
// having to know about persistence.
import SwiftUI
import OSGKeyboardShared
struct KeyboardOnboardingOverlay: View {
@Environment(\.themePalette) private var palette: ThemePalette
@ObservedObject var state: KeyboardViewController.State
var body: some View {
ZStack {
// Dim the underlying keyboard so the overlay reads as a
// distinct surface. We can't completely hide it without
// losing keyboard-system visibility, so a 60% black wash
// is the sweet spot between focus and consistency.
palette.background.opacity(0.96).ignoresSafeArea()
VStack(spacing: 0) {
header
Spacer(minLength: Spacing.sm)
Group {
switch currentStep {
case .welcome: welcomeStep
case .microphone: microphoneStep
case .speech: speechStep
case .keyboard: keyboardStep
case .api: apiStep
}
}
.transition(.opacity.combined(with: .move(edge: .trailing)))
.frame(maxWidth: .infinity, maxHeight: .infinity)
Spacer(minLength: Spacing.sm)
footer
}
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.md)
}
}
// MARK: - Steps
private enum Step: Int, CaseIterable {
case welcome = 0, microphone, speech, keyboard, api
static let count = 5
}
private var currentStep: Step {
Step(rawValue: state.onboardingPage) ?? .welcome
}
private var header: some View {
VStack(spacing: Spacing.xs) {
HStack(spacing: 6) {
ForEach(0..<Step.count, id: \.self) { idx in
Capsule()
.fill(idx <= currentStep.rawValue
? palette.accent
: palette.divider)
.frame(height: 4)
}
}
.padding(.horizontal, Spacing.xs)
Text(ExtL10n.string("keyboard.onboarding.title"))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textSecondary)
}
}
// MARK: - Step content
private var welcomeStep: some View {
stepBody(
iconSystemName: "waveform.badge.mic",
title: "keyboard.onboarding.welcome.title",
body: "keyboard.onboarding.welcome.body"
)
}
private var microphoneStep: some View {
stepBody(
iconSystemName: "mic.fill",
title: "keyboard.onboarding.mic.title",
body: "keyboard.onboarding.mic.body"
)
}
private var speechStep: some View {
stepBody(
iconSystemName: "ear",
title: "keyboard.onboarding.speech.title",
body: "keyboard.onboarding.speech.body"
)
}
private var keyboardStep: some View {
VStack(spacing: Spacing.md) {
Image(systemName: "keyboard")
.font(.system(size: 36, weight: .light))
.foregroundStyle(palette.accent)
Text(ExtL10n.text("keyboard.onboarding.keyboard.title"))
.font(TypeStyle.headline)
.foregroundStyle(palette.textPrimary)
.multilineTextAlignment(.center)
Text(ExtL10n.text("keyboard.onboarding.keyboard.body"))
.font(TypeStyle.caption1)
.foregroundStyle(palette.textSecondary)
.multilineTextAlignment(.center)
.fixedSize(horizontal: false, vertical: true)
Button {
state.openSystemSettings()
} label: {
HStack(spacing: 6) {
Image(systemName: "arrow.up.right.square")
Text(ExtL10n.text("keyboard.onboarding.keyboard.openSettings"))
}
.font(TypeStyle.caption1.weight(.semibold))
.foregroundStyle(.white)
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.xs + 2)
.background(palette.accent, in: Capsule())
}
.accessibilityLabel(ExtL10n.string("keyboard.onboarding.keyboard.openSettings"))
}
.frame(maxWidth: .infinity)
}
private var apiStep: some View {
VStack(spacing: Spacing.md) {
Image(systemName: "key.fill")
.font(.system(size: 32, weight: .light))
.foregroundStyle(palette.accent)
Text(ExtL10n.text("keyboard.onboarding.api.title"))
.font(TypeStyle.headline)
.foregroundStyle(palette.textPrimary)
.multilineTextAlignment(.center)
Text(ExtL10n.text("keyboard.onboarding.api.body"))
.font(TypeStyle.caption1)
.foregroundStyle(palette.textSecondary)
.multilineTextAlignment(.center)
.fixedSize(horizontal: false, vertical: true)
Text(ExtL10n.text("keyboard.onboarding.api.skipHint"))
.font(TypeStyle.caption2)
.foregroundStyle(palette.textSecondary)
.multilineTextAlignment(.center)
}
.frame(maxWidth: .infinity)
}
private func stepBody(
iconSystemName: String,
title: String,
body: String
) -> some View {
VStack(spacing: Spacing.md) {
Image(systemName: iconSystemName)
.font(.system(size: 36, weight: .light))
.foregroundStyle(palette.accent)
Text(ExtL10n.text(title))
.font(TypeStyle.headline)
.foregroundStyle(palette.textPrimary)
.multilineTextAlignment(.center)
Text(ExtL10n.text(body))
.font(TypeStyle.caption1)
.foregroundStyle(palette.textSecondary)
.multilineTextAlignment(.center)
.fixedSize(horizontal: false, vertical: true)
}
.frame(maxWidth: .infinity)
}
// MARK: - Footer
private var footer: some View {
HStack(spacing: Spacing.xs) {
if currentStep != .welcome {
Button(ExtL10n.text("keyboard.onboarding.back")) {
state.onboardingPage = max(0, currentStep.rawValue - 1)
}
.font(TypeStyle.caption1)
.foregroundStyle(palette.textSecondary)
.frame(minHeight: 36)
}
Spacer(minLength: 0)
primaryButton
}
}
@ViewBuilder
private var primaryButton: some View {
switch currentStep {
case .welcome:
Button(ExtL10n.text("keyboard.onboarding.getStarted")) {
state.onboardingPage = 1
}
.buttonStyle(OverlayPrimaryButtonStyle(palette: palette))
case .microphone:
Button(ExtL10n.text("keyboard.onboarding.mic.grant")) {
state.requestMicPermission()
// Optimistically advance if permission is denied the
// status text on the next viewWillAppear will reflect it.
state.onboardingPage = 2
}
.buttonStyle(OverlayPrimaryButtonStyle(palette: palette))
case .speech:
Button(ExtL10n.text("keyboard.onboarding.speech.grant")) {
state.requestSpeechPermission()
state.onboardingPage = 3
}
.buttonStyle(OverlayPrimaryButtonStyle(palette: palette))
case .keyboard:
// Step 3 is auto-advanced by viewWillAppear once the user
// has enabled the keyboard in Settings.app. We don't show
// a "Continue" button here that would re-trigger the
// confusion we're solving.
Button(ExtL10n.text("keyboard.onboarding.keyboard.openSettings")) {
state.openSystemSettings()
}
.buttonStyle(OverlayPrimaryButtonStyle(palette: palette))
case .api:
HStack(spacing: Spacing.xs) {
Button(ExtL10n.text("keyboard.onboarding.api.skip")) {
state.completeOnboarding()
}
.font(TypeStyle.caption1)
.foregroundStyle(palette.textSecondary)
.padding(.horizontal, Spacing.sm)
.frame(minHeight: 36)
Button(ExtL10n.text("keyboard.onboarding.api.openHostApp")) {
state.openSettings()
}
.buttonStyle(OverlayPrimaryButtonStyle(palette: palette))
}
}
}
}
private struct OverlayPrimaryButtonStyle: ButtonStyle {
let palette: ThemePalette
func makeBody(configuration: Configuration) -> some View {
configuration.label
.font(TypeStyle.caption1.weight(.semibold))
.foregroundStyle(.white)
.padding(.horizontal, Spacing.md)
.padding(.vertical, Spacing.xs + 2)
.background(palette.accent.opacity(configuration.isPressed ? 0.7 : 1.0),
in: Capsule())
.frame(minHeight: 36)
.contentShape(Capsule())
}
}
+38 -15
View File
@@ -75,25 +75,40 @@ public struct KeyboardRootView: View {
}
public var body: some View {
VStack(spacing: 0) {
headerBand
ZStack {
VStack(spacing: 0) {
headerBand
Color.clear
.frame(height: KeyboardLayoutMetrics.actionClusterTopGap)
Color.clear
.frame(height: KeyboardLayoutMetrics.actionClusterTopGap)
micActionRow
.frame(height: KeyboardLayoutMetrics.actionClusterHeight)
micActionRow
.frame(height: KeyboardLayoutMetrics.actionClusterHeight)
Color.clear
.frame(height: KeyboardLayoutMetrics.actionClusterBottomGap)
Color.clear
.frame(height: KeyboardLayoutMetrics.actionClusterBottomGap)
}
.padding(.top, KeyboardLayoutMetrics.outerPaddingTop)
.padding(.bottom, KeyboardLayoutMetrics.outerPaddingBottom)
// chrome
.background(Color.clear)
.frame(height: Self.totalHeight)
// Feed the resolved palette to all nested chips/buttons.
.environment(\.themePalette, palette)
// v0.3.0: in-keyboard first-launch onboarding. Mounted as
// an overlay so the normal keyboard chrome stays
// responsive underneath (mic button still works, chip
// taps register). Only rendered until
// `state.hasCompletedOnboarding` flips to true; from
// then on the overlay is unmounted and never re-rendered.
if !state.hasCompletedOnboarding {
KeyboardOnboardingOverlay(state: state)
.environment(\.themePalette, palette)
.transition(.opacity)
}
}
.padding(.top, KeyboardLayoutMetrics.outerPaddingTop)
.padding(.bottom, KeyboardLayoutMetrics.outerPaddingBottom)
// chrome
.background(Color.clear)
.frame(height: Self.totalHeight)
// Feed the resolved palette to all nested chips/buttons.
.environment(\.themePalette, palette)
.animation(.easeInOut(duration: 0.18), value: state.hasCompletedOnboarding)
}
/// Top chip row + transcript / hint line.
@@ -131,6 +146,14 @@ public struct KeyboardRootView: View {
LocaleChip(localeId: state.localeId) { newId in
state.setLocale(newId)
}
// v0.3.0: detected app context the per-app polish mode.
// The chip mirrors `AppGroupStore.detectedAppContext` and
// writes overrides back so the next LLM call uses the new
// tone. Hidden during onboarding (the overlay reads better
// without chip clutter).
if state.hasCompletedOnboarding {
AppContextChip(state: state)
}
// v0.3: always show the translation chip when the active
// engine can run the cloud LLM step off-by-default keeps
// the menu reachable so the user can pick a target language