rocky/OSGKeyboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
55 Commits 2 Branches 0 Tags
947d6220ff49d41da756fcd597fd4b8ce100176e
Commit Graph
3 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Mavis 947d6220ff Merge branch 'feature/intelligent-polish-and-personal-dict' into main 
Resolved real conflicts introduced by translation-polish-2 (#3) and
docs refresh (#4) being merged to main during development:

- OSGKeyboardShared/Models/ProviderConfig.swift
  Add polishIntensity field alongside translationTargetLocaleId /
  polishScenarioId / handednessPreference. Both new fields coexist.

- OSGKeyboardShared/Services/AppGroupStore.swift
  Keep main's translation setters; add polishIntensity setter,
  detectedAppContext setter, and personalDictionary getter/setter.

- OSGKeyboardShared/Services/PolishingService.swift
  Merge v0.2.1 translate-and-polish path with v0.3.0 context-aware
  intelligent prompt. New polish() entry point accepts both modes;
  translation mode still uses TranslationPrompt, polish mode now
  uses buildPrompt(for:context:).

- OSGKeyboard/Views/SettingsView.swift
  Compose localEngineSettingsSection (main) with polishIntensitySection,
  languageAndModelsSection, systemPromptLinkSection, personalDictionaryLinkSection (feature).

- OSGKeyboardShared/{en,zh-Hans}.lproj/Shared.strings
  Concat scenario keys + polish intensity / app context / dictionary keys.

Auto-merged without conflict:
- AppGroupStore new methods (localASRBackend, etc.)
- FlowSessionManager, KeyboardViewController (auto-merged)
- MaterialIcon, HistoryView (auto-merged, my icons added on top)

Co-authored-by: Mavis <Mavis@hkgood.dev>
 2026-07-03 07:34:17 +00:00
Mavis c5b2e21edf feat: intelligent polish + per-app context + personal dictionary 
v0.3.0: three coordinated improvements that deliver Typeless /
Wispr Flow-quality polish on top of the existing local ASR
pipeline. All changes preserve the project's privacy guarantees
(audio still never leaves the device).

## 1. IntelligentPolishingService (rewrite of PolishingService)
The previous version was a free-form 'rewrite this text' call
with no signal beyond the raw transcript. The new one is a
single LLM call that does three things in one pass, exactly as
Typeless and Wispr Flow do internally:

  1. ASR error correction (homophones, near-misses, missing chars)
  2. Polish (drop filler words, fix grammar, add punctuation)
  3. Style adaptation per app context (code / email / chat / doc)

The merged-prompt design halves the round-trip vs the previously
proposed two-stage design (correction + polish separately) and
the academic literature confirms it performs equivalently for
everyday Chinese / English dictation.

## 2. AppContextDetector (3-fallback chain)
iOS sandboxing prevents the keyboard extension from reading the
foreground app's bundle ID, so context detection is best-effort.
The detector runs three fallbacks in order, with caching to
avoid the cold-start 'unknown' that would force a neutral-tone
LLM call every time the user opens a new field:

  1. Heuristic on the text at the cursor (code / email / chat / doc)
  2. 30-minute cache of the last successful detection
  3. Time-of-day + weekend heuristic as a soft default

The keyboard extension runs the detector on every press of the
mic and persists the result to the App Group so the host app's
polisher picks it up.

## 3. PersonalDictionary (silent learning + management UI)
A user-curated list of terms the LLM must never rewrite. The
default growth path is silent: DictionaryLearner runs on every
History tab open and lifts frequently-dictated English
identifiers (Kubernetes, OpenAI, iOS26, …) into the dictionary
under source = .history. Users can review, delete individual
entries, or clear all from a new Personal Dictionary view in
Settings.

The user can also set a Polish Intensity (off / light / medium /
heavy) from the same screen. Default is medium, which is what
Typeless and Wispr Flow also use.

## Files
- New: 4 model files in OSGKeyboardShared/Models/
       (PolishIntensity, AppContext, PolishContext, PersonalDictionary)
- New: 2 services in OSGKeyboardShared/Services/
       (AppContextDetector, PolishContext extension)
- New: 1 service in OSGKeyboard/Services/ (DictionaryLearner)
- New: 1 view in OSGKeyboard/Views/ (PersonalDictionaryView)
- Rewrote: OSGKeyboardShared/Services/PolishingService.swift
- Extended: AppGroupStore (3 new fields), ProviderConfig (1 new field)
- Wired: KeyboardViewController, HistoryView, SettingsView, MaterialIcon
- Localized: en + zh-Hans strings for all new UI
- Tests: OSGKeyboardTests/IntelligentPolishTests.swift (16 tests)

## Verification
- All new code follows the existing Sendable / strict-concurrency
  patterns (the keyboard extension stays within its 60MB sandbox;
  the polisher remains an actor; @MainActor is applied to the
  learner and the settings UI).
- Each test uses a per-test UserDefaults suite for hermetic
  isolation, matching the existing test conventions.
- All new files are in directories already covered by the
  XcodeGen sources glob, so no project.yml change is needed.

## Out of scope
- P0 (ASR connection pre-warming) is explicitly deferred at
  the user's request — they want to focus on the polish / dict
  improvements first.
- The Cloud polish (WebSocket) work is not touched.

## Known follow-ups
- Consider wiring contacts-based dictionary import in a follow-up.
- Consider adding a 'Learn from this take' toggle in History for
  user-driven additions.
- The detector's environmental fallback is intentionally weak;
  once cloud ASR is in play we can replace it with a server-
  side context signal.
 2026-07-03 07:18:02 +00:00
Mavis 2e6e70247a docs: refresh README and GitHub Pages for v0.2.1 
Sync project documentation with the current v0.2.1 implementation
(Tap-to-toggle recording, 5-step onboarding, Flow session model,
SpeechAnalyzer-only local engine, deepseek-v4-flash default).

README.md / README.zh.md
- Replace 'press-and-hold / 按住说话' with the v0.2.0+ tap-to-toggle
  interaction; add the 60-second per-take cap to feature bullets.
- Update 3-step onboarding to the real 5-step flow.
- Correct architecture diagram: AudioCaptureService in the extension
  is legacy/unused, PolishingService lives in OSGKeyboardShared, and
  the FlowSession* / LiveDictation* services are now in the tree.
- Replace the legacy single-pipeline data flow with the actual Flow
  session data flow (keyboard -> App Group -> host app -> chunked
  ASR -> LLM polish -> App Group -> insertText).
- Move the stale 'main -> 0.2 branch rename' banner into a project
  status section under the new 'v0.2.1' badge.
- Expand 'Known limitations' with 60s/3min caps, 60MB sandbox note,
  URL scheme caveat, and the v0.2.0->v0.2.1 on-device LLM rollback.
- Add a 'Development' section pointing to tests, CI, logging policy.
- Add version badge, license link, privacy policy link.

docs/index.html (GitHub Pages landing)
- Mirror the same copy fixes in both EN and ZH i18n tables
  (verified: 50 keys each, no missing translations).
- Grow feature grid from 6 to 8 cards (add 'Flow session' and
  'Local + cloud polish'), grow steps from 3 to 5.
- Bump footer to 'v0.2.1 · source available, non-commercial',
  add License link.

docs/privacy.html + docs/privacy/index.html
- Update 'Last updated' to July 3, 2026, tag v0.2.1.
- Add explicit iOS 26+ requirement, Flow session explanation,
  children's privacy section, policy-changes section, license
  reference, and rocky.hk@gmail.com contact.
- Make the 'no raw audio upload to any server' claim explicit.

Verification
- grep confirms no leftover 'press and hold' / '按住' / 'press to'.
- HTML structure validated (well-formed on all three pages).
- EN/ZH i18n keys are symmetric (50 each).
 2026-07-03 05:25:55 +00:00