rocky/OSGKeyboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(translation): add post-ASR translation mode for cloud engine

Browse Source 
Adds an opt-in translation pipeline that reuses the existing
PolishingService + LLMClient + AppGroupStore chain. Translation is
implemented as a new PolishMode (.translate(targetLocaleId:)); all
existing call sites are unchanged.

Settings:
- New TranslationPickerRow in the language tab (Toggle + 10-locale
  picker: en/zh-Hans/zh-Hant/ja/ko/fr/de/es/ru/pt), persisted to the
  App Group so the keyboard extension can read it during live dictation.
- 5 new strings per language (en + zh-Hans).

Keyboard:
- New TranslationChip on the top bar to the right of LocaleChip;
  same Menu pattern, lets users toggle or quickly switch target
  language without leaving the keyboard.
- PolishingService dispatches .translate with a parameterised prompt
  (en/zh variants selected by provider id); PolishingService.error
  gains a translationNotAvailable case so local-engine users get a
  clear inline warning when the toggle is on but cloud is off.
- 6 new strings per language (en + zh-Hans) for the chip + banner.

Local engine policy:
- Translation is cloud-only by design (local engine stays ASR-only
  to honour the no-roundtrip promise). Chip shows a 'cloud required'
  state and raw transcript still inserts on failure — no data loss.

Build:
- OSGKeyboardShared adds TranslationLanguage enum (10 locales) and
  TranslationPrompt factory.
- 4 new files, 9 modified. xcodebuild scheme=OSGKeyboard
  config=Debug destination=iPhone 17 Simulator: BUILD SUCCEEDED
  (0 warning, 0 error).

Also pins DEVELOPMENT_TEAM in project.yml for TestFlight uploads
(3 targets; Team X329MZU23S).
This commit is contained in:
Rocky
2026-06-25 12:45:44 +08:00
parent dc9697bf3d
commit deddb49d56
17 changed files with 620 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
OSGKeyboardShared/Services/AppGroupStore.swift
+32
View File
@@ -39,6 +39,9 @@ public struct AppGroupStore: @unchecked Sendable {
static let uiLanguage = "config.uiLanguage"
// v0.2.0: opt-in cloud polish step after local-mode ASR.
static let localModeCloudPolishEnabled = "config.localModeCloudPolishEnabled"
// v0.2.1: translation toggle + target locale id (e.g. "en").
static let translationEnabled = "config.translationEnabled"
static let translationTargetLocaleId = "config.translationTargetLocaleId"
}
// MARK: - Reads
@@ -104,6 +107,22 @@ public struct AppGroupStore: @unchecked Sendable {
AppUILanguage.fromStored(defaults.string(forKey: Key.uiLanguage))
}
/// v0.2.1: whether the keyboard should translate the post-ASR transcript
/// before inserting it. Honored only when `engineMode == "cloud"` — see
/// `ProviderConfig.isTranslationEffective` for the effective predicate.
public var translationEnabled: Bool {
guard defaults.object(forKey: Key.translationEnabled) != nil else {
return false
}
return defaults.bool(forKey: Key.translationEnabled)
}
/// v0.2.1: target locale id the translate-and-polish prompt should
/// produce (e.g. `"en"`). Defaults to `"en"` when nothing is stored.
public var translationTargetLocaleId: String {
defaults.string(forKey: Key.translationTargetLocaleId) ?? "en"
}
// MARK: - Writes
public func setModeId(_ id: String) {
@@ -126,6 +145,19 @@ public struct AppGroupStore: @unchecked Sendable {
defaults.set(language.rawValue, forKey: Key.uiLanguage)
}
/// v0.2.1: persist translation toggle. The keyboard extension reads
/// this on every `load()` and `refreshRuntimeFlags()` so the chip
/// reflects the latest value without a host-app round-trip.
public func setTranslationEnabled(_ enabled: Bool) {
defaults.set(enabled, forKey: Key.translationEnabled)
}
/// v0.2.1: persist target locale id (e.g. `"en"`, `"ja"`). Same
/// read cadence as `setTranslationEnabled`.
public func setTranslationTargetLocaleId(_ id: String) {
defaults.set(id, forKey: Key.translationTargetLocaleId)
}
// MARK: - Client
public func makeClient() -> LLMClient {
OSGKeyboardShared/Services/KeyboardState.swift
+20
View File
@@ -87,6 +87,20 @@ public final class KeyboardState: ObservableObject {
/// CoreML local engine. Always `false` now — there are no weights
/// for the host app to preload.
@Published public var localModelsLoaded: Bool = false
/// v0.2.1: translation toggle mirrored from `ProviderConfig`. The
/// pipeline asks `isTranslationEffective` before honouring it —
/// the local engine ignores translation regardless of this flag.
@Published public var translationEnabled: Bool = false
/// v0.2.1: target locale id the translate-and-polish prompt should
/// produce (e.g. `"en"`, `"ja"`). Mirrored from `ProviderConfig`.
@Published public var translationTargetLocaleId: String = TranslationLanguageCatalog.defaultLocaleId
/// v0.2.1: effective predicate — translation is honoured only on
/// the cloud engine. The keyboard's chip / picker read this so the
/// UI can show a "需要云端" hint when the toggle is on while the
/// local engine is active.
public var isTranslationEffective: Bool {
translationEnabled && engineMode == "cloud"
}
/// Convenience shorthand used by the pipeline and views.
public var isLocalEngine: Bool { engineMode == "local" }
@@ -101,6 +115,12 @@ public final class KeyboardState: ObservableObject {
public var setLocale: (String) -> Void = { _ in }
public var setEngineMode: (String) -> Void = { _ in }
public var setLocalASRBackend: (LocalASRBackend) -> Void = { _ in }
/// v0.2.1: persist translation toggle. Wired in
/// `KeyboardViewController.installStateActions`.
public var setTranslationEnabled: (Bool) -> Void = { _ in }
/// v0.2.1: persist translation target locale id. Same wiring as
/// `setTranslationEnabled`.
public var setTranslationTargetLocaleId: (String) -> Void = { _ in }
public var insertNewline: () -> Void = {}
public var insertSpace: () -> Void = {}
public var deleteBackward: () -> Void = {}
OSGKeyboardShared/Services/PolishingService.swift
+43 -5
View File
@@ -29,6 +29,19 @@ public actor PolishingService {
/// telling them to fill it in; we deliver the raw transcript
/// so no data is lost.
case missingAPIKey
/// v0.2.1: the user requested translation but the active engine
/// can't honour it (e.g. `engineMode == "local"`). The keyboard
/// surfaces a short hint and falls back to the plain polish path.
case translationNotAvailable
}
/// v0.2.1: what the LLM should do with the raw transcript. The
/// polish path stays the default so every existing call site keeps
/// its current behaviour — translation is opt-in via the `translate`
/// case and gets a target-locale parameter baked into the prompt.
public enum PolishMode: Equatable, Sendable {
case polish
case translate(targetLocaleId: String)
}
private let store: AppGroupStore
@@ -52,10 +65,20 @@ public actor PolishingService {
self.timeout = timeout ?? (LLMClientFactory.defaultRequestTimeout + 1)
}
public func polish(_ raw: String) async throws -> String {
public func polish(_ raw: String, mode: PolishMode = .polish) async throws -> String {
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { throw PolishError.noTranscript }
// Translation requires the cloud engine (and therefore an API
// key + base URL). When the user toggles translation on while
// the local engine is active we refuse the mode so the keyboard
// can fall back to a plain polish (or raw ASR) and surface a
// short hint. This keeps the local engine's "ASR only" promise
// intact.
if case .translate = mode, store.engineMode != "cloud" {
throw PolishError.translationNotAvailable
}
// Local engine: ASR-only unless the user opted into cloud
// polish via `localModeCloudPolishEnabled`. The cloud polish
// path still requires an API key; if the Keychain is empty we
@@ -66,15 +89,15 @@ public actor PolishingService {
guard !store.apiKey.isEmpty else {
throw PolishError.missingAPIKey
}
return try await polishRemote(trimmed)
return try await polishRemote(trimmed, mode: mode)
}
return try await polishRemote(trimmed)
return try await polishRemote(trimmed, mode: mode)
}
private func polishRemote(_ trimmed: String) async throws -> String {
private func polishRemote(_ trimmed: String, mode: PolishMode) async throws -> String {
let client = injectedClient ?? store.makeClient()
let prompt = store.systemPrompt
let prompt = resolvedSystemPrompt(for: mode)
let budget = effectiveTimeout(for: trimmed)
return try await withThrowingTaskGroup(of: String.self) { group in
@@ -91,6 +114,21 @@ public actor PolishingService {
}
}
/// v0.2.1: pick the right system prompt for the requested mode.
/// Translation mode swaps in the parameterized translate-and-polish
/// prompt (see `TranslationPrompt.make`); polish mode keeps the
/// existing `store.systemPrompt` behaviour so every other call site
/// is byte-identical to before.
private func resolvedSystemPrompt(for mode: PolishMode) -> String {
switch mode {
case .polish:
return store.systemPrompt
case .translate(let targetLocaleId):
let target = TranslationLanguageCatalog.resolve(targetLocaleId)
return TranslationPrompt.make(target: target, providerId: store.providerId)
}
}
/// Scale polish budget with transcript length (3-minute Flow utterances).
private func effectiveTimeout(for text: String) -> TimeInterval {
let scaled = timeout + (Double(text.count) / 200.0) * 2.0
OSGKeyboardShared/Services/TranslationPrompt.swift
+61
View File
@@ -0,0 +1,61 @@
// TranslationPrompt.swift
// OSGKeyboard · Shared
//
// Builds the system prompt the LLM sees when the user has the
// translation toggle on. Re-uses the same per-provider "primary
// language" split the polish prompt uses (`AppGroupStore.defaultSystemPrompt`)
// so Chinese-native LLMs (DeepSeek, Qwen, GLM, Moonshot) get a Chinese
// prompt and English-native LLMs (OpenAI) get an English one — the LLM
// is most reliable when the instructions are written in its strongest
// language.
//
// The "translate AND polish" blend is intentional: ASR transcripts are
// noisy (homophone errors, broken segmentation, dropped particles), so
// the prompt asks the model to clean the noise while translating.
// Keeping those two concerns in one prompt matches how our existing
// polish prompt already mixes "preserve meaning" with "fix punctuation /
// drop filler".
import Foundation
public enum TranslationPrompt {
/// Build the translate-and-polish system prompt.
///
/// - Parameters:
/// - target: target language entry resolved via `TranslationLanguageCatalog`.
/// - providerId: provider preset id (e.g. `"deepseek"`, `"openai"`);
/// drives the language the prompt is written in.
public static func make(target: TranslationLanguage, providerId: String) -> String {
let isChineseNative = ["zhipu", "moonshot", "qwen", "deepseek"].contains(providerId)
return isChineseNative ? chinesePrompt(target: target) : englishPrompt(target: target)
}
// MARK: - Chinese prompt (for DeepSeek / Qwen / GLM / Moonshot)
private static func chinesePrompt(target: TranslationLanguage) -> String {
"""
你是一位语音输入翻译与润色助手。用户用 ASR 转写了一段可能含噪声的口述:
1) 先识别原话的主要语言(若不确定则按用户给定的方向处理);
2) 将内容翻译为「\(target.promptLanguageName)」,保留原意,不增删事实、不臆测;
3) 顺带修复 ASR 噪声(同音错字、漏字、断句错乱),让译文读起来自然;
4) 保留枚举结构(第一…第二…),使用「\(target.promptLanguageName)」的列表惯例;
5) 简洁,不超过原文 1.5 倍;去掉无意义的口头禅(嗯、啊、那个);
6) 只输出译文正文,不要解释、不要加引号、不要前缀"以下是翻译"。
"""
}
// MARK: - English prompt (for OpenAI / OpenAI-compatible non-Chinese)
private static func englishPrompt(target: TranslationLanguage) -> String {
"""
You are a voice-input translation and polishing assistant. The user has spoken informally and the transcript may contain ASR noise:
1) Identify the input language; if unclear, assume the user wants translation INTO \(target.promptLanguageName);
2) Translate the content INTO \(target.promptLanguageName), preserving meaning; do not invent facts or omit content;
3) Fix ASR noise (homophone errors, missing characters, broken segmentation) so the translation reads naturally;
4) Preserve enumeration ("first ... second ...") using \(target.promptLanguageName) list conventions;
5) Keep it concise — no longer than 1.5x the spoken length; drop filler words (um, uh, like);
6) Output ONLY the translation. No quotes, no preamble, no explanation.
"""
}
}