feat(translation): add post-ASR translation mode for cloud engine
Adds an opt-in translation pipeline that reuses the existing PolishingService + LLMClient + AppGroupStore chain. Translation is implemented as a new PolishMode (.translate(targetLocaleId:)); all existing call sites are unchanged. Settings: - New TranslationPickerRow in the language tab (Toggle + 10-locale picker: en/zh-Hans/zh-Hant/ja/ko/fr/de/es/ru/pt), persisted to the App Group so the keyboard extension can read it during live dictation. - 5 new strings per language (en + zh-Hans). Keyboard: - New TranslationChip on the top bar to the right of LocaleChip; same Menu pattern, lets users toggle or quickly switch target language without leaving the keyboard. - PolishingService dispatches .translate with a parameterised prompt (en/zh variants selected by provider id); PolishingService.error gains a translationNotAvailable case so local-engine users get a clear inline warning when the toggle is on but cloud is off. - 6 new strings per language (en + zh-Hans) for the chip + banner. Local engine policy: - Translation is cloud-only by design (local engine stays ASR-only to honour the no-roundtrip promise). Chip shows a 'cloud required' state and raw transcript still inserts on failure — no data loss. Build: - OSGKeyboardShared adds TranslationLanguage enum (10 locales) and TranslationPrompt factory. - 4 new files, 9 modified. xcodebuild scheme=OSGKeyboard config=Debug destination=iPhone 17 Simulator: BUILD SUCCEEDED (0 warning, 0 error). Also pins DEVELOPMENT_TEAM in project.yml for TestFlight uploads (3 targets; Team X329MZU23S).
This commit is contained in:
@@ -39,6 +39,12 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
// in the local engine. Default `false` — keeps the local engine
|
||||
// truly local unless the user explicitly opts in.
|
||||
static let localModeCloudPolishEnabled = "config.localModeCloudPolishEnabled"
|
||||
// v0.2.1: optional translation step after ASR. When enabled, the
|
||||
// post-ASR transcript is routed through the same LLM with a
|
||||
// translate-and-polish prompt targeting `translationTargetLocaleId`.
|
||||
// Mutually exclusive with the local-only promise — see `TranslationPolicy`.
|
||||
static let translationEnabled = "config.translationEnabled"
|
||||
static let translationTargetLocaleId = "config.translationTargetLocaleId"
|
||||
}
|
||||
|
||||
@Published public var providerId: String {
|
||||
@@ -115,6 +121,29 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
@Published public var uiLanguage: AppUILanguage {
|
||||
didSet { defaults.set(uiLanguage.rawValue, forKey: Key.uiLanguage) }
|
||||
}
|
||||
/// v0.2.1: whether to translate the transcript into
|
||||
/// `translationTargetLocaleId` before insertion. Persisted in the App
|
||||
/// Group so the keyboard extension can honour it (and so the chip on
|
||||
/// the keyboard reflects the user's choice without a host-app round-
|
||||
/// trip). Default `false` — translation is opt-in.
|
||||
@Published public var translationEnabled: Bool {
|
||||
didSet { defaults.set(translationEnabled, forKey: Key.translationEnabled) }
|
||||
}
|
||||
/// v0.2.1: BCP-47-ish target language id (e.g. `en`, `ja`, `ko`) the
|
||||
/// translate-and-polish prompt should produce. Default `"en"`.
|
||||
/// Persisted in the App Group for the same reason as `translationEnabled`.
|
||||
@Published public var translationTargetLocaleId: String {
|
||||
didSet { defaults.set(translationTargetLocaleId, forKey: Key.translationTargetLocaleId) }
|
||||
}
|
||||
|
||||
/// v0.2.1: hard gate that decides whether the translation feature can
|
||||
/// actually run. The keyboard honors `translationEnabled` only when
|
||||
/// `engineMode == "cloud"` — the local engine is contractually ASR-
|
||||
/// only, so translation is silently ignored (and the UI shows a hint)
|
||||
/// even when the toggle is on.
|
||||
public var isTranslationEffective: Bool {
|
||||
translationEnabled && engineMode == "cloud"
|
||||
}
|
||||
|
||||
public var isConfigured: Bool {
|
||||
// Local engine (on-device ASR only) doesn't need an API key,
|
||||
@@ -193,6 +222,15 @@ public final class ProviderConfig: ObservableObject, @unchecked Sendable {
|
||||
self.uiLanguage = AppUILanguage.fromStored(
|
||||
resolvedDefaults.string(forKey: Key.uiLanguage)
|
||||
)
|
||||
// v0.2.1: translation toggle + target locale. Both default in a
|
||||
// backwards-compatible way so existing installs keep their old
|
||||
// behaviour (`false`/English) without prompting.
|
||||
if resolvedDefaults.object(forKey: Key.translationEnabled) == nil {
|
||||
self.translationEnabled = false
|
||||
} else {
|
||||
self.translationEnabled = resolvedDefaults.bool(forKey: Key.translationEnabled)
|
||||
}
|
||||
self.translationTargetLocaleId = resolvedDefaults.string(forKey: Key.translationTargetLocaleId) ?? "en"
|
||||
|
||||
// Cloud no longer exposes off/transcribe; migrate legacy values.
|
||||
if self.engineMode == "cloud", self.modeId != "polish" {
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
// TranslationLanguage.swift
|
||||
// OSGKeyboard · Shared
|
||||
//
|
||||
// Catalog of target languages the translation feature can produce.
|
||||
//
|
||||
// Kept deliberately small (~10 entries) to match the kind of choices
|
||||
// the user makes in the Settings picker / keyboard chip. We don't try
|
||||
// to expose every BCP-47 locale — the prompt just needs a target
|
||||
// language name, and a curated list reads better than a 100-row scroll.
|
||||
//
|
||||
// `id` is what gets persisted to the App Group. `promptLanguageName`
|
||||
// is the human-readable target name injected into the prompt (e.g.
|
||||
// the LLM sees "English", not "en"). `nativeName` is the endonym we
|
||||
// show in the picker UI ("日本語" instead of "Japanese").
|
||||
|
||||
import Foundation
|
||||
|
||||
public struct TranslationLanguage: Identifiable, Hashable, Sendable {
|
||||
public let id: String
|
||||
public let promptLanguageName: String
|
||||
public let nativeName: String
|
||||
|
||||
public init(id: String, promptLanguageName: String, nativeName: String) {
|
||||
self.id = id
|
||||
self.promptLanguageName = promptLanguageName
|
||||
self.nativeName = nativeName
|
||||
}
|
||||
}
|
||||
|
||||
public enum TranslationLanguageCatalog {
|
||||
/// Default target language id used on fresh installs.
|
||||
public static let defaultLocaleId = "en"
|
||||
|
||||
/// Curated set. Order matters — the picker / chip render top-to-
|
||||
/// bottom, and `defaultLocaleId` is the default selection.
|
||||
public static let all: [TranslationLanguage] = [
|
||||
TranslationLanguage(id: "en", promptLanguageName: "English", nativeName: "English"),
|
||||
TranslationLanguage(id: "zh-Hans", promptLanguageName: "Simplified Chinese", nativeName: "简体中文"),
|
||||
TranslationLanguage(id: "zh-Hant", promptLanguageName: "Traditional Chinese", nativeName: "繁體中文"),
|
||||
TranslationLanguage(id: "ja", promptLanguageName: "Japanese", nativeName: "日本語"),
|
||||
TranslationLanguage(id: "ko", promptLanguageName: "Korean", nativeName: "한국어"),
|
||||
TranslationLanguage(id: "fr", promptLanguageName: "French", nativeName: "Français"),
|
||||
TranslationLanguage(id: "de", promptLanguageName: "German", nativeName: "Deutsch"),
|
||||
TranslationLanguage(id: "es", promptLanguageName: "Spanish", nativeName: "Español"),
|
||||
TranslationLanguage(id: "ru", promptLanguageName: "Russian", nativeName: "Русский"),
|
||||
TranslationLanguage(id: "pt", promptLanguageName: "Portuguese", nativeName: "Português"),
|
||||
]
|
||||
|
||||
/// Resolve a stored locale id to its catalog entry. Falls back to
|
||||
/// `defaultLocaleId` when the id is missing or unknown — matches the
|
||||
/// pattern used elsewhere (e.g. `ASRLocaleLabels`) so the keyboard
|
||||
/// never crashes on a stale persisted value.
|
||||
public static func resolve(_ id: String) -> TranslationLanguage {
|
||||
if let match = all.first(where: { $0.id == id }) {
|
||||
return match
|
||||
}
|
||||
return all.first { $0.id == defaultLocaleId } ?? all[0]
|
||||
}
|
||||
}
|
||||
@@ -39,6 +39,9 @@ public struct AppGroupStore: @unchecked Sendable {
|
||||
static let uiLanguage = "config.uiLanguage"
|
||||
// v0.2.0: opt-in cloud polish step after local-mode ASR.
|
||||
static let localModeCloudPolishEnabled = "config.localModeCloudPolishEnabled"
|
||||
// v0.2.1: translation toggle + target locale id (e.g. "en").
|
||||
static let translationEnabled = "config.translationEnabled"
|
||||
static let translationTargetLocaleId = "config.translationTargetLocaleId"
|
||||
}
|
||||
|
||||
// MARK: - Reads
|
||||
@@ -104,6 +107,22 @@ public struct AppGroupStore: @unchecked Sendable {
|
||||
AppUILanguage.fromStored(defaults.string(forKey: Key.uiLanguage))
|
||||
}
|
||||
|
||||
/// v0.2.1: whether the keyboard should translate the post-ASR transcript
|
||||
/// before inserting it. Honored only when `engineMode == "cloud"` — see
|
||||
/// `ProviderConfig.isTranslationEffective` for the effective predicate.
|
||||
public var translationEnabled: Bool {
|
||||
guard defaults.object(forKey: Key.translationEnabled) != nil else {
|
||||
return false
|
||||
}
|
||||
return defaults.bool(forKey: Key.translationEnabled)
|
||||
}
|
||||
|
||||
/// v0.2.1: target locale id the translate-and-polish prompt should
|
||||
/// produce (e.g. `"en"`). Defaults to `"en"` when nothing is stored.
|
||||
public var translationTargetLocaleId: String {
|
||||
defaults.string(forKey: Key.translationTargetLocaleId) ?? "en"
|
||||
}
|
||||
|
||||
// MARK: - Writes
|
||||
|
||||
public func setModeId(_ id: String) {
|
||||
@@ -126,6 +145,19 @@ public struct AppGroupStore: @unchecked Sendable {
|
||||
defaults.set(language.rawValue, forKey: Key.uiLanguage)
|
||||
}
|
||||
|
||||
/// v0.2.1: persist translation toggle. The keyboard extension reads
|
||||
/// this on every `load()` and `refreshRuntimeFlags()` so the chip
|
||||
/// reflects the latest value without a host-app round-trip.
|
||||
public func setTranslationEnabled(_ enabled: Bool) {
|
||||
defaults.set(enabled, forKey: Key.translationEnabled)
|
||||
}
|
||||
|
||||
/// v0.2.1: persist target locale id (e.g. `"en"`, `"ja"`). Same
|
||||
/// read cadence as `setTranslationEnabled`.
|
||||
public func setTranslationTargetLocaleId(_ id: String) {
|
||||
defaults.set(id, forKey: Key.translationTargetLocaleId)
|
||||
}
|
||||
|
||||
// MARK: - Client
|
||||
|
||||
public func makeClient() -> LLMClient {
|
||||
|
||||
@@ -87,6 +87,20 @@ public final class KeyboardState: ObservableObject {
|
||||
/// CoreML local engine. Always `false` now — there are no weights
|
||||
/// for the host app to preload.
|
||||
@Published public var localModelsLoaded: Bool = false
|
||||
/// v0.2.1: translation toggle mirrored from `ProviderConfig`. The
|
||||
/// pipeline asks `isTranslationEffective` before honouring it —
|
||||
/// the local engine ignores translation regardless of this flag.
|
||||
@Published public var translationEnabled: Bool = false
|
||||
/// v0.2.1: target locale id the translate-and-polish prompt should
|
||||
/// produce (e.g. `"en"`, `"ja"`). Mirrored from `ProviderConfig`.
|
||||
@Published public var translationTargetLocaleId: String = TranslationLanguageCatalog.defaultLocaleId
|
||||
/// v0.2.1: effective predicate — translation is honoured only on
|
||||
/// the cloud engine. The keyboard's chip / picker read this so the
|
||||
/// UI can show a "需要云端" hint when the toggle is on while the
|
||||
/// local engine is active.
|
||||
public var isTranslationEffective: Bool {
|
||||
translationEnabled && engineMode == "cloud"
|
||||
}
|
||||
|
||||
/// Convenience shorthand used by the pipeline and views.
|
||||
public var isLocalEngine: Bool { engineMode == "local" }
|
||||
@@ -101,6 +115,12 @@ public final class KeyboardState: ObservableObject {
|
||||
public var setLocale: (String) -> Void = { _ in }
|
||||
public var setEngineMode: (String) -> Void = { _ in }
|
||||
public var setLocalASRBackend: (LocalASRBackend) -> Void = { _ in }
|
||||
/// v0.2.1: persist translation toggle. Wired in
|
||||
/// `KeyboardViewController.installStateActions`.
|
||||
public var setTranslationEnabled: (Bool) -> Void = { _ in }
|
||||
/// v0.2.1: persist translation target locale id. Same wiring as
|
||||
/// `setTranslationEnabled`.
|
||||
public var setTranslationTargetLocaleId: (String) -> Void = { _ in }
|
||||
public var insertNewline: () -> Void = {}
|
||||
public var insertSpace: () -> Void = {}
|
||||
public var deleteBackward: () -> Void = {}
|
||||
|
||||
@@ -29,6 +29,19 @@ public actor PolishingService {
|
||||
/// telling them to fill it in; we deliver the raw transcript
|
||||
/// so no data is lost.
|
||||
case missingAPIKey
|
||||
/// v0.2.1: the user requested translation but the active engine
|
||||
/// can't honour it (e.g. `engineMode == "local"`). The keyboard
|
||||
/// surfaces a short hint and falls back to the plain polish path.
|
||||
case translationNotAvailable
|
||||
}
|
||||
|
||||
/// v0.2.1: what the LLM should do with the raw transcript. The
|
||||
/// polish path stays the default so every existing call site keeps
|
||||
/// its current behaviour — translation is opt-in via the `translate`
|
||||
/// case and gets a target-locale parameter baked into the prompt.
|
||||
public enum PolishMode: Equatable, Sendable {
|
||||
case polish
|
||||
case translate(targetLocaleId: String)
|
||||
}
|
||||
|
||||
private let store: AppGroupStore
|
||||
@@ -52,10 +65,20 @@ public actor PolishingService {
|
||||
self.timeout = timeout ?? (LLMClientFactory.defaultRequestTimeout + 1)
|
||||
}
|
||||
|
||||
public func polish(_ raw: String) async throws -> String {
|
||||
public func polish(_ raw: String, mode: PolishMode = .polish) async throws -> String {
|
||||
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty else { throw PolishError.noTranscript }
|
||||
|
||||
// Translation requires the cloud engine (and therefore an API
|
||||
// key + base URL). When the user toggles translation on while
|
||||
// the local engine is active we refuse the mode so the keyboard
|
||||
// can fall back to a plain polish (or raw ASR) and surface a
|
||||
// short hint. This keeps the local engine's "ASR only" promise
|
||||
// intact.
|
||||
if case .translate = mode, store.engineMode != "cloud" {
|
||||
throw PolishError.translationNotAvailable
|
||||
}
|
||||
|
||||
// Local engine: ASR-only unless the user opted into cloud
|
||||
// polish via `localModeCloudPolishEnabled`. The cloud polish
|
||||
// path still requires an API key; if the Keychain is empty we
|
||||
@@ -66,15 +89,15 @@ public actor PolishingService {
|
||||
guard !store.apiKey.isEmpty else {
|
||||
throw PolishError.missingAPIKey
|
||||
}
|
||||
return try await polishRemote(trimmed)
|
||||
return try await polishRemote(trimmed, mode: mode)
|
||||
}
|
||||
|
||||
return try await polishRemote(trimmed)
|
||||
return try await polishRemote(trimmed, mode: mode)
|
||||
}
|
||||
|
||||
private func polishRemote(_ trimmed: String) async throws -> String {
|
||||
private func polishRemote(_ trimmed: String, mode: PolishMode) async throws -> String {
|
||||
let client = injectedClient ?? store.makeClient()
|
||||
let prompt = store.systemPrompt
|
||||
let prompt = resolvedSystemPrompt(for: mode)
|
||||
let budget = effectiveTimeout(for: trimmed)
|
||||
|
||||
return try await withThrowingTaskGroup(of: String.self) { group in
|
||||
@@ -91,6 +114,21 @@ public actor PolishingService {
|
||||
}
|
||||
}
|
||||
|
||||
/// v0.2.1: pick the right system prompt for the requested mode.
|
||||
/// Translation mode swaps in the parameterized translate-and-polish
|
||||
/// prompt (see `TranslationPrompt.make`); polish mode keeps the
|
||||
/// existing `store.systemPrompt` behaviour so every other call site
|
||||
/// is byte-identical to before.
|
||||
private func resolvedSystemPrompt(for mode: PolishMode) -> String {
|
||||
switch mode {
|
||||
case .polish:
|
||||
return store.systemPrompt
|
||||
case .translate(let targetLocaleId):
|
||||
let target = TranslationLanguageCatalog.resolve(targetLocaleId)
|
||||
return TranslationPrompt.make(target: target, providerId: store.providerId)
|
||||
}
|
||||
}
|
||||
|
||||
/// Scale polish budget with transcript length (3-minute Flow utterances).
|
||||
private func effectiveTimeout(for text: String) -> TimeInterval {
|
||||
let scaled = timeout + (Double(text.count) / 200.0) * 2.0
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
// TranslationPrompt.swift
|
||||
// OSGKeyboard · Shared
|
||||
//
|
||||
// Builds the system prompt the LLM sees when the user has the
|
||||
// translation toggle on. Re-uses the same per-provider "primary
|
||||
// language" split the polish prompt uses (`AppGroupStore.defaultSystemPrompt`)
|
||||
// so Chinese-native LLMs (DeepSeek, Qwen, GLM, Moonshot) get a Chinese
|
||||
// prompt and English-native LLMs (OpenAI) get an English one — the LLM
|
||||
// is most reliable when the instructions are written in its strongest
|
||||
// language.
|
||||
//
|
||||
// The "translate AND polish" blend is intentional: ASR transcripts are
|
||||
// noisy (homophone errors, broken segmentation, dropped particles), so
|
||||
// the prompt asks the model to clean the noise while translating.
|
||||
// Keeping those two concerns in one prompt matches how our existing
|
||||
// polish prompt already mixes "preserve meaning" with "fix punctuation /
|
||||
// drop filler".
|
||||
|
||||
import Foundation
|
||||
|
||||
public enum TranslationPrompt {
|
||||
|
||||
/// Build the translate-and-polish system prompt.
|
||||
///
|
||||
/// - Parameters:
|
||||
/// - target: target language entry resolved via `TranslationLanguageCatalog`.
|
||||
/// - providerId: provider preset id (e.g. `"deepseek"`, `"openai"`);
|
||||
/// drives the language the prompt is written in.
|
||||
public static func make(target: TranslationLanguage, providerId: String) -> String {
|
||||
let isChineseNative = ["zhipu", "moonshot", "qwen", "deepseek"].contains(providerId)
|
||||
return isChineseNative ? chinesePrompt(target: target) : englishPrompt(target: target)
|
||||
}
|
||||
|
||||
// MARK: - Chinese prompt (for DeepSeek / Qwen / GLM / Moonshot)
|
||||
|
||||
private static func chinesePrompt(target: TranslationLanguage) -> String {
|
||||
"""
|
||||
你是一位语音输入翻译与润色助手。用户用 ASR 转写了一段可能含噪声的口述:
|
||||
1) 先识别原话的主要语言(若不确定则按用户给定的方向处理);
|
||||
2) 将内容翻译为「\(target.promptLanguageName)」,保留原意,不增删事实、不臆测;
|
||||
3) 顺带修复 ASR 噪声(同音错字、漏字、断句错乱),让译文读起来自然;
|
||||
4) 保留枚举结构(第一…第二…),使用「\(target.promptLanguageName)」的列表惯例;
|
||||
5) 简洁,不超过原文 1.5 倍;去掉无意义的口头禅(嗯、啊、那个);
|
||||
6) 只输出译文正文,不要解释、不要加引号、不要前缀"以下是翻译"。
|
||||
"""
|
||||
}
|
||||
|
||||
// MARK: - English prompt (for OpenAI / OpenAI-compatible non-Chinese)
|
||||
|
||||
private static func englishPrompt(target: TranslationLanguage) -> String {
|
||||
"""
|
||||
You are a voice-input translation and polishing assistant. The user has spoken informally and the transcript may contain ASR noise:
|
||||
1) Identify the input language; if unclear, assume the user wants translation INTO \(target.promptLanguageName);
|
||||
2) Translate the content INTO \(target.promptLanguageName), preserving meaning; do not invent facts or omit content;
|
||||
3) Fix ASR noise (homophone errors, missing characters, broken segmentation) so the translation reads naturally;
|
||||
4) Preserve enumeration ("first ... second ...") using \(target.promptLanguageName) list conventions;
|
||||
5) Keep it concise — no longer than 1.5x the spoken length; drop filler words (um, uh, like);
|
||||
6) Output ONLY the translation. No quotes, no preamble, no explanation.
|
||||
"""
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user