feat(typing): wire PersonalDictionary into Chinese Pinyin via Rime sidecar
Redeploy an osg_personal import table on dictionary add/delete/sync so Chinese, English typing, and ASR share one curated lexicon (next keyboard open).
This commit is contained in:
@@ -8,9 +8,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
- **Personal dictionary → Chinese Pinyin**: adding or deleting terms redeploys an `osg_personal` Rime sidecar (local pinyin from the bundled dict; same-code pin-to-top; Latin names also surface in Chinese mode); English typing and ASR keep using the same dictionary automatically. Takes effect the next time the keyboard opens. / **个性词库 → 中文拼音**:增删词条会重部署 `osg_personal` Rime 旁路词表(复用打包词典本地注音;同码置顶;拉丁专名在中文键盘也可出候选);英文打字与 ASR 继续自动共用同一词库。下次打开键盘生效。
|
||||||
- **Typing key sound & haptics**: letter / modifier / space / return / delete keys play system click sounds on press-down; Settings → General → Haptics offers Off / Light (default) / Strong role-based feedback. / **打字按键音效与震动**:字母 / 修饰 / 空格 / 回车 / 删除键按下即播系统咔嗒音;设置 → 通用 → 震动提供关 / 轻(默认) / 强 的角色分层触感。
|
- **Typing key sound & haptics**: letter / modifier / space / return / delete keys play system click sounds on press-down; Settings → General → Haptics offers Off / Light (default) / Strong role-based feedback. / **打字按键音效与震动**:字母 / 修饰 / 空格 / 回车 / 删除键按下即播系统咔嗒音;设置 → 通用 → 震动提供关 / 轻(默认) / 强 的角色分层触感。
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
- **Rime personal-dict import**: SharedSupport now imports `osg_personal` into `osg_pinyin` (resource version `2.3.0`). / **Rime 个性词导入**:SharedSupport 将 `osg_personal` 导入 `osg_pinyin`(资源版本 `2.3.0`)。
|
||||||
- **Two-level polish intensity**: Light is the default and restores full fidelity, question, and insertion-context safeguards for fun styles; Heavy keeps the formatting-only creative path for Dating, Flex, Corp, Diba, and XHS. / **两档润色强度**:默认轻度,为趣味风格启用完整保真、问句与落点上下文守卫;重度保持直男癌、装逼、大厂、帝吧和小红书仅格式化后执行人格的创意链路。
|
- **Two-level polish intensity**: Light is the default and restores full fidelity, question, and insertion-context safeguards for fun styles; Heavy keeps the formatting-only creative path for Dating, Flex, Corp, Diba, and XHS. / **两档润色强度**:默认轻度,为趣味风格启用完整保真、问句与落点上下文守卫;重度保持直男癌、装逼、大厂、帝吧和小红书仅格式化后执行人格的创意链路。
|
||||||
- **Built-in polish style JSON**: ship each built-in personality as `Resources/PolishStyles/*.json` plus a manifest; the loader strips the retired fun-foundation placeholder while the composer owns the single shared formatting layer. / **内置润色风格 JSON**:每个内置人格改为 `Resources/PolishStyles/*.json` + manifest;加载器移除已退役的趣味共享占位符,唯一共享格式化层由 Composer 负责。
|
- **Built-in polish style JSON**: ship each built-in personality as `Resources/PolishStyles/*.json` plus a manifest; the loader strips the retired fun-foundation placeholder while the composer owns the single shared formatting layer. / **内置润色风格 JSON**:每个内置人格改为 `Resources/PolishStyles/*.json` + manifest;加载器移除已退役的趣味共享占位符,唯一共享格式化层由 Composer 负责。
|
||||||
- **Dating V6 alignment**: restore heartbeat goals, action definitions, chat paragraphing, and the “吃饭了吗” example in `builtin.dating.json`; practical safeguards no longer override those Dating instructions. / **直男癌对齐 V6**:在 `builtin.dating.json` 恢复终极目标、心动动作定义、聊天分段与「吃饭了吗」示例;实用润色守卫不再覆盖 Dating 指令。
|
- **Dating V6 alignment**: restore heartbeat goals, action definitions, chat paragraphing, and the “吃饭了吗” example in `builtin.dating.json`; practical safeguards no longer override those Dating instructions. / **直男癌对齐 V6**:在 `builtin.dating.json` 恢复终极目标、心动动作定义、聊天分段与「吃饭了吗」示例;实用润色守卫不再覆盖 Dating 指令。
|
||||||
|
|||||||
@@ -175,4 +175,93 @@ final class LibrimeIntegrationTests: XCTestCase {
|
|||||||
)
|
)
|
||||||
fuzzyBridge.finalizeRuntime()
|
fuzzyBridge.finalizeRuntime()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func testPersonalDictionarySidecarPinsSameCodeCandidates() throws {
|
||||||
|
let fileManager = FileManager.default
|
||||||
|
let root = fileManager.temporaryDirectory
|
||||||
|
.appendingPathComponent("OSGRimePersonal-\(UUID().uuidString)", isDirectory: true)
|
||||||
|
let shared = root.appendingPathComponent("SharedSupport", isDirectory: true)
|
||||||
|
let user = root.appendingPathComponent("UserData", isDirectory: true)
|
||||||
|
defer { try? fileManager.removeItem(at: root) }
|
||||||
|
try fileManager.createDirectory(at: shared, withIntermediateDirectories: true)
|
||||||
|
try fileManager.createDirectory(at: user, withIntermediateDirectories: true)
|
||||||
|
|
||||||
|
let bundle = Bundle(for: LibrimeIntegrationTests.self)
|
||||||
|
let dictionaryURL = try XCTUnwrap(
|
||||||
|
bundle.url(forResource: "osg_pinyin.dict", withExtension: "yaml")
|
||||||
|
)
|
||||||
|
let baseline = try String(contentsOf: dictionaryURL, encoding: .utf8)
|
||||||
|
let patched = RimePersonalDictionaryExporter.injectingImportTables(into: baseline)
|
||||||
|
try patched.write(
|
||||||
|
to: shared.appendingPathComponent("osg_pinyin.dict.yaml"),
|
||||||
|
atomically: true,
|
||||||
|
encoding: .utf8
|
||||||
|
)
|
||||||
|
|
||||||
|
// Unique personal phrase on a common code — must outrank baseline “你好”.
|
||||||
|
let personalYAML = RimePersonalDictionaryExporter.yaml(
|
||||||
|
entries: [
|
||||||
|
.init(text: "尼好专名", code: "ni hao"),
|
||||||
|
.init(text: "ChatGPT", code: "chatgpt")
|
||||||
|
]
|
||||||
|
)
|
||||||
|
try personalYAML.write(
|
||||||
|
to: shared.appendingPathComponent("osg_personal.dict.yaml"),
|
||||||
|
atomically: true,
|
||||||
|
encoding: .utf8
|
||||||
|
)
|
||||||
|
|
||||||
|
try RimeSchemaGenerator.defaultConfiguration().write(
|
||||||
|
to: shared.appendingPathComponent("default.yaml"),
|
||||||
|
atomically: true,
|
||||||
|
encoding: .utf8
|
||||||
|
)
|
||||||
|
for schema in TypingInputSchema.allCases {
|
||||||
|
try RimeSchemaGenerator.schema(for: schema, fuzzyPairs: []).write(
|
||||||
|
to: shared.appendingPathComponent("\(schema.rawValue).schema.yaml"),
|
||||||
|
atomically: true,
|
||||||
|
encoding: .utf8
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
let deployer = OSGRimeBridge(
|
||||||
|
sharedDataDirectory: shared.path,
|
||||||
|
userDataDirectory: user.path,
|
||||||
|
distributionVersion: "tests-personal"
|
||||||
|
)
|
||||||
|
try deployer.deploy(withFullCheck: true)
|
||||||
|
deployer.finalizeRuntime()
|
||||||
|
|
||||||
|
let bridge = OSGRimeBridge(
|
||||||
|
sharedDataDirectory: shared.path,
|
||||||
|
userDataDirectory: user.path,
|
||||||
|
distributionVersion: "tests-personal"
|
||||||
|
)
|
||||||
|
try bridge.start()
|
||||||
|
XCTAssertTrue(bridge.selectSchema(TypingInputSchema.fullPinyin.rawValue))
|
||||||
|
|
||||||
|
for scalar in "nihao".utf8 {
|
||||||
|
XCTAssertTrue(bridge.processKeyCode(Int32(scalar), modifiers: 0))
|
||||||
|
}
|
||||||
|
let chinese = bridge.snapshot(withCandidateLimit: 40)
|
||||||
|
XCTAssertTrue(
|
||||||
|
chinese.candidates.contains(where: { $0.text == "尼好专名" }),
|
||||||
|
"personal Chinese missing: \(chinese.candidates.map(\.text).prefix(20))"
|
||||||
|
)
|
||||||
|
if let personal = chinese.candidates.firstIndex(where: { $0.text == "尼好专名" }),
|
||||||
|
let baselineHello = chinese.candidates.firstIndex(where: { $0.text == "你好" }) {
|
||||||
|
XCTAssertLessThan(personal, baselineHello, "personal same-code should pin above baseline")
|
||||||
|
}
|
||||||
|
|
||||||
|
bridge.clearComposition()
|
||||||
|
for scalar in "chatgpt".utf8 {
|
||||||
|
XCTAssertTrue(bridge.processKeyCode(Int32(scalar), modifiers: 0))
|
||||||
|
}
|
||||||
|
let english = bridge.snapshot(withCandidateLimit: 40)
|
||||||
|
XCTAssertTrue(
|
||||||
|
english.candidates.contains(where: { $0.text == "ChatGPT" }),
|
||||||
|
"personal English missing: \(english.candidates.map(\.text).prefix(20))"
|
||||||
|
)
|
||||||
|
bridge.finalizeRuntime()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,110 @@
|
|||||||
|
// RimePersonalDictionaryExporterTests.swift
|
||||||
|
// OSGKeyboard · Ext unit tests
|
||||||
|
|
||||||
|
import XCTest
|
||||||
|
@testable import OSGKeyboardShared
|
||||||
|
|
||||||
|
final class RimePersonalDictionaryExporterTests: XCTestCase {
|
||||||
|
private let annotator = RimePinyinAnnotator(
|
||||||
|
phraseCodes: [
|
||||||
|
"你好": "ni hao",
|
||||||
|
"阿坝": "a ba",
|
||||||
|
"官网": "guan wang"
|
||||||
|
],
|
||||||
|
characterCodes: [
|
||||||
|
"你": "ni",
|
||||||
|
"好": "hao",
|
||||||
|
"阿": "a",
|
||||||
|
"坝": "ba",
|
||||||
|
"官": "guan",
|
||||||
|
"网": "wang",
|
||||||
|
"欧": "ou",
|
||||||
|
"斯": "si",
|
||||||
|
"吉": "ji",
|
||||||
|
"键": "jian"
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
func testAnnotatesPhraseExactMatch() {
|
||||||
|
XCTAssertEqual(annotator.code(for: "阿坝"), "a ba")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testAnnotatesCharacterFallback() {
|
||||||
|
XCTAssertEqual(annotator.code(for: "欧斯吉键"), "ou si ji jian")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testAnnotatesLatinProductName() {
|
||||||
|
XCTAssertEqual(annotator.code(for: "ChatGPT"), "chatgpt")
|
||||||
|
XCTAssertEqual(annotator.code(for: "GPT-4"), "gpt")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testAnnotatesMixedChineseAndLatin() {
|
||||||
|
XCTAssertEqual(annotator.code(for: "ChatGPT官网"), "chatgpt guan wang")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testExporterPinsChineseEnglishAndLatinAlias() {
|
||||||
|
var dictionary = PersonalDictionary.empty
|
||||||
|
dictionary.entries = [
|
||||||
|
PersonalDictionary.Entry(
|
||||||
|
term: "阿坝",
|
||||||
|
aliases: ["Aba"],
|
||||||
|
category: .properNoun,
|
||||||
|
source: .manual
|
||||||
|
),
|
||||||
|
PersonalDictionary.Entry(
|
||||||
|
term: "ChatGPT",
|
||||||
|
aliases: ["聊天GP"],
|
||||||
|
category: .productName,
|
||||||
|
source: .manual
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
let rows = RimePersonalDictionaryExporter.entries(from: dictionary, annotator: annotator)
|
||||||
|
XCTAssertTrue(rows.contains { $0.text == "阿坝" && $0.code == "a ba" })
|
||||||
|
XCTAssertTrue(rows.contains { $0.text == "阿坝" && $0.code == "aba" })
|
||||||
|
XCTAssertTrue(rows.contains { $0.text == "ChatGPT" && $0.code == "chatgpt" })
|
||||||
|
// Chinese ASR alias must not become a Rime code for the English term.
|
||||||
|
XCTAssertFalse(rows.contains { $0.text == "ChatGPT" && $0.code.contains(" ") })
|
||||||
|
XCTAssertTrue(rows.allSatisfy { $0.weight == RimePersonalDictionaryExporter.pinWeight })
|
||||||
|
}
|
||||||
|
|
||||||
|
func testYamlContainsImportReadyHeaderAndRows() {
|
||||||
|
let yaml = RimePersonalDictionaryExporter.yaml(
|
||||||
|
entries: [
|
||||||
|
.init(text: "阿坝", code: "a ba"),
|
||||||
|
.init(text: "ChatGPT", code: "chatgpt")
|
||||||
|
]
|
||||||
|
)
|
||||||
|
XCTAssertTrue(yaml.contains("name: osg_personal"))
|
||||||
|
XCTAssertTrue(yaml.contains("阿坝\ta ba\t\(RimePersonalDictionaryExporter.pinWeight)"))
|
||||||
|
XCTAssertTrue(yaml.contains("ChatGPT\tchatgpt\t\(RimePersonalDictionaryExporter.pinWeight)"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testInjectImportTablesIsIdempotent() {
|
||||||
|
let baseline = """
|
||||||
|
---
|
||||||
|
name: osg_pinyin
|
||||||
|
version: "1.0"
|
||||||
|
sort: by_weight
|
||||||
|
use_preset_vocabulary: false
|
||||||
|
columns:
|
||||||
|
- text
|
||||||
|
...
|
||||||
|
啊\ta\t1
|
||||||
|
"""
|
||||||
|
let once = RimePersonalDictionaryExporter.injectingImportTables(into: baseline)
|
||||||
|
XCTAssertTrue(once.contains("import_tables:"))
|
||||||
|
XCTAssertTrue(once.contains("- osg_personal"))
|
||||||
|
let twice = RimePersonalDictionaryExporter.injectingImportTables(into: once)
|
||||||
|
XCTAssertEqual(once, twice)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testFingerprintChangesWhenEntriesChange() {
|
||||||
|
let a = RimePersonalDictionaryExporter.yaml(entries: [.init(text: "阿坝", code: "a ba")])
|
||||||
|
let b = RimePersonalDictionaryExporter.yaml(entries: [])
|
||||||
|
XCTAssertNotEqual(
|
||||||
|
RimePersonalDictionaryExporter.fingerprint(of: a),
|
||||||
|
RimePersonalDictionaryExporter.fingerprint(of: b)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,9 +1,10 @@
|
|||||||
// PersonalDictionary.swift
|
// PersonalDictionary.swift
|
||||||
// OSGKeyboard · Shared
|
// OSGKeyboard · Shared
|
||||||
//
|
//
|
||||||
// User-curated list of terms the LLM must never rewrite. Persisted
|
// User-curated list of terms shared across polish, ASR bias, English
|
||||||
// in the App Group (JSON-encoded) so both the main app's Settings
|
// typing hotwords, and (via host Rime redeploy) Chinese Pinyin candidates.
|
||||||
// UI and the keyboard extension's LLM call read the same data.
|
// Persisted in the App Group (JSON-encoded) so the main app, keyboard
|
||||||
|
// extension, and host deploy pipeline read the same data.
|
||||||
//
|
//
|
||||||
// Sources (mutually exclusive per entry):
|
// Sources (mutually exclusive per entry):
|
||||||
// - `.manual` user typed it in by hand
|
// - `.manual` user typed it in by hand
|
||||||
|
|||||||
@@ -78,6 +78,7 @@ public final class TypingInputConfiguration: ObservableObject {
|
|||||||
static let rememberLastSurface = "typing.input.rememberLastSurface"
|
static let rememberLastSurface = "typing.input.rememberLastSurface"
|
||||||
static let lastSurface = "typing.input.lastSurface"
|
static let lastSurface = "typing.input.lastSurface"
|
||||||
static let resourceVersion = "typing.rime.resourceVersion"
|
static let resourceVersion = "typing.rime.resourceVersion"
|
||||||
|
static let personalDictionaryFingerprint = "typing.rime.personalDictionaryFingerprint"
|
||||||
}
|
}
|
||||||
|
|
||||||
private let defaults: UserDefaults
|
private let defaults: UserDefaults
|
||||||
@@ -187,6 +188,21 @@ public final class TypingInputConfiguration: ObservableObject {
|
|||||||
(defaults ?? AppGroup.defaultsIfAvailable)?.set(value, forKey: Key.resourceVersion)
|
(defaults ?? AppGroup.defaultsIfAvailable)?.set(value, forKey: Key.resourceVersion)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
nonisolated public static func installedPersonalDictionaryFingerprint(
|
||||||
|
defaults: UserDefaults? = nil
|
||||||
|
) -> String? {
|
||||||
|
(defaults ?? AppGroup.defaultsIfAvailable)?
|
||||||
|
.string(forKey: Key.personalDictionaryFingerprint)
|
||||||
|
}
|
||||||
|
|
||||||
|
nonisolated public static func setInstalledPersonalDictionaryFingerprint(
|
||||||
|
_ value: String,
|
||||||
|
defaults: UserDefaults? = nil
|
||||||
|
) {
|
||||||
|
(defaults ?? AppGroup.defaultsIfAvailable)?
|
||||||
|
.set(value, forKey: Key.personalDictionaryFingerprint)
|
||||||
|
}
|
||||||
|
|
||||||
private func persistIfReady() {
|
private func persistIfReady() {
|
||||||
guard !isHydrating else { return }
|
guard !isHydrating else { return }
|
||||||
defaults.set(schema.rawValue, forKey: Key.schema)
|
defaults.set(schema.rawValue, forKey: Key.schema)
|
||||||
|
|||||||
@@ -225,6 +225,10 @@ public struct AppGroupStore: @unchecked Sendable {
|
|||||||
public func setPersonalDictionary(_ dictionary: PersonalDictionary) {
|
public func setPersonalDictionary(_ dictionary: PersonalDictionary) {
|
||||||
mutateConfiguration { $0.personalDictionary = dictionary }
|
mutateConfiguration { $0.personalDictionary = dictionary }
|
||||||
AppGroupConfigDarwin.postConfigChanged()
|
AppGroupConfigDarwin.postConfigChanged()
|
||||||
|
#if os(iOS)
|
||||||
|
// Host redeploys Rime sidecar; extension picks it up next typing open.
|
||||||
|
PersonalDictionaryRimeSync.scheduleAfterDictionaryChange()
|
||||||
|
#endif
|
||||||
}
|
}
|
||||||
|
|
||||||
public func deletePersonalDictionaryEntry(id: UUID, at date: Date = Date()) {
|
public func deletePersonalDictionaryEntry(id: UUID, at date: Date = Date()) {
|
||||||
@@ -233,6 +237,9 @@ public struct AppGroupStore: @unchecked Sendable {
|
|||||||
config.personalDictionary.deletedEntryIDs[id] = date
|
config.personalDictionary.deletedEntryIDs[id] = date
|
||||||
}
|
}
|
||||||
AppGroupConfigDarwin.postConfigChanged()
|
AppGroupConfigDarwin.postConfigChanged()
|
||||||
|
#if os(iOS)
|
||||||
|
PersonalDictionaryRimeSync.scheduleAfterDictionaryChange()
|
||||||
|
#endif
|
||||||
}
|
}
|
||||||
|
|
||||||
public var personalDictionaryICloudSyncEnabled: Bool {
|
public var personalDictionaryICloudSyncEnabled: Bool {
|
||||||
|
|||||||
@@ -0,0 +1,72 @@
|
|||||||
|
// PersonalDictionaryRimeSync.swift
|
||||||
|
// OSGKeyboard · Shared
|
||||||
|
//
|
||||||
|
// Debounces PersonalDictionary mutations on the iOS host and redeploys
|
||||||
|
// Rime so the osg_personal sidecar matches. The keyboard extension only
|
||||||
|
// picks this up the next time it opens a typing session.
|
||||||
|
|
||||||
|
import Foundation
|
||||||
|
|
||||||
|
@MainActor
|
||||||
|
public enum PersonalDictionaryRimeSync {
|
||||||
|
private static var pending: Task<Void, Never>?
|
||||||
|
private static let debounceNanoseconds: UInt64 = 750_000_000
|
||||||
|
private static let retryNanoseconds: UInt64 = 5_000_000_000
|
||||||
|
|
||||||
|
/// Call after App Group personal-dictionary writes (add / delete / sync).
|
||||||
|
/// Safe from any executor — work is hoppped onto the main actor.
|
||||||
|
public nonisolated static func scheduleAfterDictionaryChange() {
|
||||||
|
Task { @MainActor in
|
||||||
|
scheduleOnMainActor()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func deployNow() async {
|
||||||
|
pending?.cancel()
|
||||||
|
pending = nil
|
||||||
|
await deploy(retryOnMemoryPressure: false)
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func scheduleOnMainActor() {
|
||||||
|
pending?.cancel()
|
||||||
|
pending = Task {
|
||||||
|
try? await Task.sleep(nanoseconds: debounceNanoseconds)
|
||||||
|
guard !Task.isCancelled else { return }
|
||||||
|
await deploy(retryOnMemoryPressure: true)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func deploy(retryOnMemoryPressure: Bool) async {
|
||||||
|
guard HostMemoryBudget.gate("rime.personalDictionary") else {
|
||||||
|
OSGDiag.log("rime.personalDictionary deferred by memory gate", category: "boot")
|
||||||
|
if retryOnMemoryPressure {
|
||||||
|
pending?.cancel()
|
||||||
|
pending = Task {
|
||||||
|
try? await Task.sleep(nanoseconds: retryNanoseconds)
|
||||||
|
guard !Task.isCancelled else { return }
|
||||||
|
await deploy(retryOnMemoryPressure: true)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
FlowSessionBridge.setHostHeavy(true)
|
||||||
|
defer { FlowSessionBridge.setHostHeavy(false) }
|
||||||
|
|
||||||
|
let typingConfig = TypingInputConfiguration.shared.snapshot
|
||||||
|
let dictionary = AppGroupStore().personalDictionary
|
||||||
|
do {
|
||||||
|
try await RimeResourceInstaller.shared.installIfNeeded(
|
||||||
|
configuration: typingConfig,
|
||||||
|
personalDictionary: dictionary,
|
||||||
|
force: false
|
||||||
|
)
|
||||||
|
OSGDiag.log("rime.personalDictionary deploy done", category: "boot")
|
||||||
|
} catch {
|
||||||
|
OSGDiag.log(
|
||||||
|
"rime.personalDictionary deploy failed error=\(error.localizedDescription)",
|
||||||
|
category: "boot"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
// RimePersonalDictionaryExporter.swift
|
||||||
|
// OSGKeyboard · Shared
|
||||||
|
//
|
||||||
|
// Turns PersonalDictionary into an osg_personal Rime table that the
|
||||||
|
// main osg_pinyin dictionary imports. High weight keeps same-code
|
||||||
|
// personal hits above the baseline lexicon.
|
||||||
|
|
||||||
|
import CryptoKit
|
||||||
|
import Foundation
|
||||||
|
|
||||||
|
public enum RimePersonalDictionaryExporter {
|
||||||
|
public static let dictionaryName = "osg_personal"
|
||||||
|
/// Above any baseline `osg_pinyin` weight so same-code hits pin to top.
|
||||||
|
public static let pinWeight = 50_000_000
|
||||||
|
|
||||||
|
public struct Entry: Equatable, Sendable {
|
||||||
|
public let text: String
|
||||||
|
public let code: String
|
||||||
|
public let weight: Int
|
||||||
|
|
||||||
|
public init(text: String, code: String, weight: Int = RimePersonalDictionaryExporter.pinWeight) {
|
||||||
|
self.text = text
|
||||||
|
self.code = code
|
||||||
|
self.weight = weight
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds Rime rows for Chinese (pinyin-coded), English (latin-coded),
|
||||||
|
/// and Latin aliases that should surface the canonical term.
|
||||||
|
public static func entries(
|
||||||
|
from dictionary: PersonalDictionary,
|
||||||
|
annotator: RimePinyinAnnotator
|
||||||
|
) -> [Entry] {
|
||||||
|
var seen = Set<String>()
|
||||||
|
var rows: [Entry] = []
|
||||||
|
|
||||||
|
func append(text: String, code: String) {
|
||||||
|
let key = "\(text)\t\(code)"
|
||||||
|
guard seen.insert(key).inserted else { return }
|
||||||
|
rows.append(Entry(text: text, code: code))
|
||||||
|
}
|
||||||
|
|
||||||
|
for entry in dictionary.effectiveEntries {
|
||||||
|
let term = entry.term.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !term.isEmpty else { continue }
|
||||||
|
|
||||||
|
if let code = annotator.code(for: term) {
|
||||||
|
append(text: term, code: code)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Latin aliases → alternate codes for the canonical term
|
||||||
|
// (e.g. brand English → 中文专名). Chinese ASR aliases stay out.
|
||||||
|
for alias in entry.aliases {
|
||||||
|
let trimmed = alias.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !trimmed.isEmpty else { continue }
|
||||||
|
guard !RimePinyinAnnotator.containsCJK(trimmed) else { continue }
|
||||||
|
let latin = RimePinyinAnnotator.latinSpellerCode(trimmed)
|
||||||
|
guard !latin.isEmpty else { continue }
|
||||||
|
append(text: term, code: latin)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return rows.sorted {
|
||||||
|
if $0.text != $1.text {
|
||||||
|
return $0.text.localizedStandardCompare($1.text) == .orderedAscending
|
||||||
|
}
|
||||||
|
return $0.code < $1.code
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func yaml(
|
||||||
|
from dictionary: PersonalDictionary,
|
||||||
|
annotator: RimePinyinAnnotator
|
||||||
|
) -> String {
|
||||||
|
yaml(entries: entries(from: dictionary, annotator: annotator))
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func yaml(entries: [Entry]) -> String {
|
||||||
|
var lines: [String] = [
|
||||||
|
"# Generated by OSGKeyboard from PersonalDictionary. Do not edit.",
|
||||||
|
"---",
|
||||||
|
"name: \(dictionaryName)",
|
||||||
|
"version: \"1.0\"",
|
||||||
|
"sort: by_weight",
|
||||||
|
"columns:",
|
||||||
|
" - text",
|
||||||
|
" - code",
|
||||||
|
" - weight",
|
||||||
|
"..."
|
||||||
|
]
|
||||||
|
for entry in entries {
|
||||||
|
lines.append("\(entry.text)\t\(entry.code)\t\(entry.weight)")
|
||||||
|
}
|
||||||
|
lines.append("")
|
||||||
|
return lines.joined(separator: "\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func fingerprint(of yaml: String) -> String {
|
||||||
|
let digest = SHA256.hash(data: Data(yaml.utf8))
|
||||||
|
return digest.map { String(format: "%02x", $0) }.joined()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Injects `import_tables: [osg_personal]` into a baseline dict header.
|
||||||
|
public static func injectingImportTables(into baselineYAML: String) -> String {
|
||||||
|
if baselineYAML.contains("import_tables:") {
|
||||||
|
return baselineYAML
|
||||||
|
}
|
||||||
|
let needle = "use_preset_vocabulary: false\n"
|
||||||
|
let injection = """
|
||||||
|
use_preset_vocabulary: false
|
||||||
|
import_tables:
|
||||||
|
- \(dictionaryName)
|
||||||
|
|
||||||
|
"""
|
||||||
|
if let range = baselineYAML.range(of: needle) {
|
||||||
|
return baselineYAML.replacingCharacters(in: range, with: injection)
|
||||||
|
}
|
||||||
|
// Fallback: insert after the `---` document start block's name line.
|
||||||
|
let nameNeedle = "name: osg_pinyin\n"
|
||||||
|
if let range = baselineYAML.range(of: nameNeedle) {
|
||||||
|
let injectionAfterName = """
|
||||||
|
name: osg_pinyin
|
||||||
|
import_tables:
|
||||||
|
- \(dictionaryName)
|
||||||
|
|
||||||
|
"""
|
||||||
|
return baselineYAML.replacingCharacters(in: range, with: injectionAfterName)
|
||||||
|
}
|
||||||
|
return baselineYAML
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
// RimePinyinAnnotator.swift
|
||||||
|
// OSGKeyboard · Shared
|
||||||
|
//
|
||||||
|
// Builds phrase / character pinyin maps from the bundled osg_pinyin
|
||||||
|
// dictionary so PersonalDictionary terms can be coded for Rime without
|
||||||
|
// shipping a second pronunciation dataset.
|
||||||
|
|
||||||
|
import Foundation
|
||||||
|
|
||||||
|
public struct RimePinyinAnnotator: Sendable {
|
||||||
|
private let phraseCodes: [String: String]
|
||||||
|
private let characterCodes: [String: String]
|
||||||
|
|
||||||
|
public init(phraseCodes: [String: String], characterCodes: [String: String]) {
|
||||||
|
self.phraseCodes = phraseCodes
|
||||||
|
self.characterCodes = characterCodes
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parses `osg_pinyin.dict.yaml` (text / code / weight columns).
|
||||||
|
/// When duplicate texts exist, keeps the highest-weight code.
|
||||||
|
public static func load(from dictYAML: URL) throws -> RimePinyinAnnotator {
|
||||||
|
let raw = try String(contentsOf: dictYAML, encoding: .utf8)
|
||||||
|
var phraseCodes: [String: (code: String, weight: Int)] = [:]
|
||||||
|
var characterCodes: [String: (code: String, weight: Int)] = [:]
|
||||||
|
var inBody = false
|
||||||
|
|
||||||
|
for line in raw.split(separator: "\n", omittingEmptySubsequences: false) {
|
||||||
|
let trimmed = line.trimmingCharacters(in: .whitespaces)
|
||||||
|
if trimmed == "..." {
|
||||||
|
inBody = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
guard inBody, !trimmed.isEmpty, !trimmed.hasPrefix("#") else { continue }
|
||||||
|
|
||||||
|
let parts = trimmed.split(separator: "\t", omittingEmptySubsequences: false)
|
||||||
|
guard parts.count >= 2 else { continue }
|
||||||
|
let text = String(parts[0])
|
||||||
|
let code = String(parts[1]).trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !text.isEmpty, !code.isEmpty else { continue }
|
||||||
|
let weight = parts.count >= 3 ? Int(parts[2]) ?? 0 : 0
|
||||||
|
|
||||||
|
if let existing = phraseCodes[text] {
|
||||||
|
if weight >= existing.weight {
|
||||||
|
phraseCodes[text] = (code, weight)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
phraseCodes[text] = (code, weight)
|
||||||
|
}
|
||||||
|
|
||||||
|
if text.count == 1, Self.isCJKIdeograph(text.unicodeScalars.first!) {
|
||||||
|
if let existing = characterCodes[text] {
|
||||||
|
if weight >= existing.weight {
|
||||||
|
characterCodes[text] = (code, weight)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
characterCodes[text] = (code, weight)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return RimePinyinAnnotator(
|
||||||
|
phraseCodes: phraseCodes.mapValues(\.code),
|
||||||
|
characterCodes: characterCodes.mapValues(\.code)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns a Rime speller code (space-separated syllables / Latin tokens),
|
||||||
|
/// or `nil` when any CJK character cannot be annotated.
|
||||||
|
public func code(for term: String) -> String? {
|
||||||
|
let trimmed = term.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||||
|
guard !trimmed.isEmpty else { return nil }
|
||||||
|
|
||||||
|
if let exact = phraseCodes[trimmed] {
|
||||||
|
return exact
|
||||||
|
}
|
||||||
|
|
||||||
|
var parts: [String] = []
|
||||||
|
for run in Self.scriptRuns(in: trimmed) {
|
||||||
|
switch run.kind {
|
||||||
|
case .cjk:
|
||||||
|
if let phrase = phraseCodes[run.text] {
|
||||||
|
parts.append(phrase)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var syllables: [String] = []
|
||||||
|
for character in run.text {
|
||||||
|
let key = String(character)
|
||||||
|
guard let syllable = characterCodes[key] else { return nil }
|
||||||
|
syllables.append(syllable)
|
||||||
|
}
|
||||||
|
parts.append(syllables.joined(separator: " "))
|
||||||
|
case .latin:
|
||||||
|
let latin = Self.latinSpellerCode(run.text)
|
||||||
|
guard !latin.isEmpty else { continue }
|
||||||
|
parts.append(latin)
|
||||||
|
case .other:
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let joined = parts.joined(separator: " ")
|
||||||
|
.split(separator: " ", omittingEmptySubsequences: true)
|
||||||
|
.joined(separator: " ")
|
||||||
|
return joined.isEmpty ? nil : joined
|
||||||
|
}
|
||||||
|
|
||||||
|
// MARK: - Script helpers
|
||||||
|
|
||||||
|
private enum RunKind {
|
||||||
|
case cjk
|
||||||
|
case latin
|
||||||
|
case other
|
||||||
|
}
|
||||||
|
|
||||||
|
private struct ScriptRun {
|
||||||
|
let kind: RunKind
|
||||||
|
let text: String
|
||||||
|
}
|
||||||
|
|
||||||
|
private static func scriptRuns(in term: String) -> [ScriptRun] {
|
||||||
|
var runs: [ScriptRun] = []
|
||||||
|
var currentKind: RunKind?
|
||||||
|
var buffer = ""
|
||||||
|
|
||||||
|
func flush() {
|
||||||
|
guard let kind = currentKind, !buffer.isEmpty else { return }
|
||||||
|
runs.append(ScriptRun(kind: kind, text: buffer))
|
||||||
|
buffer = ""
|
||||||
|
currentKind = nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for scalar in term.unicodeScalars {
|
||||||
|
let kind: RunKind
|
||||||
|
if isCJKIdeograph(scalar) {
|
||||||
|
kind = .cjk
|
||||||
|
} else if scalar.isASCII, CharacterSet.letters.contains(scalar)
|
||||||
|
|| CharacterSet.decimalDigits.contains(scalar)
|
||||||
|
|| scalar == "-" || scalar == "'" || scalar == "_" {
|
||||||
|
kind = .latin
|
||||||
|
} else if scalar == " " || scalar == "\u{3000}" {
|
||||||
|
flush()
|
||||||
|
continue
|
||||||
|
} else {
|
||||||
|
kind = .other
|
||||||
|
}
|
||||||
|
|
||||||
|
if currentKind == nil {
|
||||||
|
currentKind = kind
|
||||||
|
buffer = String(scalar)
|
||||||
|
} else if currentKind == kind {
|
||||||
|
buffer.append(Character(scalar))
|
||||||
|
} else {
|
||||||
|
flush()
|
||||||
|
currentKind = kind
|
||||||
|
buffer = String(scalar)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
flush()
|
||||||
|
return runs
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Speller alphabet is a–z only; strip everything else and lowercase.
|
||||||
|
public static func latinSpellerCode(_ raw: String) -> String {
|
||||||
|
var output = ""
|
||||||
|
for scalar in raw.lowercased().unicodeScalars {
|
||||||
|
guard scalar.isASCII, CharacterSet.lowercaseLetters.contains(scalar) else { continue }
|
||||||
|
output.append(Character(scalar))
|
||||||
|
}
|
||||||
|
return output
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func isCJKIdeograph(_ scalar: Unicode.Scalar) -> Bool {
|
||||||
|
switch scalar.value {
|
||||||
|
case 0x3400...0x4DBF, 0x4E00...0x9FFF, 0xF900...0xFAFF:
|
||||||
|
return true
|
||||||
|
default:
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func containsCJK(_ text: String) -> Bool {
|
||||||
|
text.unicodeScalars.contains(where: isCJKIdeograph)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -55,7 +55,8 @@ public struct RimeResourcePaths: Sendable {
|
|||||||
|
|
||||||
public actor RimeResourceInstaller {
|
public actor RimeResourceInstaller {
|
||||||
public static let shared = RimeResourceInstaller()
|
public static let shared = RimeResourceInstaller()
|
||||||
public static let resourceVersion = "2.2.0"
|
/// Bump when SharedSupport layout / schema / import_tables contract changes.
|
||||||
|
public static let resourceVersion = "2.3.0"
|
||||||
|
|
||||||
public init() {}
|
public init() {}
|
||||||
|
|
||||||
@@ -71,11 +72,21 @@ public actor RimeResourceInstaller {
|
|||||||
|
|
||||||
/// Installs source data and asks librime to prebuild schemas. Call only
|
/// Installs source data and asks librime to prebuild schemas. Call only
|
||||||
/// from the host app, never from the keyboard extension.
|
/// from the host app, never from the keyboard extension.
|
||||||
|
///
|
||||||
|
/// Redeploys when `force` is set, the resource version is stale, or the
|
||||||
|
/// PersonalDictionary sidecar fingerprint changed.
|
||||||
public func installIfNeeded(
|
public func installIfNeeded(
|
||||||
configuration: TypingInputConfigurationSnapshot,
|
configuration: TypingInputConfigurationSnapshot,
|
||||||
|
personalDictionary: PersonalDictionary? = nil,
|
||||||
force: Bool = false
|
force: Bool = false
|
||||||
) throws {
|
) throws {
|
||||||
if !force, Self.isReady { return }
|
let dictionary = personalDictionary ?? AppGroupStore().personalDictionary
|
||||||
|
let personalYAML = try Self.makePersonalDictionaryYAML(from: dictionary)
|
||||||
|
let personalFingerprint = RimePersonalDictionaryExporter.fingerprint(of: personalYAML)
|
||||||
|
let personalChanged =
|
||||||
|
TypingInputConfiguration.installedPersonalDictionaryFingerprint() != personalFingerprint
|
||||||
|
|
||||||
|
if !force, Self.isReady, !personalChanged { return }
|
||||||
|
|
||||||
let paths = try RimeResourcePaths.resolve()
|
let paths = try RimeResourcePaths.resolve()
|
||||||
let fileManager = FileManager.default
|
let fileManager = FileManager.default
|
||||||
@@ -99,16 +110,32 @@ public actor RimeResourceInstaller {
|
|||||||
defer { try? fileManager.removeItem(at: staging) }
|
defer { try? fileManager.removeItem(at: staging) }
|
||||||
try fileManager.createDirectory(at: staging, withIntermediateDirectories: true)
|
try fileManager.createDirectory(at: staging, withIntermediateDirectories: true)
|
||||||
|
|
||||||
for resource in ["osg_pinyin.dict", "manifest"] {
|
guard let pinyinSource = Self.bundledURL(forResource: "osg_pinyin.dict", withExtension: "yaml") else {
|
||||||
let ext = resource == "manifest" ? "json" : "yaml"
|
throw RimeResourceError.bundledResourceMissing("osg_pinyin.dict.yaml")
|
||||||
guard let source = Self.bundledURL(forResource: resource, withExtension: ext) else {
|
}
|
||||||
throw RimeResourceError.bundledResourceMissing("\(resource).\(ext)")
|
let baseline = try String(contentsOf: pinyinSource, encoding: .utf8)
|
||||||
|
let patched = RimePersonalDictionaryExporter.injectingImportTables(into: baseline)
|
||||||
|
try patched.write(
|
||||||
|
to: staging.appendingPathComponent("osg_pinyin.dict.yaml"),
|
||||||
|
atomically: true,
|
||||||
|
encoding: .utf8
|
||||||
|
)
|
||||||
|
|
||||||
|
guard let manifestSource = Self.bundledURL(forResource: "manifest", withExtension: "json") else {
|
||||||
|
throw RimeResourceError.bundledResourceMissing("manifest.json")
|
||||||
}
|
}
|
||||||
try fileManager.copyItem(
|
try fileManager.copyItem(
|
||||||
at: source,
|
at: manifestSource,
|
||||||
to: staging.appendingPathComponent("\(resource).\(ext)")
|
to: staging.appendingPathComponent("manifest.json")
|
||||||
|
)
|
||||||
|
|
||||||
|
try personalYAML.write(
|
||||||
|
to: staging.appendingPathComponent(
|
||||||
|
"\(RimePersonalDictionaryExporter.dictionaryName).dict.yaml"
|
||||||
|
),
|
||||||
|
atomically: true,
|
||||||
|
encoding: .utf8
|
||||||
)
|
)
|
||||||
}
|
|
||||||
|
|
||||||
try RimeSchemaGenerator.defaultConfiguration().write(
|
try RimeSchemaGenerator.defaultConfiguration().write(
|
||||||
to: staging.appendingPathComponent("default.yaml"),
|
to: staging.appendingPathComponent("default.yaml"),
|
||||||
@@ -147,9 +174,20 @@ public actor RimeResourceInstaller {
|
|||||||
bridge.finalizeRuntime()
|
bridge.finalizeRuntime()
|
||||||
|
|
||||||
TypingInputConfiguration.setInstalledResourceVersion(Self.resourceVersion)
|
TypingInputConfiguration.setInstalledResourceVersion(Self.resourceVersion)
|
||||||
|
TypingInputConfiguration.setInstalledPersonalDictionaryFingerprint(personalFingerprint)
|
||||||
AppGroupConfigDarwin.postConfigChanged()
|
AppGroupConfigDarwin.postConfigChanged()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static func makePersonalDictionaryYAML(
|
||||||
|
from dictionary: PersonalDictionary
|
||||||
|
) throws -> String {
|
||||||
|
guard let pinyinSource = bundledURL(forResource: "osg_pinyin.dict", withExtension: "yaml") else {
|
||||||
|
throw RimeResourceError.bundledResourceMissing("osg_pinyin.dict.yaml")
|
||||||
|
}
|
||||||
|
let annotator = try RimePinyinAnnotator.load(from: pinyinSource)
|
||||||
|
return RimePersonalDictionaryExporter.yaml(from: dictionary, annotator: annotator)
|
||||||
|
}
|
||||||
|
|
||||||
public func syncUserData() throws {
|
public func syncUserData() throws {
|
||||||
let paths = try RimeResourcePaths.resolve()
|
let paths = try RimeResourcePaths.resolve()
|
||||||
let bridge = OSGRimeBridge(
|
let bridge = OSGRimeBridge(
|
||||||
|
|||||||
@@ -105,6 +105,7 @@
|
|||||||
"OSGKeyboardExtTests/KeyboardSurfaceStateTests",
|
"OSGKeyboardExtTests/KeyboardSurfaceStateTests",
|
||||||
"OSGKeyboardExtTests/LibrimeIntegrationTests",
|
"OSGKeyboardExtTests/LibrimeIntegrationTests",
|
||||||
"OSGKeyboardExtTests/RimeSchemaGeneratorTests",
|
"OSGKeyboardExtTests/RimeSchemaGeneratorTests",
|
||||||
|
"OSGKeyboardExtTests/RimePersonalDictionaryExporterTests",
|
||||||
"OSGKeyboardTests/CursorNavigationTests",
|
"OSGKeyboardTests/CursorNavigationTests",
|
||||||
"OSGKeyboardTests/KeyboardTranslationConfigProtectionTests"
|
"OSGKeyboardTests/KeyboardTranslationConfigProtectionTests"
|
||||||
]
|
]
|
||||||
|
|||||||
Reference in New Issue
Block a user