feat(lexicon): add custom ASR language model v1 (Sogou + AI/tech brands)

Build a first-version SFCustomLanguageModelData for on-device ASR
customization, generated offline on macOS (no app code dependency).

- Scripts/lexicon: Python builders for Sogou-derived phrases (scel +
  SogouPopularDict) and a curated bilingual AI/tech/brand seed lexicon
- export_clm.swift / prepare_clm.swift: macOS CLI tools that export the
  .bin training asset and compile it into LM + Vocab via the Speech
  framework
- Resources/CustomLanguageModel: generated phrases.tsv, manifests, the
  129k-phrase .bin, and compiled LM/Vocab assets (zh_CN)
- .gitignore: ignore lexicon build cache and Python bytecode

Note: Sogou-derived data is for internal experimentation only; the
curated AI/tech seed is MIT and safe to ship.

This commit is contained in:

Rocky

2026-07-05 17:57:18 +08:00

parent 05e005e9ce

commit 7ffa7ae3f2

18 changed files with 130951 additions and 0 deletions

									
										Scripts/lexicon/build-clm-bin.sh
									
Executable

		+6
		
												View File
												
				@@ -0,0 +1,6 @@

				#!/usr/bin/env bash

				# Build SFCustomLanguageModelData .bin on macOS (requires Speech framework).

				set -euo pipefail

				ROOT="$(cd "$(dirname "$0")/../.." && pwd)"

				cd "$ROOT"

				exec swift "$ROOT/Scripts/lexicon/export_clm.swift" "$@"