From f7a031cc1df19cfc070770be56e35f2bc51efd35 Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:04:55 +0200 Subject: [PATCH 01/19] test(manifest): register PolishStylesL10nTests in the polish group --- Tests/suite-manifest.json | 1 + 1 file changed, 1 insertion(+) diff --git a/Tests/suite-manifest.json b/Tests/suite-manifest.json index d4cf34f..05c117a 100644 --- a/Tests/suite-manifest.json +++ b/Tests/suite-manifest.json @@ -81,6 +81,7 @@ "OSGKeyboardTests/PolishStylePackTests", "OSGKeyboardTests/PolishStyleCorpusExportStoreTests", "OSGKeyboardTests/PolishStyleLearningServiceTests", + "OSGKeyboardTests/PolishStylesL10nTests", "OSGKeyboardTests/LLMClientTests", "OSGKeyboardTests/TranscriptLanguageDetectorTests", "OSGKeyboardTests/AIModeLLMClientTests", From be56d2210cca95ec4b2fd72fe88e99760005f3ef Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:05:08 +0200 Subject: [PATCH 02/19] fix(asr): keep recordings when streaming recognition fails or stalls - Keep recording after a streaming ASR error and retry the captured PCM through batch ASR before reporting a failure. - Treat an expired ASR wait as a stall so batch fallback still runs. - Prime the managed Volcengine stream on mic touch and adopt it only while fresh and open. - Auto-scroll the live transcript to the newest recognized text. --- CHANGELOG.md | 3 + OSGKeyboard/Services/FlowSessionManager.swift | 29 ++- OSGKeyboardExt/Views/AIKeyboardView.swift | 40 +++- .../Services/CloudASR/CloudASRService.swift | 215 +++++++++++++++++- .../Services/CloudASR/CloudASRStreaming.swift | 7 + .../CloudASR/ManagedVolcengineASRClient.swift | 6 + .../UtteranceBatchFallbackPolicy.swift | 7 +- OSGKeyboardTests/CloudASRServiceTests.swift | 212 +++++++++++++++++ .../UtteranceBatchFallbackPolicyTests.swift | 19 ++ 9 files changed, 518 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 46635a2..63d440d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Host-only analytics recording**: the keyboard extension no longer opens the analytics or keyboard-usage SQLite queues. The host app is the only process that records and uploads events, so hiding the keyboard cannot hold WAL locks and get killed as `0xdead10cc`. Keyboard-activated and per-keystroke usage events are no longer collected. / **仅主 App 记埋点**：键盘扩展不再打开统计与键盘用量 SQLite 队列；开库、记账与上传只在主 App 进行，收起键盘时不会因 WAL 锁被系统以 `0xdead10cc` 终止。不再采集「键盘激活」和逐次按键用量。 ### Fixed +- **Faster managed dictation feedback**: begin the managed Volcengine streaming connection while the microphone touch primes audio, reuse it when recording starts, and keep the live transcript automatically scrolled to the newest recognition text. The early connection is reused only while it is fresh (under 10 seconds) and still open; otherwise it is closed and a new one opens, so an unused early connection can no longer make the next dictation fail. / **托管听写反馈提速**：在麦克风触摸预热音频时同步建立托管火山流式连接，录音开始后直接复用，并让实时转写自动滚动到最新识别文字。提前建立的连接仅在新鲜（10 秒内）且仍可用时复用，否则关闭并重新建立，未被使用的提前连接不会再导致下一次听写失败。 +- **Recording kept when live recognition fails**: a streaming recognition error while you were still speaking ended the dictation immediately and discarded the audio. Recording now continues, and when you finish, the recorded audio is retried with batch recognition before any error is shown. After a cancelled or failed dictation, that batch retry also stopped working for the rest of the session; it now works again from the next dictation. / **实时识别失败时保留录音**：说话过程中流式识别一旦出错，本次听写会立即结束并丢弃录音。现在会继续录音，说完后先用批量识别重试已录下的音频，仍失败才提示错误。此前一次听写被取消或失败后，本次会话内的批量重试也会失效；现在从下一次听写起恢复正常。 +- **Stalled cloud ASR losing the recording**: when the streaming recognizer went quiet and the result wait expired, the utterance ended as "no speech" even though the audio was captured. A stalled recognizer now counts as a failure, so the batch ASR fallback retries the recorded audio before giving up. / **云端识别卡住时丢失录音**：流式识别无响应、等待结果超时后，即便录音已采集，本次语音也会直接报「未检测到语音」。现在识别卡住按失败处理，会先用批量识别重试已录下的音频再放弃。 - **Speak as Me output shape**: the skill could answer with raw JSON variants or echo the personal-style XML tags into the result. The prompt now requires one ready-to-send plain-text rewrite, and length follows your natural voice instead of a mechanical ±20% cap. / **「像我所说」输出形态**：该技能可能返回原始 JSON 候选，或把个人风格的 XML 标记带进结果。现在提示词要求只输出一段可直接发送的文本，长度改为跟随你的自然表达节奏，不再机械限制 ±20%。 - **Clipboard history reverting deletes**: history is one shared blob, and the main App kept an in-memory copy that only refreshed when a screen appeared — so a delete made there could write back entries the keyboard had just captured or removed, and a clear done in the keyboard could be undone entirely. Both processes now announce their writes to each other, and the main App re-reads history whenever it returns to the foreground. / **剪贴板历史撤销删除**：历史记录是整块共享数据，而主 App 仅在页面出现时刷新内存副本——因此在主 App 中删除一条，可能把键盘刚采集或刚删除的条目重新写回，在键盘里执行的清空也可能被整体撤销。现在两端写入时会互相通知，主 App 每次回到前台也会重新读取历史。 - **Keyboard crash loop on memory relief**: recording a memory milestone could request relief while relief was already running, and the two sides called each other until the keyboard extension exhausted its stack (SIGSEGV grouped under `KeyboardViewController.viewDidLoad`). Reentrancy guards on both the controller and the telemetry side break the cycle. / **键盘内存降级递归崩溃**：记录内存里程碑时可能再次请求降级，两侧互相回调直至扩展栈溢出（崩溃聚合在 `KeyboardViewController.viewDidLoad`）。控制器与遥测两侧均加重入保护，打断循环。 diff --git a/OSGKeyboard/Services/FlowSessionManager.swift b/OSGKeyboard/Services/FlowSessionManager.swift index cca96c4..4c1dccc 100644 --- a/OSGKeyboard/Services/FlowSessionManager.swift +++ b/OSGKeyboard/Services/FlowSessionManager.swift @@ -2109,6 +2109,13 @@ final class FlowSessionManager: ObservableObject { !hasUnacknowledgedTerminalResult() else { return } + bindSessionASRIfNeeded() + if let cloudASR = sessionASR as? CloudASRService { + // Same locale source as `beginUtterance`, so the prime is adoptable. + cloudASR.primeUtteranceStreaming( + locale: SpeechLocaleResolver.resolve(store.localeId) + ) + } startAudioPrime(id: command.utteranceId, origin: "micTouch") } @@ -2171,6 +2178,7 @@ final class FlowSessionManager: ObservableObject { !isUtteranceProcessing else { return } + (sessionASR as? CloudASRService)?.cancelPrimedUtteranceStreaming() audioPrimeCancellationRequested = true // `capture.start()` is not cancellation-safe. Never stop it mid-start; // the completion waiter performs cleanup unless a real utterance adopts @@ -2613,11 +2621,14 @@ final class FlowSessionManager: ObservableObject { ] .map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } .first(where: { !$0.isEmpty }) + // Without a partial, keep recording: capture is independent + // of ASR, and finalize retries the full PCM through batch + // ASR (`recognitionFailed`) before reporting the error. if let recovery { manager.lastFinal = recovery manager.debug("asr error recovered via partial len=\(recovery.count)") } else if manager.isUtteranceRecording { - manager.failUtterance(message: message, kind: .asrFailed) + manager.debug("asr error while recording — deferring to batch fallback") } case .cancelled: // A provider-side cancellation while the mic is still open @@ -2935,8 +2946,16 @@ final class FlowSessionManager: ObservableObject { try? await Task.sleep(nanoseconds: 100_000_000) } + // The recognizer never reported back within the wait: treat it as a + // failure so batch ASR can still recover the captured audio. + let asrStalled = lastFinal.isEmpty + && asrCompletedGeneration != finalizeGeneration + && !Task.isCancelled if lastFinal.isEmpty { - FlowDiagnostics.log("ASR wait elapsed — cancelling ASR task and using best available transcript") + FlowDiagnostics.log( + "ASR wait elapsed — cancelling ASR task and using best available transcript " + + "stalled=\(asrStalled ? 1 : 0)" + ) asrTask?.cancel() Task { await chunkedPipeline?.cancel() } } @@ -2973,11 +2992,13 @@ final class FlowSessionManager: ObservableObject { let wantsBatchFallback = UtteranceBatchFallbackPolicy.shouldRunBatchFallback( stitchedFinal: lastFinal, partialSnapshot: bestPartialSnapshot, - recognitionFailed: asrFailureMessage != nil + recognitionFailed: asrFailureMessage != nil, + recognitionStalled: asrStalled ) FlowTrace.pipeline( "batchFallback.decision", - "wanted=\(wantsBatchFallback ? 1 : 0) pcmSamples=\(utterancePCMSamples.count) " + "wanted=\(wantsBatchFallback ? 1 : 0) stalled=\(asrStalled ? 1 : 0) " + + "pcmSamples=\(utterancePCMSamples.count) " + "pcmRms=\(FlowTrace.rms(utterancePCMSamples)) " + "stitchedLen=\(lastFinal.count) partialLen=\(bestPartialSnapshot.count) " + "resolvedLen=\(text.count)" diff --git a/OSGKeyboardExt/Views/AIKeyboardView.swift b/OSGKeyboardExt/Views/AIKeyboardView.swift index 4b2f776..26caa6b 100644 --- a/OSGKeyboardExt/Views/AIKeyboardView.swift +++ b/OSGKeyboardExt/Views/AIKeyboardView.swift @@ -6,6 +6,7 @@ import OSGKeyboardShared import SwiftUI +import UIKit struct AIKeyboardView: View { private enum Layout { @@ -35,6 +36,7 @@ struct AIKeyboardView: View { static let pageDotSize: CGFloat = 5 static let skillPaginationBottomInset: CGFloat = 6 static let maximumSemanticSkills = 5 + static let statusViewportHeight = UIFont.systemFont(ofSize: 15).lineHeight * 3 } private struct SemanticBadgeContent { @@ -685,13 +687,37 @@ struct AIKeyboardView: View { } private func statusText(_ text: String, color: Color) -> some View { - Text(text) - .font(TypeStyle.body) - .foregroundStyle(color) - .lineLimit(2) - .multilineTextAlignment(.center) - .truncationMode(.tail) - .frame(maxWidth: .infinity) + GeometryReader { geometry in + ScrollViewReader { proxy in + ScrollView(.vertical, showsIndicators: false) { + // Match the viewport height while short, then grow upward one + // complete line at a time as the live transcript wraps. + Text(text) + .font(TypeStyle.body) + .foregroundStyle(color) + .multilineTextAlignment(.center) + .fixedSize(horizontal: false, vertical: true) + .frame( + maxWidth: .infinity, + minHeight: geometry.size.height, + alignment: .center + ) + .id("assistant.status.text") + } + .scrollDisabled(true) + .scrollEdgeEffectHidden() + .onAppear { + proxy.scrollTo("assistant.status.text", anchor: .bottom) + } + .onChange(of: text) { _, _ in + // ASR partials update per word. Jump only by the resulting + // layout delta instead of animating every partial update. + proxy.scrollTo("assistant.status.text", anchor: .bottom) + } + } + } + .frame(height: Layout.statusViewportHeight) + .frame(maxWidth: .infinity) } private var activeStatus: (text: String, color: Color)? { diff --git a/OSGKeyboardHostSupport/Services/CloudASR/CloudASRService.swift b/OSGKeyboardHostSupport/Services/CloudASR/CloudASRService.swift index c94e810..b4a55e8 100644 --- a/OSGKeyboardHostSupport/Services/CloudASR/CloudASRService.swift +++ b/OSGKeyboardHostSupport/Services/CloudASR/CloudASRService.swift @@ -11,11 +11,43 @@ import os import OSGKeyboardShared #endif +/// A primed streaming session is opened before the utterance has an owner. +/// Route its callbacks through a mutable relay, then bind the real partial +/// consumer when the mic gesture is committed to an utterance. +private final class CloudASRPartialRelay: @unchecked Sendable { + private let lock = OSAllocatedUnfairLock< + (@Sendable (String) -> Void)? + >(initialState: nil) + + func bind(_ consumer: @escaping @Sendable (String) -> Void) { + lock.withLock { $0 = consumer } + } + + func publish(_ text: String) { + let consumer = lock.withLock { $0 } + consumer?(text) + } +} + /// Uploads PCM only on the user-selected cloud engine path; provider clients /// reject missing credentials before network transmission. Personal dictionary /// entries are sent as recognition bias. Mutable client/cancellation state is /// lock-protected, which is the basis for `@unchecked Sendable`. public final class CloudASRService: ASRService, @unchecked Sendable { + private struct PrimedStreamingSession { + let id: UUID + let selection: String + let localeID: String + let createdAt: ContinuousClock.Instant + let relay: CloudASRPartialRelay + let task: Task + } + + /// The gateway closes a session that receives no audio for 15 s (counted + /// from WebSocket connect). A primed session older than this is never + /// adopted, leaving headroom for the first audio frame to arrive. + public static let defaultPrimedSessionMaxAge: Duration = .seconds(10) + private let store: any ConfigurationStore private let session: URLSession private let localFallback: ASRService @@ -23,24 +55,28 @@ public final class CloudASRService: ASRService, @unchecked Sendable { /// provider-specific local-fallback selection unchanged. private let managedClient: (any CloudASRTranscribing)? private let managedGrants: GatewayGrantCoordinator? + private let primedSessionMaxAge: Duration private let lock = OSAllocatedUnfairLock() private var client: CloudASRTranscribing? private var usesLocalFallback = false private var boundClientSelection: String? private var cancelled = false private var streamingPipeline: StreamingUtterancePipeline? + private var primedStreamingSession: PrimedStreamingSession? public init( store: any ConfigurationStore = AppGroupStore(), session: URLSession = .shared, localFallback: ASRService? = nil, managedClient: (any CloudASRTranscribing)? = nil, - managedGrants: GatewayGrantCoordinator? = nil + managedGrants: GatewayGrantCoordinator? = nil, + primedSessionMaxAge: Duration = CloudASRService.defaultPrimedSessionMaxAge ) { self.store = store self.session = session self.managedClient = managedClient self.managedGrants = managedGrants + self.primedSessionMaxAge = primedSessionMaxAge // `SpeechAnalyzerASR` is internal, so it can't appear in a public // default argument value — resolve the fallback in the body instead. self.localFallback = localFallback ?? SpeechAnalyzerASR() @@ -77,6 +113,143 @@ public final class CloudASRService: ASRService, @unchecked Sendable { } } + /// Starts the account-managed Volcengine reservation + WebSocket handshake + /// while the user's finger is still down. The utterance adopts this session + /// moments later, removing those network round trips from first-partial + /// latency without sending audio before recording actually begins. + public func primeUtteranceStreaming(locale: Locale) { + bindClientIfNeeded() + guard store.credentialSource == .managed, + let selection = lock.withLock({ boundClientSelection }), + let streamingClient = lock.withLock({ + client as? any CloudASRStreamingCapable + }) else { + return + } + + let localeID = locale.identifier(.bcp47) + let alreadyPrimed = lock.withLock { () -> Bool in + guard let current = primedStreamingSession else { return false } + return current.selection == selection + && current.localeID == localeID + && !isExpired(current) + } + guard !alreadyPrimed else { return } + + cancelPrimedUtteranceStreaming() + let id = UUID() + let relay = CloudASRPartialRelay() + let dictionary = store.personalDictionary + let task = Task.detached(priority: .userInitiated) { + try await streamingClient.openStreamingSession( + locale: locale, + dictionary: dictionary, + onPartial: { relay.publish($0) } + ) + } + lock.withLock { + primedStreamingSession = PrimedStreamingSession( + id: id, + selection: selection, + localeID: localeID, + createdAt: .now, + relay: relay, + task: task + ) + } + // Self-expire: the keyboard's cancel command may never arrive (the + // extension can be dismissed, or a start can fail before adoption). + let maxAge = primedSessionMaxAge + Task { [weak self] in + try? await Task.sleep(for: maxAge) + guard let self, let expired = self.takePrimedSession(id: id) else { return } + FlowTrace.asr("cloud.stream.primeDiscarded", "reason=expired") + Self.close(expired) + } + FlowTrace.asr( + "cloud.stream.primeStarted", + "locale=\(localeID) selection=\(selection)" + ) + } + + public func cancelPrimedUtteranceStreaming() { + guard let primed = takePrimedSession() else { return } + Self.close(primed) + } + + /// Removes the primed session; with an `id`, only if it is still that one. + private func takePrimedSession(id: UUID? = nil) -> PrimedStreamingSession? { + lock.withLock { + guard let current = primedStreamingSession, + id == nil || current.id == id else { + return nil + } + primedStreamingSession = nil + return current + } + } + + private func isExpired(_ primed: PrimedStreamingSession) -> Bool { + ContinuousClock.now - primed.createdAt >= primedSessionMaxAge + } + + /// Closes a primed session without cancelling its open task. Cancelling + /// mid-reservation can leave a reserved, never-connected gateway session + /// holding credits and a concurrency slot until its connect timeout; + /// letting the open finish and then closing releases it immediately. + private static func close(_ primed: PrimedStreamingSession) { + Task { + if let session = try? await primed.task.value { + session.cancel() + } + } + } + + /// Returns the primed session only if it still matches this utterance and + /// can accept audio. Anything else is closed and the caller opens a fresh + /// session, so priming can never make an utterance fail. + private func adoptPrimedSession( + _ primed: PrimedStreamingSession?, + selection: String?, + localeID: String, + onPartial: @escaping @Sendable (String) -> Void + ) async -> (any CloudASRStreamingSession)? { + guard let primed else { return nil } + guard primed.selection == selection, primed.localeID == localeID else { + FlowTrace.asr("cloud.stream.primeDiscarded", "reason=mismatch") + Self.close(primed) + return nil + } + guard !isExpired(primed) else { + FlowTrace.asr("cloud.stream.primeDiscarded", "reason=expired") + Self.close(primed) + return nil + } + + primed.relay.bind(onPartial) + let session: any CloudASRStreamingSession + do { + session = try await primed.task.value + } catch { + FlowTrace.asr( + "cloud.stream.primeDiscarded", + "reason=openFailed \(CloudASRLogMetadata.describe(error))" + ) + return nil + } + guard session.acceptsAudio else { + FlowTrace.asr("cloud.stream.primeDiscarded", "reason=closed") + session.cancel() + return nil + } + let ageMs = Int((ContinuousClock.now - primed.createdAt) / .milliseconds(1)) + FlowTrace.asr( + "cloud.stream.primeAdopted", + "locale=\(localeID) ageMs=\(ageMs)" + ) + return session + } + public func transcribeChunk(samples: [Float], locale: Locale) async -> ASRChunkResult { guard !samples.isEmpty else { return .success("") } if Task.isCancelled || lock.withLock({ cancelled }) { return .cancelled } @@ -129,23 +302,42 @@ public final class CloudASRService: ASRService, @unchecked Sendable { onPartial: @escaping @Sendable (String) -> Void ) async -> ChunkedUtterancePipelineOutcome { bindClientIfNeeded() + // A previous utterance's abort/failure leaves `cancelled` set, which + // would make this utterance's batch fallback (`transcribeChunk`) + // return `.cancelled` for the rest of the Flow session. + lock.withLock { cancelled = false } if usesLocalFallback { let pipeline = ChunkedUtterancePipeline(asr: localFallback, locale: locale) return await pipeline.transcribe(stream: stream, onPartial: onPartial) } guard let streamingClient = lock.withLock({ client as? CloudASRStreamingCapable }) else { + cancelPrimedUtteranceStreaming() let pipeline = ChunkedUtterancePipeline(asr: self, locale: locale) return await pipeline.transcribe(stream: stream, onPartial: onPartial) } + let selection = lock.withLock { boundClientSelection } + let localeID = locale.identifier(.bcp47) + // Every utterance either adopts the primed session or closes it. + let primed = takePrimedSession() + let session: any CloudASRStreamingSession do { - session = try await streamingClient.openStreamingSession( - locale: locale, - dictionary: store.personalDictionary, + if let adopted = await adoptPrimedSession( + primed, + selection: selection, + localeID: localeID, onPartial: onPartial - ) + ) { + session = adopted + } else { + session = try await streamingClient.openStreamingSession( + locale: locale, + dictionary: store.personalDictionary, + onPartial: onPartial + ) + } } catch where ProviderToolCancellation.matches(error) { return .cancelled } catch { @@ -249,6 +441,7 @@ public final class CloudASRService: ASRService, @unchecked Sendable { public func cancel() { lock.withLock { cancelled = true } + cancelPrimedUtteranceStreaming() let pipeline = lock.withLock { streamingPipeline } Task { await pipeline?.cancel() } localFallback.cancel() @@ -259,8 +452,10 @@ public final class CloudASRService: ASRService, @unchecked Sendable { let strategy = CloudASRModelCatalog.strategy(for: providerId) let credentialSource = store.credentialSource let selection = "\(credentialSource.rawValue):\(providerId)" - lock.withLock { - guard boundClientSelection != selection else { return } + let stalePrime = lock.withLock { () -> PrimedStreamingSession? in + guard boundClientSelection != selection else { return nil } + let stalePrime = primedStreamingSession + primedStreamingSession = nil boundClientSelection = selection if credentialSource == .managed { usesLocalFallback = false @@ -269,12 +464,16 @@ public final class CloudASRService: ASRService, @unchecked Sendable { session: session, managedGrants: managedGrants ) - return + return stalePrime } usesLocalFallback = strategy == .localFallback client = usesLocalFallback ? nil : CloudASRClientFactory.make(store: store, session: session) + return stalePrime + } + if let stalePrime { + Self.close(stalePrime) } } } diff --git a/OSGKeyboardHostSupport/Services/CloudASR/CloudASRStreaming.swift b/OSGKeyboardHostSupport/Services/CloudASR/CloudASRStreaming.swift index e3cd954..b9ba18f 100644 --- a/OSGKeyboardHostSupport/Services/CloudASR/CloudASRStreaming.swift +++ b/OSGKeyboardHostSupport/Services/CloudASR/CloudASRStreaming.swift @@ -51,6 +51,13 @@ public protocol CloudASRStreamingSession: Sendable { /// Signal end-of-audio and wait for the polish-ready final transcript. func finish() async throws -> String func cancel() + /// False once the session has failed, closed, or been cancelled, so a + /// pre-opened session is not adopted after its transport already died. + var acceptsAudio: Bool { get } +} + +public extension CloudASRStreamingSession { + var acceptsAudio: Bool { true } } /// Providers that can open an utterance-level streaming session. diff --git a/OSGKeyboardHostSupport/Services/CloudASR/ManagedVolcengineASRClient.swift b/OSGKeyboardHostSupport/Services/CloudASR/ManagedVolcengineASRClient.swift index ec60973..38ccd94 100644 --- a/OSGKeyboardHostSupport/Services/CloudASR/ManagedVolcengineASRClient.swift +++ b/OSGKeyboardHostSupport/Services/CloudASR/ManagedVolcengineASRClient.swift @@ -765,6 +765,12 @@ private final class ManagedVolcengineASRSession: CloudASRStreamingSession, @unch } } + var acceptsAudio: Bool { + lock.withLock { state in + !state.cancelled && state.failure == nil && !state.receiveClosed && !state.endSent + } + } + func cancel() { let changed = lock.withLock { state -> Bool in guard !state.cancelled else { return false } diff --git a/OSGKeyboardShared/Utilities/UtteranceBatchFallbackPolicy.swift b/OSGKeyboardShared/Utilities/UtteranceBatchFallbackPolicy.swift index 1e24178..60a259e 100644 --- a/OSGKeyboardShared/Utilities/UtteranceBatchFallbackPolicy.swift +++ b/OSGKeyboardShared/Utilities/UtteranceBatchFallbackPolicy.swift @@ -10,13 +10,18 @@ public enum UtteranceBatchFallbackPolicy { public static let defaultCharacterAdvantage = UtteranceTranscriptGuard.defaultPartialAdvantage /// True when chunked output likely lost trailing content vs the live partial. + /// + /// `recognitionStalled` means the ASR wait expired before the recognizer + /// finished (e.g. the streaming socket went quiet). That is a failure, not + /// silence, so the captured PCM still deserves a batch attempt. public static func shouldRunBatchFallback( stitchedFinal: String, partialSnapshot: String, recognitionFailed: Bool = false, + recognitionStalled: Bool = false, minimumCharacterAdvantage: Int = defaultCharacterAdvantage ) -> Bool { - if recognitionFailed { return true } + if recognitionFailed || recognitionStalled { return true } let final = stitchedFinal.trimmingCharacters(in: .whitespacesAndNewlines) let partial = partialSnapshot.trimmingCharacters(in: .whitespacesAndNewlines) diff --git a/OSGKeyboardTests/CloudASRServiceTests.swift b/OSGKeyboardTests/CloudASRServiceTests.swift index 106fe2f..88a1bc9 100644 --- a/OSGKeyboardTests/CloudASRServiceTests.swift +++ b/OSGKeyboardTests/CloudASRServiceTests.swift @@ -30,6 +30,80 @@ private final class RecordingFallbackASR: ASRService, @unchecked Sendable { } } +private final class RecordingStreamingSession: CloudASRStreamingSession, @unchecked Sendable { + private let onPartial: @Sendable (String) -> Void + /// Models a transport the gateway already closed (e.g. idle timeout). + private let isDead: Bool + + init(onPartial: @escaping @Sendable (String) -> Void, isDead: Bool = false) { + self.onPartial = onPartial + self.isDead = isDead + } + + var acceptsAudio: Bool { !isDead } + + func append(samples: [Float]) async throws { + if isDead { throw ManagedCloudASRError.idleTimeout } + if !samples.isEmpty { + onPartial("你好") + } + } + + func finish() async throws -> String { + if isDead { throw ManagedCloudASRError.idleTimeout } + return "你好" + } + + func cancel() {} +} + +/// Sessions are numbered by open order, starting at 1. +private final class RecordingStreamingClient: CloudASRStreamingCapable, @unchecked Sendable { + private let lock = OSAllocatedUnfairLock(initialState: 0) + private let deadSessions: Set + private let failingOpens: Set + + init(deadSessions: Set = [], failingOpens: Set = []) { + self.deadSessions = deadSessions + self.failingOpens = failingOpens + } + + var openCount: Int { + lock.withLock { $0 } + } + + func prepare(dictionary: PersonalDictionary) async throws {} + + func transcribe( + samples: [Float], + sampleRate: Int, + locale: Locale, + dictionary: PersonalDictionary + ) async throws -> String { + "batch" + } + + func openStreamingSession( + locale: Locale, + dictionary: PersonalDictionary, + onPartial: @escaping @Sendable (String) -> Void + ) async throws -> any CloudASRStreamingSession { + _ = locale + _ = dictionary + let index = lock.withLock { count -> Int in + count += 1 + return count + } + if failingOpens.contains(index) { + throw ManagedCloudASRError.sessionTransportFailed + } + return RecordingStreamingSession( + onPartial: onPartial, + isDead: deadSessions.contains(index) + ) + } +} + final class CloudASRServiceTests: XCTestCase { private var suiteName: String! @@ -93,4 +167,142 @@ final class CloudASRServiceTests: XCTestCase { XCTAssertEqual(text, "") XCTAssertEqual(fallback.calls, 0) } + + func testManagedStreamingAdoptsPrimedSession() async { + let client = RecordingStreamingClient() + let service = makeManagedService(client: client) + service.primeUtteranceStreaming(locale: zhLocale) + + let partials = OSAllocatedUnfairLock(initialState: [String]()) + let outcome = await service.transcribeUtteranceStreaming( + stream: oneSnapshotStream(), + locale: zhLocale + ) { partial in + partials.withLock { $0.append(partial) } + } + + guard case .success(let success) = outcome else { + return XCTFail("expected streaming success, got \(outcome)") + } + XCTAssertEqual(success.text, "你好") + XCTAssertEqual(partials.withLock { $0 }, ["你好"]) + XCTAssertEqual(client.openCount, 1) + } + + func testDeadPrimedSessionIsReplacedWithFreshSession() async { + let client = RecordingStreamingClient(deadSessions: [1]) + let service = makeManagedService(client: client) + service.primeUtteranceStreaming(locale: zhLocale) + await waitUntil { client.openCount == 1 } + + let outcome = await service.transcribeUtteranceStreaming( + stream: oneSnapshotStream(), + locale: zhLocale + ) { _ in } + + guard case .success(let success) = outcome else { + return XCTFail("expected fresh streaming success, got \(outcome)") + } + XCTAssertEqual(success.text, "你好") + XCTAssertEqual(client.openCount, 2) + } + + func testExpiredPrimedSessionIsNotAdopted() async { + let client = RecordingStreamingClient(deadSessions: [1]) + let service = makeManagedService(client: client, primedSessionMaxAge: .zero) + service.primeUtteranceStreaming(locale: zhLocale) + await waitUntil { client.openCount == 1 } + + let outcome = await service.transcribeUtteranceStreaming( + stream: oneSnapshotStream(), + locale: zhLocale + ) { _ in } + + guard case .success(let success) = outcome else { + return XCTFail("expected fresh streaming success, got \(outcome)") + } + XCTAssertEqual(success.text, "你好") + XCTAssertEqual(client.openCount, 2) + } + + func testFailedPrimeOpensFreshStreamingSession() async { + let client = RecordingStreamingClient(failingOpens: [1]) + let service = makeManagedService(client: client) + service.primeUtteranceStreaming(locale: zhLocale) + + let outcome = await service.transcribeUtteranceStreaming( + stream: oneSnapshotStream(), + locale: zhLocale + ) { _ in } + + // "你好" comes from streaming; the chunked batch fallback returns "batch". + guard case .success(let success) = outcome else { + return XCTFail("expected fresh streaming success, got \(outcome)") + } + XCTAssertEqual(success.text, "你好") + XCTAssertEqual(client.openCount, 2) + } + + func testStreamingUtteranceClearsCancellationForBatchFallback() async { + let client = RecordingStreamingClient() + let service = makeManagedService(client: client) + service.cancel() + + _ = await service.transcribeUtteranceStreaming( + stream: oneSnapshotStream(), + locale: zhLocale + ) { _ in } + let batch = await service.transcribeChunk( + samples: [Float](repeating: 0.1, count: 1_600), + locale: zhLocale + ) + + guard case .success(let text) = batch else { + return XCTFail("expected batch success after a new utterance, got \(batch)") + } + XCTAssertEqual(text, "batch") + } + + // MARK: - Helpers + + private let zhLocale = Locale(identifier: "zh-Hans") + + private func makeManagedService( + client: RecordingStreamingClient, + primedSessionMaxAge: Duration = CloudASRService.defaultPrimedSessionMaxAge + ) -> CloudASRService { + var config = AppGroupConfiguration.load(fromAvailable: defaults) + config.engineMode = "cloud" + config.credentialSource = .managed + config.save(to: defaults) + store = AppGroupStore(defaults: defaults) + return CloudASRService( + store: store, + session: .shared, + localFallback: RecordingFallbackASR(), + managedClient: client, + primedSessionMaxAge: primedSessionMaxAge + ) + } + + private func oneSnapshotStream() -> AsyncStream { + let (stream, continuation) = AsyncStream.makeStream() + continuation.yield( + AudioBufferSnapshot( + samples: [Float](repeating: 0.1, count: 1_600), + sampleRate: 16_000 + ) + ) + continuation.finish() + return stream + } + + /// Lets the detached prime task run before the utterance starts. + private func waitUntil(_ condition: () -> Bool) async { + var attempts = 0 + while !condition(), attempts < 200 { + attempts += 1 + try? await Task.sleep(for: .milliseconds(5)) + } + } } diff --git a/OSGKeyboardTests/UtteranceBatchFallbackPolicyTests.swift b/OSGKeyboardTests/UtteranceBatchFallbackPolicyTests.swift index 0f0285b..957ce90 100644 --- a/OSGKeyboardTests/UtteranceBatchFallbackPolicyTests.swift +++ b/OSGKeyboardTests/UtteranceBatchFallbackPolicyTests.swift @@ -43,6 +43,25 @@ final class UtteranceBatchFallbackPolicyTests: XCTestCase { ) } + func testShouldRunWhenRecognitionStalledWithoutAnyText() { + XCTAssertTrue( + UtteranceBatchFallbackPolicy.shouldRunBatchFallback( + stitchedFinal: "", + partialSnapshot: "", + recognitionStalled: true + ) + ) + } + + func testShouldNotRunForCompletedSilence() { + XCTAssertFalse( + UtteranceBatchFallbackPolicy.shouldRunBatchFallback( + stitchedFinal: "", + partialSnapshot: "" + ) + ) + } + func testPreferredTranscriptPicksLongestCandidate() { let resolved = UtteranceBatchFallbackPolicy.preferredTranscript( batch: "今天很好，我们一起去公园吧", From 1ddbd283715423e88189207ab2634b0b63b56a9c Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:05:21 +0200 Subject: [PATCH 03/19] feat(keyboard): show dictation progress as a fill on the microphone capsule --- CHANGELOG.md | 1 + .../Services/KeyboardFlowCoordinator.swift | 44 +++++++ OSGKeyboardExt/Views/AIKeyboardView.swift | 36 ++++++ .../DictationProgressTests.swift | 90 ++++++++++++++ .../Models/DictationProgress.swift | 113 ++++++++++++++++++ .../Services/KeyboardState.swift | 12 +- Tests/suite-manifest.json | 1 + 7 files changed, 296 insertions(+), 1 deletion(-) create mode 100644 OSGKeyboardExtTests/DictationProgressTests.swift create mode 100644 OSGKeyboardShared/Models/DictationProgress.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 63d440d..c98d4d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Changed +- **Dictation progress on the microphone**: while a dictation is being recognized and polished, the microphone capsule fills from left to right in the brand colour instead of showing a spinner. The fill steps forward when recognition finishes, slows as it nears the end, and completes only when the text is inserted; the polish estimate adapts to how long polishing usually takes on this device. / **麦克风上的听写进度**：听写识别与润色期间，麦克风胶囊改为从左到右填充品牌色，取代原来的转圈。识别完成时进度向前跳一段，越接近结束走得越慢，只有文字真正插入时才填满；润色耗时的预估会按这台设备的实际情况自动校准。 - **Messaging shift to AI voice keyboard / voice double / agentic skills**: README and the product page now lead with “Say it. It’s typed. Speaks like you. Acts for you.” — the personalization story (learned style, three-tone replies) moves ahead of typing, clipboard skills are framed as agentic actions, and typing becomes the fallback capability. / **宣传定位转向 AI 语音键盘 / 嘴替 / Agentic 技能**：README 与产品页改用「开口即文字。会说你的话，会办你的事。」定位；个人化卖点（专属风格、三语气回复）提前，剪贴板技能表述升级为 Agentic 行动，打字退为兜底能力。 - **Product page redesign**: rebuild the GitHub Pages home as a dark product stage with the new Icon Composer app icon, the three-device hero visual, and real on-device videos and UI cards replacing hand-drawn mockups; add 2.1 selling points (personal style, three-tone replies, monthly usage calendar, EventKit skills export, guided trials and credit rewards), a TestFlight link, and self-hosted inline SVG icons; bump page references to 2.1.0. / **产品页重设计**：GitHub Pages 主页改为深色产品舞台，启用 Icon Composer 新图标与三端同框主视觉，以实机视频与真实界面卡片取代手绘示意；补齐 2.1 卖点（专属风格、三语气个性回复、月度用量日历、EventKit 技能写入、引导体验与积分奖励），新增 TestFlight 入口，图标改为自托管内联 SVG；页面版本号更新至 2.1.0。 - **Host-only analytics recording**: the keyboard extension no longer opens the analytics or keyboard-usage SQLite queues. The host app is the only process that records and uploads events, so hiding the keyboard cannot hold WAL locks and get killed as `0xdead10cc`. Keyboard-activated and per-keystroke usage events are no longer collected. / **仅主 App 记埋点**：键盘扩展不再打开统计与键盘用量 SQLite 队列；开库、记账与上传只在主 App 进行，收起键盘时不会因 WAL 锁被系统以 `0xdead10cc` 终止。不再采集「键盘激活」和逐次按键用量。 diff --git a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift index 385cca2..0acb64f 100644 --- a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift +++ b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift @@ -67,6 +67,7 @@ final class KeyboardFlowCoordinator { private var currentUtteranceId: UUID? private var currentStartDeadlineAt: TimeInterval? private var currentUtteranceRequest: FlowUtteranceRequest? + private let polishLatencyCalibration = DictationPolishLatencyCalibration() private var audioPrimeID: UUID? private var audioPrimeCancelTask: Task? private var editHostConfirmed = false @@ -471,6 +472,7 @@ final class KeyboardFlowCoordinator { if state.lastTranscript.isEmpty { state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") } + beginDictationProgressIfNeeded() startFlowResultWatchdog() traceState("adoptHostBusy.processing", extra: "session=\(sessionId)") } @@ -801,6 +803,7 @@ final class KeyboardFlowCoordinator { finishLocalCancel() return } + state.dictationProgress = nil let inserted = registerCancelledUtterance(utteranceID, kind: kind) if !inserted, isAwaitingFlowResult { state.phase = .processing @@ -1101,6 +1104,7 @@ final class KeyboardFlowCoordinator { onAIRecognitionStarted(currentUtteranceId) } else if currentUtteranceRequest?.isEdit != true { state.lastTranscript = ExtL10n.string("keyboard.flow.transcribing") + beginDictationProgressIfNeeded() } startFlowResultWatchdog() recomputeMicVoiceAvailability() @@ -2152,6 +2156,8 @@ final class KeyboardFlowCoordinator { state.lastTranscript = partial if result.resolvedUtteranceMode == .aiQuestion { onAITranscript(partial, result.utteranceId, result.status) + } else if result.status == .rawReady { + markDictationPolishing(characterCount: partial.count) } } default: @@ -2159,6 +2165,43 @@ final class KeyboardFlowCoordinator { } } + // MARK: - Dictation progress + + /// Starts the microphone progress fill for a dictation wait. Keeps an + /// existing timeline for the same utterance so re-adoption does not rewind. + private func beginDictationProgressIfNeeded() { + guard let utteranceID = currentUtteranceId, + currentUtteranceRequest?.isAIQuestion != true, + currentUtteranceRequest?.isEdit != true, + state.dictationProgress?.utteranceID != utteranceID else { return } + state.dictationProgress = DictationProgressTimeline( + utteranceID: utteranceID, + transcribingStartedAt: Date().timeIntervalSince1970 + ) + } + + private func markDictationPolishing(characterCount: Int) { + guard var timeline = state.dictationProgress, + timeline.polishingStartedAt == nil else { return } + timeline.markPolishing( + at: Date().timeIntervalSince1970, + characterCount: characterCount, + expectedSeconds: polishLatencyCalibration.expectedSeconds( + forCharacterCount: characterCount + ) + ) + state.dictationProgress = timeline + } + + private func recordDictationPolishLatency() { + guard let timeline = state.dictationProgress, + let polishStart = timeline.polishingStartedAt else { return } + polishLatencyCalibration.record( + observedSeconds: Date().timeIntervalSince1970 - polishStart, + characterCount: timeline.polishCharacterCount + ) + } + private func startFlowResultWatchdog( purpose: ResultWatchdogPurpose = .delivery ) { @@ -2212,6 +2255,7 @@ final class KeyboardFlowCoordinator { self.completeAIResult(result) return } + self.recordDictationPolishLatency() self.textInserter.handleFlowTranscript( TranscriptionDelivery( text: text, diff --git a/OSGKeyboardExt/Views/AIKeyboardView.swift b/OSGKeyboardExt/Views/AIKeyboardView.swift index 26caa6b..13f598a 100644 --- a/OSGKeyboardExt/Views/AIKeyboardView.swift +++ b/OSGKeyboardExt/Views/AIKeyboardView.swift @@ -1048,6 +1048,9 @@ struct AIKeyboardView: View { .regular.tint(microphoneTint).interactive(), in: Capsule() ) + if let timeline = activeDictationProgress { + dictationProgressFill(timeline) + } microphoneContent } .frame(width: primaryButtonWidth, height: primaryButtonHeight) @@ -1096,11 +1099,41 @@ struct AIKeyboardView: View { } } + /// Dictation wait rendered as a fill; AI phases keep their own spinner. + private var activeDictationProgress: DictationProgressTimeline? { + guard state.phase == .processing, !state.aiSession.isBusy else { return nil } + return state.dictationProgress + } + + /// Leading-edge fill in the brand colour. The real result swaps the + /// capsule to the solid accent success state, which reads as reaching 100 %. + private func dictationProgressFill(_ timeline: DictationProgressTimeline) -> some View { + TimelineView(.animation) { context in + let fraction = DictationProgressEstimator.fraction( + for: timeline, + at: context.date.timeIntervalSince1970 + ) + GeometryReader { proxy in + Rectangle() + .fill(palette.accent) + .frame(width: proxy.size.width * fraction) + } + } + .clipShape(Capsule()) + .allowsHitTesting(false) + .accessibilityHidden(true) + } + @ViewBuilder private var ordinaryMicrophoneContent: some View { switch state.phase { case .recording: waveform(color: OSGColor.recordingWaveform) + case .processing where activeDictationProgress != nil: + Image(systemName: "ellipsis") + .font(.system(size: 25, weight: .bold)) + .foregroundStyle(OSGColor.fixedLightContent) + .symbolEffect(.variableColor.iterative, options: .repeating) case .requestingPermissions, .processing: ProgressView().tint(OSGColor.fixedLightContent) case .error: @@ -1146,6 +1179,9 @@ struct AIKeyboardView: View { switch state.phase { case .recording: return palette.recordRed + case .processing where activeDictationProgress != nil: + // Unfilled track of the progress fill. + return palette.accent.opacity(0.35) case .requestingPermissions, .processing: return palette.recordRed.opacity(0.72) case .error, .denied: diff --git a/OSGKeyboardExtTests/DictationProgressTests.swift b/OSGKeyboardExtTests/DictationProgressTests.swift new file mode 100644 index 0000000..d00b004 --- /dev/null +++ b/OSGKeyboardExtTests/DictationProgressTests.swift @@ -0,0 +1,90 @@ +// DictationProgressTests.swift +// OSGKeyboard · Keyboard Extension Tests +// +// Staged microphone progress fill: monotonic, never completes on its own, +// steps forward at the ASR → polish milestone, and learns polish latency. + +@testable import OSGKeyboardShared +import XCTest + +final class DictationProgressTests: XCTestCase { + private let start: TimeInterval = 1_000 + + private func timeline(polishingAfter seconds: TimeInterval? = nil, + expected: TimeInterval = 3) -> DictationProgressTimeline { + var timeline = DictationProgressTimeline(utteranceID: UUID(), transcribingStartedAt: start) + if let seconds { + timeline.markPolishing(at: start + seconds, characterCount: 40, expectedSeconds: expected) + } + return timeline + } + + func testTranscribingStaysBelowCeiling() { + let ceiling = DictationProgressEstimator.transcribingCeiling + XCTAssertLessThan(DictationProgressEstimator.fraction(for: timeline(), at: start + 3), ceiling) + // Long waits converge onto the ceiling (exp underflows to exactly it), never past it. + XCTAssertLessThanOrEqual(DictationProgressEstimator.fraction(for: timeline(), at: start + 60), ceiling) + XCTAssertEqual(DictationProgressEstimator.fraction(for: timeline(), at: start), 0) + } + + func testFractionIsMonotonicAcrossMilestone() { + let subject = timeline(polishingAfter: 0.8) + var previous = -1.0 + for step in 0...600 { + let value = DictationProgressEstimator.fraction(for: subject, at: start + Double(step) * 0.05) + XCTAssertGreaterThanOrEqual(value, previous) + previous = value + } + } + + func testPolishingStepsPastFloorAndNeverCompletes() { + let subject = timeline(polishingAfter: 0.8) + let afterBlend = DictationProgressEstimator.fraction(for: subject, at: start + 0.8 + 0.3) + XCTAssertGreaterThanOrEqual(afterBlend, DictationProgressEstimator.polishingFloor) + let muchLater = DictationProgressEstimator.fraction(for: subject, at: start + 600) + XCTAssertLessThan(muchLater, 1) + XCTAssertLessThanOrEqual(muchLater, DictationProgressEstimator.polishingCeiling) + } + + func testRepeatedRawReadyKeepsFirstMilestone() { + var subject = timeline(polishingAfter: 1) + subject.markPolishing(at: start + 5, characterCount: 999, expectedSeconds: 30) + XCTAssertEqual(subject.polishingStartedAt, start + 1) + XCTAssertEqual(subject.polishCharacterCount, 40) + } + + func testCalibrationMovesTowardObservedLatencyWithinBounds() throws { + let suite = "DictationProgressTests.\(UUID().uuidString)" + let defaults = try XCTUnwrap(UserDefaults(suiteName: suite)) + defer { defaults.removePersistentDomain(forName: suite) } + let calibration = DictationPolishLatencyCalibration(defaults: defaults) + + let initial = calibration.expectedSeconds(forCharacterCount: 100) + calibration.record(observedSeconds: 8, characterCount: 100) + XCTAssertGreaterThan(calibration.expectedSeconds(forCharacterCount: 100), initial) + + for _ in 0..<50 { calibration.record(observedSeconds: 0.01, characterCount: 0) } + XCTAssertEqual( + calibration.overheadSeconds, + DictationPolishLatencyCalibration.overheadRange.lowerBound, + accuracy: 0.001 + ) + calibration.record(observedSeconds: -1, characterCount: 0) + XCTAssertEqual( + calibration.overheadSeconds, + DictationPolishLatencyCalibration.overheadRange.lowerBound, + accuracy: 0.001 + ) + } + + @MainActor + func testTimelineClearsWhenProcessingEnds() { + let state = KeyboardState() + state.phase = .processing + state.dictationProgress = timeline() + state.phase = .processing + XCTAssertNotNil(state.dictationProgress) + state.phase = .idle + XCTAssertNil(state.dictationProgress) + } +} diff --git a/OSGKeyboardShared/Models/DictationProgress.swift b/OSGKeyboardShared/Models/DictationProgress.swift new file mode 100644 index 0000000..d4e5505 --- /dev/null +++ b/OSGKeyboardShared/Models/DictationProgress.swift @@ -0,0 +1,113 @@ +// DictationProgress.swift +// OSGKeyboard · Shared +// +// Staged, time-estimated progress for the dictation wait after the user stops +// recording. The host only reports two milestones to the keyboard (`rawReady` +// when ASR finishes, `final` when polish finishes) and polish is a single +// non-streaming request, so the fill between milestones is an estimate: it +// eases toward a ceiling it never reaches, and only the real `final` result +// completes the bar. + +import Foundation + +/// Timestamps of one dictation wait, owned by `KeyboardState`. +public struct DictationProgressTimeline: Equatable, Sendable { + public let utteranceID: UUID + public let transcribingStartedAt: TimeInterval + public private(set) var polishingStartedAt: TimeInterval? + /// Raw transcript length when polish began; scales the expected duration. + public private(set) var polishCharacterCount: Int = 0 + public private(set) var expectedPolishSeconds: TimeInterval = 0 + + public init(utteranceID: UUID, transcribingStartedAt: TimeInterval) { + self.utteranceID = utteranceID + self.transcribingStartedAt = transcribingStartedAt + } + + /// Records the ASR → polish milestone once; later `rawReady` repeats are ignored. + public mutating func markPolishing( + at time: TimeInterval, + characterCount: Int, + expectedSeconds: TimeInterval + ) { + guard polishingStartedAt == nil else { return } + polishingStartedAt = time + polishCharacterCount = characterCount + expectedPolishSeconds = expectedSeconds + } +} + +public enum DictationProgressEstimator { + /// Asymptote of the ASR stage; the bar never passes it before `rawReady`. + static let transcribingCeiling = 0.25 + /// Time constant of the ASR stage curve (local drain is usually < 1 s). + static let transcribingTimeConstant: TimeInterval = 1.0 + /// Where the polish stage starts, so `rawReady` reads as a visible step. + static let polishingFloor = 0.35 + /// Asymptote of the polish stage; only the real result fills to 100 %. + static let polishingCeiling = 0.95 + /// Duration of the ease from the ASR curve into the polish curve. + static let stageBlendSeconds: TimeInterval = 0.25 + + /// Fill fraction in `0...polishingCeiling` for `timeline` at `now`. + public static func fraction( + for timeline: DictationProgressTimeline, + at now: TimeInterval + ) -> Double { + guard let polishStart = timeline.polishingStartedAt else { + return transcribingFraction(elapsed: now - timeline.transcribingStartedAt) + } + let atMilestone = transcribingFraction( + elapsed: polishStart - timeline.transcribingStartedAt + ) + let elapsed = max(0, now - polishStart) + // At `elapsed == expected` the polish curve covers ~86 % of its range. + let timeConstant = max(timeline.expectedPolishSeconds, 0.5) / 2 + let polishing = polishingFloor + + (polishingCeiling - polishingFloor) * (1 - exp(-elapsed / timeConstant)) + let blend = min(elapsed / stageBlendSeconds, 1) + return atMilestone + (polishing - atMilestone) * blend + } + + static func transcribingFraction(elapsed: TimeInterval) -> Double { + transcribingCeiling * (1 - exp(-max(0, elapsed) / transcribingTimeConstant)) + } +} + +/// Learns this device's polish latency so the estimate tracks the user's +/// provider and network. Stored in the extension's own defaults; it is a UI +/// hint, not a setting, so it is neither synced nor shared with the host. +public struct DictationPolishLatencyCalibration { + static let overheadKey = "keyboard.dictationProgress.polishOverheadSeconds" + static let defaultOverheadSeconds: TimeInterval = 1.5 + static let secondsPerCharacter: TimeInterval = 0.012 + static let overheadRange: ClosedRange = 0.5...15 + /// Weight of the newest observation in the moving average. + static let smoothing = 0.3 + + private let defaults: UserDefaults + + public init(defaults: UserDefaults = .standard) { + self.defaults = defaults + } + + public func expectedSeconds(forCharacterCount count: Int) -> TimeInterval { + overheadSeconds + Double(max(0, count)) * Self.secondsPerCharacter + } + + public func record(observedSeconds: TimeInterval, characterCount: Int) { + guard observedSeconds.isFinite, observedSeconds > 0 else { return } + let observedOverhead = observedSeconds + - Double(max(0, characterCount)) * Self.secondsPerCharacter + let blended = overheadSeconds * (1 - Self.smoothing) + observedOverhead * Self.smoothing + defaults.set( + min(max(blended, Self.overheadRange.lowerBound), Self.overheadRange.upperBound), + forKey: Self.overheadKey + ) + } + + var overheadSeconds: TimeInterval { + let stored = defaults.double(forKey: Self.overheadKey) + return stored > 0 ? stored : Self.defaultOverheadSeconds + } +} diff --git a/OSGKeyboardShared/Services/KeyboardState.swift b/OSGKeyboardShared/Services/KeyboardState.swift index de53caf..de5277a 100644 --- a/OSGKeyboardShared/Services/KeyboardState.swift +++ b/OSGKeyboardShared/Services/KeyboardState.swift @@ -89,7 +89,17 @@ public final class KeyboardState: ObservableObject { public var id: String { rawValue } } - @Published public var phase: Phase = .idle + @Published public var phase: Phase = .idle { + didSet { + // A timeline only describes the current processing wait. + if phase != .processing, dictationProgress != nil { + dictationProgress = nil + } + } + } + /// Dictation wait shown as the microphone capsule's progress fill. + /// Nil outside dictation processing (AI, edit, and cancellation waits). + @Published public var dictationProgress: DictationProgressTimeline? /// Active chrome. Forced to `.voice` while recording / processing. @Published public var surface: Surface = .voice /// Surface the user was on when auto mode switched them to `.voice` to show diff --git a/Tests/suite-manifest.json b/Tests/suite-manifest.json index 05c117a..def694b 100644 --- a/Tests/suite-manifest.json +++ b/Tests/suite-manifest.json @@ -174,6 +174,7 @@ "OSGKeyboardExtTests/CandidatePanelExpandTests", "OSGKeyboardExtTests/ClipboardSuggestionLifecycleTests", "OSGKeyboardExtTests/KeyboardStateTests", + "OSGKeyboardExtTests/DictationProgressTests", "OSGKeyboardExtTests/KeyboardUsageTypingTests", "OSGKeyboardExtTests/KeyboardSurfaceStateTests", "OSGKeyboardExtTests/KeyHitTestingTests", From f71f598ac78bf0cb535ef32cc54d13ba91882643 Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:05:29 +0200 Subject: [PATCH 04/19] fix(keyboard): stop showing the edit hint after every insertion --- CHANGELOG.md | 1 + OSGKeyboardExt/Services/KeyboardTextInserter.swift | 6 ------ 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c98d4d9..68de96e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Host-only analytics recording**: the keyboard extension no longer opens the analytics or keyboard-usage SQLite queues. The host app is the only process that records and uploads events, so hiding the keyboard cannot hold WAL locks and get killed as `0xdead10cc`. Keyboard-activated and per-keystroke usage events are no longer collected. / **仅主 App 记埋点**：键盘扩展不再打开统计与键盘用量 SQLite 队列；开库、记账与上传只在主 App 进行，收起键盘时不会因 WAL 锁被系统以 `0xdead10cc` 终止。不再采集「键盘激活」和逐次按键用量。 ### Fixed +- **Misleading edit hint after insertion**: stop showing “Hold to edit your last input” for 10 seconds after every insertion. Holding the microphone asks AI; editing the last input uses the pencil button beside it. / **插入后的误导性编辑提示**：每次插入后不再显示 10 秒的「长按编辑上一条」。长按麦克风是向 AI 提问，编辑上一条请使用旁边的铅笔按钮。 - **Faster managed dictation feedback**: begin the managed Volcengine streaming connection while the microphone touch primes audio, reuse it when recording starts, and keep the live transcript automatically scrolled to the newest recognition text. The early connection is reused only while it is fresh (under 10 seconds) and still open; otherwise it is closed and a new one opens, so an unused early connection can no longer make the next dictation fail. / **托管听写反馈提速**：在麦克风触摸预热音频时同步建立托管火山流式连接，录音开始后直接复用，并让实时转写自动滚动到最新识别文字。提前建立的连接仅在新鲜（10 秒内）且仍可用时复用，否则关闭并重新建立，未被使用的提前连接不会再导致下一次听写失败。 - **Recording kept when live recognition fails**: a streaming recognition error while you were still speaking ended the dictation immediately and discarded the audio. Recording now continues, and when you finish, the recorded audio is retried with batch recognition before any error is shown. After a cancelled or failed dictation, that batch retry also stopped working for the rest of the session; it now works again from the next dictation. / **实时识别失败时保留录音**：说话过程中流式识别一旦出错，本次听写会立即结束并丢弃录音。现在会继续录音，说完后先用批量识别重试已录下的音频，仍失败才提示错误。此前一次听写被取消或失败后，本次会话内的批量重试也会失效；现在从下一次听写起恢复正常。 - **Stalled cloud ASR losing the recording**: when the streaming recognizer went quiet and the result wait expired, the utterance ended as "no speech" even though the audio was captured. A stalled recognizer now counts as a failure, so the batch ASR fallback retries the recorded audio before giving up. / **云端识别卡住时丢失录音**：流式识别无响应、等待结果超时后，即便录音已采集，本次语音也会直接报「未检测到语音」。现在识别卡住按失败处理，会先用批量识别重试已录下的音频再放弃。 diff --git a/OSGKeyboardExt/Services/KeyboardTextInserter.swift b/OSGKeyboardExt/Services/KeyboardTextInserter.swift index 02fe5d1..1c103d5 100644 --- a/OSGKeyboardExt/Services/KeyboardTextInserter.swift +++ b/OSGKeyboardExt/Services/KeyboardTextInserter.swift @@ -547,12 +547,6 @@ final class KeyboardTextInserter { guard !Task.isCancelled else { return } state?.assistantInsertionSucceeded = false } - let hint = ExtL10n.string("keyboard.edit.hint.available") - editHintScheduler.show( - message: hint, - isPositive: true, - duration: .seconds(10) - ) } private func clearEditHintIfPositive() { From eaacc35cf7542d561ac69587e7e4f2c30a605e43 Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:05:35 +0200 Subject: [PATCH 05/19] feat(styles): learn the personal reply style incrementally Regeneration merges the on-device learned evidence with dictation and reply choices since the last generation instead of re-reading only the newest ~2,500 characters. It unlocks after 500 new characters, fades traits that stop getting support, and fully relearns styles without saved evidence. --- CHANGELOG.md | 1 + .../Views/PersonalReplyStyleSection.swift | 76 ++++- OSGKeyboard/Views/PolishStylesView.swift | 7 +- .../Views/ReleaseNotesScreenshotHarness.swift | 17 +- OSGKeyboard/en.lproj/Localizable.strings | 1 + OSGKeyboard/zh-Hans.lproj/Localizable.strings | 1 + .../Services/PersonalStyleLearningState.swift | 96 ++++++ .../Services/PolishStyleLearningService.swift | 276 ++++++++++++++++-- ...PersonalStyleLearningStateStoreTests.swift | 103 +++++++ .../PolishStyleLearningServiceTests.swift | 241 ++++++++++++++- Tests/suite-manifest.json | 1 + 11 files changed, 777 insertions(+), 43 deletions(-) create mode 100644 OSGKeyboardShared/Services/PersonalStyleLearningState.swift create mode 100644 OSGKeyboardTests/PersonalStyleLearningStateStoreTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 68de96e..1d72262 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Managed ASR connect crash**: `URLSessionWebSocketTask.sendPing` can invoke its completion twice when the task is cancelled mid-ping, so opening a managed Volcengine session could trap on the second continuation resume (SIGTRAP). The handler now passes through a once-only gate. / **托管语音识别建连崩溃**：`URLSessionWebSocketTask.sendPing` 在 ping 在途时被取消会回调两次，打开托管火山引擎会话时第二次恢复 continuation 会触发 SIGTRAP 断言。回调现经一次性闸门过滤。 - **Auto-reply storms from Universal Clipboard**: copying on a nearby Mac re-announces one copy under several pasteboard changeCounts, and each re-announcement re-armed auto reply/translate and the auto-reply guide — including on every later keyboard open, because re-ingesting the same text kept refreshing its 30-second hint window. A capture whose text is identical to the newest history entry is now treated as the same logical copy (bookkeeping only, no re-analysis), and auto actions additionally dedupe on a content fingerprint within a 30-second window. The same text fires once; copying something else and then re-copying the text still triggers. / **跨端剪贴板引发自动回复风暴**：在附近 Mac 上复制一次会被系统以多个 changeCount 重复宣告，且每次宣告都会重新武装自动回复/翻译与自动回复引导——由于同文本被重复入库并不断刷新 30 秒提示窗口，之后每次打开键盘也会再触发。现在，与最新历史条目文本相同的采集会被视为同一次复制（仅更新记账，不重新分析），自动动作再叠加内容指纹 + 30 秒窗口去重。同一文本只触发一次；复制其他内容后再重新复制该文本仍可触发。 - **Style card captions**: resolve each built-in style’s short description through the in-app language override, so switching App Language updates the caption instead of showing a localization key or the system language. / **风格卡片说明**：内置风格的说明小字改为走应用内语言设置；切换 App 语言后说明会跟着变，而不再显示本地化 key 或跟随系统语言。 +- **Personal reply style keeps learning**: regenerating used only the newest ~2,500 characters, so new dictation pushed earlier learning out instead of adding to it. Regeneration is now incremental: it merges what the style already learned (kept on this device only) with dictation and reply choices since the last generation, unlocks after 500 new characters, and the card shows the cumulative character count plus progress since the last generation. Traits that stop getting new support fade out over a few rounds. A style without saved learning (older builds or another device) is relearned in full and replaced. / **个人回复风格持续学习**：此前重新生成只使用最新约 2,500 字，新听写会把早期学习挤掉而非叠加。现在改为增量重新生成：把该风格已学到的特征（仅保存在本机）与上次生成后的新听写和回复选择合并，新增满 500 字即可重新生成；卡片显示累计学习字数与自上次生成后的新增进度。长期没有新证据支持的特征会在几轮后自然淘汰。没有本机学习记录的风格（旧版本或其他设备生成）会全量重新学习并替换。 ## [2.2.0] - 2026-09-18 diff --git a/OSGKeyboard/Views/PersonalReplyStyleSection.swift b/OSGKeyboard/Views/PersonalReplyStyleSection.swift index 3c07115..02d8aef 100644 --- a/OSGKeyboard/Views/PersonalReplyStyleSection.swift +++ b/OSGKeyboard/Views/PersonalReplyStyleSection.swift @@ -28,16 +28,24 @@ struct PersonalReplyStyleSection: View { @State private var generationTask: Task? @State private var generationID: UUID? @State private var editingPack: PolishStylePack? + /// Evidence from the generation under review; persisted only on save. + @State private var pendingLearning: PendingLearning? @State private var errorAlert: PersonalReplyStyleErrorAlert? + private struct PendingLearning { + let packID: String + let prior: PolishStyleLearningPriorEvidence + } + private let store = AppGroupStore() + private let learningState = PersonalStyleLearningStateStore.shared private let generator: LearnedStyleGenerator init( initialEditingPack: PolishStylePack? = nil, - generator: @escaping LearnedStyleGenerator = { corpus, replyExamples, language in + generator: @escaping LearnedStyleGenerator = { corpus, replyExamples, language, prior in try await PolishStyleLearningService(store: AppGroupStore()) - .generateStyle( + .learnStyle( from: corpus, replyExamples: replyExamples, outputLanguage: language, @@ -46,7 +54,8 @@ struct PersonalReplyStyleSection: View { ? PolishStyleLearningCorpusBuilder .testBuildEffectiveCharacterCount : PolishStyleLearningCorpusBuilder - .requiredEffectiveCharacterCount + .requiredEffectiveCharacterCount, + prior: prior ) } ) { @@ -64,7 +73,7 @@ struct PersonalReplyStyleSection: View { } } } - .sheet(item: $editingPack) { pack in + .sheet(item: $editingPack, onDismiss: { pendingLearning = nil }) { pack in PolishStyleEditorSheet( pack: pack, isNew: !catalog.entries.contains(where: { $0.id == pack.id }) @@ -107,8 +116,17 @@ struct PersonalReplyStyleSection: View { : PolishStyleLearningCorpusBuilder.requiredEffectiveCharacterCount } - private func isEligible(_ corpus: PolishStyleLearningCorpus) -> Bool { - corpus.effectiveCharacterCount >= minimumCharacterCount + /// Without a prior the full corpus gate applies; with one, only the + /// characters dictated since the last saved generation count. + private func isEligible( + _ corpus: PolishStyleLearningCorpus, + prior: PolishStyleLearningPriorEvidence? = nil + ) -> Bool { + guard let prior else { + return corpus.effectiveCharacterCount >= minimumCharacterCount + } + return corpus.examples(after: prior.learnedThrough).effectiveCharacterCount + >= PolishStyleLearningCorpusBuilder.incrementalEffectiveCharacterCount } private var learnedPack: PolishStylePack? { @@ -121,7 +139,8 @@ struct PersonalReplyStyleSection: View { _ pack: PolishStylePack, corpus: PolishStyleLearningCorpus ) -> some View { - let canRegenerate = isEligible(corpus) && !isGenerating + let prior = learningState.prior(for: pack) + let canRegenerate = isEligible(corpus, prior: prior) && !isGenerating let actionShape = RoundedRectangle(cornerRadius: Radius.medium, style: .continuous) return VStack(alignment: .leading, spacing: Spacing.md) { @@ -181,6 +200,22 @@ struct PersonalReplyStyleSection: View { Self.confidencePercentage(metadata.confidence) ) ) + if let prior { + Text( + AppL10n.format( + "personalReplyStyle.newSinceLast", + Int64( + corpus.examples(after: prior.learnedThrough) + .effectiveCharacterCount + ), + Int64( + PolishStyleLearningCorpusBuilder + .incrementalEffectiveCharacterCount + ) + ) + ) + .accessibilityIdentifier("personalReplyStyle.newSinceLast") + } } .font(TypeStyle.caption2) .foregroundStyle(palette.textTertiary) @@ -328,24 +363,31 @@ struct PersonalReplyStyleSection: View { from corpus: PolishStyleLearningCorpus, replacing existingPack: PolishStylePack? = nil ) { - guard isEligible(corpus), !isGenerating, generationTask == nil else { return } + let prior = existingPack.flatMap(learningState.prior(for:)) + guard isEligible(corpus, prior: prior), + !isGenerating, + generationTask == nil else { return } let id = UUID() generationID = id isGenerating = true generationTask = Task { @MainActor in defer { finishGeneration(id: id) } do { - let generated = try await generator( + let result = try await generator( corpus, ClipboardReplyFeedbackStore.shared.learningExamples(), - config.uiLanguage + config.uiLanguage, + prior ) try Task.checkCancellation() guard generationID == id, editingPack == nil else { return } + let generated = result.pack // Always let the user inspect and edit the learned prompt before - // it is saved, synced, or enabled. + // it is saved, synced, or enabled. Saving overwrites the existing + // style, including any manual edits to it. + let reviewPack: PolishStylePack if let existingPack { - editingPack = PolishStylePack( + reviewPack = PolishStylePack( id: existingPack.id, name: generated.name, prompt: generated.prompt, @@ -356,8 +398,12 @@ struct PersonalReplyStyleSection: View { updatedAt: Date() ) } else { - editingPack = generated + reviewPack = generated } + pendingLearning = result.prior.map { + PendingLearning(packID: reviewPack.id, prior: $0) + } + editingPack = reviewPack } catch { guard generationID == id, !Task.isCancelled, @@ -395,6 +441,10 @@ struct PersonalReplyStyleSection: View { store.setPolishStyleCatalog(catalog) store.setPersonalReplyStyleId(pack.id) enabledStyleID = store.personalReplyStyleId + if let pendingLearning, pendingLearning.packID == pack.id { + learningState.save(pendingLearning.prior, for: pack) + self.pendingLearning = nil + } pushSync() return true } catch { diff --git a/OSGKeyboard/Views/PolishStylesView.swift b/OSGKeyboard/Views/PolishStylesView.swift index 9fee57d..0c2979c 100644 --- a/OSGKeyboard/Views/PolishStylesView.swift +++ b/OSGKeyboard/Views/PolishStylesView.swift @@ -7,11 +7,14 @@ import OSGKeyboardShared import SwiftUI +/// The prior is nil for a first or full relearn; with one, the generator +/// learns incrementally from examples after `learnedThrough`. typealias LearnedStyleGenerator = @MainActor @Sendable ( PolishStyleLearningCorpus, [PolishStyleReplyLearningExample], - AppUILanguage -) async throws -> PolishStylePack + AppUILanguage, + PolishStyleLearningPriorEvidence? +) async throws -> PolishStyleLearningResult private struct PolishStyleErrorAlert { let title: String diff --git a/OSGKeyboard/Views/ReleaseNotesScreenshotHarness.swift b/OSGKeyboard/Views/ReleaseNotesScreenshotHarness.swift index 758d80f..9dd5bb8 100644 --- a/OSGKeyboard/Views/ReleaseNotesScreenshotHarness.swift +++ b/OSGKeyboard/Views/ReleaseNotesScreenshotHarness.swift @@ -81,10 +81,13 @@ struct PolishStylesScreenshotHarness: View { ) } else if simulatesGeneration { PersonalReplyStyleHost( - generator: { _, _, language in + generator: { _, _, language, _ in try await Task.sleep(for: .seconds(1.8)) - return ReleaseNotesScreenshotFixture.generatedStyle( - language: language + return PolishStyleLearningResult( + pack: ReleaseNotesScreenshotFixture.generatedStyle( + language: language + ), + prior: nil ) } ) @@ -115,7 +118,7 @@ private struct PolishStylesServiceUITestHarness: View { ZStack(alignment: .topTrailing) { if showsStyles { PersonalReplyStyleHost( - generator: { corpus, replyExamples, language in + generator: { corpus, replyExamples, language, prior in let client = PolishStylesUITestScriptedLLMClient( responses: ReleaseNotesScreenshotFixture.serviceResponses( language: language, @@ -129,7 +132,7 @@ private struct PolishStylesServiceUITestHarness: View { store: AppGroupStore(), client: client ) - .generateStyle( + .learnStyle( from: corpus, replyExamples: replyExamples, outputLanguage: language, @@ -138,7 +141,8 @@ private struct PolishStylesServiceUITestHarness: View { ? PolishStyleLearningCorpusBuilder .testBuildEffectiveCharacterCount : PolishStyleLearningCorpusBuilder - .requiredEffectiveCharacterCount + .requiredEffectiveCharacterCount, + prior: prior ) } ) @@ -456,6 +460,7 @@ private enum ReleaseNotesScreenshotFixture { store.setPolishStyleCatalog(PolishStyleCatalog()) store.setActivePolishStyleId(PolishStylePackCatalog.defaultID) store.setPersonalReplyStyleId("") + PersonalStyleLearningStateStore.shared.clear() } /// Seeds varied per-day dictation totals across the current month so the diff --git a/OSGKeyboard/en.lproj/Localizable.strings b/OSGKeyboard/en.lproj/Localizable.strings index 50a93be..473e8f9 100644 --- a/OSGKeyboard/en.lproj/Localizable.strings +++ b/OSGKeyboard/en.lproj/Localizable.strings @@ -715,6 +715,7 @@ "personalReplyStyle.generatedAt" = "Generated"; "personalReplyStyle.evidenceSummary" = "%lld ASR characters · %lld reply choices · %lld final edits"; "personalReplyStyle.confidence" = "Confidence: %lld%%"; +"personalReplyStyle.newSinceLast" = "New since last generation: %lld / %lld characters"; "personalReplyStyle.regenerate" = "Regenerate"; "personalReplyStyle.regenerating" = "Regenerating…"; "personalReplyStyle.privacy" = "Sent to your configured AI only when you generate. Review and edit before saving."; diff --git a/OSGKeyboard/zh-Hans.lproj/Localizable.strings b/OSGKeyboard/zh-Hans.lproj/Localizable.strings index 7228a9e..6830909 100644 --- a/OSGKeyboard/zh-Hans.lproj/Localizable.strings +++ b/OSGKeyboard/zh-Hans.lproj/Localizable.strings @@ -715,6 +715,7 @@ "personalReplyStyle.generatedAt" = "生成于"; "personalReplyStyle.evidenceSummary" = "%lld 个 ASR 字符 · %lld 次回复选择 · %lld 次最终改稿"; "personalReplyStyle.confidence" = "置信度：%lld%%"; +"personalReplyStyle.newSinceLast" = "自上次生成后新增 %lld / %lld 字"; "personalReplyStyle.regenerate" = "重新生成"; "personalReplyStyle.regenerating" = "重新生成中…"; "personalReplyStyle.privacy" = "仅在生成时发送给你配置的 AI，保存前可预览和修改。"; diff --git a/OSGKeyboardShared/Services/PersonalStyleLearningState.swift b/OSGKeyboardShared/Services/PersonalStyleLearningState.swift new file mode 100644 index 0000000..c256103 --- /dev/null +++ b/OSGKeyboardShared/Services/PersonalStyleLearningState.swift @@ -0,0 +1,96 @@ +// PersonalStyleLearningState.swift +// OSGKeyboard · Shared +// +// Device-local memory that lets a personal style be relearned incrementally: +// the compacted evidence of the last saved generation and the corpus it has +// already absorbed. Dictation history keeps only a rolling window, so this is +// the only place older learning survives. It is never synced — a device +// without a matching record simply relearns from scratch. + +import Foundation + +/// Compacted stage-one evidence carried into the next generation. +public struct PolishStyleLearningPriorEvidence: Codable, Equatable, Sendable { + public struct Domain: Codable, Equatable, Sendable { + public let traits: [PolishStyleLearningEvidence.Trait] + public let contradictions: [PolishStyleLearningEvidence.Contradiction] + } + + public let asr: Domain + public let reply: Domain + /// Newest learned example timestamp. Only later examples count as new. + public let learnedThrough: Date + public let cumulativeASRExampleCount: Int + public let cumulativeASREffectiveCharacterCount: Int + public let cumulativeReplyExampleCount: Int + public let cumulativeReplyFinalEditCount: Int +} + +/// A reviewed pack plus the evidence to persist once the user saves it. +public struct PolishStyleLearningResult: Sendable { + public let pack: PolishStylePack + /// Nil when the generator produced no reusable evidence (fixtures). + public let prior: PolishStyleLearningPriorEvidence? + + public init(pack: PolishStylePack, prior: PolishStyleLearningPriorEvidence?) { + self.pack = pack + self.prior = prior + } +} + +@MainActor +public final class PersonalStyleLearningStateStore { + public static let shared = PersonalStyleLearningStateStore() + + private struct Record: Codable { + let packID: String + let generatedAt: Date + let prior: PolishStyleLearningPriorEvidence + } + + private static let storageKey = "personalStyle.learningState.v1" + /// Cloud-merged metadata may round dates, so match within one second. + private static let generatedAtTolerance: TimeInterval = 1 + + private let defaults: UserDefaults? + + public init(defaults: UserDefaults? = AppGroup.defaultsIfAvailable) { + self.defaults = defaults + } + + /// Returns the prior only when it belongs to exactly this saved + /// generation. A pack replaced elsewhere (another device, an older build) + /// has a different `generatedAt` and falls back to full relearning. + public func prior(for pack: PolishStylePack) -> PolishStyleLearningPriorEvidence? { + guard let generatedAt = pack.learningMetadata?.generatedAt, + let record = load(), + record.packID == pack.id, + abs(record.generatedAt.timeIntervalSince(generatedAt)) + <= Self.generatedAtTolerance else { + return nil + } + return record.prior + } + + /// Only one personal style is learned at a time, so a save replaces any + /// earlier record. + public func save(_ prior: PolishStyleLearningPriorEvidence, for pack: PolishStylePack) { + guard let defaults, + let generatedAt = pack.learningMetadata?.generatedAt, + let data = try? JSONEncoder().encode( + Record(packID: pack.id, generatedAt: generatedAt, prior: prior) + ) else { + return + } + defaults.set(data, forKey: Self.storageKey) + } + + public func clear() { + defaults?.removeObject(forKey: Self.storageKey) + } + + private func load() -> Record? { + guard let data = defaults?.data(forKey: Self.storageKey) else { return nil } + return try? JSONDecoder().decode(Record.self, from: data) + } +} diff --git a/OSGKeyboardShared/Services/PolishStyleLearningService.swift b/OSGKeyboardShared/Services/PolishStyleLearningService.swift index 3833646..ef88e3a 100644 --- a/OSGKeyboardShared/Services/PolishStyleLearningService.swift +++ b/OSGKeyboardShared/Services/PolishStyleLearningService.swift @@ -91,6 +91,9 @@ public struct PolishStyleLearningEvidence: Codable, Equatable, Sendable { case replyFinalEdit case replyCrossContextSelection case replyAcceptance + /// A trait retained from the previous generation's accumulated + /// evidence, which is no longer present as raw examples. + case priorProfile } public struct Trait: Codable, Equatable, Sendable { @@ -147,6 +150,20 @@ public struct PolishStyleLearningCorpus: Equatable, Sendable { effectiveCharacterCount >= PolishStyleLearningCorpusBuilder.requiredEffectiveCharacterCount } + + /// Examples recorded after `date`, i.e. not yet absorbed by a prior + /// generation. + public func examples(after date: Date) -> PolishStyleLearningCorpus { + let newer = examples.filter { $0.createdAt > date } + return PolishStyleLearningCorpus( + examples: newer, + effectiveCharacterCount: newer.reduce(into: 0) { count, example in + count += PolishStyleLearningCorpusBuilder.effectiveCharacterCount( + in: example.prePolishText + ) + } + ) + } } public enum PolishStyleLearningCorpusBuilder { @@ -160,6 +177,10 @@ public enum PolishStyleLearningCorpusBuilder { /// still a real gate; the previous "0 / unlimited" bypass is gone. public static let testBuildEffectiveCharacterCount = 1_250 + /// New effective characters required since the last saved generation + /// before an incremental relearn is worth a request. + public static let incrementalEffectiveCharacterCount = 500 + /// Upper bound on effective characters included in a single /// training-corpus export. Even when the user has accumulated /// significantly more history than the unlock threshold, the @@ -459,8 +480,16 @@ public actor PolishStyleLearningService { let examples: [ReplyExamplePayload] } + private struct PriorInput: Codable { + let learnedThrough: Date + let asr: PolishStyleLearningPriorEvidence.Domain + let reply: PolishStyleLearningPriorEvidence.Domain + } + private struct EvidenceRequestPayload: Codable { let schemaVersion: Int + /// Omitted from the JSON entirely on a full (non-incremental) run. + let prior: PriorInput? let asr: ASRInput let reply: ReplyInput } @@ -488,7 +517,16 @@ public actor PolishStyleLearningService { private static let maximumEvidenceItemsPerDomain = 24 private static let maximumContradictionsPerDomain = 12 private static let maximumEvidenceFieldCharacters = 320 - private static let learningSchemaVersion = 3 + private static let learningSchemaVersion = 4 + /// Each relearn without fresh support shrinks a prior trait; below the + /// floor it is no longer sent, so unsupported traits fade out in a few + /// rounds without per-trait bookkeeping. + static let priorConfidenceDecay = 0.8 + static let priorRetentionFloor = 0.15 + static let maximumPriorTraitsPerDomain = 8 + static let maximumPriorContradictionsPerDomain = 6 + /// Keeps the prior well inside the 30,000-character request budget. + static let maximumPriorCharacters = 4_000 private static let generationOptions = LLMGenerationOptions( temperature: 0.1, topP: 0.9, @@ -513,7 +551,33 @@ public actor PolishStyleLearningService { minimumEffectiveCharacterCount: Int = PolishStyleLearningCorpusBuilder.requiredEffectiveCharacterCount ) async throws -> PolishStylePack { - let requiredCharacterCount = max(0, minimumEffectiveCharacterCount) + try await learnStyle( + from: corpus, + replyExamples: replyExamples, + outputLanguage: outputLanguage, + minimumEffectiveCharacterCount: minimumEffectiveCharacterCount + ).pack + } + + /// With a `prior`, learns incrementally: only examples recorded after + /// `prior.learnedThrough` are sent, merged with the decayed prior, and + /// the incremental gate replaces `minimumEffectiveCharacterCount`. + public func learnStyle( + from fullCorpus: PolishStyleLearningCorpus, + replyExamples allReplyExamples: [PolishStyleReplyLearningExample] = [], + outputLanguage: AppUILanguage, + minimumEffectiveCharacterCount: Int = + PolishStyleLearningCorpusBuilder.requiredEffectiveCharacterCount, + prior: PolishStyleLearningPriorEvidence? = nil + ) async throws -> PolishStyleLearningResult { + let corpus = prior.map { fullCorpus.examples(after: $0.learnedThrough) } + ?? fullCorpus + let replyExamples = prior.map { prior in + allReplyExamples.filter { $0.createdAt > prior.learnedThrough } + } ?? allReplyExamples + let requiredCharacterCount = prior == nil + ? max(0, minimumEffectiveCharacterCount) + : PolishStyleLearningCorpusBuilder.incrementalEffectiveCharacterCount let verifiedCharacterCount = corpus.examples.reduce(into: 0) { count, example in count += PolishStyleLearningCorpusBuilder.effectiveCharacterCount( in: example.prePolishText @@ -540,7 +604,8 @@ public actor PolishStyleLearningService { replyExamples: selectedReplyExamples, activeStyleID: configuration.activePolishStyleId, catalog: configuration.polishStyleCatalog, - outputLanguage: outputLanguage + outputLanguage: outputLanguage, + prior: prior.map(Self.decayedPrior) ) let service = PolishingService( store: configuration, @@ -554,21 +619,30 @@ public actor PolishStyleLearningService { requiresBestEffortASRCandidate: !selectedASRExamples.isEmpty, notifiesManagedCredits: notifiesManagedCredits ) + // Counts are cumulative across incremental generations so the card + // reflects everything this style has learned, not just this round. + let asrExampleCount = (prior?.cumulativeASRExampleCount ?? 0) + + selectedASRExamples.count + let asrEffectiveCharacterCount = (prior?.cumulativeASREffectiveCharacterCount ?? 0) + + selectedASRExamples.reduce(into: 0) { count, example in + count += PolishStyleLearningCorpusBuilder.effectiveCharacterCount( + in: example.prePolishText + ) + } + let replyExampleCount = (prior?.cumulativeReplyExampleCount ?? 0) + + selectedReplyExamples.count + let replyFinalEditCount = (prior?.cumulativeReplyFinalEditCount ?? 0) + + selectedReplyExamples.filter { Self.normalized($0.finalEdit) != nil }.count + let generatedAt = Date() let metadata = PolishStylePack.LearningMetadata( schemaVersion: Self.learningSchemaVersion, evidenceStatus: evidence.status.rawValue, confidence: evidence.confidence, - asrExampleCount: selectedASRExamples.count, - asrEffectiveCharacterCount: selectedASRExamples.reduce(into: 0) { count, example in - count += PolishStyleLearningCorpusBuilder.effectiveCharacterCount( - in: example.prePolishText - ) - }, - replyExampleCount: selectedReplyExamples.count, - replyFinalEditCount: selectedReplyExamples.filter { - Self.normalized($0.finalEdit) != nil - }.count, - generatedAt: Date() + asrExampleCount: asrExampleCount, + asrEffectiveCharacterCount: asrEffectiveCharacterCount, + replyExampleCount: replyExampleCount, + replyFinalEditCount: replyFinalEditCount, + generatedAt: generatedAt ) // Always synthesize from this operation's evidence. Low-confidence // profiles use their strongest candidate traits; only genuinely empty @@ -577,13 +651,30 @@ public actor PolishStyleLearningService { evidence: evidence, metadata: metadata ) - return try await synthesizeStyle( + let pack = try await synthesizeStyle( payload: synthesisPayload, service: service, learningMetadata: metadata, outputLanguage: outputLanguage, notifiesManagedCredits: notifiesManagedCredits ) + let newestLearnedExample = ( + selectedASRExamples.map(\.createdAt) + selectedReplyExamples.map(\.createdAt) + ).max() + let learnedThrough = [newestLearnedExample, prior?.learnedThrough] + .compactMap { $0 } + .max() ?? generatedAt + return PolishStyleLearningResult( + pack: pack, + prior: Self.compactPrior( + from: evidence, + learnedThrough: learnedThrough, + cumulativeASRExampleCount: asrExampleCount, + cumulativeASREffectiveCharacterCount: asrEffectiveCharacterCount, + cumulativeReplyExampleCount: replyExampleCount, + cumulativeReplyFinalEditCount: replyFinalEditCount + ) + ) } private func extractEvidence( @@ -687,7 +778,8 @@ public actor PolishStyleLearningService { replyExamples: [PolishStyleReplyLearningExample], activeStyleID: String, catalog: PolishStyleCatalog, - outputLanguage: AppUILanguage + outputLanguage: AppUILanguage, + prior: PolishStyleLearningPriorEvidence? = nil ) throws -> String { let activeStyle = PolishStylePackCatalog.resolve( id: activeStyleID, @@ -706,6 +798,9 @@ public actor PolishStyleLearningService { ) let payload = EvidenceRequestPayload( schemaVersion: learningSchemaVersion, + prior: prior.map { + PriorInput(learnedThrough: $0.learnedThrough, asr: $0.asr, reply: $0.reply) + }, asr: ASRInput( residualBaseline: reference( for: baselineStyle, @@ -896,9 +991,33 @@ public actor PolishStyleLearningService { The ASR domain may be empty only when asr.examples itself is empty. - Set status to "sufficient" only when total confidence is at least 0.5. + INCREMENTAL MERGE (only when the payload contains prior): + - prior is this user's previously learned profile, already decayed for + age. It summarizes older evidence that is no longer in the payload. + Unlike the contamination style prompts, it is evidence about the user. + asr.examples and reply.examples contain only newer observations. + - Return one merged profile covering prior and the newer examples. + - Keep every prior trait the newer examples do not contradict, at its + given confidence or lower. Never raise a prior trait without new + support. + - When newer examples support a prior trait, keep its name, add the new + support to its supportCount, and you may raise its confidence. + - When newer examples contradict a prior trait, lower its confidence and + add a contradictions entry; drop it only when the contradiction is + clear and repeated. Keep prior contradictions unless resolved. + - Cite retained prior traits with source priorProfile and the prior + supportCount. priorProfile ranks with asrRepeatedBefore and + replyCrossContextSelection: after user edits, before single + observations. + - Keep prior ASR and reply traits in their own domains. + - When over the trait limit, drop the lowest-confidence prior traits. + - status and confidence describe the merged profile. "insufficient" + requires overall and every trait confidence at most 0.35; + "sufficient" requires overall confidence at least 0.5. + Allowed source values: asrUserEdit, asrRepeatedBefore, asrObservedBefore, replyFinalEdit, - replyCrossContextSelection, replyAcceptance. + replyCrossContextSelection, replyAcceptance, priorProfile. Return this exact Codable shape: { @@ -1096,6 +1215,118 @@ public actor PolishStyleLearningService { return references } + // MARK: - Incremental prior + + /// Ages the prior before it is sent: every trait loses confidence, and + /// traits that fall below the retention floor are dropped. Decay is local + /// and deterministic rather than delegated to the model. + static func decayedPrior( + _ prior: PolishStyleLearningPriorEvidence + ) -> PolishStyleLearningPriorEvidence { + func decayed( + _ domain: PolishStyleLearningPriorEvidence.Domain + ) -> PolishStyleLearningPriorEvidence.Domain { + PolishStyleLearningPriorEvidence.Domain( + traits: domain.traits.compactMap { trait in + let confidence = trait.confidence * priorConfidenceDecay + guard confidence >= priorRetentionFloor else { return nil } + return PolishStyleLearningEvidence.Trait( + name: trait.name, + description: trait.description, + confidence: confidence, + supportCount: trait.supportCount + ) + }, + contradictions: domain.contradictions + ) + } + return PolishStyleLearningPriorEvidence( + asr: decayed(prior.asr), + reply: decayed(prior.reply), + learnedThrough: prior.learnedThrough, + cumulativeASRExampleCount: prior.cumulativeASRExampleCount, + cumulativeASREffectiveCharacterCount: prior.cumulativeASREffectiveCharacterCount, + cumulativeReplyExampleCount: prior.cumulativeReplyExampleCount, + cumulativeReplyFinalEditCount: prior.cumulativeReplyFinalEditCount + ) + } + + /// Reduces validated evidence to what the next relearn needs: the + /// strongest traits and a few contradictions per domain, within + /// `maximumPriorCharacters`. Evidence items are not kept; the next run + /// cites retained traits as `priorProfile`. + static func compactPrior( + from evidence: PolishStyleLearningEvidence, + learnedThrough: Date, + cumulativeASRExampleCount: Int, + cumulativeASREffectiveCharacterCount: Int, + cumulativeReplyExampleCount: Int, + cumulativeReplyFinalEditCount: Int + ) -> PolishStyleLearningPriorEvidence { + func strongest( + _ traits: [PolishStyleLearningEvidence.Trait] + ) -> [PolishStyleLearningEvidence.Trait] { + Array( + traits + .filter { $0.confidence >= priorRetentionFloor } + .sorted { $0.confidence > $1.confidence } + .prefix(maximumPriorTraitsPerDomain) + ) + } + + var asrTraits = strongest(evidence.asr.traits) + var replyTraits = strongest(evidence.reply.traits) + var asrContradictions = Array( + evidence.asr.contradictions.prefix(maximumPriorContradictionsPerDomain) + ) + var replyContradictions = Array( + evidence.reply.contradictions.prefix(maximumPriorContradictionsPerDomain) + ) + + func makePrior() -> PolishStyleLearningPriorEvidence { + PolishStyleLearningPriorEvidence( + asr: .init(traits: asrTraits, contradictions: asrContradictions), + reply: .init(traits: replyTraits, contradictions: replyContradictions), + learnedThrough: learnedThrough, + cumulativeASRExampleCount: cumulativeASRExampleCount, + cumulativeASREffectiveCharacterCount: cumulativeASREffectiveCharacterCount, + cumulativeReplyExampleCount: cumulativeReplyExampleCount, + cumulativeReplyFinalEditCount: cumulativeReplyFinalEditCount + ) + } + + var prior = makePrior() + // Over budget: shed contradictions first, then the weakest trait + // across both domains, until the prior fits. + while priorCharacterCount(prior) > maximumPriorCharacters { + if !replyContradictions.isEmpty { + replyContradictions.removeLast() + } else if !asrContradictions.isEmpty { + asrContradictions.removeLast() + } else if let weakestASR = asrTraits.last, + weakestASR.confidence <= (replyTraits.last?.confidence ?? .infinity) { + asrTraits.removeLast() + } else if !replyTraits.isEmpty { + replyTraits.removeLast() + } else { + break + } + prior = makePrior() + } + return prior + } + + static func priorCharacterCount(_ prior: PolishStyleLearningPriorEvidence) -> Int { + let encoder = JSONEncoder() + encoder.outputFormatting = [.sortedKeys, .withoutEscapingSlashes] + let domains = [prior.asr, prior.reply] + guard let data = try? encoder.encode(domains), + let text = String(data: data, encoding: .utf8) else { + return 0 + } + return text.count + } + private static func boundedExample( _ example: PolishStyleLearningExample ) -> PolishStyleLearningExample { @@ -1303,7 +1534,8 @@ public actor PolishStyleLearningService { allowedSources: [ .asrUserEdit, .asrRepeatedBefore, - .asrObservedBefore + .asrObservedBefore, + .priorProfile ] ), isValid( @@ -1311,7 +1543,8 @@ public actor PolishStyleLearningService { allowedSources: [ .replyFinalEdit, .replyCrossContextSelection, - .replyAcceptance + .replyAcceptance, + .priorProfile ] ) else { return false @@ -1369,7 +1602,8 @@ public actor PolishStyleLearningService { switch item.source { case .asrRepeatedBefore, .replyCrossContextSelection: return item.supportCount >= 2 - case .asrUserEdit, .asrObservedBefore, .replyFinalEdit, .replyAcceptance: + case .asrUserEdit, .asrObservedBefore, .replyFinalEdit, .replyAcceptance, + .priorProfile: return true } } @@ -1388,7 +1622,7 @@ public actor PolishStyleLearningService { switch source { case .asrUserEdit, .replyFinalEdit: return 0 - case .asrRepeatedBefore, .replyCrossContextSelection: + case .asrRepeatedBefore, .replyCrossContextSelection, .priorProfile: return 1 case .asrObservedBefore, .replyAcceptance: return 2 diff --git a/OSGKeyboardTests/PersonalStyleLearningStateStoreTests.swift b/OSGKeyboardTests/PersonalStyleLearningStateStoreTests.swift new file mode 100644 index 0000000..48a495b --- /dev/null +++ b/OSGKeyboardTests/PersonalStyleLearningStateStoreTests.swift @@ -0,0 +1,103 @@ +// PersonalStyleLearningStateStoreTests.swift +// OSGKeyboard · Tests +// +// Verifies the device-local prior only applies to the exact saved generation. + +@testable import OSGKeyboardShared +import XCTest + +@MainActor +final class PersonalStyleLearningStateStoreTests: XCTestCase { + private var suiteName: String! + private var defaults: UserDefaults! + private var store: PersonalStyleLearningStateStore! + + override func setUp() { + super.setUp() + suiteName = "com.osgkeyboard.personal-style-state.\(UUID().uuidString)" + defaults = UserDefaults(suiteName: suiteName)! + defaults.removePersistentDomain(forName: suiteName) + store = PersonalStyleLearningStateStore(defaults: defaults) + } + + override func tearDown() { + defaults.removePersistentDomain(forName: suiteName) + super.tearDown() + } + + func testSavedPriorMatchesSamePackAndGeneration() { + let pack = Self.pack(id: "user.style", generatedAt: Date(timeIntervalSince1970: 100)) + store.save(Self.prior, for: pack) + + XCTAssertEqual(store.prior(for: pack), Self.prior) + } + + func testPriorSurvivesManualEditThatKeepsMetadata() { + let pack = Self.pack(id: "user.style", generatedAt: Date(timeIntervalSince1970: 100)) + store.save(Self.prior, for: pack) + var edited = pack + edited.prompt = "# 角色\n手动修改\n# 风格边界\n自然\n# 示例\n输入 → 输出" + + XCTAssertEqual(store.prior(for: edited), Self.prior) + } + + func testPriorIsIgnoredForAnotherGenerationOrPack() { + store.save( + Self.prior, + for: Self.pack(id: "user.style", generatedAt: Date(timeIntervalSince1970: 100)) + ) + + XCTAssertNil( + store.prior( + for: Self.pack(id: "user.style", generatedAt: Date(timeIntervalSince1970: 500)) + ) + ) + XCTAssertNil( + store.prior( + for: Self.pack(id: "user.other", generatedAt: Date(timeIntervalSince1970: 100)) + ) + ) + XCTAssertNil(store.prior(for: PolishStylePack(id: "user.style", name: "Legacy", prompt: "p"))) + } + + func testClearRemovesPrior() { + let pack = Self.pack(id: "user.style", generatedAt: Date(timeIntervalSince1970: 100)) + store.save(Self.prior, for: pack) + store.clear() + + XCTAssertNil(store.prior(for: pack)) + } + + private static let prior = PolishStyleLearningPriorEvidence( + asr: .init( + traits: [ + .init(name: "简短直接", description: "反复使用短句", confidence: 0.7, supportCount: 3) + ], + contradictions: [] + ), + reply: .init(traits: [], contradictions: []), + learnedThrough: Date(timeIntervalSince1970: 90), + cumulativeASRExampleCount: 5, + cumulativeASREffectiveCharacterCount: 2_500, + cumulativeReplyExampleCount: 0, + cumulativeReplyFinalEditCount: 0 + ) + + private static func pack(id: String, generatedAt: Date) -> PolishStylePack { + PolishStylePack( + id: id, + name: "我的说话风格", + prompt: "# 角色\n自然\n# 风格边界\n自然\n# 示例\n输入 → 输出", + learningMetadata: PolishStylePack.LearningMetadata( + schemaVersion: 4, + evidenceStatus: "sufficient", + confidence: 0.8, + asrExampleCount: 5, + asrEffectiveCharacterCount: 2_500, + replyExampleCount: 0, + replyFinalEditCount: 0, + generatedAt: generatedAt + ) + ) + } +} diff --git a/OSGKeyboardTests/PolishStyleLearningServiceTests.swift b/OSGKeyboardTests/PolishStyleLearningServiceTests.swift index eea1a12..4497b7a 100644 --- a/OSGKeyboardTests/PolishStyleLearningServiceTests.swift +++ b/OSGKeyboardTests/PolishStyleLearningServiceTests.swift @@ -288,7 +288,7 @@ final class PolishStyleLearningServiceTests: XCTestCase { XCTAssertTrue(client.requests.allSatisfy { $0.options?.maxTokens == 4_096 }) let metadata = try XCTUnwrap(generated.learningMetadata) - XCTAssertEqual(metadata.schemaVersion, 3) + XCTAssertEqual(metadata.schemaVersion, 4) XCTAssertEqual(metadata.evidenceStatus, "sufficient") XCTAssertEqual(metadata.confidence, 0.86) XCTAssertEqual(metadata.asrExampleCount, 1) @@ -342,6 +342,191 @@ final class PolishStyleLearningServiceTests: XCTestCase { XCTAssertFalse(client.requests[1].text.contains("newest-marker")) } + // MARK: - Incremental learning + + func testFullLearningReturnsPriorThroughNewestLearnedExample() async throws { + let newest = Date(timeIntervalSince1970: 3_000) + let corpus = PolishStyleLearningCorpus( + examples: [ + PolishStyleLearningExample( + prePolishText: String(repeating: "测试语料", count: 625), + finalText: "完整语料", + polishStyleID: nil, + createdAt: newest + ) + ], + effectiveCharacterCount: 2_500 + ) + let client = StyleLearningCapturingClient( + responses: [Self.sufficientEvidenceResponse, Self.generatedStyleResponse] + ) + let service = PolishStyleLearningService(store: store, client: client) + + let result = try await service.learnStyle(from: corpus, outputLanguage: .chinese) + + XCTAssertFalse(client.requests[0].text.contains(#""prior":"#)) + let prior = try XCTUnwrap(result.prior) + XCTAssertEqual(prior.learnedThrough, newest) + XCTAssertEqual(prior.asr.traits.map(\.name), ["简短直接"]) + XCTAssertEqual(prior.reply.traits.map(\.name), ["轻松回复"]) + XCTAssertEqual(prior.cumulativeASREffectiveCharacterCount, 2_500) + } + + func testIncrementalLearningSendsOnlyNewExamplesWithDecayedPrior() async throws { + let learnedThrough = Date(timeIntervalSince1970: 2_000) + let newest = Date(timeIntervalSince1970: 3_000) + let corpus = PolishStyleLearningCorpus( + examples: [ + PolishStyleLearningExample( + prePolishText: String(repeating: "旧", count: 3_000), + finalText: "old-marker", + polishStyleID: nil, + createdAt: Date(timeIntervalSince1970: 1_000) + ), + PolishStyleLearningExample( + prePolishText: String(repeating: "新", count: 600), + finalText: "new-marker", + polishStyleID: nil, + createdAt: newest + ) + ], + effectiveCharacterCount: 3_600 + ) + let replyExamples = [ + PolishStyleReplyLearningExample( + receivedMessage: "old-reply-marker", + ordinaryCandidate: "普通候选", + selection: .ordinary, + createdAt: Date(timeIntervalSince1970: 1_500) + ), + PolishStyleReplyLearningExample( + receivedMessage: "new-reply-marker", + ordinaryCandidate: "普通候选", + selection: .ordinary, + finalEdit: "用户改稿", + createdAt: Date(timeIntervalSince1970: 2_500) + ) + ] + let client = StyleLearningCapturingClient( + responses: [Self.incrementalEvidenceResponse, Self.generatedStyleResponse] + ) + let service = PolishStyleLearningService(store: store, client: client) + + let result = try await service.learnStyle( + from: corpus, + replyExamples: replyExamples, + outputLanguage: .chinese, + prior: Self.prior(learnedThrough: learnedThrough) + ) + + let extractor = client.requests[0] + XCTAssertTrue(extractor.prompt.contains("INCREMENTAL MERGE")) + XCTAssertTrue(extractor.text.contains(#""prior":"#)) + XCTAssertTrue(extractor.text.contains("new-marker")) + XCTAssertFalse(extractor.text.contains("old-marker")) + XCTAssertTrue(extractor.text.contains("new-reply-marker")) + XCTAssertFalse(extractor.text.contains("old-reply-marker")) + // 0.5 decays to 0.4; the 0.15 trait falls below the floor and is dropped. + XCTAssertTrue(extractor.text.contains(#""confidence":0.4"#)) + XCTAssertTrue(extractor.text.contains("早期简短直接")) + XCTAssertFalse(extractor.text.contains("即将淘汰")) + + let metadata = try XCTUnwrap(result.pack.learningMetadata) + XCTAssertEqual(metadata.asrEffectiveCharacterCount, 2_500 + 600) + XCTAssertEqual(metadata.asrExampleCount, 10 + 1) + XCTAssertEqual(metadata.replyExampleCount, 4 + 1) + XCTAssertEqual(metadata.replyFinalEditCount, 2 + 1) + let nextPrior = try XCTUnwrap(result.prior) + XCTAssertEqual(nextPrior.learnedThrough, newest) + XCTAssertEqual(nextPrior.cumulativeASREffectiveCharacterCount, 3_100) + } + + func testIncrementalLearningRequiresFiveHundredNewCharacters() async { + let corpus = PolishStyleLearningCorpus( + examples: [ + PolishStyleLearningExample( + prePolishText: String(repeating: "旧", count: 3_000), + finalText: "旧", + polishStyleID: nil, + createdAt: Date(timeIntervalSince1970: 1_000) + ), + PolishStyleLearningExample( + prePolishText: String(repeating: "新", count: 100), + finalText: "新", + polishStyleID: nil, + createdAt: Date(timeIntervalSince1970: 3_000) + ) + ], + effectiveCharacterCount: 3_100 + ) + let client = StyleLearningCapturingClient(response: "{}") + let service = PolishStyleLearningService(store: store, client: client) + + do { + _ = try await service.learnStyle( + from: corpus, + outputLanguage: .chinese, + prior: Self.prior(learnedThrough: Date(timeIntervalSince1970: 2_000)) + ) + XCTFail("Expected the incremental gate to count only new characters") + } catch let error as PolishStyleLearningError { + XCTAssertEqual(error, .insufficientCorpus(required: 500, actual: 100)) + } catch { + XCTFail("Unexpected error: \(error)") + } + XCTAssertTrue(client.requests.isEmpty) + } + + func testIncrementalEvidenceAcceptsPriorProfileSource() throws { + let evidence = try PolishStyleLearningService.parseEvidence( + Self.incrementalEvidenceResponse + ) + + XCTAssertEqual(evidence.asr.evidence.first?.source, .priorProfile) + XCTAssertEqual(evidence.reply.evidence.first?.source, .priorProfile) + } + + func testCompactPriorKeepsStrongestTraitsWithinBudget() { + let traits = (0..<12).map { index in + PolishStyleLearningEvidence.Trait( + name: "trait-\(index)", + // 500 characters each, so eight traits overflow the budget. + description: String(repeating: "描述", count: 250), + confidence: 0.2 + Double(index) * 0.05, + supportCount: index + 1 + ) + } + let evidence = PolishStyleLearningEvidence( + status: .sufficient, + confidence: 0.8, + asr: .init(traits: traits, evidence: [], contradictions: []), + reply: .init(traits: [], evidence: [], contradictions: []) + ) + + let prior = PolishStyleLearningService.compactPrior( + from: evidence, + learnedThrough: Date(), + cumulativeASRExampleCount: 1, + cumulativeASREffectiveCharacterCount: 1, + cumulativeReplyExampleCount: 0, + cumulativeReplyFinalEditCount: 0 + ) + + XCTAssertLessThan( + prior.asr.traits.count, + PolishStyleLearningService.maximumPriorTraitsPerDomain + ) + XCTAssertLessThanOrEqual( + PolishStyleLearningService.priorCharacterCount(prior), + PolishStyleLearningService.maximumPriorCharacters + ) + XCTAssertEqual(prior.asr.traits.first?.name, "trait-11") + XCTAssertEqual( + prior.asr.traits.map(\.confidence), + prior.asr.traits.map(\.confidence).sorted(by: >) + ) + } + func testServiceRecomputesReadinessInsteadOfTrustingCallerCount() async { let corpus = PolishStyleLearningCorpus( examples: [ @@ -761,6 +946,60 @@ final class PolishStyleLearningServiceTests: XCTestCase { } """## + private static let incrementalEvidenceResponse = ##""" + { + "status":"sufficient", + "confidence":0.8, + "asr":{ + "traits":[ + {"name":"早期简短直接","description":"早期画像与新样本都显示简短直接","confidence":0.6,"supportCount":5} + ], + "evidence":[ + {"source":"priorProfile","summary":"早期画像保留简短直接","supportCount":4}, + {"source":"asrObservedBefore","summary":"新样本继续使用短句","supportCount":1} + ], + "contradictions":[] + }, + "reply":{ + "traits":[ + {"name":"轻松回复","description":"早期画像中的轻松回复偏好","confidence":0.5,"supportCount":2} + ], + "evidence":[ + {"source":"priorProfile","summary":"早期画像保留轻松回复","supportCount":2} + ], + "contradictions":[] + } + } + """## + + private static func prior(learnedThrough: Date) -> PolishStyleLearningPriorEvidence { + PolishStyleLearningPriorEvidence( + asr: .init( + traits: [ + .init( + name: "早期简短直接", + description: "早期语料反复出现短句", + confidence: 0.5, + supportCount: 4 + ), + .init( + name: "即将淘汰", + description: "多轮没有新支持的旧特征", + confidence: 0.15, + supportCount: 1 + ) + ], + contradictions: [] + ), + reply: .init(traits: [], contradictions: []), + learnedThrough: learnedThrough, + cumulativeASRExampleCount: 10, + cumulativeASREffectiveCharacterCount: 2_500, + cumulativeReplyExampleCount: 4, + cumulativeReplyFinalEditCount: 2 + ) + } + private static let insufficientEvidenceResponse = ##""" { "status":"insufficient", diff --git a/Tests/suite-manifest.json b/Tests/suite-manifest.json index def694b..ffa9677 100644 --- a/Tests/suite-manifest.json +++ b/Tests/suite-manifest.json @@ -81,6 +81,7 @@ "OSGKeyboardTests/PolishStylePackTests", "OSGKeyboardTests/PolishStyleCorpusExportStoreTests", "OSGKeyboardTests/PolishStyleLearningServiceTests", + "OSGKeyboardTests/PersonalStyleLearningStateStoreTests", "OSGKeyboardTests/PolishStylesL10nTests", "OSGKeyboardTests/LLMClientTests", "OSGKeyboardTests/TranscriptLanguageDetectorTests", From 03bfbbe08b075813542737851ea2f39ff438d8d2 Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:06:02 +0200 Subject: [PATCH 06/19] feat(account): add QR invitation poster and 8-character invitation codes - Invite opens an appearance-matched poster with the app icon, inviter name, QR code, and short code; tap for a full-brightness QR view; save (add-only Photos access), share, or copy the link. Mac shows the poster in a popover and can save it as PNG. - Accounts get look-alike-free 8-character codes. "I have a code" accepts a code, link, or pasted message during the server-defined binding window. - Permanently rejected redemptions show a specific message and drop the stored code instead of retrying on every sign-in. --- CHANGELOG.md | 6 + OSGKeyboard/Info.plist | 2 + .../Utilities/ScreenBrightnessBoost.swift | 88 +++++ .../Views/Account/AccountCenterModels.swift | 163 ++++++++- .../Account/AccountCenterUITestHarness.swift | 15 +- .../Views/Account/AccountCenterView.swift | 78 ++--- .../Views/Account/AccountRewardsCard.swift | 50 ++- .../Account/AccountSessionCoordinator.swift | 112 +++++- OSGKeyboard/Views/Account/InvitePoster.swift | 329 +++++++++++++++++ .../Account/InviteQRFullScreenView.swift | 62 ++++ .../Views/Account/InviteShareSheet.swift | 306 ++++++++++++++++ .../Views/Account/ReferralCodeEntry.swift | 288 +++++++++++++++ OSGKeyboard/en.lproj/InfoPlist.strings | 1 + OSGKeyboard/en.lproj/Localizable.strings | 29 ++ OSGKeyboard/zh-Hans.lproj/InfoPlist.strings | 1 + OSGKeyboard/zh-Hans.lproj/Localizable.strings | 29 ++ OSGKeyboardMac/MacAccountView.swift | 165 +++++++-- .../DesignSystem/ColorLibrary.swift | 7 + OSGKeyboardShared/en.lproj/Shared.strings | 15 + .../zh-Hans.lproj/Shared.strings | 15 + .../AccountCenterViewModelTests.swift | 331 ++++++++++++++++++ OSGKeyboardTests/InvitePosterTests.swift | 123 +++++++ OSGKeyboardTests/ReferralProfileTests.swift | 53 +++ .../ScreenBrightnessBoostTests.swift | 81 +++++ Tests/suite-manifest.json | 4 +- project.yml | 2 + 26 files changed, 2269 insertions(+), 86 deletions(-) create mode 100644 OSGKeyboard/Utilities/ScreenBrightnessBoost.swift create mode 100644 OSGKeyboard/Views/Account/InvitePoster.swift create mode 100644 OSGKeyboard/Views/Account/InviteQRFullScreenView.swift create mode 100644 OSGKeyboard/Views/Account/InviteShareSheet.swift create mode 100644 OSGKeyboard/Views/Account/ReferralCodeEntry.swift create mode 100644 OSGKeyboardTests/InvitePosterTests.swift create mode 100644 OSGKeyboardTests/ScreenBrightnessBoostTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d72262..c277e06 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- **QR invitation poster**: the Invite button now opens a minimal poster that automatically matches light or dark appearance, uses the current App Icon, names the inviter (falling back to “OSG User”), and shows the QR code plus the short invitation code in Chinese or English. Tap the poster to show the code full screen at full brightness for face-to-face scanning; your brightness returns when you close it. You can also save the poster to Photos (add-only access), share the image, or copy the link. On Mac, the account page shows the same poster in a popover and can save it as a PNG. / **二维码邀请海报**：「邀请」按钮会打开自动适配黑白外观的极简海报，使用当前 App Icon，显示邀请人用户名（未设置时为「OSG 用户」），并以中英文展示二维码与短邀请码；点海报可全屏显示二维码并自动调到最亮，方便当面扫码，关闭后恢复原亮度；也可保存到相册（仅申请添加权限）、分享图片或复制链接。Mac 账户页以弹出面板展示同款海报，并可存为 PNG。 +- **8-character invitation codes**: every account now has a short code such as `K7M2-Q9XA`, printed on the poster, the Mac poster popover, and the invitation landing page. The code skips look-alike characters (0/O, 1/I/L), so it can be read aloud. Tap “I have a code” to type it, or paste an invitation link or a whole chat message instead; the clipboard is never read on its own. A friend who installed from the App Store therefore still gets credited. For the first 7 days after registering, and until an inviter is bound, the home card shows “Enter a code, or invite more friends for credits” with the “I have a code” button, and Invite moves to the bottom of that sheet; the window follows the server’s campaign deadline. The landing page copies the link and opens the App Store in one tap, and keeps the link-open paths that older app builds rely on. / **8 位邀请码**：每个账号都有一个形如 `K7M2-Q9XA` 的短邀请码，印在海报、Mac 海报面板与邀请落地页上；不含易混字符（0/O、1/I/L），念出来也不会输错。点「我有邀请码」即可输入，也可直接粘贴邀请链接或整段聊天消息；App 不会主动读取剪贴板。好友从 App Store 安装后仍能计入邀请。注册后 7 天内且尚未绑定邀请人时，首页卡片显示「输入邀请码，或邀请更多好友获取积分」与「我有邀请码」按钮，「邀请」入口移到弹窗底部；期限以服务器活动截止时间为准。落地页一键「复制邀请并下载」，并保留旧版本 App 依赖的链接打开方式。 + ### Changed - **Dictation progress on the microphone**: while a dictation is being recognized and polished, the microphone capsule fills from left to right in the brand colour instead of showing a spinner. The fill steps forward when recognition finishes, slows as it nears the end, and completes only when the text is inserted; the polish estimate adapts to how long polishing usually takes on this device. / **麦克风上的听写进度**：听写识别与润色期间，麦克风胶囊改为从左到右填充品牌色，取代原来的转圈。识别完成时进度向前跳一段，越接近结束走得越慢，只有文字真正插入时才填满；润色耗时的预估会按这台设备的实际情况自动校准。 - **Messaging shift to AI voice keyboard / voice double / agentic skills**: README and the product page now lead with “Say it. It’s typed. Speaks like you. Acts for you.” — the personalization story (learned style, three-tone replies) moves ahead of typing, clipboard skills are framed as agentic actions, and typing becomes the fallback capability. / **宣传定位转向 AI 语音键盘 / 嘴替 / Agentic 技能**：README 与产品页改用「开口即文字。会说你的话，会办你的事。」定位；个人化卖点（专属风格、三语气回复）提前，剪贴板技能表述升级为 Agentic 行动，打字退为兜底能力。 @@ -15,6 +19,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed - **Misleading edit hint after insertion**: stop showing “Hold to edit your last input” for 10 seconds after every insertion. Holding the microphone asks AI; editing the last input uses the pencil button beside it. / **插入后的误导性编辑提示**：每次插入后不再显示 10 秒的「长按编辑上一条」。长按麦克风是向 AI 提问，编辑上一条请使用旁边的铅笔按钮。 +- **Saving invitation posters**: declare add-only Photos access in the built app and preflight the privacy description before calling PhotoKit, preventing Save Image from terminating the app when the generated project configuration is stale. / **保存邀请海报**：在 App 构建配置中声明“仅添加照片”权限，并在调用 PhotoKit 前检查权限说明，避免生成的工程配置过期时点击“保存图片”导致 App 被系统终止。 +- **Invitation redemption retrying forever**: an invitation the server permanently rejected (unknown code, self-referral, already bound, or binding window expired) stayed stored and was retried on every sign-in. These rejections now show a specific message and drop the stored code; network failures still retry. / **邀请兑换无限重试**：服务器明确拒绝的邀请（邀请码无效、邀请自己、已绑定或超出绑定期限）会一直保留并在每次登录时重试。现在这些情况会给出具体提示并清除暂存的邀请码；网络失败仍会重试。 - **Faster managed dictation feedback**: begin the managed Volcengine streaming connection while the microphone touch primes audio, reuse it when recording starts, and keep the live transcript automatically scrolled to the newest recognition text. The early connection is reused only while it is fresh (under 10 seconds) and still open; otherwise it is closed and a new one opens, so an unused early connection can no longer make the next dictation fail. / **托管听写反馈提速**：在麦克风触摸预热音频时同步建立托管火山流式连接，录音开始后直接复用，并让实时转写自动滚动到最新识别文字。提前建立的连接仅在新鲜（10 秒内）且仍可用时复用，否则关闭并重新建立，未被使用的提前连接不会再导致下一次听写失败。 - **Recording kept when live recognition fails**: a streaming recognition error while you were still speaking ended the dictation immediately and discarded the audio. Recording now continues, and when you finish, the recorded audio is retried with batch recognition before any error is shown. After a cancelled or failed dictation, that batch retry also stopped working for the rest of the session; it now works again from the next dictation. / **实时识别失败时保留录音**：说话过程中流式识别一旦出错，本次听写会立即结束并丢弃录音。现在会继续录音，说完后先用批量识别重试已录下的音频，仍失败才提示错误。此前一次听写被取消或失败后，本次会话内的批量重试也会失效；现在从下一次听写起恢复正常。 - **Stalled cloud ASR losing the recording**: when the streaming recognizer went quiet and the result wait expired, the utterance ended as "no speech" even though the audio was captured. A stalled recognizer now counts as a failure, so the batch ASR fallback retries the recorded audio before giving up. / **云端识别卡住时丢失录音**：流式识别无响应、等待结果超时后，即便录音已采集，本次语音也会直接报「未检测到语音」。现在识别卡住按失败处理，会先用批量识别重试已录下的音频再放弃。 diff --git a/OSGKeyboard/Info.plist b/OSGKeyboard/Info.plist index b42f93f..da35774 100644 --- a/OSGKeyboard/Info.plist +++ b/OSGKeyboard/Info.plist @@ -108,6 +108,8 @@ OSGKeyboard uses Contacts only when you choose Create Contact for a copied phone number. NSMicrophoneUsageDescription OSGKeyboard uses the microphone for voice dictation, including active Flow sessions while you type in other apps. + NSPhotoLibraryAddUsageDescription + OSGKeyboard saves your invitation poster to Photos only when you choose Save Image. NSRemindersFullAccessUsageDescription OSGKeyboard adds reminders only when you choose Extract To-dos for copied text. NSSpeechRecognitionUsageDescription diff --git a/OSGKeyboard/Utilities/ScreenBrightnessBoost.swift b/OSGKeyboard/Utilities/ScreenBrightnessBoost.swift new file mode 100644 index 0000000..8be8b27 --- /dev/null +++ b/OSGKeyboard/Utilities/ScreenBrightnessBoost.swift @@ -0,0 +1,88 @@ +// ScreenBrightnessBoost.swift +// OSGKeyboard · Main App +// +// Reference-counted full-brightness hold for content another device scans, +// such as the invitation QR code. iOS keeps an app-set brightness after the +// app leaves the foreground, so the hold restores the user's level while the +// app is in the background and reapplies it on return. + +import SwiftUI +import UIKit + +@MainActor +protocol ScreenBrightnessControlling: AnyObject { + var brightness: CGFloat { get set } +} + +extension UIScreen: ScreenBrightnessControlling {} + +@MainActor +final class ScreenBrightnessBoost { + static let shared = ScreenBrightnessBoost(screenProvider: foregroundScreen) + + private let screenProvider: @MainActor () -> ScreenBrightnessControlling? + private var holdCount = 0 + private var isInForeground = true + /// Set only while boosted, so the user's level is captured exactly once. + private var boost: (screen: ScreenBrightnessControlling, originalBrightness: CGFloat)? + + init(screenProvider: @escaping @MainActor () -> ScreenBrightnessControlling?) { + self.screenProvider = screenProvider + } + + func acquire() { + holdCount += 1 + update() + } + + func release() { + guard holdCount > 0 else { return } + holdCount -= 1 + update() + } + + func setInForeground(_ inForeground: Bool) { + isInForeground = inForeground + update() + } + + private func update() { + let wantsBoost = holdCount > 0 && isInForeground + if wantsBoost, boost == nil, let screen = screenProvider() { + boost = (screen, screen.brightness) + screen.brightness = 1 + } else if !wantsBoost, let boost { + boost.screen.brightness = boost.originalBrightness + self.boost = nil + } + } + + private static func foregroundScreen() -> ScreenBrightnessControlling? { + UIApplication.shared.connectedScenes + .compactMap { $0 as? UIWindowScene } + .first { $0.activationState == .foregroundActive }? + .screen + } +} + +extension View { + /// Holds full screen brightness while this view is on screen. + func boostsScreenBrightness() -> some View { + modifier(ScreenBrightnessBoostModifier()) + } +} + +private struct ScreenBrightnessBoostModifier: ViewModifier { + @Environment(\.scenePhase) private var scenePhase + + func body(content: Content) -> some View { + content + .onAppear { ScreenBrightnessBoost.shared.acquire() } + .onDisappear { ScreenBrightnessBoost.shared.release() } + // `.inactive` is skipped on purpose: Control Center makes the scene + // inactive, and restoring there would fight a manual adjustment. + .onChange(of: scenePhase) { _, phase in + ScreenBrightnessBoost.shared.setInForeground(phase != .background) + } + } +} diff --git a/OSGKeyboard/Views/Account/AccountCenterModels.swift b/OSGKeyboard/Views/Account/AccountCenterModels.swift index b7e8462..a5914a5 100644 --- a/OSGKeyboard/Views/Account/AccountCenterModels.swift +++ b/OSGKeyboard/Views/Account/AccountCenterModels.swift @@ -80,19 +80,34 @@ struct ReferralCode: Codable, Equatable, Sendable { let inviteURL: URL let campaignID: String? let createdAt: Date + /// 8-character alias friends can type; nil until the server assigns one. + let shortCode: String? private enum CodingKeys: String, CodingKey { case code case inviteURL = "inviteUrl" case campaignID = "campaignId" case createdAt + case shortCode } - init(code: String, inviteURL: URL, campaignID: String?, createdAt: Date) { + init( + code: String, + inviteURL: URL, + campaignID: String?, + createdAt: Date, + shortCode: String? = nil + ) { self.code = code self.inviteURL = inviteURL self.campaignID = campaignID self.createdAt = createdAt + self.shortCode = shortCode.flatMap(ReferralShortCode.normalize) + } + + /// `XXXX-XXXX` form for posters and share sheets. + var displayShortCode: String? { + shortCode.map(ReferralShortCode.display) } init(from decoder: Decoder) throws { @@ -115,6 +130,10 @@ struct ReferralCode: Codable, Equatable, Sendable { self.inviteURL = inviteURL campaignID = try container.decodeIfPresent(String.self, forKey: .campaignID) self.createdAt = createdAt + // A malformed alias only hides the short code; the link still works. + shortCode = (try? container.decodeIfPresent(String.self, forKey: .shortCode)) + .flatMap { $0 } + .flatMap(ReferralShortCode.normalize) } func encode(to encoder: Encoder) throws { @@ -123,9 +142,10 @@ struct ReferralCode: Codable, Equatable, Sendable { try container.encode(inviteURL, forKey: .inviteURL) try container.encodeIfPresent(campaignID, forKey: .campaignID) try container.encode(Self.string(from: createdAt), forKey: .createdAt) + try container.encodeIfPresent(shortCode, forKey: .shortCode) } - private static func date(from value: String) -> Date? { + static func date(from value: String) -> Date? { let fractional = ISO8601DateFormatter() fractional.formatOptions = [.withInternetDateTime, .withFractionalSeconds] let standard = ISO8601DateFormatter() @@ -133,7 +153,7 @@ struct ReferralCode: Codable, Equatable, Sendable { return fractional.date(from: value) ?? standard.date(from: value) } - private static func string(from date: Date) -> String { + static func string(from date: Date) -> String { let formatter = ISO8601DateFormatter() formatter.formatOptions = [.withInternetDateTime, .withFractionalSeconds] return formatter.string(from: date) @@ -214,6 +234,40 @@ enum ReferralJSONValue: Codable, Equatable, Sendable { struct ReferralProfile: Codable, Equatable, Sendable { let code: ReferralCode let binding: ReferralBinding? + /// Last moment this account may still bind an inviter. The server omits it + /// once bound or when no campaign is active; older servers never send it. + let bindingDeadline: Date? + + private enum CodingKeys: String, CodingKey { + case code + case binding + case bindingDeadline + } + + init(code: ReferralCode, binding: ReferralBinding?, bindingDeadline: Date? = nil) { + self.code = code + self.binding = binding + self.bindingDeadline = bindingDeadline + } + + init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + code = try container.decode(ReferralCode.self, forKey: .code) + binding = try container.decodeIfPresent(ReferralBinding.self, forKey: .binding) + bindingDeadline = (try? container.decodeIfPresent(String.self, forKey: .bindingDeadline)) + .flatMap { $0 } + .flatMap(ReferralCode.date(from:)) + } + + func encode(to encoder: Encoder) throws { + var container = encoder.container(keyedBy: CodingKeys.self) + try container.encode(code, forKey: .code) + try container.encodeIfPresent(binding, forKey: .binding) + try container.encodeIfPresent( + bindingDeadline.map(ReferralCode.string(from:)), + forKey: .bindingDeadline + ) + } } struct AccountCenterSnapshot: Equatable, Sendable { @@ -448,6 +502,38 @@ enum ReferralUniversalLink { return code } + /// Extracts a code from typed or pasted text: a short code, a bare long + /// code, an invitation link, or a chat message that contains one of them. + /// Links are matched with the same host and path rules as Universal Links, + /// so arbitrary URLs never qualify. Short codes are returned normalized + /// (uppercase, no hyphen); the server resolves them to the long code. + static func code(fromUserInput text: String) -> String? { + let separators = CharacterSet.whitespacesAndNewlines + .union(CharacterSet(charactersIn: "，。；：、！？「」『』“”‘’<>()（）【】《》")) + let tokens = text + .components(separatedBy: separators) + .filter { !$0.isEmpty } + for token in tokens { + if isValid(code: token) { + return token + } + // Chat apps often glue the link to the preceding sentence. + let link = token.range(of: "https://", options: .caseInsensitive) + .map { String(token[$0.lowerBound...]) } ?? token + if let url = URL(string: link), let code = code(from: url) { + return code + } + } + // Short codes go last: an 8-letter word in a pasted message must not + // shadow the link that message carries. + return ReferralShortCode.firstMatch(in: text) + } + + /// Anything the bind endpoint accepts: a long code or a normalized short code. + static func isRedeemable(code: String) -> Bool { + isValid(code: code) || ReferralShortCode.isCanonical(code) + } + static func isValid(code: String) -> Bool { guard code.utf8.count == codeLength else { return false } return code.utf8.allSatisfy { byte in @@ -461,6 +547,73 @@ enum ReferralUniversalLink { } +/// 8-character typeable alias of a referral code. Mirrors the server's +/// `ReferralShortCode`: the alphabet drops 0/O and 1/I/L so a code read aloud +/// or copied from a poster cannot be mistyped into another valid code. +enum ReferralShortCode { + static let length = 8 + static let alphabet: Set = Set("23456789ABCDEFGHJKMNPQRSTUVWXYZ") + + static func isCanonical(_ value: String) -> Bool { + value.count == length && value.allSatisfy(alphabet.contains) + } + + /// Accepts `K7M2Q9XA` or `k7m2-q9xa`; returns the stored uppercase form. + static func normalize(_ raw: String) -> String? { + var value = raw.trimmingCharacters(in: .whitespacesAndNewlines).uppercased() + if value.count == length + 1, value[value.index(value.startIndex, offsetBy: 4)] == "-" { + value.remove(at: value.index(value.startIndex, offsetBy: 4)) + } + return isCanonical(value) ? value : nil + } + + static func display(_ value: String) -> String { + guard isCanonical(value) else { return value } + return "\(value.prefix(4))-\(value.suffix(4))" + } + + /// Finds a short code inside free text such as "邀请码 K7M2-Q9XA" or a + /// Chinese sentence glued to the code, which whitespace splitting misses. + static func firstMatch(in text: String) -> String? { + // Runs of code-like ASCII; anything else (spaces, CJK, punctuation) + // separates them, so a run is exactly one candidate. "_" stays inside + // a run so a fragment of a long code never reads as a short code. + let runs = text.split { character in + !(character.isASCII + && (character.isLetter || character.isNumber || character == "-" || character == "_")) + } + return runs.lazy.compactMap { normalize(String($0)) }.first + } +} + +/// When a signed-in account may still bind an inviter. Prefers the server's +/// `bindingDeadline`, which reflects the active campaign; without it, falls +/// back to the default `REFERRAL_BINDING_DAYS`. The server stays +/// authoritative, so a mismatch still surfaces as a redemption error. +enum ReferralBindingWindow { + static let duration: TimeInterval = 7 * 24 * 60 * 60 + + static func isOpen(for account: AccountSession, now: Date = Date()) -> Bool { + // Zero means the account record has not loaded yet. + guard account.createdAtEpochSeconds > 0 else { return false } + let registeredAt = Date(timeIntervalSince1970: TimeInterval(account.createdAtEpochSeconds)) + return now.timeIntervalSince(registeredAt) < duration + } + + /// Open only while unbound and before the deadline the server reported. + static func isOpen( + for account: AccountSession, + profile: ReferralProfile, + now: Date = Date() + ) -> Bool { + guard profile.binding == nil else { return false } + if let deadline = profile.bindingDeadline { + return now <= deadline + } + return isOpen(for: account, now: now) + } +} + @MainActor protocol PendingReferralCodeStoring: AnyObject { var code: String? { get } @@ -480,7 +633,7 @@ final class UserDefaultsPendingReferralCodeStore: PendingReferralCodeStoring { var code: String? { guard let value = defaults.string(forKey: Self.storageKey), - ReferralUniversalLink.isValid(code: value) + ReferralUniversalLink.isRedeemable(code: value) else { return nil } @@ -488,7 +641,7 @@ final class UserDefaultsPendingReferralCodeStore: PendingReferralCodeStoring { } func save(_ code: String) { - guard ReferralUniversalLink.isValid(code: code) else { return } + guard ReferralUniversalLink.isRedeemable(code: code) else { return } defaults.set(code, forKey: Self.storageKey) } diff --git a/OSGKeyboard/Views/Account/AccountCenterUITestHarness.swift b/OSGKeyboard/Views/Account/AccountCenterUITestHarness.swift index d314d09..78e26ff 100644 --- a/OSGKeyboard/Views/Account/AccountCenterUITestHarness.swift +++ b/OSGKeyboard/Views/Account/AccountCenterUITestHarness.swift @@ -29,7 +29,8 @@ struct AccountCenterUITestHarness: View { "--account-snapshot-failure" ), startsSignedOut: startsSignIn || staysSignedOut, - signInDelay: startsSignIn ? .seconds(5) : .zero + signInDelay: startsSignIn ? .seconds(5) : .zero, + isInsideReferralWindow: arguments.contains("--account-referral-window") ) self.startsSignIn = startsSignIn let store = AccountCenterUITestCreditStore(accountID: service.account.accountID) @@ -122,15 +123,18 @@ private actor AccountCenterUITestService: private let shouldFailSnapshot: Bool private let startsSignedOut: Bool private let signInDelay: Duration + private let isInsideReferralWindow: Bool init( shouldFailSnapshot: Bool = false, startsSignedOut: Bool = false, - signInDelay: Duration = .zero + signInDelay: Duration = .zero, + isInsideReferralWindow: Bool = false ) { self.shouldFailSnapshot = shouldFailSnapshot self.startsSignedOut = startsSignedOut self.signInDelay = signInDelay + self.isInsideReferralWindow = isInsideReferralWindow } func restoreSession() async throws -> AccountSession? { @@ -180,9 +184,12 @@ private actor AccountCenterUITestService: code: "UITestInvite_123456789", inviteURL: URL(string: "https://osglab.com/i/UITestInvite_123456789")!, campaignID: nil, - createdAt: Date(timeIntervalSince1970: 1_700_000_000) + createdAt: Date(timeIntervalSince1970: 1_700_000_000), + shortCode: "K7M2Q9XA" ), - binding: nil + binding: nil, + // The fixed account is years old; a server deadline reopens the window. + bindingDeadline: isInsideReferralWindow ? Date().addingTimeInterval(3 * 86_400) : nil ) } diff --git a/OSGKeyboard/Views/Account/AccountCenterView.swift b/OSGKeyboard/Views/Account/AccountCenterView.swift index 56f74cd..ff6b5b7 100644 --- a/OSGKeyboard/Views/Account/AccountCenterView.swift +++ b/OSGKeyboard/Views/Account/AccountCenterView.swift @@ -111,12 +111,7 @@ struct AccountCenterView: View { AccountAppleAuthorizationButton(purpose: .signIn) .disabled(coordinator.operation != nil) - if coordinator.pendingReferralCode != nil { - Label(AppL10n.string("account.referral.pendingAfterSignIn"), systemImage: "link") - .font(.footnote) - .foregroundStyle(palette.textSecondary) - .frame(maxWidth: .infinity, alignment: .leading) - } + ReferralCodeEntryRow() Text(AppL10n.string("account.signedOut.localUnaffected")) .font(.footnote) @@ -136,6 +131,10 @@ struct AccountCenterView: View { ScrollView { CardPageContent { accountSummarySection(session: session) + SignedInReferralCodeCard( + profile: coordinator.referralProfile, + session: session + ) accountRewardsContent(session: session) securitySection } @@ -769,62 +768,63 @@ struct AccountCreditProgress: View { struct AccountInvitationButton: View { @Environment(\.themePalette) private var palette + @EnvironmentObject private var coordinator: AccountSessionCoordinator @State private var showsShareDrawer = false let invitationURL: URL? + var shortCode: String? var body: some View { Button { guard invitationURL != nil else { return } showsShareDrawer = true } label: { - Label(AppL10n.string("account.referral.inviteTitle"), systemImage: "person.badge.plus") - .font(TypeStyle.bodyEmph) - .foregroundStyle(OSGColor.fixedLightContent) - .padding(.horizontal, Spacing.sm) - .frame(minHeight: 38) - .background( - palette.success, - in: RoundedRectangle( - cornerRadius: Radius.medium, - style: .continuous - ) - ) + AccountReferralActionLabel( + titleKey: "account.referral.inviteTitle", + systemImage: "person.badge.plus" + ) } .buttonStyle(.plain) .disabled(invitationURL == nil) .accessibilityIdentifier("account.referral.share") .sheet(isPresented: $showsShareDrawer) { if let invitationURL { - AccountInvitationActivityView(invitationURL: invitationURL) - .presentationDetents([.medium, .large]) + InviteShareSheet( + invitationURL: invitationURL, + shortCode: shortCode, + referralProfile: coordinator.referralProfile + ) + .presentationDetents([.large]) .presentationDragIndicator(.visible) } } } } -private struct AccountInvitationActivityView: UIViewControllerRepresentable { - let invitationURL: URL +/// Green rounded-rectangle label shared by the Invite and "I have a code" +/// buttons so the two read as one family on the account card. +struct AccountReferralActionLabel: View { + @Environment(\.themePalette) private var palette - func makeUIViewController(context: Context) -> UIActivityViewController { - let controller = UIActivityViewController( - activityItems: [invitationURL], - applicationActivities: nil - ) - controller.completionWithItemsHandler = { _, completed, _, _ in - guard completed else { return } - Task { @MainActor in - AnalyticsHostService.shared.client.recordReferralShared() - } - } - return controller + let titleKey: String + let systemImage: String + /// Full-width form for a sheet's primary action. + var fillsWidth = false + + var body: some View { + Label(AppL10n.string(titleKey), systemImage: systemImage) + .font(TypeStyle.bodyEmph) + .foregroundStyle(OSGColor.fixedLightContent) + .padding(.horizontal, Spacing.sm) + .frame(maxWidth: fillsWidth ? .infinity : nil, minHeight: fillsWidth ? 44 : 38) + .background( + palette.success, + in: RoundedRectangle( + cornerRadius: Radius.medium, + style: .continuous + ) + ) } - - func updateUIViewController( - _ uiViewController: UIActivityViewController, - context: Context - ) {} } #if DEBUG diff --git a/OSGKeyboard/Views/Account/AccountRewardsCard.swift b/OSGKeyboard/Views/Account/AccountRewardsCard.swift index 118effc..cc09780 100644 --- a/OSGKeyboard/Views/Account/AccountRewardsCard.swift +++ b/OSGKeyboard/Views/Account/AccountRewardsCard.swift @@ -63,6 +63,9 @@ struct AccountRewardsCard: View { AccountAppleAuthorizationButton(purpose: .signIn) .disabled(coordinator.operation != nil) + + Divider().background(palette.divider) + ReferralCodeEntryRow() } .padding(Spacing.md) } @@ -90,10 +93,18 @@ struct AccountRewardsCard: View { } Divider().background(palette.divider) - AccountReferralLinkView(viewModel: coordinator.referralProfile) + AccountReferralLinkView( + viewModel: coordinator.referralProfile, + session: signedInSession + ) } } + private var signedInSession: AccountSession? { + guard case let .signedIn(session) = coordinator.sessionPhase else { return nil } + return session + } + private func creditSummary(_ snapshot: AccountCenterSnapshot) -> some View { let totalCredits = creditTotal(snapshot) return Button(action: onOpenAccount) { @@ -185,6 +196,9 @@ struct AccountRewardsCard: View { private struct AccountReferralLinkView: View { @Environment(\.themePalette) private var palette @ObservedObject var viewModel: ReferralProfileViewModel + let session: AccountSession? + + @State private var isCodeEntryPresented = false var body: some View { Group { @@ -217,12 +231,27 @@ private struct AccountReferralLinkView: View { private func loadedContent(_ profile: ReferralProfile) -> some View { VStack(alignment: .leading, spacing: Spacing.sm) { HStack(spacing: Spacing.md) { - Text(AppL10n.string("account.referral.equalRewardDescription")) - .font(TypeStyle.body.weight(.bold)) - .foregroundStyle(palette.textSecondary) - .fixedSize(horizontal: false, vertical: true) + Text(AppL10n.string( + showsCodeEntry(profile) + ? "account.referral.codeEntry.windowDescription" + : "account.referral.equalRewardDescription" + )) + .font(TypeStyle.body.weight(.bold)) + .foregroundStyle(palette.textSecondary) + .fixedSize(horizontal: false, vertical: true) Spacer(minLength: Spacing.xs) - AccountInvitationButton(invitationURL: profile.code.inviteURL) + if showsCodeEntry(profile) { + // Inviting stays reachable from the bottom of the sheet. + ReferralCodeEntryButton( + isPresented: $isCodeEntryPresented, + invitation: profile.code + ) + } else { + AccountInvitationButton( + invitationURL: profile.code.inviteURL, + shortCode: profile.code.displayShortCode + ) + } } if viewModel.isRefreshing { @@ -239,6 +268,15 @@ private struct AccountReferralLinkView: View { } } + /// New accounts that can still bind an inviter see "I have a code" in + /// place of Invite. The open sheet keeps the button mounted, so a + /// redemption that closes the window cannot tear the sheet down mid-confirmation. + private func showsCodeEntry(_ profile: ReferralProfile) -> Bool { + if isCodeEntryPresented { return true } + guard let session else { return false } + return ReferralBindingWindow.isOpen(for: session, profile: profile) + } + private func retryRow( messageKey: String, actionKey: String diff --git a/OSGKeyboard/Views/Account/AccountSessionCoordinator.swift b/OSGKeyboard/Views/Account/AccountSessionCoordinator.swift index 406528c..eb856b2 100644 --- a/OSGKeyboard/Views/Account/AccountSessionCoordinator.swift +++ b/OSGKeyboard/Views/Account/AccountSessionCoordinator.swift @@ -10,9 +10,20 @@ import Foundation #if canImport(OSGKeyboardShared) import OSGKeyboardShared #endif +#if canImport(OSGKeyboardHostSupport) +import OSGKeyboardHostSupport +#endif @MainActor final class AccountSessionCoordinator: ObservableObject { + /// Result of an invitation the user typed or pasted instead of opening a link. + enum ReferralInputOutcome: Equatable { + case invalidInput + case savedForSignIn + case redeemed + case failed(messageKey: String) + } + enum SessionPhase: Equatable { case restoring case signedOut @@ -208,11 +219,8 @@ final class AccountSessionCoordinator: ObservableObject { func handleIncomingURL(_ url: URL) -> Bool { guard let code = ReferralUniversalLink.code(from: url) else { return false } - pendingReferralStore.save(code) - pendingReferralCode = code + stagePendingReferral(code) shouldPresentAccountCenter = true - operationErrorKey = nil - operationErrorDetail = nil analyticsClient.recordInviteOpened() if isSignedIn { @@ -224,6 +232,26 @@ final class AccountSessionCoordinator: ObservableObject { return true } + /// Manual path for invitations that never reached the app as a Universal + /// Link — typically a friend who scanned a poster, installed from the App + /// Store, and typed the short code (or pasted the link). The outcome is + /// returned to the caller instead of `operationErrorKey` so the code-entry + /// sheet can explain it inline rather than through the global alert. + func submitReferralInput(_ text: String) async -> ReferralInputOutcome { + guard let code = ReferralUniversalLink.code(fromUserInput: text) else { + return .invalidInput + } + stagePendingReferral(code) + guard isSignedIn else { return .savedForSignIn } + + if let messageKey = await redeemPendingReferral() { + return .failed(messageKey: messageKey) + } + await referralProfile.refresh() + await refreshAccountData(force: true) + return .redeemed + } + func consumeAccountCenterPresentation() -> Bool { guard shouldPresentAccountCenter else { return false } shouldPresentAccountCenter = false @@ -544,22 +572,84 @@ final class AccountSessionCoordinator: ObservableObject { sessionRevision &+= 1 } + private func stagePendingReferral(_ code: String) { + pendingReferralStore.save(code) + pendingReferralCode = code + operationErrorKey = nil + operationErrorDetail = nil + } + private func redeemPendingReferralIfNeeded() async { - guard isSignedIn, let code = pendingReferralStore.code else { return } + if let messageKey = await redeemPendingReferral() { + operationErrorKey = messageKey + } + } + + /// Returns nil on success (or when nothing is pending), otherwise the + /// message key. Rejections the server will repeat forever drop the stored + /// code; transient failures keep it for the next sign-in or launch. + private func redeemPendingReferral() async -> String? { + guard isSignedIn, let code = pendingReferralStore.code else { return nil } operation = .redeemingReferral + defer { operation = nil } do { try await centerService.redeemReferral(code: code) pendingReferralStore.clear() pendingReferralCode = nil + return nil } catch { - pendingReferralCode = code - operationErrorKey = errorMessageKey( - for: error, - fallback: "account.error.redeemReferral" - ) + let failure = Self.referralRedemptionFailure(for: error) + if failure.isPermanent { + pendingReferralStore.clear() + pendingReferralCode = nil + } else { + pendingReferralCode = code + } + return failure.messageKey } - operation = nil + } + + private static func referralRedemptionFailure( + for error: Error + ) -> (messageKey: String, isPermanent: Bool) { + if error as? AccountIntegrationError == .unavailable { + return ("account.error.unavailable", false) + } + guard let apiError = error as? AccountAPIError else { + return ("account.error.redeemReferral", false) + } + // Referral routes answer with a bare `{"error": message}` body, so most + // rejections arrive as `.server` and are told apart by status code. + switch apiError { + case .invalidRequest: + return ("account.referral.redeem.error.invalid", true) + case .conflict(let message): + return (conflictMessageKey(for: message), true) + case let .server(statusCode, _, message): + switch statusCode { + case 400, 404: + return ("account.referral.redeem.error.invalid", true) + case 409: + return (conflictMessageKey(for: message), true) + case 422: + return ("account.referral.redeem.error.expired", true) + default: + return ("account.error.redeemReferral", false) + } + default: + return ("account.error.redeemReferral", false) + } + } + + private static func conflictMessageKey(for message: String) -> String { + if message.localizedCaseInsensitiveContains("self-referral") { + return "account.referral.redeem.error.self" + } + if message.localizedCaseInsensitiveContains("already bound") { + return "account.referral.redeem.error.alreadyBound" + } + return "account.referral.redeem.error.ineligible" } private func errorMessageKey(for error: Error, fallback: String) -> String { diff --git a/OSGKeyboard/Views/Account/InvitePoster.swift b/OSGKeyboard/Views/Account/InvitePoster.swift new file mode 100644 index 0000000..f6a0979 --- /dev/null +++ b/OSGKeyboard/Views/Account/InvitePoster.swift @@ -0,0 +1,329 @@ +// InvitePoster.swift +// OSGKeyboard · Main App + Mac +// +// Shareable invitation artwork: the app icon, inviter name, a QR code for the +// account's invitation link, and the short code for friends who type it +// instead of scanning. The QR code is generated on device with Core Image, +// so the link is never sent anywhere to be rendered. The Mac target compiles +// this file at the source level (see project.yml), so it must stay free of +// UIKit and AppKit. + +import CoreImage +import CoreImage.CIFilterBuiltins +import SwiftUI +#if canImport(OSGKeyboardShared) +import OSGKeyboardShared +#endif + +enum InviteQRCode { + /// Pixels per QR module. Integer scaling keeps module edges sharp. + fileprivate static let moduleScale: CGFloat = 12 + + static func makeImage(for url: URL) -> CGImage? { + let filter = CIFilter.qrCodeGenerator() + filter.message = Data(url.absoluteString.utf8) + // "M" (15%) survives chat-app recompression. Nothing is drawn over the + // code, so a higher level would only shrink the modules. + filter.correctionLevel = "M" + guard let output = filter.outputImage else { return nil } + let scaled = output.transformed( + by: CGAffineTransform(scaleX: moduleScale, y: moduleScale) + ) + // QRGenerator produces a one-channel image that SwiftUI can treat as a + // template and recolor white in dark mode. Materialize explicit black + // and white pixels so both poster appearances remain scannable. + let falseColor = CIFilter.falseColor() + falseColor.inputImage = scaled + falseColor.color0 = CIColor.black + falseColor.color1 = CIColor.white + guard let colored = falseColor.outputImage else { return nil } + return CIContext().createCGImage(colored, from: colored.extent) + } +} + +/// Fixed 3:4 layout so the exported image is the same on every device and +/// is never cropped in chat previews or Moments. +struct InvitePosterView: View { + static let size = CGSize(width: 360, height: 480) + + let qrImage: CGImage + let language: AppUILanguage? + let inviterName: String? + let colorScheme: ColorScheme + /// `XXXX-XXXX`; lets a friend who cannot scan type the code instead. + var shortCode: String? + + var body: some View { + VStack(spacing: 0) { + Spacer(minLength: 28) + + // Use the active Icon Composer artwork, not the horizontal logo. + Image("AppIcon_Assets/OSGKeyboard Icon2") + .resizable() + .scaledToFit() + .frame(width: 64, height: 64) + .accessibilityHidden(true) + + Text( + SharedL10n.format( + "invite.poster.inviter", + language: language, + resolvedInviterName + ) + ) + .font(.system(size: 18, weight: .medium, design: .default)) + .foregroundStyle(primaryContent) + .multilineTextAlignment(.center) + .lineLimit(2) + .lineSpacing(3) + .minimumScaleFactor(0.75) + .padding(.top, 24) + + // Dark modules on a light card scan most reliably, including in + // WeChat's long-press recognizer. + InviteQRCodeView(image: qrImage) + .frame(width: 156, height: 156) + .padding(Spacing.sm) + .background( + OSGColor.fixedLightContent, + in: Rectangle() + ) + .padding(.top, 28) + + Text(SharedL10n.string("invite.poster.scan", language: language)) + .font(.system(size: 12, weight: .regular, design: .default)) + .foregroundStyle(secondaryContent) + .multilineTextAlignment(.center) + .padding(.top, 28) + + if let shortCode { + Text(SharedL10n.format("invite.poster.code", language: language, shortCode)) + .font(.system(size: 18, weight: .bold, design: .monospaced)) + .foregroundStyle(primaryContent) + .padding(.top, 10) + } + + Spacer(minLength: 32) + } + .padding(.horizontal, Spacing.xxl) + .frame(width: Self.size.width, height: Self.size.height) + .background(InvitePosterBackground(colorScheme: colorScheme)) + } + + private var resolvedInviterName: String { + let trimmed = inviterName?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty + ? SharedL10n.string("invite.poster.defaultInviter", language: language) + : trimmed + } + + private var primaryContent: Color { + colorScheme == .dark ? Color(white: 0.96) : Color(white: 0.08) + } + + private var secondaryContent: Color { + primaryContent.opacity(0.62) + } +} + +/// Draws each QR module as a vector rectangle. `ImageRenderer` can otherwise +/// reinterpret a CGImage as a template on complex dark backgrounds and erase +/// its black modules. Vector modules stay black in every appearance. +private struct InviteQRCodeView: View { + private let sideCount: Int + private let darkModules: [Bool] + + init(image: CGImage) { + let matrix = Self.moduleMatrix(from: image) + sideCount = matrix.sideCount + darkModules = matrix.darkModules + } + + var body: some View { + Canvas { context, size in + guard sideCount > 0 else { return } + let moduleWidth = size.width / CGFloat(sideCount) + let moduleHeight = size.height / CGFloat(sideCount) + var modules = Path() + + for row in 0.. (sideCount: Int, darkModules: [Bool]) { + let sideCount = max(image.width / Int(InviteQRCode.moduleScale), 1) + var pixels = [UInt8](repeating: 0, count: sideCount * sideCount * 4) + guard let context = CGContext( + data: &pixels, + width: sideCount, + height: sideCount, + bitsPerComponent: 8, + bytesPerRow: sideCount * 4, + space: CGColorSpaceCreateDeviceRGB(), + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue + ) else { + return (0, []) + } + context.interpolationQuality = .none + context.draw( + image, + in: CGRect(x: 0, y: 0, width: sideCount, height: sideCount) + ) + let darkModules = (0..<(sideCount * sideCount)).map { index in + let offset = index * 4 + let luminance = Int(pixels[offset]) + + Int(pixels[offset + 1]) + + Int(pixels[offset + 2]) + return luminance < 3 * 128 + } + return (sideCount, darkModules) + } +} + +/// A restrained paper surface: warm Nordic stock in light mode and graphite +/// stock in dark mode. The daylight bands and fibers stay below text contrast, +/// so the background adds depth without becoming decoration. +private struct InvitePosterBackground: View { + let colorScheme: ColorScheme + + var body: some View { + GeometryReader { proxy in + ZStack { + base + + LinearGradient( + colors: daylightColors, + startPoint: .topLeading, + endPoint: .bottomTrailing + ) + + Ellipse() + .fill(topTrailingShadow) + .frame(width: 310, height: 250) + .offset(x: 178, y: -186) + .blur(radius: 62) + + Ellipse() + .fill(bottomLeadingShadow) + .frame(width: 300, height: 260) + .offset(x: -182, y: 224) + .blur(radius: 68) + + Rectangle() + .fill(diagonalShadow) + .frame(width: proxy.size.width * 1.55, height: 82) + .rotationEffect(.degrees(-18)) + .offset(x: 72, y: -118) + .blur(radius: 34) + + paperFibers(in: proxy.size) + } + } + } + + private var base: Color { + colorScheme == .dark + ? Color(red: 0.055, green: 0.054, blue: 0.050) + : Color(red: 0.965, green: 0.953, blue: 0.922) + } + + private var daylightColors: [Color] { + if colorScheme == .dark { + return [ + .white.opacity(0.085), + .clear, + .black.opacity(0.30) + ] + } + return [ + .white.opacity(0.62), + .clear, + Color(red: 0.52, green: 0.48, blue: 0.41).opacity(0.18) + ] + } + + private var topTrailingShadow: Color { + colorScheme == .dark + ? .black.opacity(0.38) + : Color(red: 0.43, green: 0.40, blue: 0.35).opacity(0.24) + } + + private var bottomLeadingShadow: Color { + colorScheme == .dark + ? .black.opacity(0.28) + : Color(red: 0.48, green: 0.44, blue: 0.37).opacity(0.15) + } + + private var diagonalShadow: Color { + colorScheme == .dark + ? .white.opacity(0.055) + : Color.black.opacity(0.055) + } + + private func paperFibers(in size: CGSize) -> some View { + ZStack { + ForEach(0..<48, id: \.self) { index in + Rectangle() + .fill(fiberColor) + .frame(width: 2 + 7 * fraction(index * 19), height: 0.35) + .position( + x: size.width * fraction(index * 37), + y: size.height * fraction(index * 61) + ) + } + } + .allowsHitTesting(false) + } + + private var fiberColor: Color { + colorScheme == .dark + ? Color.white.opacity(0.025) + : Color.black.opacity(0.025) + } + + private func fraction(_ value: Int) -> CGFloat { + CGFloat(value % 101) / 100 + } +} + +enum InvitePosterRenderer { + /// 3× the point layout: a 1080×1440 px image. + static let exportScale: CGFloat = 3 + + @MainActor + static func render( + inviteURL: URL, + shortCode: String? = nil, + inviterName: String? = nil, + language: AppUILanguage?, + colorScheme: ColorScheme + ) -> CGImage? { + guard let qrImage = InviteQRCode.makeImage(for: inviteURL) else { return nil } + let renderer = ImageRenderer( + content: InvitePosterView( + qrImage: qrImage, + language: language, + inviterName: inviterName, + colorScheme: colorScheme, + shortCode: shortCode + ) + ) + renderer.scale = exportScale + return renderer.cgImage + } +} diff --git a/OSGKeyboard/Views/Account/InviteQRFullScreenView.swift b/OSGKeyboard/Views/Account/InviteQRFullScreenView.swift new file mode 100644 index 0000000..0684d26 --- /dev/null +++ b/OSGKeyboard/Views/Account/InviteQRFullScreenView.swift @@ -0,0 +1,62 @@ +// InviteQRFullScreenView.swift +// OSGKeyboard · Main App +// +// Face-to-face mode for the invitation: only the QR code, as large as the +// screen allows, on white at full brightness so a friend's camera locks on +// from arm's length. Tapping anywhere closes it. + +import OSGKeyboardShared +import SwiftUI + +struct InviteQRFullScreenView: View { + @Environment(\.dismiss) private var dismiss + + let qrImage: CGImage + let language: AppUILanguage? + + /// Past this size a phone held at arm's length no longer gains anything. + private static let maxCodeSide: CGFloat = 440 + + var body: some View { + GeometryReader { proxy in + let side = min( + min(proxy.size.width, proxy.size.height) - Spacing.xxl * 2, + Self.maxCodeSide + ) + VStack(spacing: Spacing.lg) { + Image(decorative: qrImage, scale: 1) + .interpolation(.none) + .resizable() + .scaledToFit() + .frame(width: side, height: side) + .accessibilityLabel(AppL10n.string("account.referral.poster.accessibility")) + + Text(SharedL10n.string("invite.poster.hint", language: language)) + .font(TypeStyle.headline) + .foregroundStyle(OSGColor.fixedDarkContent.opacity(0.6)) + } + .frame(maxWidth: .infinity, maxHeight: .infinity) + } + .background(OSGColor.fixedLightContent.ignoresSafeArea()) + .overlay(alignment: .topTrailing) { + Button { + dismiss() + } label: { + Image(systemName: "xmark") + .font(.system(size: 17, weight: .semibold)) + .foregroundStyle(OSGColor.fixedDarkContent.opacity(0.6)) + .frame(width: 44, height: 44) + } + .buttonStyle(.plain) + .padding(Spacing.md) + .accessibilityLabel(AppL10n.string("common.done")) + .accessibilityIdentifier("account.referral.poster.fullScreen.close") + } + .contentShape(Rectangle()) + .onTapGesture { dismiss() } + .boostsScreenBrightness() + .statusBarHidden() + .accessibilityAction(.escape) { dismiss() } + .accessibilityIdentifier("account.referral.poster.fullScreen") + } +} diff --git a/OSGKeyboard/Views/Account/InviteShareSheet.swift b/OSGKeyboard/Views/Account/InviteShareSheet.swift new file mode 100644 index 0000000..9c2ed72 --- /dev/null +++ b/OSGKeyboard/Views/Account/InviteShareSheet.swift @@ -0,0 +1,306 @@ +// InviteShareSheet.swift +// OSGKeyboard · Main App +// +// Opened by the Invite button. Shows the invitation poster, which opens a +// full-screen code for face-to-face scanning when tapped, plus the three ways an +// invitation usually leaves the phone: saving the image to send in a chat app, +// the system share sheet, or the plain link. + +import OSGKeyboardShared +import Photos +import SwiftUI +import UIKit + +struct InviteShareSheet: View { + @Environment(\.themePalette) private var palette + @Environment(\.dismiss) private var dismiss + @Environment(\.colorScheme) private var colorScheme + @EnvironmentObject private var coordinator: AccountSessionCoordinator + @ObservedObject private var config = ProviderConfig.shared + + let invitationURL: URL + /// Display form (`XXXX-XXXX`), printed on the poster when available. + var shortCode: String? + @ObservedObject var referralProfile: ReferralProfileViewModel + + @State private var poster: UIImage? + @State private var qrImage: CGImage? + @State private var isShowingFullScreenCode = false + @State private var isSharingPoster = false + @State private var isSaving = false + @State private var status: Status? + + private enum Status: Equatable { + case success(messageKey: String) + case failure(messageKey: String) + } + + var body: some View { + NavigationStack { + ScrollView { + VStack(spacing: Spacing.lg) { + posterPreview + actions + statusMessage + } + .padding(Spacing.lg) + .frame(maxWidth: .infinity) + } + .background(palette.background.ignoresSafeArea()) + .navigationTitle(AppL10n.string("account.referral.poster.title")) + .navigationBarTitleDisplayMode(.inline) + .toolbar { + ToolbarItem(placement: .confirmationAction) { + Button(AppL10n.string("common.done")) { dismiss() } + .tint(palette.textPrimary) + } + } + } + .task(id: posterRenderID) { + qrImage = InviteQRCode.makeImage(for: invitationURL) + poster = InvitePosterRenderer + .render( + inviteURL: invitationURL, + shortCode: resolvedShortCode, + inviterName: inviterName, + language: config.uiLanguage, + colorScheme: colorScheme + ) + .map { UIImage(cgImage: $0) } + } + .task { + // A cached pre-short-code profile may be visible for the first + // frame. Refresh when the sheet opens and rerender once the server + // supplies the account's assigned alias. + guard shortCode == nil else { return } + await referralProfile.refresh() + } + .fullScreenCover(isPresented: $isShowingFullScreenCode) { + if let qrImage { + InviteQRFullScreenView(qrImage: qrImage, language: config.uiLanguage) + } + } + .sheet(isPresented: $isSharingPoster) { + if let poster { + InvitePosterActivityView(poster: poster) + .presentationDetents([.medium, .large]) + .presentationDragIndicator(.visible) + } + } + .accessibilityIdentifier("account.referral.poster") + } + + /// Tapping the poster opens the full-screen code for face-to-face scanning. + private var posterPreview: some View { + let shape = RoundedRectangle(cornerRadius: Radius.xl, style: .continuous) + return Button { + isShowingFullScreenCode = true + } label: { + Group { + if let poster { + Image(uiImage: poster) + .resizable() + .scaledToFit() + } else { + ZStack { + colorScheme == .dark + ? OSGColor.fixedDarkContent + : OSGColor.fixedLightContent + ProgressView() + .tint( + colorScheme == .dark + ? OSGColor.fixedLightContent + : OSGColor.fixedDarkContent + ) + } + } + } + .aspectRatio( + InvitePosterView.size.width / InvitePosterView.size.height, + contentMode: .fit + ) + .overlay(alignment: .bottomTrailing) { + Image(systemName: "arrow.up.left.and.arrow.down.right") + .font(.system(size: 13, weight: .semibold)) + .foregroundStyle(posterPrimaryContent) + .frame(width: 32, height: 32) + .background(posterPrimaryContent.opacity(0.16), in: Circle()) + .padding(Spacing.sm) + .accessibilityHidden(true) + } + .clipShape(shape) + } + .buttonStyle(.plain) + .disabled(qrImage == nil) + .frame(maxWidth: 320) + .cardElevation() + .accessibilityLabel(AppL10n.string("account.referral.poster.accessibility")) + .accessibilityHint(AppL10n.string("account.referral.poster.enlargeHint")) + .accessibilityIdentifier("account.referral.poster.preview") + } + + private var inviterName: String? { + guard case let .signedIn(session) = coordinator.sessionPhase else { return nil } + return session.displayName + } + + private var resolvedShortCode: String? { + if let shortCode { return shortCode } + guard case let .loaded(profile) = referralProfile.state else { return nil } + return profile.code.displayShortCode + } + + private var posterRenderID: String { + "\(colorScheme)-\(resolvedShortCode ?? "")-\(inviterName ?? "")" + } + + private var posterPrimaryContent: Color { + colorScheme == .dark + ? OSGColor.fixedLightContent + : OSGColor.fixedDarkContent + } + + private var actions: some View { + VStack(spacing: Spacing.sm) { + Button { + Task { await savePoster() } + } label: { + Group { + if isSaving { + ProgressView() + .tint(palette.textOnAccent) + } else { + Label( + AppL10n.string("account.referral.poster.save"), + systemImage: "square.and.arrow.down" + ) + } + } + .primaryButton() + } + .buttonStyle(.plain) + .disabled(poster == nil || isSaving) + .accessibilityIdentifier("account.referral.poster.save") + + HStack(spacing: Spacing.sm) { + Button { + isSharingPoster = true + } label: { + Label( + AppL10n.string("account.referral.poster.share"), + systemImage: "square.and.arrow.up" + ) + .secondaryButton() + } + .buttonStyle(.plain) + .disabled(poster == nil) + .accessibilityIdentifier("account.referral.poster.share") + + Button(action: copyLink) { + Label( + AppL10n.string("account.referral.poster.copyLink"), + systemImage: "link" + ) + .secondaryButton() + } + .buttonStyle(.plain) + .accessibilityIdentifier("account.referral.poster.copyLink") + } + } + .frame(maxWidth: 360) + } + + @ViewBuilder + private var statusMessage: some View { + switch status { + case .success(let messageKey): + Label(AppL10n.string(messageKey), systemImage: "checkmark.circle.fill") + .font(TypeStyle.footnote) + .foregroundStyle(palette.success) + case .failure(let messageKey): + Label(AppL10n.string(messageKey), systemImage: "exclamationmark.triangle.fill") + .font(TypeStyle.footnote) + .foregroundStyle(palette.danger) + .multilineTextAlignment(.center) + case nil: + Text(AppL10n.string("account.referral.poster.footnote")) + .font(TypeStyle.footnote) + .foregroundStyle(palette.textTertiary) + .multilineTextAlignment(.center) + } + } + + private func savePoster() async { + guard let poster, !isSaving else { return } + isSaving = true + defer { isSaving = false } + + // PhotoKit terminates the process instead of throwing when this usage + // description is missing from the built app. Keep this preflight so a + // future project-generation regression becomes a visible error. + guard let usageDescription = Bundle.main.object( + forInfoDictionaryKey: "NSPhotoLibraryAddUsageDescription" + ) as? String, !usageDescription.isEmpty else { + status = .failure(messageKey: "account.referral.poster.saveFailed") + return + } + + // Add-only access: the app never needs to read the user's library. + let authorization = await PHPhotoLibrary.requestAuthorization(for: .addOnly) + guard authorization == .authorized || authorization == .limited else { + status = .failure(messageKey: "account.referral.poster.saveDenied") + return + } + guard let pngData = poster.pngData() else { + status = .failure(messageKey: "account.referral.poster.saveFailed") + return + } + do { + try await Self.addToPhotos(pngData: pngData) + status = .success(messageKey: "account.referral.poster.saved") + AnalyticsHostService.shared.client.recordReferralShared() + } catch { + status = .failure(messageKey: "account.referral.poster.saveFailed") + } + } + + /// Photos runs the change block on its own transaction queue. It must be + /// built outside the view's main-actor isolation: Swift 6 traps when a + /// main-actor closure runs off the main thread. + private nonisolated static func addToPhotos(pngData: Data) async throws { + try await PHPhotoLibrary.shared().performChanges { + PHAssetCreationRequest.forAsset().addResource(with: .photo, data: pngData, options: nil) + } + } + + private func copyLink() { + UIPasteboard.general.string = invitationURL.absoluteString + status = .success(messageKey: "account.referral.poster.linkCopied") + AnalyticsHostService.shared.client.recordReferralShared() + } +} + +/// Shares the poster image only. Chat apps' share extensions often refuse a +/// mixed image + URL payload, and the link is one tap away via Copy Link. +private struct InvitePosterActivityView: UIViewControllerRepresentable { + let poster: UIImage + + func makeUIViewController(context: Context) -> UIActivityViewController { + let controller = UIActivityViewController( + activityItems: [poster], + applicationActivities: nil + ) + controller.completionWithItemsHandler = { _, completed, _, _ in + guard completed else { return } + Task { @MainActor in + AnalyticsHostService.shared.client.recordReferralShared() + } + } + return controller + } + + func updateUIViewController( + _ uiViewController: UIActivityViewController, + context: Context + ) {} +} diff --git a/OSGKeyboard/Views/Account/ReferralCodeEntry.swift b/OSGKeyboard/Views/Account/ReferralCodeEntry.swift new file mode 100644 index 0000000..8712348 --- /dev/null +++ b/OSGKeyboard/Views/Account/ReferralCodeEntry.swift @@ -0,0 +1,288 @@ +// ReferralCodeEntry.swift +// OSGKeyboard · Main App +// +// Manual path for invitations that never opened the app directly. A friend +// who scans a poster usually installs from the App Store, so the Universal +// Link does not fire; instead they type the 8-character code printed on the +// poster and the landing page. Pasting the full link into the field works +// too. The clipboard is never read on its own. + +import OSGKeyboardShared +import SwiftUI + +/// Green "I have a code" button, styled like the Invite button. The caller +/// owns `isPresented` so it can keep the button on screen while the sheet is +/// open, even if a successful redemption closes the binding window. +struct ReferralCodeEntryButton: View { + @Binding var isPresented: Bool + /// Signed-in accounts also get an Invite entry at the bottom of the sheet. + var invitation: ReferralCode? + + var body: some View { + Button { + isPresented = true + } label: { + AccountReferralActionLabel( + titleKey: "account.referral.codeEntry.button", + systemImage: "gift" + ) + } + .buttonStyle(.plain) + .accessibilityIdentifier("account.referral.codeEntry") + .sheet(isPresented: $isPresented) { + ReferralCodeEntrySheet(invitation: invitation) + } + } +} + +/// Text field + confirm. Outcomes are shown inline rather than through the +/// global alert; success closes the sheet after a short confirmation. +struct ReferralCodeEntrySheet: View { + typealias Outcome = AccountSessionCoordinator.ReferralInputOutcome + + @Environment(\.dismiss) private var dismiss + @Environment(\.themePalette) private var palette + @EnvironmentObject private var coordinator: AccountSessionCoordinator + + var invitation: ReferralCode? + + @State private var text = "" + @State private var isSubmitting = false + @State private var outcome: Outcome? + @FocusState private var isFieldFocused: Bool + + /// Long enough to read the confirmation before the sheet closes. + private static let successDismissDelay: Duration = .milliseconds(1_200) + + var body: some View { + NavigationStack { + ScrollView { + VStack(alignment: .leading, spacing: Spacing.md) { + Text(AppL10n.string("account.referral.codeEntry.sheetBody")) + .font(TypeStyle.footnote) + .foregroundStyle(palette.textSecondary) + .fixedSize(horizontal: false, vertical: true) + + codeField + confirmButton + feedback + + if coordinator.isSignedIn, let invitation { + inviteMore(invitation) + } + } + .padding(.horizontal, Spacing.lg) + .padding(.vertical, Spacing.md) + } + .scrollBounceBehavior(.basedOnSize) + .background(palette.background.ignoresSafeArea()) + .navigationTitle(AppL10n.string("account.referral.codeEntry.sheetTitle")) + .navigationBarTitleDisplayMode(.inline) + .toolbar { + ToolbarItem(placement: .cancellationAction) { + Button(AppL10n.string("common.cancel")) { dismiss() } + .tint(palette.textPrimary) + } + } + .onAppear { isFieldFocused = true } + } + .presentationDetents([.medium, .large]) + .presentationDragIndicator(.visible) + .interactiveDismissDisabled(isSubmitting) + .accessibilityIdentifier("account.referral.codeEntry.sheet") + } + + private var codeField: some View { + TextField(AppL10n.string("account.referral.codeEntry.placeholder"), text: $text) + .font(.system(.title3, design: .monospaced, weight: .semibold)) + .foregroundStyle(palette.textPrimary) + .tint(palette.textPrimary) + .focused($isFieldFocused) + // Codes are uppercase; links pasted into the field are unaffected. + .textInputAutocapitalization(.characters) + .autocorrectionDisabled() + .keyboardType(.asciiCapable) + .submitLabel(.done) + .onSubmit { Task { await submit() } } + .onChange(of: text) { _, _ in outcome = nil } + .settingsListRow() + .background( + palette.formSurface, + in: RoundedRectangle(cornerRadius: Radius.large, style: .continuous) + ) + .accessibilityLabel(AppL10n.string("account.referral.codeEntry.fieldLabel")) + .accessibilityIdentifier("account.referral.codeEntry.field") + } + + private var confirmButton: some View { + Button { + Task { await submit() } + } label: { + Group { + if isSubmitting { + ProgressView() + .tint(OSGColor.fixedLightContent) + .frame(maxWidth: .infinity, minHeight: 44) + .background( + palette.success, + in: RoundedRectangle(cornerRadius: Radius.medium, style: .continuous) + ) + } else { + AccountReferralActionLabel( + titleKey: "account.referral.codeEntry.confirm", + systemImage: "checkmark", + fillsWidth: true + ) + } + } + } + .buttonStyle(.plain) + .disabled(!canSubmit) + .opacity(canSubmit || isSubmitting ? 1 : 0.5) + .accessibilityIdentifier("account.referral.codeEntry.confirm") + } + + @ViewBuilder + private var feedback: some View { + switch outcome { + case .invalidInput: + ReferralFeedbackLabel(messageKey: "account.referral.codeEntry.invalid", isSuccess: false) + case .failed(let messageKey): + ReferralFeedbackLabel(messageKey: messageKey, isSuccess: false) + case .redeemed: + ReferralFeedbackLabel(messageKey: "account.referral.codeEntry.redeemed", isSuccess: true) + case .savedForSignIn: + ReferralFeedbackLabel(messageKey: "account.referral.pendingAfterSignIn", isSuccess: true) + case nil: + EmptyView() + } + } + + /// The "invite in the sheet" decision: during the binding window the card + /// shows "I have a code" instead of Invite, so inviting stays one tap away. + private func inviteMore(_ invitation: ReferralCode) -> some View { + VStack(alignment: .leading, spacing: Spacing.md) { + Divider().background(palette.divider) + HStack(spacing: Spacing.md) { + Text(AppL10n.string("account.referral.codeEntry.inviteMore")) + .font(TypeStyle.footnote) + .foregroundStyle(palette.textSecondary) + .fixedSize(horizontal: false, vertical: true) + Spacer(minLength: Spacing.xs) + AccountInvitationButton( + invitationURL: invitation.inviteURL, + shortCode: invitation.displayShortCode + ) + } + } + .padding(.top, Spacing.sm) + } + + private var canSubmit: Bool { + !isSubmitting + && coordinator.operation == nil + && !text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + } + + private func submit() async { + guard canSubmit else { return } + isSubmitting = true + let result = await coordinator.submitReferralInput(text) + isSubmitting = false + outcome = result + guard result == .redeemed || result == .savedForSignIn else { return } + isFieldFocused = false + try? await Task.sleep(for: Self.successDismissDelay) + dismiss() + } +} + +/// "Have an invitation?" row. Signed out it sits under the Apple button on +/// Home and in the account center, and the code is held until sign-in. +struct ReferralCodeEntryRow: View { + @Environment(\.themePalette) private var palette + @EnvironmentObject private var coordinator: AccountSessionCoordinator + + var invitation: ReferralCode? + /// Pass a binding when the parent must know the sheet is open. + var isEntryPresented: Binding? + + @State private var isLocallyPresented = false + + private var presentation: Binding { + isEntryPresented ?? $isLocallyPresented + } + + var body: some View { + VStack(alignment: .leading, spacing: Spacing.xs) { + HStack(alignment: .center, spacing: Spacing.sm) { + VStack(alignment: .leading, spacing: Spacing.xxs) { + Text(AppL10n.string("account.referral.codeEntry.title")) + .font(TypeStyle.bodyEmph) + .foregroundStyle(palette.textPrimary) + Text(AppL10n.string( + coordinator.isSignedIn + ? "account.referral.codeEntry.bodySignedIn" + : "account.referral.codeEntry.body" + )) + .font(TypeStyle.footnote) + .foregroundStyle(palette.textSecondary) + .fixedSize(horizontal: false, vertical: true) + } + .frame(maxWidth: .infinity, alignment: .leading) + + ReferralCodeEntryButton(isPresented: presentation, invitation: invitation) + .disabled(coordinator.operation != nil) + } + + if coordinator.pendingReferralCode != nil, !coordinator.isSignedIn { + ReferralFeedbackLabel(messageKey: "account.referral.pendingAfterSignIn", isSuccess: true) + } + } + } +} + +/// Signed-in variant for the account center: only accounts that have not +/// bound an inviter and are still inside the binding window see it. +struct SignedInReferralCodeCard: View { + @ObservedObject var profile: ReferralProfileViewModel + let session: AccountSession + + @State private var isEntryPresented = false + + var body: some View { + if isEntryPresented || isEligible { + ReferralCodeEntryRow(invitation: loadedCode, isEntryPresented: $isEntryPresented) + .padding(Spacing.md) + .surfaceCard() + .accessibilityIdentifier("account.referral.codeEntryCard") + } + } + + private var loadedCode: ReferralCode? { + guard case let .loaded(loaded) = profile.state else { return nil } + return loaded.code + } + + private var isEligible: Bool { + guard case let .loaded(loaded) = profile.state else { return false } + return ReferralBindingWindow.isOpen(for: session, profile: loaded) + } +} + +struct ReferralFeedbackLabel: View { + @Environment(\.themePalette) private var palette + + let messageKey: String + let isSuccess: Bool + + var body: some View { + Label( + AppL10n.string(messageKey), + systemImage: isSuccess ? "checkmark.circle.fill" : "exclamationmark.triangle.fill" + ) + .font(TypeStyle.footnote) + .foregroundStyle(isSuccess ? palette.success : palette.danger) + .fixedSize(horizontal: false, vertical: true) + } +} diff --git a/OSGKeyboard/en.lproj/InfoPlist.strings b/OSGKeyboard/en.lproj/InfoPlist.strings index e9fdc90..42cdd80 100644 --- a/OSGKeyboard/en.lproj/InfoPlist.strings +++ b/OSGKeyboard/en.lproj/InfoPlist.strings @@ -5,3 +5,4 @@ "NSContactsUsageDescription" = "OSGKeyboard uses Contacts only when you choose Create Contact for a copied phone number."; "NSRemindersFullAccessUsageDescription" = "OSGKeyboard adds reminders only when you choose Extract To-dos for copied text."; "NSCalendarsWriteOnlyAccessUsageDescription" = "OSGKeyboard adds calendar events only when you choose Extract Events for copied text. It never reads your existing events."; +"NSPhotoLibraryAddUsageDescription" = "OSGKeyboard saves your invitation poster to Photos only when you choose Save Image."; diff --git a/OSGKeyboard/en.lproj/Localizable.strings b/OSGKeyboard/en.lproj/Localizable.strings index 473e8f9..0daf4b6 100644 --- a/OSGKeyboard/en.lproj/Localizable.strings +++ b/OSGKeyboard/en.lproj/Localizable.strings @@ -983,6 +983,35 @@ "account.referral.pendingAfterSignIn" = "Your invitation is saved and will be redeemed after sign-in."; "account.referral.rewardGranted" = "You received %@ credits from the invitation."; "account.referral.rewardGrantedGeneric" = "Your invitation reward has been added to your balance."; +"account.referral.poster.title" = "Invite Friends"; +"account.referral.poster.accessibility" = "Invitation poster with a QR code for your invitation link"; +"account.referral.poster.enlargeHint" = "Shows the QR code full screen for scanning in person."; +"account.referral.poster.save" = "Save Image"; +"account.referral.poster.share" = "Share"; +"account.referral.poster.copyLink" = "Copy Link"; +"account.referral.poster.footnote" = "Send the image to a friend, or tap the poster to show a large code to scan in person. Once your friend signs up and completes a first qualifying use, you both earn credits."; +"account.referral.poster.saved" = "Saved to Photos"; +"account.referral.poster.saveDenied" = "Allow OSGKeyboard to add photos in Settings, then try again."; +"account.referral.poster.saveFailed" = "Couldn’t save the image. Please try again."; +"account.referral.poster.linkCopied" = "Invitation link copied"; +"account.referral.codeEntry.title" = "Have an invitation?"; +"account.referral.codeEntry.body" = "Enter your friend’s code, then sign in to accept it."; +"account.referral.codeEntry.bodySignedIn" = "Enter your friend’s code and you both earn credits."; +"account.referral.codeEntry.button" = "I have a code"; +"account.referral.codeEntry.windowDescription" = "Enter a code, or invite more friends for credits"; +"account.referral.codeEntry.sheetTitle" = "Enter Invitation Code"; +"account.referral.codeEntry.sheetBody" = "Enter the 8-character code from your friend’s poster or invitation page, or paste the invitation link."; +"account.referral.codeEntry.placeholder" = "e.g. K7M2-Q9XA"; +"account.referral.codeEntry.fieldLabel" = "Invitation code"; +"account.referral.codeEntry.confirm" = "Confirm"; +"account.referral.codeEntry.invalid" = "That doesn’t look like an invitation code. Check it and try again."; +"account.referral.codeEntry.redeemed" = "Invitation accepted. You both earn credits after qualifying usage."; +"account.referral.codeEntry.inviteMore" = "You can also invite more friends for credits"; +"account.referral.redeem.error.invalid" = "This invitation code is invalid, or the campaign has ended."; +"account.referral.redeem.error.expired" = "The time to add an invitation to this account has passed."; +"account.referral.redeem.error.self" = "You can’t use your own invitation."; +"account.referral.redeem.error.alreadyBound" = "This account has already accepted another invitation."; +"account.referral.redeem.error.ineligible" = "This account can’t accept this invitation."; "account.security.section" = "Account controls"; "account.signOut.action" = "Sign Out"; "account.signOut.confirmTitle" = "Sign out?"; diff --git a/OSGKeyboard/zh-Hans.lproj/InfoPlist.strings b/OSGKeyboard/zh-Hans.lproj/InfoPlist.strings index 1f33d98..f4d32fa 100644 --- a/OSGKeyboard/zh-Hans.lproj/InfoPlist.strings +++ b/OSGKeyboard/zh-Hans.lproj/InfoPlist.strings @@ -5,3 +5,4 @@ "NSContactsUsageDescription" = "仅当你为复制的电话号码选择“新建联系人”时，OSGKeyboard 才会使用通讯录。"; "NSRemindersFullAccessUsageDescription" = "仅当你为复制的文本选择“提取待办”时，OSGKeyboard 才会添加提醒事项。"; "NSCalendarsWriteOnlyAccessUsageDescription" = "仅当你为复制的文本选择“提取日程”时，OSGKeyboard 才会添加日历事件，不会读取你已有的日程。"; +"NSPhotoLibraryAddUsageDescription" = "仅当你选择“保存图片”时，OSGKeyboard 才会把邀请海报存入相册。"; diff --git a/OSGKeyboard/zh-Hans.lproj/Localizable.strings b/OSGKeyboard/zh-Hans.lproj/Localizable.strings index 6830909..1ed358a 100644 --- a/OSGKeyboard/zh-Hans.lproj/Localizable.strings +++ b/OSGKeyboard/zh-Hans.lproj/Localizable.strings @@ -983,6 +983,35 @@ "account.referral.pendingAfterSignIn" = "邀请已暂存，登录后会自动兑换。"; "account.referral.rewardGranted" = "恭喜获得 %@ 积分"; "account.referral.rewardGrantedGeneric" = "邀请奖励已发放，积分已到账。"; +"account.referral.poster.title" = "邀请好友"; +"account.referral.poster.accessibility" = "带有你的邀请链接二维码的邀请海报"; +"account.referral.poster.enlargeHint" = "全屏显示二维码，方便当面扫码。"; +"account.referral.poster.save" = "保存图片"; +"account.referral.poster.share" = "分享"; +"account.referral.poster.copyLink" = "复制链接"; +"account.referral.poster.footnote" = "把图片发给好友，或点海报放大二维码当面扫。好友注册并完成首次有效使用后，你们都能获得积分。"; +"account.referral.poster.saved" = "已保存到相册"; +"account.referral.poster.saveDenied" = "请在“设置”中允许 OSGKeyboard 添加照片，然后重试。"; +"account.referral.poster.saveFailed" = "图片保存失败，请重试。"; +"account.referral.poster.linkCopied" = "邀请链接已复制"; +"account.referral.codeEntry.title" = "有好友邀请？"; +"account.referral.codeEntry.body" = "输入好友的邀请码后登录，即可接受邀请"; +"account.referral.codeEntry.bodySignedIn" = "输入好友的邀请码，你们都能获得积分。"; +"account.referral.codeEntry.button" = "我有邀请码"; +"account.referral.codeEntry.windowDescription" = "输入邀请码，或邀请更多好友获取积分"; +"account.referral.codeEntry.sheetTitle" = "输入邀请码"; +"account.referral.codeEntry.sheetBody" = "输入好友海报或邀请页上的 8 位邀请码，也可以直接粘贴邀请链接。"; +"account.referral.codeEntry.placeholder" = "例如 K7M2-Q9XA"; +"account.referral.codeEntry.fieldLabel" = "邀请码"; +"account.referral.codeEntry.confirm" = "确认"; +"account.referral.codeEntry.invalid" = "没有识别到邀请码，请核对后重试。"; +"account.referral.codeEntry.redeemed" = "已接受好友邀请，完成一次有效使用后双方均可获得积分。"; +"account.referral.codeEntry.inviteMore" = "也可以邀请更多好友获取积分"; +"account.referral.redeem.error.invalid" = "邀请码无效，或活动已结束。"; +"account.referral.redeem.error.expired" = "这个账号已超过补填邀请的期限。"; +"account.referral.redeem.error.self" = "不能使用自己的邀请。"; +"account.referral.redeem.error.alreadyBound" = "这个账号已经接受过其他好友的邀请。"; +"account.referral.redeem.error.ineligible" = "这个账号暂时不能接受该邀请。"; "account.security.section" = "账号操作"; "account.signOut.action" = "退出登录"; "account.signOut.confirmTitle" = "退出登录？"; diff --git a/OSGKeyboardMac/MacAccountView.swift b/OSGKeyboardMac/MacAccountView.swift index 38d380a..16a4eac 100644 --- a/OSGKeyboardMac/MacAccountView.swift +++ b/OSGKeyboardMac/MacAccountView.swift @@ -18,6 +18,7 @@ // is server-authoritative either way. import SwiftUI +import UniformTypeIdentifiers #if os(macOS) import AppKit #endif @@ -46,7 +47,11 @@ struct MacAccountSettingsSection: View { case let .signedIn(session): identityRow(session) creditRows - MacInviteLinkRow(profile: coordinator.referralProfile, language: lang) + MacInviteLinkRow( + profile: coordinator.referralProfile, + inviterName: session.displayName, + language: lang + ) referralCountRows accountActionsRow deleteRows @@ -58,9 +63,7 @@ struct MacAccountSettingsSection: View { MacL10n.string("mac.account.error.title", language: lang), isPresented: operationErrorBinding ) { - Button(MacL10n.string("mac.account.error.dismiss", language: lang)) { - coordinator.dismissOperationError() - } + Button(MacL10n.string("mac.account.error.dismiss", language: lang)) {} } message: { if let key = coordinator.operationErrorKey { // The code is appended verbatim: on Mac there is no TestFlight @@ -268,7 +271,12 @@ struct MacAccountSettingsSection: View { Binding( get: { coordinator.operationErrorKey != nil }, set: { isPresented in - if !isPresented { coordinator.dismissOperationError() } + guard !isPresented else { return } + // SwiftUI writes this binding while updating the alert. + // Defer ObservableObject mutations until that update ends. + Task { @MainActor in + coordinator.dismissOperationError() + } } ) } @@ -283,10 +291,12 @@ private struct MacInviteLinkRow: View { @ObservedObject var profile: ReferralProfileViewModel @Environment(\.themePalette) private var palette + let inviterName: String? let language: AppUILanguage @State private var didCopy = false @State private var copyResetTask: Task? + @State private var isShowingPoster = false var body: some View { content @@ -312,23 +322,46 @@ private struct MacInviteLinkRow: View { title: MacL10n.string("mac.account.invite.link", language: language), subtitle: loaded.code.inviteURL.absoluteString ) { - Button { - copy(loaded.code.inviteURL) - } label: { - Label( - MacL10n.string( - didCopy - ? "mac.account.invite.copied" - : "mac.account.invite.copy", + HStack(spacing: Spacing.md) { + Button { + isShowingPoster = true + } label: { + Label( + MacL10n.string("mac.account.invite.qr", language: language), + systemImage: "qrcode" + ) + .font(MacSettingsType.rowLabel) + } + .buttonStyle(.plain) + .foregroundStyle(palette.accent) + .accessibilityIdentifier("mac.account.invite.qr") + .popover(isPresented: $isShowingPoster, arrowEdge: .bottom) { + MacInvitePosterPopover( + inviteURL: loaded.code.inviteURL, + shortCode: loaded.code.displayShortCode, + inviterName: inviterName, language: language - ), - systemImage: didCopy ? "checkmark" : "doc.on.doc" - ) - .font(MacSettingsType.rowLabel) + ) + } + + Button { + copy(loaded.code.inviteURL) + } label: { + Label( + MacL10n.string( + didCopy + ? "mac.account.invite.copied" + : "mac.account.invite.copy", + language: language + ), + systemImage: didCopy ? "checkmark" : "doc.on.doc" + ) + .font(MacSettingsType.rowLabel) + } + .buttonStyle(.plain) + .foregroundStyle(palette.accent) + .accessibilityIdentifier("mac.account.invite.copy") } - .buttonStyle(.plain) - .foregroundStyle(palette.accent) - .accessibilityIdentifier("mac.account.invite.copy") } case let .failed(messageKey): @@ -361,3 +394,95 @@ private struct MacInviteLinkRow: View { } } } + +/// The same poster as iOS. On a Mac it is mostly scanned straight off the +/// screen with a phone, or saved as a PNG to send from a chat app. +private struct MacInvitePosterPopover: View { + @Environment(\.themePalette) private var palette + @Environment(\.colorScheme) private var colorScheme + + let inviteURL: URL + let shortCode: String? + let inviterName: String? + let language: AppUILanguage + + @State private var poster: CGImage? + @State private var saveFailed = false + + var body: some View { + VStack(spacing: Spacing.md) { + Group { + if let poster { + Image(decorative: poster, scale: InvitePosterRenderer.exportScale) + .resizable() + .scaledToFit() + } else { + ProgressView().controlSize(.small) + } + } + .frame(width: 270, height: 360) + .clipShape(RoundedRectangle(cornerRadius: Radius.large, style: .continuous)) + + // Selectable so it can be copied into a chat on the Mac. + if let shortCode { + Text(SharedL10n.format("invite.poster.code", language: language, shortCode)) + .font(.system(.body, design: .monospaced, weight: .semibold)) + .foregroundStyle(palette.textPrimary) + .textSelection(.enabled) + .accessibilityIdentifier("mac.account.invite.shortCode") + } + + Text(MacL10n.string("mac.account.invite.qrHint", language: language)) + .font(MacSettingsType.hint) + .foregroundStyle(palette.textSecondary) + .multilineTextAlignment(.center) + .fixedSize(horizontal: false, vertical: true) + + Button(MacL10n.string("mac.account.invite.savePoster", language: language)) { + savePoster() + } + .disabled(poster == nil) + .accessibilityIdentifier("mac.account.invite.savePoster") + + if saveFailed { + Text(MacL10n.string("mac.account.invite.saveFailed", language: language)) + .font(MacSettingsType.hint) + .foregroundStyle(palette.danger) + } + } + .padding(Spacing.lg) + .frame(width: 310) + .task(id: colorScheme) { + poster = InvitePosterRenderer.render( + inviteURL: inviteURL, + shortCode: shortCode, + inviterName: inviterName, + language: language, + colorScheme: colorScheme + ) + } + } + + private func savePoster() { + #if os(macOS) + guard let poster else { return } + let panel = NSSavePanel() + panel.allowedContentTypes = [.png] + panel.canCreateDirectories = true + panel.nameFieldStringValue = "OSGKeyboard-Invite.png" + guard panel.runModal() == .OK, let url = panel.url else { return } + + let bitmap = NSBitmapImageRep(cgImage: poster) + guard let data = bitmap.representation(using: .png, properties: [:]) else { + saveFailed = true + return + } + do { + try data.write(to: url, options: .atomic) + saveFailed = false + } catch { + saveFailed = true + } + #endif + } +} diff --git a/OSGKeyboardShared/DesignSystem/ColorLibrary.swift b/OSGKeyboardShared/DesignSystem/ColorLibrary.swift index 58ebdc4..c3fb819 100644 --- a/OSGKeyboardShared/DesignSystem/ColorLibrary.swift +++ b/OSGKeyboardShared/DesignSystem/ColorLibrary.swift @@ -17,6 +17,13 @@ public enum OSGColor { public static let fixedLightContent = Color.white public static let fixedDarkContent = Color.black + // MARK: - Invitation poster + + /// Shared artwork must look identical for every recipient, so these do not + /// follow the app appearance. The background matches the app icon. + public static let invitePosterBackground = color(0x252527) + public static let invitePosterHint = color(0xA1A1A6) + // MARK: - iOS app theme public static let darkBackground = color(0x0A0A0B) diff --git a/OSGKeyboardShared/en.lproj/Shared.strings b/OSGKeyboardShared/en.lproj/Shared.strings index b60df32..8ba79dd 100644 --- a/OSGKeyboardShared/en.lproj/Shared.strings +++ b/OSGKeyboardShared/en.lproj/Shared.strings @@ -498,6 +498,16 @@ "mac.account.invite.rewarded" = "Rewarded invites"; "mac.account.invite.pending" = "Pending invites"; "mac.account.invite.footnote" = "Both of you get credits once an invited friend qualifies."; +"mac.account.invite.qr" = "QR Code"; +"mac.account.invite.qrHint" = "Scan with a phone, or save the poster and send it to a friend."; +"mac.account.invite.savePoster" = "Save Poster…"; +"mac.account.invite.saveFailed" = "Couldn’t save the poster."; +"invite.poster.slogan" = "Your AI voice double"; +"invite.poster.hint" = "Scan to try it and earn credits"; +"invite.poster.defaultInviter" = "OSG User"; +"invite.poster.inviter" = "%@ invites you to try\nOSG AI Keyboard"; +"invite.poster.scan" = "Scan to download and enter the code to earn credits"; +"invite.poster.code" = "Invitation code: %@"; "mac.account.delete.title" = "Delete Account"; "mac.account.delete.body" = "Deleting removes your account and any remaining credits. This cannot be undone."; "mac.account.delete.action" = "Delete Account…"; @@ -523,6 +533,11 @@ "account.referral.error.notFound" = "Your invitation link isn’t available yet."; "account.referral.error.session" = "Sign in again to load your invitation link."; "account.referral.error.unavailable" = "The invitation service is temporarily unavailable."; +"account.referral.redeem.error.invalid" = "This invitation link is invalid, or the campaign has ended."; +"account.referral.redeem.error.expired" = "The time to add an invitation to this account has passed."; +"account.referral.redeem.error.self" = "You can’t use your own invitation."; +"account.referral.redeem.error.alreadyBound" = "This account has already accepted another invitation."; +"account.referral.redeem.error.ineligible" = "This account can’t accept this invitation."; /* Rime / typing engine errors */ "error.rime.appGroupUnavailable" = "App Group unavailable."; diff --git a/OSGKeyboardShared/zh-Hans.lproj/Shared.strings b/OSGKeyboardShared/zh-Hans.lproj/Shared.strings index 25c53d8..2808b0a 100644 --- a/OSGKeyboardShared/zh-Hans.lproj/Shared.strings +++ b/OSGKeyboardShared/zh-Hans.lproj/Shared.strings @@ -498,6 +498,16 @@ "mac.account.invite.rewarded" = "已奖励邀请"; "mac.account.invite.pending" = "待生效邀请"; "mac.account.invite.footnote" = "受邀好友达成条件后，双方都可获得积分。"; +"mac.account.invite.qr" = "二维码"; +"mac.account.invite.qrHint" = "用手机扫码，或保存海报发给好友。"; +"mac.account.invite.savePoster" = "保存海报…"; +"mac.account.invite.saveFailed" = "海报保存失败。"; +"invite.poster.slogan" = "你的 AI 嘴替"; +"invite.poster.hint" = "扫一扫 · 下载体验领积分"; +"invite.poster.defaultInviter" = "OSG 用户"; +"invite.poster.inviter" = "%@ 邀你体验\nOSG AI 输入法"; +"invite.poster.scan" = "扫码下载，输入邀请码获取积分"; +"invite.poster.code" = "邀请码：%@"; "mac.account.delete.title" = "注销账号"; "mac.account.delete.body" = "注销会删除你的账号及剩余积分，且无法撤销。"; "mac.account.delete.action" = "注销账号…"; @@ -523,6 +533,11 @@ "account.referral.error.notFound" = "邀请链接暂不可用。"; "account.referral.error.session" = "请重新登录后加载邀请链接。"; "account.referral.error.unavailable" = "邀请服务暂时不可用。"; +"account.referral.redeem.error.invalid" = "邀请链接无效，或活动已结束。"; +"account.referral.redeem.error.expired" = "这个账号已超过补填邀请的期限。"; +"account.referral.redeem.error.self" = "不能使用自己的邀请。"; +"account.referral.redeem.error.alreadyBound" = "这个账号已经接受过其他好友的邀请。"; +"account.referral.redeem.error.ineligible" = "这个账号暂时不能接受该邀请。"; /* 输入法引擎错误 */ "error.rime.appGroupUnavailable" = "App Group 不可用。"; diff --git a/OSGKeyboardTests/AccountCenterViewModelTests.swift b/OSGKeyboardTests/AccountCenterViewModelTests.swift index 4336c5a..969a3b1 100644 --- a/OSGKeyboardTests/AccountCenterViewModelTests.swift +++ b/OSGKeyboardTests/AccountCenterViewModelTests.swift @@ -5,6 +5,7 @@ import AuthenticationServices @testable import OSGKeyboard +@testable import OSGKeyboardHostSupport import XCTest final class AccountCenterViewModelTests: XCTestCase { @@ -34,6 +35,185 @@ final class AccountCenterViewModelTests: XCTestCase { } } + func testPastedInputAcceptsCodeLinkAndChatMessage() { + let inputs = [ + validCode, + " \(validCode)\n", + "https://osglab.com/i/\(validCode)", + "https://osglab.com/i/\(validCode)?source=poster", + "快来试试 OSGKeyboard：https://osglab.com/i/\(validCode)", + "邀请你加入https://osglab.com/i/\(validCode)，一起领积分", + "Join me: " + ] + + for input in inputs { + XCTAssertEqual( + ReferralUniversalLink.code(fromUserInput: input), + validCode, + "Failed to extract from \(input)" + ) + } + } + + func testPastedInputRejectsForeignLinksAndPlainText() { + let inputs = [ + "", + "hello", + "https://example.com/i/\(validCode)", + "http://osglab.com/i/\(validCode)", + "https://osglab.com/i/short", + String(validCode.dropLast()) + ] + + for input in inputs { + XCTAssertNil( + ReferralUniversalLink.code(fromUserInput: input), + "Unexpectedly accepted \(input)" + ) + } + } + + func testTypedInputAcceptsShortCodeInAnyCaseWithOrWithoutHyphen() { + let inputs = [ + "K7M2Q9XA", + "k7m2-q9xa", + " K7M2-Q9XA \n", + "邀请码K7M2-Q9XA", + "我的邀请码：K7M2-Q9XA，快来" + ] + + for input in inputs { + XCTAssertEqual( + ReferralUniversalLink.code(fromUserInput: input), + "K7M2Q9XA", + "Failed to extract from \(input)" + ) + } + } + + func testTypedInputRejectsAmbiguousOrMalformedShortCodes() { + let inputs = [ + "K7M2Q9X", + "K7M2Q9XAB", + "K7M-2Q9XA", + "K0M2Q9XA", + "KOM2Q9XA", + "K1M2Q9XA", + "KLM2Q9XA" + ] + + for input in inputs { + XCTAssertNil( + ReferralUniversalLink.code(fromUserInput: input), + "Unexpectedly accepted \(input)" + ) + } + } + + func testInvitationLinkWinsOverShortCodeLikeWordsInTheSameMessage() { + // "HEXAGRAM" uses only short-code characters. + let input = "HEXAGRAM https://osglab.com/i/\(validCode)" + + XCTAssertEqual(ReferralUniversalLink.code(fromUserInput: input), validCode) + } + + func testShortCodeDisplayAddsHyphen() { + XCTAssertEqual(ReferralShortCode.display("K7M2Q9XA"), "K7M2-Q9XA") + XCTAssertEqual(ReferralShortCode.display("bogus"), "bogus") + } + + @MainActor + func testPendingStoreKeepsShortCodesForRedemptionAfterSignIn() throws { + let suiteName = "PendingReferralShortCode.\(UUID().uuidString)" + let defaults = try XCTUnwrap(UserDefaults(suiteName: suiteName)) + defer { defaults.removePersistentDomain(forName: suiteName) } + let store = UserDefaultsPendingReferralCodeStore(defaults: defaults) + + store.save("K7M2Q9XA") + + XCTAssertEqual(store.code, "K7M2Q9XA") + } + + func testBindingWindowFollowsServerDeadlineWhenPresent() throws { + let account = AccountSession(accountID: UUID(), createdAtEpochSeconds: 1_700_000_000) + let deadline = Date(timeIntervalSince1970: 1_700_003_600) + let profile = ReferralProfile( + code: try makeReferralCode(), + binding: nil, + bindingDeadline: deadline + ) + + XCTAssertTrue(ReferralBindingWindow.isOpen(for: account, profile: profile, now: deadline)) + XCTAssertFalse(ReferralBindingWindow.isOpen( + for: account, + profile: profile, + now: deadline.addingTimeInterval(1) + )) + } + + func testBindingWindowFallsBackToSevenDaysWithoutServerDeadline() throws { + let account = AccountSession(accountID: UUID(), createdAtEpochSeconds: 1_700_000_000) + let profile = ReferralProfile(code: try makeReferralCode(), binding: nil) + let start = Date(timeIntervalSince1970: 1_700_000_000) + + XCTAssertTrue(ReferralBindingWindow.isOpen( + for: account, + profile: profile, + now: start.addingTimeInterval(6 * 24 * 60 * 60) + )) + } + + func testBindingWindowClosesOnceBound() throws { + let account = AccountSession(accountID: UUID(), createdAtEpochSeconds: 1_700_000_000) + let binding = try JSONDecoder().decode( + ReferralBinding.self, + from: Data(#"{"inviterUserId":"x"}"#.utf8) + ) + let profile = ReferralProfile( + code: try makeReferralCode(), + binding: binding, + bindingDeadline: .distantFuture + ) + + XCTAssertFalse(ReferralBindingWindow.isOpen( + for: account, + profile: profile, + now: Date(timeIntervalSince1970: 1_700_000_001) + )) + } + + private func makeReferralCode() throws -> ReferralCode { + ReferralCode( + code: validCode, + inviteURL: try XCTUnwrap(URL(string: "https://osglab.com/i/\(validCode)")), + campaignID: nil, + createdAt: Date(timeIntervalSince1970: 1_700_000_000), + shortCode: "K7M2Q9XA" + ) + } + + func testBindingWindowClosesSevenDaysAfterRegistration() { + let registeredAt: Int64 = 1_700_000_000 + let account = AccountSession( + accountID: UUID(), + createdAtEpochSeconds: registeredAt + ) + let start = Date(timeIntervalSince1970: TimeInterval(registeredAt)) + + XCTAssertTrue(ReferralBindingWindow.isOpen( + for: account, + now: start.addingTimeInterval(6 * 24 * 60 * 60) + )) + XCTAssertFalse(ReferralBindingWindow.isOpen( + for: account, + now: start.addingTimeInterval(7 * 24 * 60 * 60) + )) + XCTAssertFalse(ReferralBindingWindow.isOpen( + for: AccountSession(accountID: UUID(), createdAtEpochSeconds: 0), + now: start + )) + } + func testReferralSummaryCountsEveryServerStatus() { let referrals = [ makeReferral(status: .pending), @@ -439,6 +619,151 @@ final class AccountCenterViewModelTests: XCTestCase { XCTAssertEqual(coordinator.operationErrorKey, "account.error.redeemReferral") } + @MainActor + func testPermanentRedemptionRejectionsDropTheStoredCode() async { + let cases: [(AccountAPIError, String)] = [ + ( + .server(statusCode: 404, code: "http_error", message: "Referral code does not exist"), + "account.referral.redeem.error.invalid" + ), + ( + .server(statusCode: 422, code: "http_error", message: "Referral binding window has expired"), + "account.referral.redeem.error.expired" + ), + ( + .server(statusCode: 409, code: "http_error", message: "Self-referral is not allowed"), + "account.referral.redeem.error.self" + ), + ( + .server( + statusCode: 409, + code: "http_error", + message: "This account is already bound to another inviter" + ), + "account.referral.redeem.error.alreadyBound" + ), + ( + .conflict("This account is not eligible for referral rewards"), + "account.referral.redeem.error.ineligible" + ) + ] + + for (error, expectedKey) in cases { + let account = AccountSession( + accountID: UUID(), + createdAtEpochSeconds: 1_700_000_000 + ) + let service = AccountServiceSpy( + restoredSession: account, + snapshot: makeSnapshot(account: account), + redemptionError: error + ) + let store = InMemoryPendingReferralStore(code: validCode) + let coordinator = AccountSessionCoordinator( + dependencies: AccountDependencies( + sessionService: service, + centerService: service + ), + pendingReferralStore: store + ) + + await coordinator.restoreIfNeeded() + + XCTAssertNil(store.code, "Kept code after \(error)") + XCTAssertNil(coordinator.pendingReferralCode) + XCTAssertEqual(coordinator.operationErrorKey, expectedKey) + } + } + + @MainActor + func testPastedInvitationWhileSignedOutIsSavedForSignIn() async { + let service = AccountServiceSpy(restoredSession: nil) + let store = InMemoryPendingReferralStore() + let coordinator = AccountSessionCoordinator( + dependencies: AccountDependencies( + sessionService: service, + centerService: service + ), + pendingReferralStore: store + ) + await coordinator.restoreIfNeeded() + + let invalid = await coordinator.submitReferralInput("not an invitation") + let saved = await coordinator.submitReferralInput( + "Join me https://osglab.com/i/\(validCode)" + ) + + XCTAssertEqual(invalid, .invalidInput) + XCTAssertEqual(saved, .savedForSignIn) + XCTAssertEqual(store.code, validCode) + XCTAssertEqual(coordinator.pendingReferralCode, validCode) + XCTAssertFalse(coordinator.consumeAccountCenterPresentation()) + let redeemedCodes = await service.redeemedCodes() + XCTAssertEqual(redeemedCodes, []) + } + + @MainActor + func testPastedInvitationWhileSignedInRedeemsImmediately() async { + let account = AccountSession( + accountID: UUID(), + createdAtEpochSeconds: 1_700_000_000 + ) + let service = AccountServiceSpy( + restoredSession: account, + snapshot: makeSnapshot(account: account) + ) + let store = InMemoryPendingReferralStore() + let coordinator = AccountSessionCoordinator( + dependencies: AccountDependencies( + sessionService: service, + centerService: service + ), + pendingReferralStore: store + ) + await coordinator.restoreIfNeeded() + + let outcome = await coordinator.submitReferralInput(validCode) + + XCTAssertEqual(outcome, .redeemed) + XCTAssertNil(store.code) + XCTAssertNil(coordinator.pendingReferralCode) + XCTAssertNil(coordinator.operationErrorKey) + let redeemedCodes = await service.redeemedCodes() + XCTAssertEqual(redeemedCodes, [validCode]) + } + + @MainActor + func testPastedInvitationRejectionIsReturnedInsteadOfAlerting() async { + let account = AccountSession( + accountID: UUID(), + createdAtEpochSeconds: 1_700_000_000 + ) + let service = AccountServiceSpy( + restoredSession: account, + snapshot: makeSnapshot(account: account), + redemptionError: .server( + statusCode: 422, + code: "http_error", + message: "Referral binding window has expired" + ) + ) + let store = InMemoryPendingReferralStore() + let coordinator = AccountSessionCoordinator( + dependencies: AccountDependencies( + sessionService: service, + centerService: service + ), + pendingReferralStore: store + ) + await coordinator.restoreIfNeeded() + + let outcome = await coordinator.submitReferralInput(validCode) + + XCTAssertEqual(outcome, .failed(messageKey: "account.referral.redeem.error.expired")) + XCTAssertNil(store.code) + XCTAssertNil(coordinator.operationErrorKey) + } + @MainActor func testSignOutClearsRemoteAccountStateOnly() async { let account = AccountSession( @@ -800,6 +1125,7 @@ private actor AccountServiceSpy: AccountSessionServicing, AccountCenterServicing private let centerSnapshot: AccountCenterSnapshot? private let refreshedSnapshot: AccountCenterSnapshot? private let shouldFailRedemption: Bool + private let redemptionError: AccountAPIError? private let shouldFailAccountRefresh: Bool private let shouldFailSignOut: Bool private let signOutDelayNanoseconds: UInt64 @@ -822,6 +1148,7 @@ private actor AccountServiceSpy: AccountSessionServicing, AccountCenterServicing snapshot: AccountCenterSnapshot? = nil, refreshedSnapshot: AccountCenterSnapshot? = nil, shouldFailRedemption: Bool = false, + redemptionError: AccountAPIError? = nil, shouldFailAccountRefresh: Bool = false, shouldFailSignOut: Bool = false, signOutDelayNanoseconds: UInt64 = 0, @@ -836,6 +1163,7 @@ private actor AccountServiceSpy: AccountSessionServicing, AccountCenterServicing centerSnapshot = snapshot self.refreshedSnapshot = refreshedSnapshot self.shouldFailRedemption = shouldFailRedemption + self.redemptionError = redemptionError self.shouldFailAccountRefresh = shouldFailAccountRefresh self.shouldFailSignOut = shouldFailSignOut self.signOutDelayNanoseconds = signOutDelayNanoseconds @@ -925,6 +1253,9 @@ private actor AccountServiceSpy: AccountSessionServicing, AccountCenterServicing if shouldFailRedemption { throw AccountServiceSpyError.failed } + if let redemptionError { + throw redemptionError + } redeemed.append(code) } diff --git a/OSGKeyboardTests/InvitePosterTests.swift b/OSGKeyboardTests/InvitePosterTests.swift new file mode 100644 index 0000000..1ab4e0e --- /dev/null +++ b/OSGKeyboardTests/InvitePosterTests.swift @@ -0,0 +1,123 @@ +// InvitePosterTests.swift +// OSGKeyboardTests +// +// The poster is only useful if a camera can read it back, so these tests +// decode the generated QR code instead of only checking that an image exists. + +import CoreImage +@testable import OSGKeyboard +import OSGKeyboardShared +import XCTest + +final class InvitePosterTests: XCTestCase { + private let inviteURL = URL( + string: "https://osglab.com/i/Abcdefghij_1234567890-?source=server" + )! + + func testQRCodeDecodesToTheExactInvitationLink() throws { + let image = try XCTUnwrap(InviteQRCode.makeImage(for: inviteURL)) + + XCTAssertEqual(decodedMessages(in: image), [inviteURL.absoluteString]) + } + + @MainActor + func testPosterExportsAtFixedSizeAndStaysScannable() throws { + let poster = try XCTUnwrap( + InvitePosterRenderer.render( + inviteURL: inviteURL, + inviterName: nil, + language: .chinese, + colorScheme: .light + ) + ) + + XCTAssertEqual(poster.width, 1080) + XCTAssertEqual(poster.height, 1440) + XCTAssertEqual(decodedMessages(in: poster), [inviteURL.absoluteString]) + XCTAssertGreaterThan(cornerLuminance(of: poster), 0.95) + } + + @MainActor + func testDarkPosterWithShortCodeAndLongNameStaysScannable() throws { + let poster = try XCTUnwrap( + InvitePosterRenderer.render( + inviteURL: inviteURL, + shortCode: "K7M2-Q9XA", + inviterName: "一位用户名比较长但仍然需要完整显示的朋友", + language: .chinese, + colorScheme: .dark + ) + ) + + XCTAssertEqual(poster.width, 1080) + XCTAssertEqual(poster.height, 1440) + XCTAssertEqual(decodedMessages(in: poster), [inviteURL.absoluteString]) + XCTAssertGreaterThan( + darkPixelFraction( + in: poster, + crop: CGRect(x: 300, y: 600, width: 480, height: 500) + ), + 0.15 + ) + // The graphite paper carries a subtle daylight wash, but remains + // distinctly dark rather than approaching the warm-paper surface. + XCTAssertLessThan(cornerLuminance(of: poster), 0.18) + } + + private func decodedMessages(in image: CGImage) -> [String] { + let detector = CIDetector( + ofType: CIDetectorTypeQRCode, + context: nil, + options: [CIDetectorAccuracy: CIDetectorAccuracyHigh] + ) + return (detector?.features(in: CIImage(cgImage: image)) ?? []) + .compactMap { ($0 as? CIQRCodeFeature)?.messageString } + } + + private func cornerLuminance(of image: CGImage) -> Double { + guard let corner = image.cropping(to: CGRect(x: 0, y: 0, width: 1, height: 1)) else { + return -1 + } + var pixel = [UInt8](repeating: 0, count: 4) + let context = CGContext( + data: &pixel, + width: 1, + height: 1, + bitsPerComponent: 8, + bytesPerRow: 4, + space: CGColorSpaceCreateDeviceRGB(), + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue + ) + context?.draw(corner, in: CGRect(x: 0, y: 0, width: 1, height: 1)) + return (0.2126 * Double(pixel[0]) + + 0.7152 * Double(pixel[1]) + + 0.0722 * Double(pixel[2])) / 255 + } + + private func darkPixelFraction(in image: CGImage, crop: CGRect) -> Double { + guard let cropped = image.cropping(to: crop) else { return 0 } + let width = 120 + let height = 120 + var pixels = [UInt8](repeating: 0, count: width * height * 4) + guard let context = CGContext( + data: &pixels, + width: width, + height: height, + bitsPerComponent: 8, + bytesPerRow: width * 4, + space: CGColorSpaceCreateDeviceRGB(), + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue + ) else { + return 0 + } + context.interpolationQuality = .none + context.draw(cropped, in: CGRect(x: 0, y: 0, width: width, height: height)) + let darkCount = (0..<(width * height)).count { index in + let offset = index * 4 + return Int(pixels[offset]) + + Int(pixels[offset + 1]) + + Int(pixels[offset + 2]) < 3 * 100 + } + return Double(darkCount) / Double(width * height) + } +} diff --git a/OSGKeyboardTests/ReferralProfileTests.swift b/OSGKeyboardTests/ReferralProfileTests.swift index b3a7a0a..1d2332c 100644 --- a/OSGKeyboardTests/ReferralProfileTests.swift +++ b/OSGKeyboardTests/ReferralProfileTests.swift @@ -37,6 +37,59 @@ final class ReferralProfileTests: XCTestCase { XCTAssertEqual(profile.code.inviteURL.absoluteString, inviteURL) XCTAssertNil(profile.code.campaignID) XCTAssertNotNil(profile.binding) + XCTAssertNil(profile.code.shortCode) + XCTAssertNil(profile.bindingDeadline) + } + + func testReferralProfileDecodesShortCodeAndBindingDeadlineAndSurvivesCache() throws { + let data = Data( + """ + { + "code": { + "code": "Abcdefghij_1234567890-", + "inviteUrl": "https://osglab.com/i/Abcdefghij_1234567890-", + "createdAt": "2026-08-20T12:34:56Z", + "shortCode": "K7M2Q9XA" + }, + "bindingDeadline": "2026-08-27T12:34:56Z" + } + """.utf8 + ) + + let profile = try JSONDecoder().decode(ReferralProfile.self, from: data) + let cached = try JSONDecoder().decode( + ReferralProfile.self, + from: JSONEncoder().encode(profile) + ) + + XCTAssertEqual(profile.code.shortCode, "K7M2Q9XA") + XCTAssertEqual(profile.code.displayShortCode, "K7M2-Q9XA") + XCTAssertEqual( + profile.bindingDeadline, + ISO8601DateFormatter().date(from: "2026-08-27T12:34:56Z") + ) + XCTAssertEqual(cached, profile) + } + + func testMalformedShortCodeIsDroppedWithoutRejectingTheProfile() throws { + let data = Data( + """ + { + "code": { + "code": "Abcdefghij_1234567890-", + "inviteUrl": "https://osglab.com/i/Abcdefghij_1234567890-", + "createdAt": "2026-08-20T12:34:56Z", + "shortCode": "0OIL0OIL" + }, + "bindingDeadline": "not a date" + } + """.utf8 + ) + + let profile = try JSONDecoder().decode(ReferralProfile.self, from: data) + + XCTAssertNil(profile.code.shortCode) + XCTAssertNil(profile.bindingDeadline) } func testLiveReferralServiceUsesAuthenticatedGETWithBoundedTimeout() async throws { diff --git a/OSGKeyboardTests/ScreenBrightnessBoostTests.swift b/OSGKeyboardTests/ScreenBrightnessBoostTests.swift new file mode 100644 index 0000000..3fd984b --- /dev/null +++ b/OSGKeyboardTests/ScreenBrightnessBoostTests.swift @@ -0,0 +1,81 @@ +// ScreenBrightnessBoostTests.swift +// OSGKeyboardTests +// +// Whatever order holds and scene changes arrive in, the user's own +// brightness must come back once nothing needs the boost. + +@testable import OSGKeyboard +import XCTest + +@MainActor +final class ScreenBrightnessBoostTests: XCTestCase { + private final class FakeScreen: ScreenBrightnessControlling { + var brightness: CGFloat + init(brightness: CGFloat) { self.brightness = brightness } + } + + func testHoldRaisesBrightnessAndReleaseRestoresIt() { + let screen = FakeScreen(brightness: 0.4) + let boost = ScreenBrightnessBoost { screen } + + boost.acquire() + XCTAssertEqual(screen.brightness, 1) + + boost.release() + XCTAssertEqual(screen.brightness, 0.4) + } + + func testOverlappingHoldsKeepTheOriginalLevel() { + let screen = FakeScreen(brightness: 0.3) + let boost = ScreenBrightnessBoost { screen } + + // A new view appears before the previous one disappears. + boost.acquire() + boost.acquire() + boost.release() + XCTAssertEqual(screen.brightness, 1) + + boost.release() + XCTAssertEqual(screen.brightness, 0.3) + } + + func testBackgroundRestoresAndForegroundReapplies() { + let screen = FakeScreen(brightness: 0.5) + let boost = ScreenBrightnessBoost { screen } + boost.acquire() + + boost.setInForeground(false) + XCTAssertEqual(screen.brightness, 0.5) + + boost.setInForeground(true) + XCTAssertEqual(screen.brightness, 1) + + boost.release() + XCTAssertEqual(screen.brightness, 0.5) + } + + func testUnbalancedReleaseLeavesBrightnessAlone() { + let screen = FakeScreen(brightness: 0.6) + let boost = ScreenBrightnessBoost { screen } + + boost.release() + boost.acquire() + XCTAssertEqual(screen.brightness, 1) + + boost.release() + XCTAssertEqual(screen.brightness, 0.6) + } + + func testMissingScreenIsRetriedOnTheNextChange() { + var screen: FakeScreen? + let boost = ScreenBrightnessBoost { screen } + + boost.acquire() + screen = FakeScreen(brightness: 0.2) + boost.setInForeground(true) + XCTAssertEqual(screen?.brightness, 1) + + boost.release() + XCTAssertEqual(screen?.brightness, 0.2) + } +} diff --git a/Tests/suite-manifest.json b/Tests/suite-manifest.json index ffa9677..7ccde0f 100644 --- a/Tests/suite-manifest.json +++ b/Tests/suite-manifest.json @@ -32,7 +32,9 @@ "OSGKeyboardTests/AccountSignInCoordinatorTests", "OSGKeyboardTests/AccountSnapshotLoaderTests", "OSGKeyboardTests/DeviceIntegrityTests", - "OSGKeyboardTests/ReferralProfileTests" + "OSGKeyboardTests/ReferralProfileTests", + "OSGKeyboardTests/InvitePosterTests", + "OSGKeyboardTests/ScreenBrightnessBoostTests" ] }, "managed_gateway": { diff --git a/project.yml b/project.yml index b682c36..c253cf0 100644 --- a/project.yml +++ b/project.yml @@ -188,6 +188,7 @@ targets: NSContactsUsageDescription: "OSGKeyboard uses Contacts only when you choose Create Contact for a copied phone number." NSRemindersFullAccessUsageDescription: "OSGKeyboard adds reminders only when you choose Extract To-dos for copied text." NSCalendarsFullAccessUsageDescription: "OSGKeyboard adds calendar events only when you choose Extract Events for copied text." + NSPhotoLibraryAddUsageDescription: "OSGKeyboard saves your invitation poster to Photos only when you choose Save Image." UIBackgroundModes: - audio BGTaskSchedulerPermittedIdentifiers: @@ -579,6 +580,7 @@ targets: - path: OSGKeyboard/Views/Account/AccountCreditPurchaseManager.swift - path: OSGKeyboard/Views/Account/LiveAccountServices.swift - path: OSGKeyboard/Views/Account/ReferralProfileViewModel.swift + - path: OSGKeyboard/Views/Account/InvitePoster.swift - path: OSGKeyboard/Views/Account/AccountAppleAuthorizationFactories.swift - path: OSGKeyboard/Services/AccountAnalyticsBearerBridge.swift - path: OSGKeyboardHostSupport From d540a58594cbb326438dee2a26cdf05acebbeb1a Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:06:03 +0200 Subject: [PATCH 07/19] fix(account): defer operation-error dismissal until the alert update ends --- OSGKeyboard/Views/MainAppRoot.swift | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/OSGKeyboard/Views/MainAppRoot.swift b/OSGKeyboard/Views/MainAppRoot.swift index 7ebdf5c..f2b3348 100644 --- a/OSGKeyboard/Views/MainAppRoot.swift +++ b/OSGKeyboard/Views/MainAppRoot.swift @@ -97,9 +97,7 @@ struct MainAppRoot: View { AppL10n.string("account.error.title"), isPresented: accountOperationErrorBinding ) { - Button(AppL10n.string("common.done")) { - accountSession.dismissOperationError() - } + Button(AppL10n.string("common.done")) {} } message: { if let key = accountSession.operationErrorKey { Text(AppL10n.string(key)) @@ -326,7 +324,11 @@ struct MainAppRoot: View { get: { accountSession.operationErrorKey != nil }, set: { isPresented in if !isPresented { - accountSession.dismissOperationError() + // SwiftUI writes this binding while updating the alert. + // Defer ObservableObject mutations until that update ends. + Task { @MainActor in + accountSession.dismissOperationError() + } } } ) From ad79c2ea13c9add5b6e4ba31c8421a139781ad56 Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:06:19 +0200 Subject: [PATCH 08/19] chore(flow): expand host-return whitelist and add a DEBUG host bundle probe - Rebuild the return-to-host whitelist from the CN/US top charts plus Apple system apps; drop the canOpenURL pre-check (LSApplicationQueriesSchemes is capped at 50) and let open() report failure. - Never record an OSG bundle id as the host to return to. - DEBUG only: probe the host bundle id through the keyboard arbiter callback and surface the log on a Home test card. Release builds keep the previous behaviour (no host identification). --- OSGKeyboard/Services/AppURLHandler.swift | 16 +- OSGKeyboard/Services/HostReturnService.swift | 17 +- .../Components/HostBundleProbeCard.swift | 94 ++++ OSGKeyboard/Views/HomeView.swift | 4 + OSGKeyboard/en.lproj/Localizable.strings | 108 ++++- OSGKeyboard/zh-Hans.lproj/Localizable.strings | 110 ++++- OSGKeyboardExt/KeyboardViewController.swift | 5 + .../OSGKeyboardExt-Bridging-Header.h | 6 + .../Services/KeyboardFlowCoordinator.swift | 1 + .../Utilities/HostBundleProbe.swift | 68 +++ .../Utilities/OSGHostArbiterProbe.h | 38 ++ .../Utilities/OSGHostArbiterProbe.m | 133 ++++++ .../Services/HostAppURLRegistry.swift | 445 +++++++----------- .../Utilities/HostBundleProbeLog.swift | 47 ++ .../HostAppURLRegistryTests.swift | 41 +- project.yml | 1 + 16 files changed, 807 insertions(+), 327 deletions(-) create mode 100644 OSGKeyboard/Views/Components/HostBundleProbeCard.swift create mode 100644 OSGKeyboardExt/OSGKeyboardExt-Bridging-Header.h create mode 100644 OSGKeyboardExt/Utilities/HostBundleProbe.swift create mode 100644 OSGKeyboardExt/Utilities/OSGHostArbiterProbe.h create mode 100644 OSGKeyboardExt/Utilities/OSGHostArbiterProbe.m create mode 100644 OSGKeyboardShared/Utilities/HostBundleProbeLog.swift diff --git a/OSGKeyboard/Services/AppURLHandler.swift b/OSGKeyboard/Services/AppURLHandler.swift index 4c2e777..831e3a2 100644 --- a/OSGKeyboard/Services/AppURLHandler.swift +++ b/OSGKeyboard/Services/AppURLHandler.swift @@ -112,12 +112,16 @@ final class AppSceneDelegate: NSObject, UIWindowSceneDelegate { MainActor.assumeIsolated { for item in items { // `sourceApplication` is only non-nil when the caller belongs to - // the same Apple Developer Team (our own keyboard extension) — - // exactly what the host-return whitelist relies on. Record it - // ONLY for the `startflow` handoff: overwriting it for every - // deep link (e.g. `osgkeyboard://settings`) could point a later - // cold-start "return to host" at the wrong app. - if let source = item.source, item.url.host == "startflow" { + // the same Apple Developer Team, so for the keyboard handoff it is + // our own extension, not the app the user was typing in. The + // keyboard publishes the real host before opening `startflow`; + // never overwrite that with an OSG bundle id. Record it ONLY for + // the `startflow` handoff: overwriting it for every deep link + // (e.g. `osgkeyboard://settings`) could point a later cold-start + // "return to host" at the wrong app. + if let source = item.source, + item.url.host == "startflow", + !HostAppURLRegistry.isOwnBundleId(source) { FlowSessionBridge.setPendingHostBundleId(source) } AppOpenURLRouter.shared.route(item.url) diff --git a/OSGKeyboard/Services/HostReturnService.swift b/OSGKeyboard/Services/HostReturnService.swift index c207032..4c9de48 100644 --- a/OSGKeyboard/Services/HostReturnService.swift +++ b/OSGKeyboard/Services/HostReturnService.swift @@ -2,6 +2,10 @@ // OSGKeyboard · Main App // // Opens a whitelisted host-app URL after a cold-start Flow handoff. +// +// No `canOpenURL` pre-check: iOS caps `LSApplicationQueriesSchemes` at 50 +// entries (already spent) and the whitelist is far larger. `open` needs no +// declaration and reports failure through its completion instead. import OSGKeyboardShared import UIKit @@ -13,12 +17,11 @@ enum HostReturnService { let bundleId = FlowSessionBridge.pendingHostBundleId() guard let entry = HostAppURLRegistry.lookup(bundleId: bundleId), let url = entry.returnURL else { - return false - } - guard UIApplication.shared.canOpenURL(url) else { + logReturn(bundleId: bundleId, outcome: "no-entry") return false } UIApplication.shared.open(url, options: [:]) { success in + logReturn(bundleId: bundleId, outcome: success ? "opened" : "open-failed") if success { FlowSessionBridge.clearPendingHostBundleId() } @@ -28,7 +31,7 @@ enum HostReturnService { @MainActor static func openHost(entry: HostAppEntry) -> Bool { - guard let url = entry.returnURL, UIApplication.shared.canOpenURL(url) else { + guard let url = entry.returnURL else { return false } UIApplication.shared.open(url, options: [:]) { success in @@ -43,4 +46,10 @@ enum HostReturnService { static func pendingHostEntry() -> HostAppEntry? { HostAppURLRegistry.lookup(bundleId: FlowSessionBridge.pendingHostBundleId()) } + + private static func logReturn(bundleId: String?, outcome: String) { + #if DEBUG + HostBundleProbeLog.append(event: "return", bundleId: bundleId, detail: outcome) + #endif + } } diff --git a/OSGKeyboard/Views/Components/HostBundleProbeCard.swift b/OSGKeyboard/Views/Components/HostBundleProbeCard.swift new file mode 100644 index 0000000..cbaf90d --- /dev/null +++ b/OSGKeyboard/Views/Components/HostBundleProbeCard.swift @@ -0,0 +1,94 @@ +// HostBundleProbeCard.swift +// OSGKeyboard · Main App +// +// 临时调试卡片（仅 DEBUG）：展示键盘扩展识别到的宿主 bundle ID， +// 用于真机验证 iOS 26.4+ 宿主识别。验证结束后删除。 + +#if DEBUG +import OSGKeyboardShared +import SwiftUI + +struct HostBundleProbeCard: View { + @Environment(\.themePalette) private var palette: ThemePalette + @State private var entries: [HostBundleProbeLog.Entry] = [] + @State private var draft = "" + + private static let timeFormatter: DateFormatter = { + let formatter = DateFormatter() + formatter.dateFormat = "HH:mm:ss.SSS" + return formatter + }() + + var body: some View { + VStack(alignment: .leading, spacing: Spacing.sm) { + HStack { + Text("宿主识别测试（临时）") + .font(TypeStyle.footnote.weight(.semibold)) + .foregroundStyle(palette.textTertiary) + Spacer(minLength: Spacing.xs) + Button("清空") { + HostBundleProbeLog.clear() + reload() + } + .font(TypeStyle.caption2) + .foregroundStyle(palette.accent) + } + + // 在 OSG 自己里唤起键盘时，预期识别结果是主 App 的 bundle ID。 + TextField("点这里唤起 OSG 键盘", text: $draft) + .textFieldStyle(.roundedBorder) + + if entries.isEmpty { + Text("暂无记录：去其他 App 唤起 OSG 键盘后回到这里查看") + .font(TypeStyle.caption2) + .foregroundStyle(palette.textTertiary) + } else { + VStack(alignment: .leading, spacing: 6) { + ForEach(entries) { entry in + entryRow(entry) + } + } + .textSelection(.enabled) + } + } + .padding(Spacing.md) + .frame(maxWidth: .infinity, alignment: .leading) + .surfaceCard() + // 卡片可见期间每秒刷新，在本 App 内唤起键盘时也能实时看到结果。 + .task { + while !Task.isCancelled { + reload() + try? await Task.sleep(for: .seconds(1)) + } + } + } + + private func entryRow(_ entry: HostBundleProbeLog.Entry) -> some View { + VStack(alignment: .leading, spacing: 1) { + HStack(spacing: Spacing.xs) { + Text(Self.timeFormatter.string(from: entry.date)) + .foregroundStyle(palette.textTertiary) + Text(entry.event) + .foregroundStyle(entry.event == "appear" ? palette.textSecondary : palette.accent) + if let detail = entry.detail { + Text(detail) + .foregroundStyle(palette.textTertiary) + .lineLimit(1) + } + } + .font(.system(size: 11, design: .monospaced)) + + if entry.event != "appear" { + Text(entry.bundleId ?? "nil") + .font(.system(size: 13, weight: .semibold, design: .monospaced)) + .foregroundStyle(entry.bundleId == nil ? palette.warning : palette.textPrimary) + } + } + } + + private func reload() { + let latest = HostBundleProbeLog.entries() + if latest != entries { entries = latest } + } +} +#endif diff --git a/OSGKeyboard/Views/HomeView.swift b/OSGKeyboard/Views/HomeView.swift index 4ed623b..7acc3f7 100644 --- a/OSGKeyboard/Views/HomeView.swift +++ b/OSGKeyboard/Views/HomeView.swift @@ -369,6 +369,10 @@ struct HomeView: View { layout: UsageStatsClusterLayout, compact: Bool ) -> some View { + #if DEBUG + HostBundleProbeCard() + #endif + AccountRewardsCard { path.append(HomeRoute.account) } diff --git a/OSGKeyboard/en.lproj/Localizable.strings b/OSGKeyboard/en.lproj/Localizable.strings index 0daf4b6..aea8db4 100644 --- a/OSGKeyboard/en.lproj/Localizable.strings +++ b/OSGKeyboard/en.lproj/Localizable.strings @@ -829,6 +829,9 @@ "hostApp.wecom" = "WeCom"; "hostApp.dingtalk" = "DingTalk"; "hostApp.lark" = "Lark"; +"hostApp.tencentMeeting" = "Tencent Meeting"; +"hostApp.qqmail" = "QQ Mail"; +"hostApp.wps" = "WPS Office"; "hostApp.telegram" = "Telegram"; "hostApp.whatsapp" = "WhatsApp"; "hostApp.line" = "LINE"; @@ -836,28 +839,56 @@ "hostApp.slack" = "Slack"; "hostApp.teams" = "Microsoft Teams"; "hostApp.discord" = "Discord"; -"hostApp.notion" = "Notion"; -"hostApp.bear" = "Bear"; -"hostApp.obsidian" = "Obsidian"; -"hostApp.drafts" = "Drafts"; -"hostApp.gmail" = "Gmail"; -"hostApp.outlook" = "Outlook"; -"hostApp.chrome" = "Chrome"; -"hostApp.weibo" = "Weibo"; -"hostApp.xiaohongshu" = "RED"; "hostApp.signal" = "Signal"; "hostApp.kakaotalk" = "KakaoTalk"; "hostApp.viber" = "Viber"; "hostApp.zalo" = "Zalo"; -"hostApp.skype" = "Skype"; "hostApp.zoom" = "Zoom"; +"hostApp.tinder" = "Tinder"; +"hostApp.notion" = "Notion"; +"hostApp.bear" = "Bear"; +"hostApp.obsidian" = "Obsidian"; +"hostApp.drafts" = "Drafts"; "hostApp.things" = "Things"; "hostApp.todoist" = "Todoist"; "hostApp.evernote" = "Evernote"; "hostApp.onenote" = "OneNote"; +"hostApp.word" = "Microsoft Word"; +"hostApp.excel" = "Microsoft Excel"; +"hostApp.gmail" = "Gmail"; +"hostApp.outlook" = "Outlook"; "hostApp.spark" = "Spark"; +"hostApp.yahooMail" = "Yahoo Mail"; +"hostApp.googleDocs" = "Google Docs"; +"hostApp.googleSheets" = "Google Sheets"; +"hostApp.googleDrive" = "Google Drive"; +"hostApp.googleTranslate" = "Google Translate"; +"hostApp.google" = "Google"; +"hostApp.chrome" = "Chrome"; "hostApp.firefox" = "Firefox"; "hostApp.edge" = "Edge"; +"hostApp.chatgpt" = "ChatGPT"; +"hostApp.claude" = "Claude"; +"hostApp.weibo" = "Weibo"; +"hostApp.xiaohongshu" = "RED"; +"hostApp.zhihu" = "Zhihu"; +"hostApp.bilibili" = "Bilibili"; +"hostApp.douyin" = "Douyin"; +"hostApp.douyinLite" = "Douyin Lite"; +"hostApp.kuaishou" = "Kuaishou"; +"hostApp.kuaishouLite" = "Kuaishou Lite"; +"hostApp.toutiao" = "Toutiao"; +"hostApp.douban" = "Douban"; +"hostApp.baidu" = "Baidu"; +"hostApp.youku" = "Youku"; +"hostApp.tencentVideo" = "Tencent Video"; +"hostApp.iqiyi" = "iQIYI"; +"hostApp.mangoTV" = "Mango TV"; +"hostApp.qqMusic" = "QQ Music"; +"hostApp.neteaseMusic" = "NetEase Cloud Music"; +"hostApp.kugou" = "Kugou Music"; +"hostApp.ximalaya" = "Ximalaya"; +"hostApp.weread" = "WeRead"; "hostApp.facebook" = "Facebook"; "hostApp.instagram" = "Instagram"; "hostApp.x" = "X"; @@ -865,10 +896,61 @@ "hostApp.snapchat" = "Snapchat"; "hostApp.reddit" = "Reddit"; "hostApp.pinterest" = "Pinterest"; -"hostApp.zhihu" = "Zhihu"; -"hostApp.bilibili" = "Bilibili"; -"hostApp.douyin" = "Douyin"; "hostApp.tiktok" = "TikTok"; +"hostApp.youtube" = "YouTube"; +"hostApp.linkedin" = "LinkedIn"; +"hostApp.tumblr" = "Tumblr"; +"hostApp.twitch" = "Twitch"; +"hostApp.spotify" = "Spotify"; +"hostApp.netflix" = "Netflix"; +"hostApp.capcut" = "CapCut"; +"hostApp.duolingo" = "Duolingo"; +"hostApp.taobao" = "Taobao"; +"hostApp.tmall" = "Tmall"; +"hostApp.jd" = "JD.com"; +"hostApp.pinduoduo" = "Pinduoduo"; +"hostApp.xianyu" = "Xianyu"; +"hostApp.alibaba1688" = "1688"; +"hostApp.vipshop" = "Vipshop"; +"hostApp.meituan" = "Meituan"; +"hostApp.meituanWaimai" = "Meituan Waimai"; +"hostApp.eleme" = "Taobao Shangou"; +"hostApp.dianping" = "Dianping"; +"hostApp.wuba" = "58.com"; +"hostApp.didi" = "DiDi"; +"hostApp.amap" = "Amap"; +"hostApp.baiduMap" = "Baidu Maps"; +"hostApp.tencentMap" = "Tencent Maps"; +"hostApp.ctrip" = "Ctrip"; +"hostApp.baiduNetdisk" = "Baidu Netdisk"; +"hostApp.ucBrowser" = "UC Browser"; +"hostApp.qqBrowser" = "QQ Browser"; +"hostApp.alipay" = "Alipay"; +"hostApp.unionpay" = "UnionPay"; +"hostApp.cmb" = "China Merchants Bank"; +"hostApp.ccb" = "China Construction Bank"; +"hostApp.abc" = "Agricultural Bank of China"; +"hostApp.amazon" = "Amazon"; +"hostApp.walmart" = "Walmart"; +"hostApp.ebay" = "eBay"; +"hostApp.aliexpress" = "AliExpress"; +"hostApp.uber" = "Uber"; +"hostApp.lyft" = "Lyft"; +"hostApp.doordash" = "DoorDash"; +"hostApp.airbnb" = "Airbnb"; +"hostApp.googleMaps" = "Google Maps"; +"hostApp.paypal" = "PayPal"; +"hostApp.venmo" = "Venmo"; +"hostApp.cashApp" = "Cash App"; +"hostApp.messages" = "Messages"; +"hostApp.notes" = "Notes"; +"hostApp.mail" = "Mail"; +"hostApp.reminders" = "Reminders"; +"hostApp.calendar" = "Calendar"; +"hostApp.shortcuts" = "Shortcuts"; +"hostApp.files" = "Files"; +"hostApp.maps" = "Maps"; +"hostApp.appStore" = "App Store"; "flow.warning.polishDegradedQuality" = "Polish output failed validation; inserted a conservative version."; "flow.error.editLastInputFailed" = "Could not complete the edit. Please try again."; "flow.error.aiQuestionFailed" = "AI response failed. Please try again."; diff --git a/OSGKeyboard/zh-Hans.lproj/Localizable.strings b/OSGKeyboard/zh-Hans.lproj/Localizable.strings index 1ed358a..6832db0 100644 --- a/OSGKeyboard/zh-Hans.lproj/Localizable.strings +++ b/OSGKeyboard/zh-Hans.lproj/Localizable.strings @@ -829,6 +829,9 @@ "hostApp.wecom" = "企业微信"; "hostApp.dingtalk" = "钉钉"; "hostApp.lark" = "飞书"; +"hostApp.tencentMeeting" = "腾讯会议"; +"hostApp.qqmail" = "QQ邮箱"; +"hostApp.wps" = "WPS Office"; "hostApp.telegram" = "Telegram"; "hostApp.whatsapp" = "WhatsApp"; "hostApp.line" = "LINE"; @@ -836,28 +839,56 @@ "hostApp.slack" = "Slack"; "hostApp.teams" = "Microsoft Teams"; "hostApp.discord" = "Discord"; -"hostApp.notion" = "Notion"; -"hostApp.bear" = "Bear"; -"hostApp.obsidian" = "Obsidian"; -"hostApp.drafts" = "Drafts"; -"hostApp.gmail" = "Gmail"; -"hostApp.outlook" = "Outlook"; -"hostApp.chrome" = "Chrome"; -"hostApp.weibo" = "微博"; -"hostApp.xiaohongshu" = "小红书"; "hostApp.signal" = "Signal"; "hostApp.kakaotalk" = "KakaoTalk"; "hostApp.viber" = "Viber"; "hostApp.zalo" = "Zalo"; -"hostApp.skype" = "Skype"; "hostApp.zoom" = "Zoom"; +"hostApp.tinder" = "Tinder"; +"hostApp.notion" = "Notion"; +"hostApp.bear" = "熊掌记"; +"hostApp.obsidian" = "Obsidian"; +"hostApp.drafts" = "Drafts"; "hostApp.things" = "Things"; "hostApp.todoist" = "Todoist"; -"hostApp.evernote" = "印象笔记"; +"hostApp.evernote" = "Evernote"; "hostApp.onenote" = "OneNote"; +"hostApp.word" = "Word"; +"hostApp.excel" = "Excel"; +"hostApp.gmail" = "Gmail"; +"hostApp.outlook" = "Outlook"; "hostApp.spark" = "Spark"; +"hostApp.yahooMail" = "雅虎邮箱"; +"hostApp.googleDocs" = "Google 文档"; +"hostApp.googleSheets" = "Google 表格"; +"hostApp.googleDrive" = "Google 云端硬盘"; +"hostApp.googleTranslate" = "Google 翻译"; +"hostApp.google" = "Google"; +"hostApp.chrome" = "Chrome"; "hostApp.firefox" = "Firefox"; "hostApp.edge" = "Edge"; +"hostApp.chatgpt" = "ChatGPT"; +"hostApp.claude" = "Claude"; +"hostApp.weibo" = "微博"; +"hostApp.xiaohongshu" = "小红书"; +"hostApp.zhihu" = "知乎"; +"hostApp.bilibili" = "哔哩哔哩"; +"hostApp.douyin" = "抖音"; +"hostApp.douyinLite" = "抖音极速版"; +"hostApp.kuaishou" = "快手"; +"hostApp.kuaishouLite" = "快手极速版"; +"hostApp.toutiao" = "今日头条"; +"hostApp.douban" = "豆瓣"; +"hostApp.baidu" = "百度"; +"hostApp.youku" = "优酷视频"; +"hostApp.tencentVideo" = "腾讯视频"; +"hostApp.iqiyi" = "爱奇艺"; +"hostApp.mangoTV" = "芒果TV"; +"hostApp.qqMusic" = "QQ音乐"; +"hostApp.neteaseMusic" = "网易云音乐"; +"hostApp.kugou" = "酷狗音乐"; +"hostApp.ximalaya" = "喜马拉雅"; +"hostApp.weread" = "微信读书"; "hostApp.facebook" = "Facebook"; "hostApp.instagram" = "Instagram"; "hostApp.x" = "X"; @@ -865,10 +896,61 @@ "hostApp.snapchat" = "Snapchat"; "hostApp.reddit" = "Reddit"; "hostApp.pinterest" = "Pinterest"; -"hostApp.zhihu" = "知乎"; -"hostApp.bilibili" = "哔哩哔哩"; -"hostApp.douyin" = "抖音"; "hostApp.tiktok" = "TikTok"; +"hostApp.youtube" = "YouTube"; +"hostApp.linkedin" = "领英"; +"hostApp.tumblr" = "Tumblr"; +"hostApp.twitch" = "Twitch"; +"hostApp.spotify" = "Spotify"; +"hostApp.netflix" = "Netflix"; +"hostApp.capcut" = "CapCut"; +"hostApp.duolingo" = "多邻国"; +"hostApp.taobao" = "淘宝"; +"hostApp.tmall" = "天猫"; +"hostApp.jd" = "京东"; +"hostApp.pinduoduo" = "拼多多"; +"hostApp.xianyu" = "闲鱼"; +"hostApp.alibaba1688" = "阿里巴巴 1688"; +"hostApp.vipshop" = "唯品会"; +"hostApp.meituan" = "美团"; +"hostApp.meituanWaimai" = "美团外卖"; +"hostApp.eleme" = "淘宝闪购"; +"hostApp.dianping" = "大众点评"; +"hostApp.wuba" = "58同城"; +"hostApp.didi" = "滴滴出行"; +"hostApp.amap" = "高德地图"; +"hostApp.baiduMap" = "百度地图"; +"hostApp.tencentMap" = "腾讯地图"; +"hostApp.ctrip" = "携程旅行"; +"hostApp.baiduNetdisk" = "百度网盘"; +"hostApp.ucBrowser" = "UC浏览器"; +"hostApp.qqBrowser" = "QQ浏览器"; +"hostApp.alipay" = "支付宝"; +"hostApp.unionpay" = "云闪付"; +"hostApp.cmb" = "招商银行"; +"hostApp.ccb" = "中国建设银行"; +"hostApp.abc" = "中国农业银行"; +"hostApp.amazon" = "亚马逊"; +"hostApp.walmart" = "沃尔玛"; +"hostApp.ebay" = "eBay"; +"hostApp.aliexpress" = "速卖通"; +"hostApp.uber" = "Uber"; +"hostApp.lyft" = "Lyft"; +"hostApp.doordash" = "DoorDash"; +"hostApp.airbnb" = "爱彼迎"; +"hostApp.googleMaps" = "Google 地图"; +"hostApp.paypal" = "PayPal"; +"hostApp.venmo" = "Venmo"; +"hostApp.cashApp" = "Cash App"; +"hostApp.messages" = "信息"; +"hostApp.notes" = "备忘录"; +"hostApp.mail" = "邮件"; +"hostApp.reminders" = "提醒事项"; +"hostApp.calendar" = "日历"; +"hostApp.shortcuts" = "快捷指令"; +"hostApp.files" = "文件"; +"hostApp.maps" = "地图"; +"hostApp.appStore" = "App Store"; "flow.warning.polishDegradedQuality" = "本次润色结果未通过校验，已插入保守处理版本。"; "flow.error.editLastInputFailed" = "未能完成编辑，请重试。"; "flow.error.aiQuestionFailed" = "AI 回答失败，请重试。"; diff --git a/OSGKeyboardExt/KeyboardViewController.swift b/OSGKeyboardExt/KeyboardViewController.swift index b997fd1..24cc549 100644 --- a/OSGKeyboardExt/KeyboardViewController.swift +++ b/OSGKeyboardExt/KeyboardViewController.swift @@ -155,6 +155,9 @@ public final class KeyboardViewController: UIInputViewController { public override func viewDidLoad() { super.viewDidLoad() + #if DEBUG + HostBundleProbe.installLogging() + #endif state.showsSystemGlobeKey = UIDevice.current.userInterfaceIdiom == .pad // Voice-first keyboard — hide the misleading "English" subtitle in Settings. primaryLanguage = "mis" @@ -227,6 +230,7 @@ public final class KeyboardViewController: UIInputViewController { public override func viewWillDisappear(_ animated: Bool) { super.viewWillDisappear(animated) isKeyboardPresentationActive = false + HostAppIdentity.invalidate() typingDocumentPresentationID = nil typingSessionStorage?.endDocumentPresentation() assistantFieldActionRefreshTask?.cancel() @@ -353,6 +357,7 @@ public final class KeyboardViewController: UIInputViewController { self.flowCoordinator.ensurePiPReadyOnKeyboardOpen() } #if DEBUG + HostBundleProbe.recordPresentationAndPing() WhatsNewDemoDriver.resetForNewPresentation() WhatsNewDemoDriver.startIfNeeded( state: state, diff --git a/OSGKeyboardExt/OSGKeyboardExt-Bridging-Header.h b/OSGKeyboardExt/OSGKeyboardExt-Bridging-Header.h new file mode 100644 index 0000000..e8e7464 --- /dev/null +++ b/OSGKeyboardExt/OSGKeyboardExt-Bridging-Header.h @@ -0,0 +1,6 @@ +// OSGKeyboardExt-Bridging-Header.h +// OSGKeyboard · Keyboard Extension +// +// Exposes extension-local Objective-C helpers to Swift. + +#import "Utilities/OSGHostArbiterProbe.h" diff --git a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift index 0acb64f..e17f280 100644 --- a/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift +++ b/OSGKeyboardExt/Services/KeyboardFlowCoordinator.swift @@ -1138,6 +1138,7 @@ final class KeyboardFlowCoordinator { state.lastTranscript = "" recomputeMicVoiceAvailability() FlowSessionBridge.markPiPArmAttempt() + HostAppIdentity.publishReturnTarget() OSGDiag.log( "beginFlowStart → openHostApp(startflow) recordAfterHandoff=\(recordAfterHandoff) " + "\(OSGDiag.memoryTag())", diff --git a/OSGKeyboardExt/Utilities/HostBundleProbe.swift b/OSGKeyboardExt/Utilities/HostBundleProbe.swift new file mode 100644 index 0000000..80c029f --- /dev/null +++ b/OSGKeyboardExt/Utilities/HostBundleProbe.swift @@ -0,0 +1,68 @@ +// HostBundleProbe.swift +// OSGKeyboard · Keyboard Extension +// +// 识别当前宿主 App 的 bundle ID，供冷启动跳转主 App 后自动跳回。 +// 目前依赖仅在 DEBUG 编译的私有 arbiter 探针（`OSGHostArbiterProbe`）， +// Release 构建里识别结果恒为 nil，行为与接入前一致。 + +import Foundation +import OSGKeyboardShared + +enum HostAppIdentity { + /// 最近一次识别到的宿主；识别不到、或宿主是 OSG 自己时为 nil。 + static var currentBundleId: String? { + #if DEBUG + guard let bundleId = OSGHostArbiterProbe.lastCapturedBundleId, + !HostAppURLRegistry.isOwnBundleId(bundleId) else { + return nil + } + return bundleId + #else + return nil + #endif + } + + /// 跳转主 App（`startflow`）前调用：把当前宿主写进 App Group，主 App 就绪后据此跳回。 + /// 识别不到时写 nil，清掉上一次残留，避免跳回错误的 App。 + static func publishReturnTarget() { + let bundleId = currentBundleId + FlowSessionBridge.setPendingHostBundleId(bundleId) + #if DEBUG + let returnable = HostAppURLRegistry.lookup(bundleId: bundleId) != nil + HostBundleProbeLog.append( + event: "startflow", + bundleId: bundleId, + detail: returnable ? "returnable" : "no-return-url" + ) + #endif + } + + /// 键盘收起时作废识别结果，下次弹出由新回调重新填充。 + static func invalidate() { + #if DEBUG + OSGHostArbiterProbe.invalidateCapturedBundleId() + #endif + } +} + +#if DEBUG +/// 临时调试：把探针回调写进 App Group，主 App 首页的测试卡片读取展示。 +enum HostBundleProbe { + /// 只需安装一次；回调可能来自非主线程，闭包必须是 nonisolated。 + static func installLogging() { + OSGHostArbiterProbe.setCaptureHandler { @Sendable bundleId, detail in + HostBundleProbeLog.append(event: "arbiter", bundleId: bundleId, detail: detail) + } + } + + /// 键盘弹出时记一条标记，再主动触发回调，便于把记录和每次弹出对应起来。 + static func recordPresentationAndPing() { + HostBundleProbeLog.append( + event: "appear", + bundleId: nil, + detail: OSGHostArbiterProbe.isInstalled ? "hook=installed" : "hook=missing" + ) + OSGHostArbiterProbe.pingArbiter() + } +} +#endif diff --git a/OSGKeyboardExt/Utilities/OSGHostArbiterProbe.h b/OSGKeyboardExt/Utilities/OSGHostArbiterProbe.h new file mode 100644 index 0000000..c199b87 --- /dev/null +++ b/OSGKeyboardExt/Utilities/OSGHostArbiterProbe.h @@ -0,0 +1,38 @@ +// OSGHostArbiterProbe.h +// OSGKeyboard · Keyboard Extension +// +// 临时调试探针（仅 DEBUG）：iOS 26.4+ 通过 _UIKeyboardArbiterClient 回调 +// 读取宿主 App 的 bundle ID。原理参考 editorss/KeyboardHostBundleID（MIT）。 +// 必须在 +load 安装，否则可能错过键盘首次切换宿主时的回调。 + +#import + +#ifdef DEBUG + +NS_ASSUME_NONNULL_BEGIN + +/// 回调可能在 arbiter 的私有队列上触发，不保证主线程。 +typedef void (^OSGHostArbiterCaptureHandler)(NSString *_Nullable bundleId, NSString *_Nullable detail); + +@interface OSGHostArbiterProbe : NSObject + +/// 钩子是否已成功安装（私有类或方法不存在时为 NO）。 +@property (class, nonatomic, readonly) BOOL isInstalled; + +/// 最近一次回调读到的宿主 bundle ID；回调读不到或已作废时为 nil。线程安全。 +@property (class, nonatomic, readonly, nullable) NSString *lastCapturedBundleId; + +/// 作废上一次的识别结果（键盘收起时调用，避免把上一个宿主当成当前宿主）。 ++ (void)invalidateCapturedBundleId; + +/// 每次 arbiter 回调都会调用；设置为 nil 取消。 ++ (void)setCaptureHandler:(nullable OSGHostArbiterCaptureHandler)handler; + +/// 主动让 arbiter 检查连接，可能触发一次回调，但不保证。 ++ (void)pingArbiter; + +@end + +NS_ASSUME_NONNULL_END + +#endif diff --git a/OSGKeyboardExt/Utilities/OSGHostArbiterProbe.m b/OSGKeyboardExt/Utilities/OSGHostArbiterProbe.m new file mode 100644 index 0000000..2520c5b --- /dev/null +++ b/OSGKeyboardExt/Utilities/OSGHostArbiterProbe.m @@ -0,0 +1,133 @@ +// OSGHostArbiterProbe.m +// OSGKeyboard · Keyboard Extension +// +// 临时调试探针（仅 DEBUG），见头文件说明。所有私有符号都在运行时按字符串查找， +// 任何一步找不到都静默放弃，并保证原始实现照常执行。 + +#import "OSGHostArbiterProbe.h" + +#ifdef DEBUG + +#import +#import +#import +#import + +static NSString *const kArbiterClientClassName = @"_UIKeyboardArbiterClient"; +static NSString *const kDestinationClassName = @"_UIKeyboardArbiterClientInputDestination"; +static NSString *const kEnabledSelectorName = @"enabled"; +static NSString *const kSharedClientSelectorName = @"automaticSharedArbiterClient"; +static NSString *const kCheckConnectionSelectorName = @"checkConnection"; +static NSString *const kKeyboardChangedSelectorName = @"queue_keyboardChanged:onComplete:"; +static NSString *const kSourceBundleIdKey = @"_sourceBundleIdentifier"; + +static os_unfair_lock sLock = OS_UNFAIR_LOCK_INIT; +static OSGHostArbiterCaptureHandler _Nullable sHandler = nil; +static NSString *_Nullable sLastCapturedBundleId = nil; +static IMP _Nullable sOriginalKeyboardChangedIMP = NULL; +static BOOL sInstalled = NO; + +#pragma mark - Trampolines + +// 替换 -[_UIKeyboardArbiterClientInputDestination queue_keyboardChanged:onComplete:]。 +// bundle ID 在参数 change 上，不在 self 上。 +static void OSGSwizzledKeyboardChanged(id self, SEL _cmd, id change, id completion) { + NSString *bundleId = nil; + if (change) { + @try { + id value = [change valueForKey:kSourceBundleIdKey]; + if ([value isKindOfClass:[NSString class]]) { + bundleId = [(NSString *)value copy]; + } + } @catch (__unused NSException *exception) { + // 字段改名时 KVC 会抛异常；吞掉，保证原始实现继续执行。 + } + } + NSString *detail = change ? NSStringFromClass([change class]) : @"change=nil"; + + os_unfair_lock_lock(&sLock); + // 读不到 ID 的回调同样覆盖旧值：宿主已经变了，旧 ID 不能再用。 + sLastCapturedBundleId = bundleId.length > 0 ? bundleId : nil; + OSGHostArbiterCaptureHandler handler = sHandler; + os_unfair_lock_unlock(&sLock); + if (handler) { + handler(bundleId, detail); + } + + if (sOriginalKeyboardChangedIMP) { + ((void (*)(id, SEL, id, id))sOriginalKeyboardChangedIMP)(self, _cmd, change, completion); + } +} + +// 替换 +[_UIKeyboardArbiterClient enabled]：iOS 26.4 用它控制是否派发上面的回调。 +static BOOL OSGAlwaysEnabled(__unused id self, __unused SEL _cmd) { + return YES; +} + +@implementation OSGHostArbiterProbe + ++ (void)load { + if (@available(iOS 26.4, *)) { + [self installHooks]; + } +} + ++ (void)installHooks { + Class clientClass = NSClassFromString(kArbiterClientClassName); + if (clientClass) { + Method enabled = class_getClassMethod(clientClass, NSSelectorFromString(kEnabledSelectorName)); + if (enabled) { + method_setImplementation(enabled, (IMP)&OSGAlwaysEnabled); + } + } + + Class destinationClass = NSClassFromString(kDestinationClassName); + if (!destinationClass) return; + Method keyboardChanged = class_getInstanceMethod( + destinationClass, + NSSelectorFromString(kKeyboardChangedSelectorName) + ); + if (!keyboardChanged) return; + sOriginalKeyboardChangedIMP = method_getImplementation(keyboardChanged); + method_setImplementation(keyboardChanged, (IMP)&OSGSwizzledKeyboardChanged); + sInstalled = YES; +} + ++ (BOOL)isInstalled { + return sInstalled; +} + ++ (NSString *)lastCapturedBundleId { + os_unfair_lock_lock(&sLock); + NSString *bundleId = [sLastCapturedBundleId copy]; + os_unfair_lock_unlock(&sLock); + return bundleId; +} + ++ (void)invalidateCapturedBundleId { + os_unfair_lock_lock(&sLock); + sLastCapturedBundleId = nil; + os_unfair_lock_unlock(&sLock); +} + ++ (void)setCaptureHandler:(OSGHostArbiterCaptureHandler)handler { + os_unfair_lock_lock(&sLock); + sHandler = [handler copy]; + os_unfair_lock_unlock(&sLock); +} + ++ (void)pingArbiter { + Class clientClass = NSClassFromString(kArbiterClientClassName); + SEL sharedSelector = NSSelectorFromString(kSharedClientSelectorName); + if (!clientClass || ![clientClass respondsToSelector:sharedSelector]) return; + + id client = ((id (*)(id, SEL))objc_msgSend)(clientClass, sharedSelector); + SEL checkSelector = NSSelectorFromString(kCheckConnectionSelectorName); + if (client && [client respondsToSelector:checkSelector]) { + ((void (*)(id, SEL))objc_msgSend)(client, checkSelector); + } +} + +@end + +#endif diff --git a/OSGKeyboardShared/Services/HostAppURLRegistry.swift b/OSGKeyboardShared/Services/HostAppURLRegistry.swift index 3f7cba3..1a620a9 100644 --- a/OSGKeyboardShared/Services/HostAppURLRegistry.swift +++ b/OSGKeyboardShared/Services/HostAppURLRegistry.swift @@ -3,6 +3,10 @@ // // Public URL-scheme whitelist for returning to a known host app after a // cold-start Flow session handoff. One bundle id maps to one preferred URL. +// +// Bundle ids are checked against the App Store lookup API. URL schemes have no +// public source of truth; a wrong one only makes the open fail (the user stays +// in OSG), it never sends the user to a different app. import Foundation @@ -25,287 +29,159 @@ public struct HostAppEntry: Sendable, Equatable { } public enum HostAppURLRegistry { - /// Curated whitelist for high-frequency host apps (IM, work, notes). + /// Curated whitelist: high-frequency typing hosts first, then the CN / US + /// App Store top charts, then Apple system apps. public static let entries: [HostAppEntry] = [ // Tier 1 — China IM / work - HostAppEntry( - bundleId: "com.tencent.xin", - displayNameKey: "hostApp.wechat", - returnURLString: "weixin://", - tier: 1 - ), - HostAppEntry( - bundleId: "com.tencent.mqq", - displayNameKey: "hostApp.qq", - returnURLString: "mqq://", - tier: 1 - ), - HostAppEntry( - bundleId: "com.tencent.wework", - displayNameKey: "hostApp.wecom", - returnURLString: "wxwork://", - tier: 1 - ), - HostAppEntry( - bundleId: "com.laiwang.DingTalk", - displayNameKey: "hostApp.dingtalk", - returnURLString: "dingtalk://", - tier: 1 - ), - HostAppEntry( - bundleId: "com.bytedance.ee.lark", - displayNameKey: "hostApp.lark", - returnURLString: "lark://", - tier: 1 - ), - // Tier 2 — global IM / collaboration - HostAppEntry( - bundleId: "ph.telegra.Telegraph", - displayNameKey: "hostApp.telegram", - returnURLString: "tg://", - tier: 2 - ), - HostAppEntry( - bundleId: "net.whatsapp.WhatsApp", - displayNameKey: "hostApp.whatsapp", - returnURLString: "whatsapp://", - tier: 2 - ), - HostAppEntry( - bundleId: "jp.naver.line", - displayNameKey: "hostApp.line", - returnURLString: "line://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.facebook.Messenger", - displayNameKey: "hostApp.messenger", - returnURLString: "fb-messenger://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.tinyspeck.chatlyio", - displayNameKey: "hostApp.slack", - returnURLString: "slack://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.microsoft.skype.teams", - displayNameKey: "hostApp.teams", - returnURLString: "msteams://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.hammerandchisel.discord", - displayNameKey: "hostApp.discord", - returnURLString: "discord://", - tier: 2 - ), - // Tier 3 — notes / mail / browser - HostAppEntry( - bundleId: "notion.id", - displayNameKey: "hostApp.notion", - returnURLString: "notion://", - tier: 3 - ), - HostAppEntry( - bundleId: "net.shinyfrog.bear", - displayNameKey: "hostApp.bear", - returnURLString: "bear://", - tier: 3 - ), - HostAppEntry( - bundleId: "md.obsidian", - displayNameKey: "hostApp.obsidian", - returnURLString: "obsidian://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.agiletortoise.Drafts5", - displayNameKey: "hostApp.drafts", - returnURLString: "drafts5://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.google.Gmail", - displayNameKey: "hostApp.gmail", - returnURLString: "googlegmail://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.microsoft.Office.Outlook", - displayNameKey: "hostApp.outlook", - returnURLString: "ms-outlook://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.google.chrome.ios", - displayNameKey: "hostApp.chrome", - returnURLString: "googlechrome://", - tier: 3 - ), - // Tier 4 — China social - HostAppEntry( - bundleId: "com.sina.weibo", - displayNameKey: "hostApp.weibo", - returnURLString: "sinaweibo://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.xingin.discover", - displayNameKey: "hostApp.xiaohongshu", - returnURLString: "xhsdiscover://", - tier: 4 - ), - // Tier 2 (cont.) — global IM / calls - HostAppEntry( - bundleId: "org.whispersystems.signal", - displayNameKey: "hostApp.signal", - returnURLString: "sgnl://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.iwilab.KakaoTalk", - displayNameKey: "hostApp.kakaotalk", - returnURLString: "kakaotalk://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.viber", - displayNameKey: "hostApp.viber", - returnURLString: "viber://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.vng.zaloapp", - displayNameKey: "hostApp.zalo", - returnURLString: "zalo://", - tier: 2 - ), - HostAppEntry( - bundleId: "com.skype.skype", - displayNameKey: "hostApp.skype", - returnURLString: "skype://", - tier: 2 - ), - HostAppEntry( - bundleId: "us.zoom.videomeetings", - displayNameKey: "hostApp.zoom", - returnURLString: "zoomus://", - tier: 2 - ), - // Tier 3 (cont.) — notes / mail / browser - HostAppEntry( - bundleId: "com.culturedcode.ThingsiPhone", - displayNameKey: "hostApp.things", - returnURLString: "things://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.todoist.ios", - displayNameKey: "hostApp.todoist", - returnURLString: "todoist://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.evernote.iPhone.Evernote", - displayNameKey: "hostApp.evernote", - returnURLString: "evernote://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.microsoft.onenote", - displayNameKey: "hostApp.onenote", - returnURLString: "onenote://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.readdle.smartemail", - displayNameKey: "hostApp.spark", - returnURLString: "readdle-spark://", - tier: 3 - ), - HostAppEntry( - bundleId: "org.mozilla.ios.Firefox", - displayNameKey: "hostApp.firefox", - returnURLString: "firefox://", - tier: 3 - ), - HostAppEntry( - bundleId: "com.microsoft.msedge", - displayNameKey: "hostApp.edge", - returnURLString: "microsoft-edge://", - tier: 3 - ), - // Tier 4 (cont.) — global / China social - HostAppEntry( - bundleId: "com.facebook.Facebook", - displayNameKey: "hostApp.facebook", - returnURLString: "fb://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.burbn.instagram", - displayNameKey: "hostApp.instagram", - returnURLString: "instagram://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.atebits.Tweetie2", - displayNameKey: "hostApp.x", - returnURLString: "twitter://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.burbn.barcelona", - displayNameKey: "hostApp.threads", - returnURLString: "barcelona://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.toyopagroup.picaboo", - displayNameKey: "hostApp.snapchat", - returnURLString: "snapchat://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.reddit.Reddit", - displayNameKey: "hostApp.reddit", - returnURLString: "reddit://", - tier: 4 - ), - HostAppEntry( - bundleId: "pinterest", - displayNameKey: "hostApp.pinterest", - returnURLString: "pinterest://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.zhihu.ios", - displayNameKey: "hostApp.zhihu", - returnURLString: "zhihu://", - tier: 4 - ), - HostAppEntry( - bundleId: "tv.danmaku.bili", - displayNameKey: "hostApp.bilibili", - returnURLString: "bilibili://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.ss.iphone.ugc.Aweme", - displayNameKey: "hostApp.douyin", - returnURLString: "snssdk1128://", - tier: 4 - ), - HostAppEntry( - bundleId: "com.zhiliaoapp.musically", - displayNameKey: "hostApp.tiktok", - returnURLString: "tiktok://", - tier: 4 - ) + app("com.tencent.xin", "wechat", "weixin://", tier: 1), + app("com.tencent.mqq", "qq", "mqq://", tier: 1), + app("com.tencent.ww", "wecom", "wxwork://", tier: 1), + app("com.laiwang.DingTalk", "dingtalk", "dingtalk://", tier: 1), + app("com.bytedance.ee.lark", "lark", "lark://", tier: 1), + app("com.tencent.meeting", "tencentMeeting", "wemeet://", tier: 1), + app("com.tencent.qqmail", "qqmail", "qqmail://", tier: 1), + app("com.kingsoft.www.office.wpsoffice", "wps", "wpsoffice://", tier: 1), + + // Tier 2 — global IM / calls / dating + app("ph.telegra.Telegraph", "telegram", "tg://", tier: 2), + app("net.whatsapp.WhatsApp", "whatsapp", "whatsapp://", tier: 2), + app("jp.naver.line", "line", "line://", tier: 2), + app("com.facebook.Messenger", "messenger", "fb-messenger://", tier: 2), + app("com.tinyspeck.chatlyio", "slack", "slack://", tier: 2), + app("com.microsoft.skype.teams", "teams", "msteams://", tier: 2), + app("com.hammerandchisel.discord", "discord", "discord://", tier: 2), + app("org.whispersystems.signal", "signal", "sgnl://", tier: 2), + app("com.iwilab.KakaoTalk", "kakaotalk", "kakaotalk://", tier: 2), + app("com.viber", "viber", "viber://", tier: 2), + app("vn.com.vng.zingalo", "zalo", "zalo://", tier: 2), + app("us.zoom.videomeetings", "zoom", "zoomus://", tier: 2), + app("com.cardify.tinder", "tinder", "tinder://", tier: 2), + + // Tier 3 — notes / mail / browser / office / AI + app("notion.id", "notion", "notion://", tier: 3), + app("net.shinyfrog.bear", "bear", "bear://", tier: 3), + app("md.obsidian", "obsidian", "obsidian://", tier: 3), + app("com.agiletortoise.Drafts5", "drafts", "drafts5://", tier: 3), + app("com.culturedcode.ThingsiPhone", "things", "things://", tier: 3), + app("com.todoist.ios", "todoist", "todoist://", tier: 3), + app("com.evernote.iPhone.Evernote", "evernote", "evernote://", tier: 3), + app("com.microsoft.onenote", "onenote", "onenote://", tier: 3), + app("com.microsoft.Office.Word", "word", "ms-word://", tier: 3), + app("com.microsoft.Office.Excel", "excel", "ms-excel://", tier: 3), + app("com.google.Gmail", "gmail", "googlegmail://", tier: 3), + app("com.microsoft.Office.Outlook", "outlook", "ms-outlook://", tier: 3), + app("com.readdle.smartemail", "spark", "readdle-spark://", tier: 3), + app("com.yahoo.Aerogram", "yahooMail", "ymail://", tier: 3), + app("com.google.Docs", "googleDocs", "googledocs://", tier: 3), + app("com.google.Sheets", "googleSheets", "googlesheets://", tier: 3), + app("com.google.Drive", "googleDrive", "googledrive://", tier: 3), + app("com.google.Translate", "googleTranslate", "googletranslate://", tier: 3), + app("com.google.GoogleMobile", "google", "google://", tier: 3), + app("com.google.chrome.ios", "chrome", "googlechrome://", tier: 3), + app("org.mozilla.ios.Firefox", "firefox", "firefox://", tier: 3), + app("com.microsoft.msedge", "edge", "microsoft-edge://", tier: 3), + app("com.openai.chat", "chatgpt", "chatgpt://", tier: 3), + app("com.anthropic.claude", "claude", "claude://", tier: 3), + + // Tier 4 — China social / content + app("com.sina.weibo", "weibo", "sinaweibo://", tier: 4), + app("com.xingin.discover", "xiaohongshu", "xhsdiscover://", tier: 4), + app("com.zhihu.ios", "zhihu", "zhihu://", tier: 4), + app("tv.danmaku.bilianime", "bilibili", "bilibili://", tier: 4), + app("com.ss.iphone.ugc.Aweme", "douyin", "snssdk1128://", tier: 4), + app("com.ss.iphone.ugc.aweme.lite", "douyinLite", "snssdk2329://", tier: 4), + app("com.jiangjia.gif", "kuaishou", "kwai://", tier: 4), + app("com.kuaishou.nebula", "kuaishouLite", "ksnebula://", tier: 4), + app("com.ss.iphone.article.News", "toutiao", "snssdk141://", tier: 4), + app("com.douban.frodo", "douban", "douban://", tier: 4), + app("com.baidu.BaiduMobile", "baidu", "baiduboxapp://", tier: 4), + app("com.youku.YouKu", "youku", "youku://", tier: 4), + app("com.tencent.live4iphone", "tencentVideo", "tenvideo://", tier: 4), + app("com.qiyi.iphone", "iqiyi", "iqiyi://", tier: 4), + app("com.hunantv.imgotv", "mangoTV", "imgotv://", tier: 4), + app("com.tencent.QQMusic", "qqMusic", "qqmusic://", tier: 4), + app("com.netease.cloudmusic", "neteaseMusic", "orpheus://", tier: 4), + app("com.kugou.kugou1002", "kugou", "kugouURL://", tier: 4), + app("com.gemd.iting", "ximalaya", "iting://", tier: 4), + app("com.tencent.weread", "weread", "weread://", tier: 4), + + // Tier 4 — global social / content + app("com.facebook.Facebook", "facebook", "fb://", tier: 4), + app("com.burbn.instagram", "instagram", "instagram://", tier: 4), + app("com.atebits.Tweetie2", "x", "twitter://", tier: 4), + app("com.burbn.barcelona", "threads", "barcelona://", tier: 4), + app("com.toyopagroup.picaboo", "snapchat", "snapchat://", tier: 4), + app("com.reddit.Reddit", "reddit", "reddit://", tier: 4), + app("pinterest", "pinterest", "pinterest://", tier: 4), + app("com.zhiliaoapp.musically", "tiktok", "tiktok://", tier: 4), + app("com.google.ios.youtube", "youtube", "youtube://", tier: 4), + app("com.linkedin.LinkedIn", "linkedin", "linkedin://", tier: 4), + app("com.tumblr.tumblr", "tumblr", "tumblr://", tier: 4), + app("tv.twitch", "twitch", "twitch://", tier: 4), + app("com.spotify.client", "spotify", "spotify://", tier: 4), + app("com.netflix.Netflix", "netflix", "nflx://", tier: 4), + app("com.lemon.lvoverseas", "capcut", "capcut://", tier: 4), + app("com.duolingo.DuolingoMobile", "duolingo", "duolingo://", tier: 4), + + // Tier 5 — China shopping / local services / travel / finance + app("com.taobao.taobao4iphone", "taobao", "taobao://", tier: 5), + app("com.taobao.tmall", "tmall", "tmall://", tier: 5), + app("com.360buy.jdmobile", "jd", "openapp.jdmobile://", tier: 5), + app("com.xunmeng.pinduoduo", "pinduoduo", "pinduoduo://", tier: 5), + app("com.taobao.fleamarket", "xianyu", "fleamarket://", tier: 5), + app("com.alibaba.wireless", "alibaba1688", "wireless1688://", tier: 5), + app("com.vipshop.iphone", "vipshop", "vipshop://", tier: 5), + app("com.meituan.imeituan", "meituan", "imeituan://", tier: 5), + app("com.meituan.itakeaway", "meituanWaimai", "meituanwaimai://", tier: 5), + app("me.ele.ios.eleme", "eleme", "eleme://", tier: 5), + app("com.dianping.dpscope", "dianping", "dianping://", tier: 5), + app("com.taofang.iphone", "wuba", "wbmain://", tier: 5), + app("com.xiaojukeji.didi", "didi", "diditaxi://", tier: 5), + app("com.autonavi.amap", "amap", "iosamap://", tier: 5), + app("com.baidu.map", "baiduMap", "baidumap://", tier: 5), + app("com.tencent.sosomap", "tencentMap", "qqmap://", tier: 5), + app("ctrip.com", "ctrip", "ctrip://", tier: 5), + app("com.baidu.netdisk", "baiduNetdisk", "baiduyun://", tier: 5), + app("com.ucweb.iphone.lowversion", "ucBrowser", "ucbrowser://", tier: 5), + app("com.tencent.mttlite", "qqBrowser", "mttbrowser://", tier: 5), + app("com.alipay.iphoneclient", "alipay", "alipay://", tier: 5), + app("com.unionpay.chsp", "unionpay", "upwallet://", tier: 5), + app("com.cmbchina.MPBBank", "cmb", "cmbmobilebank://", tier: 5), + app("com.ccb.ccbDemo", "ccb", "ccbmobilebank://", tier: 5), + app("com.bankabc.iphonerelease", "abc", "bankabc://", tier: 5), + + // Tier 5 — US shopping / local services / finance + app("com.amazon.Amazon", "amazon", "com.amazon.mobile.shopping://", tier: 5), + app("com.walmart.electronics", "walmart", "walmart://", tier: 5), + app("com.ebay.iphone", "ebay", "ebay://", tier: 5), + app("com.alibaba.iAliexpress", "aliexpress", "aliexpress://", tier: 5), + app("com.ubercab.UberClient", "uber", "uber://", tier: 5), + app("com.zimride.instant", "lyft", "lyft://", tier: 5), + app("doordash.DoorDashConsumer", "doordash", "doordash://", tier: 5), + app("com.airbnb.app", "airbnb", "airbnb://", tier: 5), + app("com.google.Maps", "googleMaps", "comgooglemaps://", tier: 5), + app("com.yourcompany.PPClient", "paypal", "paypal://", tier: 5), + app("net.kortina.labs.Venmo", "venmo", "venmo://", tier: 5), + app("com.squareup.cash", "cashApp", "cashme://", tier: 5), + + // Tier 6 — Apple system apps. Safari has no scheme that reopens the + // current tab (every https/x-web-search URL opens a new page), so it is + // deliberately absent. + app("com.apple.MobileSMS", "messages", "messages://", tier: 6), + app("com.apple.mobilenotes", "notes", "mobilenotes://", tier: 6), + app("com.apple.mobilemail", "mail", "message://", tier: 6), + app("com.apple.reminders", "reminders", "x-apple-reminderkit://", tier: 6), + app("com.apple.mobilecal", "calendar", "calshow://", tier: 6), + app("com.apple.shortcuts", "shortcuts", "shortcuts://", tier: 6), + app("com.apple.DocumentsApp", "files", "shareddocuments://", tier: 6), + app("com.apple.Maps", "maps", "maps://", tier: 6), + app("com.apple.AppStore", "appStore", "itms-apps://", tier: 6) ] + private static func app(_ bundleId: String, _ name: String, _ url: String, tier: Int) -> HostAppEntry { + HostAppEntry(bundleId: bundleId, displayNameKey: "hostApp.\(name)", returnURLString: url, tier: tier) + } + private static let byBundleId: [String: HostAppEntry] = { Dictionary(uniqueKeysWithValues: entries.map { ($0.bundleId, $0) }) }() @@ -315,15 +191,10 @@ public enum HostAppURLRegistry { return byBundleId[bundleId] } - /// URL schemes declared in `LSApplicationQueriesSchemes` for `canOpenURL`. - public static var querySchemes: [String] { - Array( - Set( - entries.compactMap { entry -> String? in - guard let url = entry.returnURL, let scheme = url.scheme else { return nil } - return scheme - } - ) - ).sorted() + /// OSG's own targets (host app, keyboard, Mac) are never a return destination. + public static func isOwnBundleId(_ bundleId: String) -> Bool { + bundleId == ownBundleIdRoot || bundleId.hasPrefix(ownBundleIdRoot + ".") } + + private static let ownBundleIdRoot = "com.osgkeyboard" } diff --git a/OSGKeyboardShared/Utilities/HostBundleProbeLog.swift b/OSGKeyboardShared/Utilities/HostBundleProbeLog.swift new file mode 100644 index 0000000..cf91399 --- /dev/null +++ b/OSGKeyboardShared/Utilities/HostBundleProbeLog.swift @@ -0,0 +1,47 @@ +// HostBundleProbeLog.swift +// OSGKeyboard · Shared +// +// 临时调试：键盘扩展把 arbiter 回调识别到的宿主 bundle ID 写进 App Group， +// 主 App 首页读出来展示，用于真机验证 iOS 26.4+ 宿主识别方案。验证结束后删除。 + +#if DEBUG +import Foundation + +public enum HostBundleProbeLog { + public struct Entry: Codable, Equatable, Identifiable, Sendable { + public let id: UUID + public let date: Date + /// `appear`：键盘弹出（随后主动触发一次回调）；`arbiter`：系统回调。 + public let event: String + /// 回调里读到的宿主 bundle ID；读不到时为 nil。 + public let bundleId: String? + /// 回调参数的类名等辅助信息，方便字段失效时排查。 + public let detail: String? + } + + private static let key = "debug.hostBundleProbe.entries" + private static let limit = 40 + + /// 新记录插在最前面，只保留最近 `limit` 条。 + public static func append(event: String, bundleId: String?, detail: String? = nil) { + guard let defaults = AppGroup.defaultsIfAvailable else { return } + let entry = Entry(id: UUID(), date: Date(), event: event, bundleId: bundleId, detail: detail) + let updated = Array(([entry] + entries()).prefix(limit)) + guard let data = try? JSONEncoder().encode(updated) else { return } + defaults.set(data, forKey: key) + } + + /// 按时间倒序返回。 + public static func entries() -> [Entry] { + guard let data = AppGroup.defaultsIfAvailable?.data(forKey: key), + let decoded = try? JSONDecoder().decode([Entry].self, from: data) else { + return [] + } + return decoded + } + + public static func clear() { + AppGroup.defaultsIfAvailable?.removeObject(forKey: key) + } +} +#endif diff --git a/OSGKeyboardTests/HostAppURLRegistryTests.swift b/OSGKeyboardTests/HostAppURLRegistryTests.swift index 0156eda..3d1088b 100644 --- a/OSGKeyboardTests/HostAppURLRegistryTests.swift +++ b/OSGKeyboardTests/HostAppURLRegistryTests.swift @@ -11,18 +11,53 @@ final class HostAppURLRegistryTests: XCTestCase { XCTAssertEqual(entry?.displayNameKey, "hostApp.wechat") } + /// App Store bundle ids; the earlier `com.tencent.wework` / `tv.danmaku.bili` never matched. + func testCorrectedBundleIds() { + XCTAssertEqual(HostAppURLRegistry.lookup(bundleId: "com.tencent.ww")?.returnURLString, "wxwork://") + XCTAssertEqual(HostAppURLRegistry.lookup(bundleId: "tv.danmaku.bilianime")?.returnURLString, "bilibili://") + } + + func testSystemAppLookup() { + XCTAssertEqual(HostAppURLRegistry.lookup(bundleId: "com.apple.mobilenotes")?.returnURLString, "mobilenotes://") + XCTAssertNil(HostAppURLRegistry.lookup(bundleId: "com.apple.mobilesafari")) + } + func testUnknownBundleReturnsNil() { - XCTAssertNil(HostAppURLRegistry.lookup(bundleId: "com.apple.MobileSMS")) + XCTAssertNil(HostAppURLRegistry.lookup(bundleId: "com.example.unknown")) XCTAssertNil(HostAppURLRegistry.lookup(bundleId: nil)) XCTAssertNil(HostAppURLRegistry.lookup(bundleId: "")) } - func testQuerySchemesIncludesWeChat() { - XCTAssertTrue(HostAppURLRegistry.querySchemes.contains("weixin")) + func testEveryEntryHasParseableReturnURL() { + for entry in HostAppURLRegistry.entries { + XCTAssertNotNil(entry.returnURL?.scheme, entry.bundleId) + } + } + + func testOwnBundleIdsAreNeverReturnTargets() { + XCTAssertTrue(HostAppURLRegistry.isOwnBundleId("com.osgkeyboard")) + XCTAssertTrue(HostAppURLRegistry.isOwnBundleId("com.osgkeyboard.ios")) + XCTAssertTrue(HostAppURLRegistry.isOwnBundleId("com.osgkeyboard.ios.keyboard")) + XCTAssertFalse(HostAppURLRegistry.isOwnBundleId("com.osgkeyboardx.ios")) + XCTAssertFalse(HostAppURLRegistry.isOwnBundleId("com.tencent.xin")) } func testEntriesHaveUniqueBundleIds() { let ids = HostAppURLRegistry.entries.map(\.bundleId) XCTAssertEqual(Set(ids).count, ids.count) } + + func testEveryEntryHasLocalizedDisplayName() throws { + let bundle = Bundle(identifier: "com.osgkeyboard.ios") ?? .main + for language in ["en", "zh-Hans"] { + let path = try XCTUnwrap(bundle.path(forResource: language, ofType: "lproj"), language) + let strings = try XCTUnwrap( + NSDictionary(contentsOfFile: "\(path)/Localizable.strings") as? [String: String], + language + ) + for entry in HostAppURLRegistry.entries { + XCTAssertNotNil(strings[entry.displayNameKey], "\(language): \(entry.displayNameKey)") + } + } + } } diff --git a/project.yml b/project.yml index c253cf0..ab80317 100644 --- a/project.yml +++ b/project.yml @@ -353,6 +353,7 @@ targets: settings: base: PRODUCT_BUNDLE_IDENTIFIER: com.osgkeyboard.ios.keyboard + SWIFT_OBJC_BRIDGING_HEADER: OSGKeyboardExt/OSGKeyboardExt-Bridging-Header.h TARGETED_DEVICE_FAMILY: "1,2" SUPPORTS_MACCATALYST: NO SUPPORTS_MAC_DESIGNED_FOR_IPHONE_IPAD: NO From a4c6102f5ccc6e0703fc66c7590748e596d8045c Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:06:24 +0200 Subject: [PATCH 09/19] chore(previews): record English feature previews and add App Store creative asset scripts --- .../Views/ClipboardHistoryDemoView.swift | 54 +- OSGKeyboard/Views/EditDemoView.swift | 31 +- .../Views/FeaturePreviewHostDocument.swift | 20 + Scripts/appstore-creative/compose.py | 322 +++++++++++ Scripts/appstore-creative/header.html | 506 ++++++++++++++++++ Scripts/appstore-creative/render.cjs | 104 ++++ Scripts/appstore-creative/search.html | 365 +++++++++++++ Scripts/record_feature_previews.sh | 16 +- docs/FEATURE_PREVIEW_RECORDING.md | 20 + 9 files changed, 1415 insertions(+), 23 deletions(-) create mode 100644 Scripts/appstore-creative/compose.py create mode 100644 Scripts/appstore-creative/header.html create mode 100644 Scripts/appstore-creative/render.cjs create mode 100644 Scripts/appstore-creative/search.html diff --git a/OSGKeyboard/Views/ClipboardHistoryDemoView.swift b/OSGKeyboard/Views/ClipboardHistoryDemoView.swift index 3a3cf93..7c8734c 100644 --- a/OSGKeyboard/Views/ClipboardHistoryDemoView.swift +++ b/OSGKeyboard/Views/ClipboardHistoryDemoView.swift @@ -4,7 +4,8 @@ // What's New 1.7.0 recording host. Voice chrome + clipboard panel are the // **real** extension views (`KeyboardTopControls`, `RecordButton`, // `KeyboardTranslationMenuButton`, `ClipboardHistoryPanelView`, …) driven by -// scripted `KeyboardState`. Launch with `--clipboard-demo`. +// scripted `KeyboardState`. Launch with `--clipboard-demo` and optional +// `--whats-new-lang=zh|en`. #if DEBUG import OSGKeyboardShared @@ -35,10 +36,17 @@ struct ClipboardHistoryDemoView: View { /// Closing beat: hand the copied text to the real AI skill row. @State private var showsSkills = false + private static let language = FeaturePreviewFlags.language + private static var isEnglish: Bool { language == .en } + private var palette: ThemePalette { colorScheme == .dark ? Palette.dark : Palette.light } + init() { + FeaturePreviewFlags.applyRecordingLanguage() + } + var body: some View { ZStack { if FeaturePreviewFlags.isFullscreen { @@ -48,13 +56,10 @@ struct ClipboardHistoryDemoView: View { } VStack(spacing: 0) { if FeaturePreviewFlags.isFullscreen { - // Chinese-only recording — see the `zh-Hans` locale pin on - // this view's body; the sample conversation has no English - // variant, so the host chrome stays Chinese to match. FeaturePreviewHostDocument( kind: .messages, - title: "信息", - language: .zh, + title: Self.isEnglish ? "Messages" : "信息", + language: Self.language, text: hostDraft, incoming: Self.incomingMessage ) @@ -71,7 +76,7 @@ struct ClipboardHistoryDemoView: View { } } .environment(\.themePalette, palette) - .environment(\.locale, Locale(identifier: "zh-Hans")) + .environment(\.locale, Locale(identifier: Self.isEnglish ? "en" : "zh-Hans")) .preferredColorScheme(.light) .task { await runTimeline() } } @@ -290,7 +295,9 @@ struct ClipboardHistoryDemoView: View { }) else { return } await runReplySkill( skill: generic, - answer: "好的，预算版我整理一下，明天上午发你。" + answer: Self.isEnglish + ? "Sure, I'll pull the budget version together and send it tomorrow morning." + : "好的，预算版我整理一下，明天上午发你。" ) state.pendingClipboardSkillID = nil try? await sleep(0.6) @@ -337,6 +344,7 @@ struct ClipboardHistoryDemoView: View { /// Demo copy per reply style. Keyed by skill so the answer matches whatever /// chip the ranker put in the row. private static func answer(for skillID: String) -> String { + if isEnglish { return englishAnswer(for: skillID) } switch skillID { case AIClipboardSkillCatalog.playfulReplyID: return "行嘞，预算表这就去梳妆打扮，明早准时上桌。" @@ -353,6 +361,23 @@ struct ClipboardHistoryDemoView: View { } } + private static func englishAnswer(for skillID: String) -> String { + switch skillID { + case AIClipboardSkillCatalog.playfulReplyID: + return "On it! The budget sheet is getting dressed up and will be on your desk by morning." + case AIClipboardSkillCatalog.businessReplyID: + return "Noted. I'll finalize the budget version tonight and send it over tomorrow morning." + case AIClipboardSkillCatalog.empathyReplyID: + return "Totally get that you need this soon. I'll prioritize it and send it first thing tomorrow." + case AIClipboardSkillCatalog.acceptTaskID: + return "Got it, I'll take this on and send the budget version tomorrow morning." + case AIClipboardSkillCatalog.clarifyRequestID: + return "Sure. Do you need the version with staffing costs, or just procurement?" + default: + return "Sure, I'll put it together and send it tomorrow morning." + } + } + /// One tap on a reply-style chip: the row gives way to the thinking state, /// the answer streams in, then it is offered for insertion. private func runReplySkill(skill: AIClipboardSkill, answer: String) async { @@ -394,6 +419,8 @@ struct ClipboardHistoryDemoView: View { state.layoutWidth = 390 state.usesIPadLayoutMetrics = false state.showsSystemGlobeKey = false + // The skill ranker localizes chip titles off this. + state.uiLanguage = Self.isEnglish ? .english : .chinese // `AIKeyboardView.showsClipboardSkills` gates on this — without it the // real skill row never appears no matter what the ranker returns. state.clipboardHistoryEnabled = true @@ -414,7 +441,10 @@ struct ClipboardHistoryDemoView: View { private func seedHistory() { history.clearAll() - _ = history.ingest(rawText: "订单号 OSG-20260811-8842", changeCount: 3) + _ = history.ingest( + rawText: Self.isEnglish ? "Order OSG-20260811-8842" : "订单号 OSG-20260811-8842", + changeCount: 3 + ) _ = history.ingest(rawText: "https://osglab.com", changeCount: 2) _ = history.ingest(rawText: Self.incomingMessage, changeCount: 1) history.reload() @@ -422,7 +452,11 @@ struct ClipboardHistoryDemoView: View { /// The message the clip replies to. Copying it is what puts the reply /// skills in the row, so it has to be the newest clipboard entry. - static let incomingMessage = "方案我看了，明天能不能把预算那一版也发我？" + static var incomingMessage: String { + isEnglish + ? "Read the proposal. Could you also send me the budget version tomorrow?" + : "方案我看了，明天能不能把预算那一版也发我？" + } private func sleep(_ seconds: Double) async throws { try await Task.sleep(nanoseconds: UInt64(seconds * 2.4 * 1_000_000_000)) diff --git a/OSGKeyboard/Views/EditDemoView.swift b/OSGKeyboard/Views/EditDemoView.swift index 67d5851..450ea94 100644 --- a/OSGKeyboard/Views/EditDemoView.swift +++ b/OSGKeyboard/Views/EditDemoView.swift @@ -4,15 +4,27 @@ // What's New "Edit last input" recording host. Uses the **real** // `LastInputEditView` (compiled into the app target) driven by a scripted // `KeyboardState` — no ASR / LLM. Opening beat shows the voice mic + hint. -// Launch with `--edit-demo`. Not shipped in Release. +// Launch with `--edit-demo` and optional `--whats-new-lang=zh|en`. +// Not shipped in Release. #if DEBUG import OSGKeyboardShared import SwiftUI struct EditDemoView: View { - private static let originalText = "明天下午三点开会讨论方案" - private static let editedText = "各位好，明天下午三点在 A 会议室召开方案讨论会，请准时参加。" + private static let language = FeaturePreviewFlags.language + + private static var originalText: String { + language == .en + ? "meeting tomorrow at 3 to discuss the plan" + : "明天下午三点开会讨论方案" + } + + private static var editedText: String { + language == .en + ? "Hi all, we'll review the proposal tomorrow at 3 PM in Room A. Please be on time." + : "各位好，明天下午三点在 A 会议室召开方案讨论会，请准时参加。" + } private enum Scene: Equatable { case hint @@ -24,6 +36,10 @@ struct EditDemoView: View { /// Mirrors the host document for `--preview-fullscreen` recordings. @State private var hostText: String = EditDemoView.originalText + init() { + FeaturePreviewFlags.applyRecordingLanguage() + } + private var source: EditSessionSource { let reference = EditableInputReference( displayText: Self.originalText, @@ -45,13 +61,10 @@ struct EditDemoView: View { } VStack(spacing: 0) { if FeaturePreviewFlags.isFullscreen { - // Chinese-only recording: the sample text below has no - // English variant, so the host chrome is pinned to match - // (see the `zh-Hans` locale pin on this view's body). FeaturePreviewHostDocument( kind: .notes, - title: "备忘录", - language: .zh, + title: Self.language == .en ? "Notes" : "备忘录", + language: Self.language, text: hostText ) } else { @@ -74,7 +87,7 @@ struct EditDemoView: View { } } .environment(\.themePalette, palette) - .environment(\.locale, Locale(identifier: "zh-Hans")) + .environment(\.locale, Locale(identifier: Self.language == .en ? "en" : "zh-Hans")) .preferredColorScheme(.light) .task { await runTimeline() } } diff --git a/OSGKeyboard/Views/FeaturePreviewHostDocument.swift b/OSGKeyboard/Views/FeaturePreviewHostDocument.swift index cf598b1..e503ef2 100644 --- a/OSGKeyboard/Views/FeaturePreviewHostDocument.swift +++ b/OSGKeyboard/Views/FeaturePreviewHostDocument.swift @@ -18,6 +18,26 @@ enum FeaturePreviewFlags { static var isFullscreen: Bool { ProcessInfo.processInfo.arguments.contains("--preview-fullscreen") } + + /// Recording language from `--whats-new-lang=zh|en`; Chinese when absent. + static var language: WhatsNewDemoScenario.Language { + let prefix = "--whats-new-lang=" + guard let argument = ProcessInfo.processInfo.arguments.first( + where: { $0.hasPrefix(prefix) } + ) else { + return .zh + } + return WhatsNewDemoScenario.Language( + rawValue: String(argument.dropFirst(prefix.count)) + ) ?? .zh + } + + /// Points the extension strings (`ExtL10n` reads the App Group setting) + /// at the recording language before the first frame renders. + @MainActor + static func applyRecordingLanguage() { + ProviderConfig.shared.uiLanguage = language == .en ? .english : .chinese + } } /// A Notes- or Messages-like document that reacts to the demo timeline. diff --git a/Scripts/appstore-creative/compose.py b/Scripts/appstore-creative/compose.py new file mode 100644 index 0000000..3dc7d56 --- /dev/null +++ b/Scripts/appstore-creative/compose.py @@ -0,0 +1,322 @@ +#!/usr/bin/env python3 +"""Build the App Store creative assets (product page header + search results). + +Specs (App Store Connect › Creative assets specifications): + * Header video 21:9 3840×1646 30 fps, 5–30 s, mp4 + * Search video 3:2 1920×1280…3840×2560 + * Universal still 16:9 5244×2950 PNG (no video allowed at this size) +Safe areas are taken from Apple's creative-asset Sketch template. + +Scenes are deterministic canvas pages (header.html / search.html) rendered by +render.cjs; search results composite the real simulator recordings from +`Scripts/record_feature_previews.sh` into the device screens. + +Usage: + NODE_PATH=.tmp/appstore-assets/node_modules \\ + python3 Scripts/appstore-creative/compose.py [header|search|stills ...] [--lang zh,en] + +Output: .tmp/appstore-creative/out/ +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + +from PIL import Image, ImageDraw + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parents[1] +WORK = ROOT / ".tmp" / "appstore-creative" / "work" +OUT = ROOT / ".tmp" / "appstore-creative" / "out" +RAW = { + "zh": ROOT / ".tmp" / "feature-previews" / "raw", + "en": ROOT / ".tmp" / "feature-previews" / "raw-en", +} + +FPS = 30 +DURATION = 10.0 # every scene loops on this period +LOOP_BLEND = 0.6 # tail-over-head crossfade that closes each screen loop +POSTER_T = 7.4 # header: full line written, sheen finished + +HEADER_SIZE = (3840, 1646) +HEADER_SAFE = "1097,493,1646,661" +UNIVERSAL_SIZE = (5244, 2950) +UNIVERSAL_SAFE = "1921,660,1402,962" +SEARCH_SIZE = (3840, 2560) +# H.264 above Level 5.1 is not universally decodable; ship a 5.1 copy too. +SEARCH_COMPAT = (3072, 2048) + + +@dataclass(frozen=True) +class Window: + """Segments of a raw recording, in real (CFR) seconds, played back to back. + + Cuts only land inside static holds (processing, generating, answer + review), where neighbouring frames are identical, so they read as one + continuous take. Segments must add up to DURATION + LOOP_BLEND. + """ + + raw: str + segments: tuple[tuple[float, float], ...] + + +# Read off 0.5 s contact sheets of the CFR footage. Re-time after re-recording. +WINDOWS: dict[str, dict[str, Window]] = { + "zh": { + # 复制的消息 → 技能行 → 思考 → 答案 → 回复气泡落进对话 + "clipboard": Window("clipboard-agent", ((17.0, 27.6),)), + # 聆听 → 处理 → 编辑后结果 → 应用到备忘录 + "voice": Window("voice-polish", ((6.3, 9.6), (10.9, 14.6), (16.9, 20.5))), + # 长按提问 → 生成 → 答案 → 插入对话 + "ask": Window("ask-ai", ((12.8, 16.6), (20.2, 23.6), (25.9, 29.3))), + }, + "en": { + "clipboard": Window("clipboard-agent", ((17.5, 28.1),)), + "voice": Window("voice-polish", ((6.3, 9.6), (11.0, 14.7), (17.6, 21.2))), + "ask": Window("ask-ai", ((12.2, 15.9), (21.0, 24.6), (29.2, 32.5))), + }, +} + +# sRGB transfer matches the simulator footage and the PNG stills, so players +# do not shift the greens between the video and its poster. +COLOR_TAGS = [ + "-colorspace", "bt709", "-color_primaries", "bt709", + "-color_trc", "iec61966-2-1", "-color_range", "tv", +] +# Frame side data wins over the CLI tags after `scale`; pin it explicitly or +# the stream ships with unknown primaries / transfer. +SET_COLOR = "setparams=color_primaries=bt709:color_trc=iec61966-2-1:colorspace=bt709:range=tv" + + +def run(cmd: list[str]) -> None: + proc = subprocess.run(cmd, capture_output=True, text=True) + if proc.returncode != 0: + sys.stderr.write(" ".join(map(str, cmd)) + "\n" + proc.stderr[-4000:] + "\n") + raise SystemExit(f"command failed ({proc.returncode})") + + +def render(page: str, query: str, out: Path, *, still: float | None = None, + layout: Path | None = None) -> None: + cmd = ["node", str(HERE / "render.cjs"), "--page", page, "--query", query, "--out", str(out)] + if still is None: + cmd += ["--video", "--fps", str(FPS)] + else: + cmd += ["--still", f"{still}"] + if layout: + cmd += ["--layout", str(layout)] + print(f" render {page} {query} -> {out.name}", flush=True) + subprocess.run(cmd, check=True, env={**os.environ}) + + +def flatten_png(path: Path) -> None: + """App Store Connect rejects images with an alpha channel.""" + Image.open(path).convert("RGB").save(path, optimize=True) + + +def deliver(src: Path, dest: Path, size: tuple[int, int], level: str) -> None: + w, h = size + run([ + "ffmpeg", "-y", "-i", str(src), + "-vf", f"scale={w}:{h}:flags=lanczos:out_color_matrix=bt709:out_range=tv," + f"format=yuv420p,{SET_COLOR}", + "-an", "-r", str(FPS), "-t", f"{DURATION:.3f}", + "-c:v", "libx264", "-preset", "slow", "-profile:v", "high", "-level", level, + "-crf", "14", "-maxrate", "60M", "-bufsize", "120M", + *COLOR_TAGS, "-movflags", "+faststart", str(dest), + ]) + + +def probe(path: Path) -> str: + info = json.loads(subprocess.check_output([ + "ffprobe", "-v", "error", "-select_streams", "v:0", + "-show_entries", "stream=codec_name,profile,level,width,height,r_frame_rate,nb_frames", + "-show_entries", "format=duration,size,nb_streams", "-of", "json", str(path), + ], text=True)) + s, f = info["streams"][0], info["format"] + return (f"{path.name}: {s['width']}x{s['height']} {s['codec_name']} {s.get('profile')} " + f"L{s.get('level')} {s['r_frame_rate']} {float(f['duration']):.2f}s " + f"{int(f['size']) / 1e6:.1f} MB streams={f['nb_streams']}") + + +# -------------------------------------------------------------------------- +# Header + + +def build_header(lang: str) -> None: + w, h = HEADER_SIZE + q = f"w={w}&h={h}&safe={HEADER_SAFE}&lang={lang}" + master = WORK / f"header-{lang}.mov" + render("header.html", q, master) + dest = OUT / f"OSG-Header-{w}x{h}-{lang}.mp4" + deliver(master, dest, HEADER_SIZE, "5.1") + poster = OUT / f"OSG-Header-{w}x{h}-{lang}-poster.png" + render("header.html", q, poster, still=POSTER_T) + flatten_png(poster) + print(" " + probe(dest)) + + +def build_universal(lang: str) -> None: + w, h = UNIVERSAL_SIZE + dest = OUT / f"OSG-Universal-{w}x{h}-{lang}.png" + render("header.html", f"w={w}&h={h}&safe={UNIVERSAL_SAFE}&lang={lang}&hero=1", + dest, still=POSTER_T) + flatten_png(dest) + # Static 21:9 header in the same hero state, for an image-only upload. + hw, hh = HEADER_SIZE + still = OUT / f"OSG-Header-{hw}x{hh}-{lang}-still.png" + render("header.html", f"w={hw}&h={hh}&safe={HEADER_SAFE}&lang={lang}&hero=1", + still, still=POSTER_T) + flatten_png(still) + + +# -------------------------------------------------------------------------- +# Search results + + +def cfr(lang: str, raw: str) -> Path: + src = RAW[lang] / f"{raw}.mov" + if not src.exists(): + raise SystemExit(f"missing recording: {src}") + dest = WORK / "cfr" / lang / f"{raw}.mp4" + if dest.exists() and dest.stat().st_mtime >= src.stat().st_mtime: + return dest + dest.parent.mkdir(parents=True, exist_ok=True) + # simctl footage is VFR; normalise once so window times are real seconds. + run([ + "ffmpeg", "-y", "-i", str(src), "-vf", f"fps={FPS},setsar=1", "-vsync", "cfr", + "-c:v", "libx264", "-pix_fmt", "yuv420p", "-crf", "12", "-preset", "medium", + *COLOR_TAGS, str(dest), + ]) + return dest + + +def screen_mask(w: int, h: int, r: int, dest: Path) -> Path: + """Anti-aliased rounded-rect alpha mask (4× supersampled).""" + ss = 4 + big = Image.new("L", (w * ss, h * ss), 0) + ImageDraw.Draw(big).rounded_rectangle((0, 0, w * ss - 1, h * ss - 1), radius=r * ss, fill=255) + big.resize((w, h), Image.LANCZOS).save(dest) + return dest + + +def screen_loop(lang: str, screen: dict) -> Path: + """Trim → scale → close the loop → round the corners, as ProRes 4444.""" + win = WINDOWS[lang][screen["id"]] + src = cfr(lang, win.raw) + w, h, r = screen["w"], screen["h"], screen["r"] + mask = screen_mask(w, h, r, WORK / f"mask-{w}x{h}.png") + dest = WORK / f"screen-{lang}-{screen['id']}.mov" + body = DURATION + total = DURATION + LOOP_BLEND + span = sum(end - start for start, end in win.segments) + if abs(span - total) > 0.01: + raise SystemExit(f"{lang}/{screen['id']}: segments sum to {span:.2f}s, need {total:.2f}s") + + n = len(win.segments) + chain = f"[0:v]split={n}" + "".join(f"[s{i}]" for i in range(n)) + ";" + for i, (start, end) in enumerate(win.segments): + chain += f"[s{i}]trim=start={start:.3f}:end={end:.3f},setpts=PTS-STARTPTS[g{i}];" + chain += "".join(f"[g{i}]" for i in range(n)) + f"concat=n={n}:v=1:a=0," + chain += ( + f"scale={w}:{h}:flags=lanczos,split=2[a][b];" + f"[a]trim=start=0:end={body:.3f},setpts=PTS-STARTPTS[head];" + f"[b]trim=start={body:.3f}:end={total:.3f},setpts=PTS-STARTPTS," + f"format=yuva444p,fade=t=out:st=0:d={LOOP_BLEND:.3f}:alpha=1[tail];" + f"[head][tail]overlay=0:0:eof_action=pass,format=yuva444p[looped];" + f"[1:v]format=gray,loop=-1:size=1,trim=duration={body:.3f}[m];" + f"[looped][m]alphamerge[v]" + ) + run([ + "ffmpeg", "-y", "-i", str(src), "-i", str(mask), "-filter_complex", chain, + "-map", "[v]", "-t", f"{body:.3f}", "-r", str(FPS), + "-c:v", "prores_ks", "-profile:v", "4", "-pix_fmt", "yuva444p10le", str(dest), + ]) + return dest + + +def build_search(lang: str) -> None: + w, h = SEARCH_SIZE + base_q = f"w={w}&h={h}&lang={lang}" + layout_path = WORK / f"search-layout-{lang}.json" + under = WORK / f"search-under-{lang}.mov" + over = WORK / f"search-over-{lang}.mov" + scene_mtime = (HERE / "search.html").stat().st_mtime + fresh = all(p.exists() and p.stat().st_mtime > scene_mtime for p in (under, over, layout_path)) + if not fresh: + render("search.html", base_q + "&layer=under", under, layout=layout_path) + render("search.html", base_q + "&layer=over", over) + layout = json.loads(layout_path.read_text()) + + screens = [screen_loop(lang, s) for s in layout["screens"]] + + inputs = ["-i", str(under)] + for s in screens: + inputs += ["-i", str(s)] + inputs += ["-i", str(over)] + chain = "[0:v]format=yuva444p10le[base0];" + for i, s in enumerate(layout["screens"], start=1): + chain += f"[base{i - 1}][{i}:v]overlay={s['x']}:{s['y']}:format=auto[base{i}];" + n = len(screens) + chain += f"[base{n}][{n + 1}:v]overlay=0:0:format=auto,format=yuv444p10le,{SET_COLOR}[v]" + + master = WORK / f"search-{lang}.mov" + run([ + "ffmpeg", "-y", *inputs, "-filter_complex", chain, "-map", "[v]", + "-t", f"{DURATION:.3f}", "-r", str(FPS), + "-c:v", "prores_ks", "-profile:v", "4", "-pix_fmt", "yuv444p10le", + *COLOR_TAGS, str(master), + ]) + + full = OUT / f"OSG-Search-{w}x{h}-{lang}.mp4" + deliver(master, full, SEARCH_SIZE, "6.0") + cw, ch = SEARCH_COMPAT + compat = OUT / f"OSG-Search-{cw}x{ch}-{lang}.mp4" + deliver(master, compat, SEARCH_COMPAT, "5.1") + + poster = OUT / f"OSG-Search-{w}x{h}-{lang}-poster.png" + run(["ffmpeg", "-y", "-ss", "4.0", "-i", str(master), "-frames:v", "1", + "-vf", "scale=out_color_matrix=bt709:in_range=tv:out_range=pc,format=rgb24", str(poster)]) + flatten_png(poster) + print(" " + probe(full)) + print(" " + probe(compat)) + + +# -------------------------------------------------------------------------- + + +def main(argv: list[str]) -> int: + langs = ["zh", "en"] + if "--lang" in argv: + i = argv.index("--lang") + langs = argv[i + 1].split(",") + del argv[i:i + 2] + jobs = argv or ["header", "stills", "search"] + if "redeliver-header" in jobs: + # Re-encode the delivery mp4s from existing masters (no re-render). + for lang in langs: + w, h = HEADER_SIZE + dest = OUT / f"OSG-Header-{w}x{h}-{lang}.mp4" + deliver(WORK / f"header-{lang}.mov", dest, HEADER_SIZE, "5.1") + print(" " + probe(dest)) + WORK.mkdir(parents=True, exist_ok=True) + OUT.mkdir(parents=True, exist_ok=True) + for lang in langs: + if "header" in jobs: + print(f"[header {lang}]", flush=True) + build_header(lang) + if "stills" in jobs: + print(f"[universal {lang}]", flush=True) + build_universal(lang) + if "search" in jobs: + print(f"[search {lang}]", flush=True) + build_search(lang) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/Scripts/appstore-creative/header.html b/Scripts/appstore-creative/header.html new file mode 100644 index 0000000..962b9d7 --- /dev/null +++ b/Scripts/appstore-creative/header.html @@ -0,0 +1,506 @@ + + + + + + + + + + + + diff --git a/Scripts/appstore-creative/render.cjs b/Scripts/appstore-creative/render.cjs new file mode 100644 index 0000000..284d04e --- /dev/null +++ b/Scripts/appstore-creative/render.cjs @@ -0,0 +1,104 @@ +#!/usr/bin/env node +// Renders a deterministic canvas scene (window.render(t)) to PNG stills or +// a looping video piped straight into ffmpeg. +// +// NODE_PATH= node render.cjs \ +// --page header.html --query "w=3840&h=1646&lang=zh" \ +// --still 7.0 --out poster.png [--debug-safe] +// ... --video --fps 30 --out frames.mov (ProRes 4444 intermediate) +// +// Chromium comes from the Playwright cache; override with CHROMIUM_PATH. + +const { chromium } = require('playwright-core'); +const { spawn } = require('child_process'); +const path = require('path'); +const fs = require('fs'); +const os = require('os'); + +function arg(name, fallback) { + const i = process.argv.indexOf(`--${name}`); + if (i === -1) return fallback; + const v = process.argv[i + 1]; + return v === undefined || v.startsWith('--') ? true : v; +} + +function chromiumPath() { + if (process.env.CHROMIUM_PATH) return process.env.CHROMIUM_PATH; + const cache = path.join(os.homedir(), 'Library/Caches/ms-playwright'); + const dirs = fs.readdirSync(cache).filter((d) => /^chromium-\d+$/.test(d)).sort().reverse(); + for (const d of dirs) { + const p = path.join(cache, d, + 'chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing'); + if (fs.existsSync(p)) return p; + } + throw new Error('No Playwright Chromium found; set CHROMIUM_PATH'); +} + +(async () => { + const page = path.resolve(__dirname, arg('page', 'header.html')); + const query = arg('query', ''); + const out = path.resolve(arg('out')); + const params = new URLSearchParams(query); + const W = +params.get('w'); + const H = +params.get('h'); + + const browser = await chromium.launch({ + executablePath: chromiumPath(), + args: ['--force-color-profile=srgb', '--disable-gpu-vsync'], + }); + const ctx = await browser.newContext({ viewport: { width: W, height: H }, deviceScaleFactor: 1 }); + const tab = await ctx.newPage(); + tab.on('console', (m) => console.log('[page]', m.text())); + tab.on('pageerror', (e) => { console.error('[page error]', e); process.exit(1); }); + await tab.goto(`file://${page}?${query}`); + await tab.evaluate(() => window.READY); + + const grab = async (t, debugSafe) => { + const b64 = await tab.evaluate(async ({ t, debugSafe }) => { + await window.render(t); + if (debugSafe) window.SAFE_DEBUG(); + const c = document.getElementById('c'); + return c.toDataURL('image/png').split(',')[1]; + }, { t, debugSafe }); + return Buffer.from(b64, 'base64'); + }; + + if (arg('layout')) { + const layout = await tab.evaluate(() => window.LAYOUT); + fs.writeFileSync(path.resolve(arg('layout')), JSON.stringify(layout, null, 2)); + } + + if (arg('still') !== undefined) { + const t = parseFloat(arg('still')); + fs.writeFileSync(out, await grab(t, !!arg('debug-safe'))); + console.log(`wrote ${out}`); + } else if (arg('video')) { + const fps = +arg('fps', 30); + const duration = await tab.evaluate(() => window.DURATION); + const frames = Math.round(duration * fps); + // Lossless-ish intermediate; delivery encodes happen in a second pass. + const ff = spawn('ffmpeg', [ + '-y', '-hide_banner', '-loglevel', 'error', + '-f', 'image2pipe', '-framerate', String(fps), '-c:v', 'png', '-i', '-', + // Same matrix + tags as the simulator footage it gets composited with. + '-vf', 'scale=out_color_matrix=bt709:out_range=tv', + '-c:v', 'prores_ks', '-profile:v', '4', '-pix_fmt', 'yuva444p10le', + '-colorspace', 'bt709', '-color_primaries', 'bt709', + '-color_trc', 'iec61966-2-1', '-color_range', 'tv', + '-vendor', 'apl0', out, + ], { stdio: ['pipe', 'inherit', 'inherit'] }); + const started = Date.now(); + for (let i = 0; i < frames; i++) { + const buf = await grab(i / fps, false); + if (!ff.stdin.write(buf)) await new Promise((r) => ff.stdin.once('drain', r)); + if (i % 30 === 0) { + const s = ((Date.now() - started) / 1000).toFixed(0); + console.log(`frame ${i}/${frames} (${s}s)`); + } + } + ff.stdin.end(); + await new Promise((r, j) => ff.on('close', (c) => (c === 0 ? r() : j(new Error(`ffmpeg ${c}`))))); + console.log(`wrote ${out}`); + } + await browser.close(); +})().catch((e) => { console.error(e); process.exit(1); }); diff --git a/Scripts/appstore-creative/search.html b/Scripts/appstore-creative/search.html new file mode 100644 index 0000000..876ae4c --- /dev/null +++ b/Scripts/appstore-creative/search.html @@ -0,0 +1,365 @@ + + + + + + + + + + + + diff --git a/Scripts/record_feature_previews.sh b/Scripts/record_feature_previews.sh index 2930943..3f4e1fa 100755 --- a/Scripts/record_feature_previews.sh +++ b/Scripts/record_feature_previews.sh @@ -18,6 +18,8 @@ # # Usage: # Scripts/record_feature_previews.sh [slug ...] +# PREVIEW_LANG=en Scripts/record_feature_previews.sh [slug ...] +# -> English demo copy, footage in `.tmp/feature-previews/raw-en` set -euo pipefail ROOT="$(cd "$(dirname "$0")/.." && pwd)" @@ -27,17 +29,23 @@ UDID="${1:?usage: record_feature_previews.sh [slug ...]}" shift || true BUNDLE="com.osgkeyboard.ios" -RAW_DIR="$ROOT/.tmp/feature-previews/raw" +PREVIEW_LANG="${PREVIEW_LANG:-zh}" +if [[ "$PREVIEW_LANG" == "zh" ]]; then + RAW_DIR="$ROOT/.tmp/feature-previews/raw" +else + RAW_DIR="$ROOT/.tmp/feature-previews/raw-$PREVIEW_LANG" +fi mkdir -p "$RAW_DIR" +LANG_ARG="--whats-new-lang=$PREVIEW_LANG" # slug | seconds | launch args CLIPS=( - "voice-polish|23|--edit-demo --preview-fullscreen" + "voice-polish|23|--edit-demo --preview-fullscreen $LANG_ARG" # Static page — the motion is a post-production push-in, so this only needs # to be long enough to fill the App Store 15s minimum after trimming. "personal-style|22|--polish-styles-screenshot" - "clipboard-agent|46|--clipboard-demo --preview-fullscreen" - "ask-ai|36|--ai-demo --preview-fullscreen --whats-new-lang=zh" + "clipboard-agent|46|--clipboard-demo --preview-fullscreen $LANG_ARG" + "ask-ai|36|--ai-demo --preview-fullscreen $LANG_ARG" ) want() { diff --git a/docs/FEATURE_PREVIEW_RECORDING.md b/docs/FEATURE_PREVIEW_RECORDING.md index 44b45a4..378e503 100644 --- a/docs/FEATURE_PREVIEW_RECORDING.md +++ b/docs/FEATURE_PREVIEW_RECORDING.md @@ -59,6 +59,8 @@ xcrun simctl install "$UDID" \ # 3) 录制（脚本会先校验 App Group 容器，缺失直接报错退出） Scripts/record_feature_previews.sh "$UDID" # 全部四段 Scripts/record_feature_previews.sh "$UDID" ask-ai # 只录一段 +# 英文文案（--whats-new-lang=en），素材落在 .tmp/feature-previews/raw-en +PREVIEW_LANG=en Scripts/record_feature_previews.sh "$UDID" voice-polish clipboard-agent ask-ai # 4) 合成。需要 Pillow —— 这版 ffmpeg 没有 libfreetype / libass， # 所有文字都由 PIL 渲染成 RGBA PNG 再 overlay。 @@ -137,6 +139,24 @@ app 里**没有**「点一次回复，弹出多条候选」这种交互：点一 `zoompan` 表达式同时做进和出——它的实际出帧数对不上 `duration * FPS`，片子会 停在半路，循环处肉眼可见地跳。现在的做法是只渲染推近的一半再 `reverse` 拼接。 +## App Store 创意素材（Header / Search Results） + +同一批原始素材（中英文各一套）还驱动 iOS 27 起的 App Store 创意素材，脚本在 +`Scripts/appstore-creative/`：`header.html` / `search.html` 是逐帧确定的 canvas +场景，`render.cjs` 用 Playwright 的 Chromium 渲染，`compose.py` 把录屏合进设备 +屏幕并按 App Store Connect 规格出片。 + +```bash +(cd .tmp/appstore-assets && npm install playwright-core) +NODE_PATH=$PWD/.tmp/appstore-assets/node_modules \ + python3 Scripts/appstore-creative/compose.py # header + stills + search +``` + +- 5244×2950 只接受 PNG；视频 Header 只能是 3840×1646。 +- 安全区取自 Apple 官方 Sketch 模板，`compose.py` 顶部有数值。 +- 3840×2560 的 H.264 需要 Level 6，另出一份 3072×2048 / Level 5.1 兼容版。 +- `compose.py` 的 `WINDOWS` 是照 0.5 秒联系表量出来的，重录后要重新对时。 + ## 改动了 demo 时间轴之后 `Scripts/feature_preview_clips.py` 里的裁切点和字幕时间是照着实际录像量出来的。 From 9e19060a8a5489dde288e28d5c382fd58db25c99 Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:07:12 +0200 Subject: [PATCH 10/19] chore(account): note why the Mac target compiles InvitePoster.swift --- project.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/project.yml b/project.yml index ab80317..0ea4d2f 100644 --- a/project.yml +++ b/project.yml @@ -581,6 +581,7 @@ targets: - path: OSGKeyboard/Views/Account/AccountCreditPurchaseManager.swift - path: OSGKeyboard/Views/Account/LiveAccountServices.swift - path: OSGKeyboard/Views/Account/ReferralProfileViewModel.swift + # Invitation QR poster: pure SwiftUI + Core Image, shared with iOS. - path: OSGKeyboard/Views/Account/InvitePoster.swift - path: OSGKeyboard/Views/Account/AccountAppleAuthorizationFactories.swift - path: OSGKeyboard/Services/AccountAnalyticsBearerBridge.swift From f463f78296adff849d1f920363b552e1bef9c92b Mon Sep 17 00:00:00 2001 From: Rocky Date: Tue, 20 Oct 2026 20:07:35 +0200 Subject: [PATCH 11/19] chore(version): set repository version to 2.1.5 (build 125) --- AGENTS.md | 2 +- README.en.md | 4 ++-- README.md | 4 ++-- docs/index.html | 6 +++--- docs/llms.txt | 2 +- project.yml | 4 ++-- 6 files changed, 11 insertions(+), 11 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 4c53767..53fad0d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ writing commit messages that will ship to users. ### Version format -The current source-of-truth version is **2.1.0 (build 110)**. Releases use stable SemVer: +The current source-of-truth version is **2.1.5 (build 125)**. Releases use stable SemVer: | Field | File | Rule | |-------|------|------| diff --git a/README.en.md b/README.en.md index 1a9645c..ae04a76 100644 --- a/README.en.md +++ b/README.en.md @@ -11,7 +11,7 @@ OSGKeyboard is an AI voice keyboard for iPhone, iPad, and Mac: say it and it's t ![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple) ![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple) ![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift) -![Version](https://img.shields.io/badge/version-2.1.0-3aa05a) +![Version](https://img.shields.io/badge/version-2.1.5-3aa05a) ![License](https://img.shields.io/badge/license-Source%20Available-blue) [Website](https://hkgood.github.io/OSGKeyboard/?lang=en) · [TestFlight Beta](https://testflight.apple.com/join/c2Bz4qK9) · [中文](./README.md) · [Privacy Policy](https://hkgood.github.io/OSGKeyboard/privacy/) · [Changelog](./CHANGELOG.md) @@ -28,7 +28,7 @@ OSGKeyboard is an AI voice keyboard for iPhone, iPad, and Mac: say it and it's t > The public Mac DMG is the signed and notarized historical version 1.1. The macOS target in this repository has moved forward with the 2.2 codebase; build with Xcode 26 for the latest source capabilities. > -> The repository version follows [`project.yml`](./project.yml) (2.1.0, build 110). The `2.2.0` and later entries in CHANGELOG are already reflected in the capability descriptions below. +> The repository version follows [`project.yml`](./project.yml) (2.1.5, build 125). The `2.2.0` and later entries in CHANGELOG are already reflected in the capability descriptions below. ## One keyboard, four ways to input diff --git a/README.md b/README.md index aa2093e..0acf53b 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ OSGKeyboard 是面向 iPhone、iPad 与 Mac 的 AI 语音键盘：开口即文 ![Platform](https://img.shields.io/badge/iOS%20%2F%20iPadOS-26%2B-0078D4?logo=apple) ![Platform](https://img.shields.io/badge/macOS-15%2B-555?logo=apple) ![Swift](https://img.shields.io/badge/Swift-6.0-FA7343?logo=swift) -![Version](https://img.shields.io/badge/version-2.1.0-3aa05a) +![Version](https://img.shields.io/badge/version-2.1.5-3aa05a) ![License](https://img.shields.io/badge/license-Source%20Available-blue) [官网](https://hkgood.github.io/OSGKeyboard/) · [TestFlight 公测](https://testflight.apple.com/join/c2Bz4qK9) · [English](./README.en.md) · [隐私政策](https://hkgood.github.io/OSGKeyboard/privacy/) · [更新记录](./CHANGELOG.md) @@ -28,7 +28,7 @@ OSGKeyboard 是面向 iPhone、iPad 与 Mac 的 AI 语音键盘：开口即文 > 公开下载的 Mac DMG 是已签名并公证的历史版本 1.1。仓库中的 macOS target 随整体源码演进到 2.2；最新源码能力请用 Xcode 26 构建。 > -> 当前仓库版本以 [`project.yml`](./project.yml) 为准（`2.1.0` · build 110）。CHANGELOG 中 `2.2.0` 及更新的条目已经包含在本 README 的能力描述中。 +> 当前仓库版本以 [`project.yml`](./project.yml) 为准（`2.1.5` · build 125）。CHANGELOG 中 `2.2.0` 及更新的条目已经包含在本 README 的能力描述中。 ## 一把键盘，四种输入方式 diff --git a/docs/index.html b/docs/index.html index 4686739..d1fc4af 100644 --- a/docs/index.html +++ b/docs/index.html @@ -93,7 +93,7 @@ "operatingSystem": "iOS 26, iPadOS 26, macOS 15", "applicationCategory": "UtilitiesApplication", "applicationSubCategory": "Keyboard", - "softwareVersion": "2.1.0", + "softwareVersion": "2.1.5", "description": "OSGKeyboard is an agentic keyboard and AI voice keyboard (语音输入法 / 嘴替输入法) for iPhone, iPad, and Mac: on-device dictation by default, an AI assistant with preview-before-insert, a learned personal style that sounds like you, agentic clipboard skills that act on what you copy, Chinese and English typing, providers configured with your own API key, and optional managed credits. Source available, not open source.", "downloadUrl": "https://apps.apple.com/app/osgkeyboard/id6781553267", "installUrl": "https://apps.apple.com/app/osgkeyboard/id6781553267", @@ -899,8 +899,8 @@