Add privacy-safe product analytics
CI / verify (push) Has been cancelled
CI / publish (push) Has been cancelled

Establish an idempotent analytics pipeline and internal decision dashboard while keeping event metadata allowlisted and account deletion enforceable.
This commit is contained in:
Rocky
2026-08-20 15:20:01 +08:00
parent 231c5040a5
commit 4b465e0e5e
38 changed files with 4275 additions and 80 deletions
+3
View File
@@ -4,6 +4,9 @@
- Apple email and avatar data are not requested or stored.
- Deleting an account removes its session, profile, referral, grant, and mutable
account records in the same local transaction before Apple revocation is retried.
- Product analytics installations linked to the account and all of their events
are deleted by database cascade. The service stores only the digest of a
random installation UUID and never stores user content in analytics events.
- Pseudonymous immutable credit-ledger entries, StoreKit transaction audit data,
and time-limited anti-abuse tombstones remain after deletion where required to
prevent replay, preserve financial integrity, and stop repeated trial abuse.
+175
View File
@@ -0,0 +1,175 @@
# OSGKeyboard product analytics metrics dictionary
This document is the canonical definition of product metrics. All dates and
cohorts use UTC calendar boundaries. Counts are based on distinct accounts when
an installation is linked, otherwise on the pseudonymous installation.
## North-star metric
### Weekly AI active users (WAIU)
The number of distinct users that successfully complete at least one AI feature
during a UTC calendar week.
- Managed AI and ASR use server-settled `credit_usage_records`.
- Local and BYOK use accepted `AI_FEATURE_SUCCEEDED` client events.
- Managed client success events provide feature breakdowns but are not added to
the server-settled total, preventing double counting.
- Week-over-week growth is `(current WAIU - previous WAIU) / previous WAIU`.
A missing previous population is reported without a percentage.
## Growth and activation
### New installations
Distinct installations whose first accepted `FIRST_OPEN` event occurred in the
selected period. Acquisition channel is fixed by the first non-`UNKNOWN`
channel observed for the installation.
Allowed channels:
- `APP_STORE_ORGANIC`
- `REFERRAL`
- `SOCIAL_CONTENT`
- `UNKNOWN`
### New accounts
Accounts whose `accounts.created_at` falls in the selected period.
### 24-hour AI activation rate
The percentage of new installations that successfully complete any AI feature
within 24 hours of their first open. The numerator uses the same value-event
rules as WAIU.
### Time to first value
Elapsed time from `FIRST_OPEN` to the first successful AI feature. The dashboard
reports the median in minutes. Users without a successful AI feature are not
included in the median and remain visible in the activation denominator.
## Activity
### AI DAU, WAU and MAU
Distinct value-active users in the last 1, 7 and 30 UTC days ending at the
report's `until` timestamp.
### DAU/MAU stickiness
`AI DAU / AI MAU`. The value is null when MAU is zero.
### Successful AI requests
The sum of settled managed requests and successful local/BYOK client events.
Managed client success events are excluded from this total.
### Successful AI requests per active user
`successful AI requests / distinct value-active users` for the selected period.
## Retention
The cohort date is the UTC date of a user's first successful AI feature.
Retention is value retention, not application-open retention.
- `D1`: active on cohort date + 1 day.
- `D7`: active on cohort date + 7 days.
- `D30`: active on cohort date + 30 days.
Each retention rate uses the original cohort size as denominator. A day that has
not fully elapsed at the report's `until` timestamp is returned as unavailable,
not zero. Channel and first-feature breakdowns are optional dimensions and must
not alter the base cohort definition.
## AI feature usage
Allowed feature types:
- `TRANSCRIPTION`
- `POLISH`
- `AI_ASSISTANT`
- `AGENT`
- `HOTWORD`
- `OTHER`
Allowed execution modes:
- `MANAGED`
- `LOCAL`
- `BYOK`
Feature distributions use accepted client events because server billing only
distinguishes `ASR` and `LLM`. Server-settled aggregates remain authoritative
for managed totals, credits, token counts and ASR duration.
## Credit consumption
### Daily total credit consumption
The sum of non-negative `credit_usage_records.charged_credits` by UTC date.
### Average daily credits per AI active user
For each UTC date, divide total settled credits by distinct managed AI users,
then average those daily values across days containing at least one active user.
### Median user-day credits
The median of per-account daily settled credits. This is shown beside the mean
to prevent a small number of heavy users from distorting typical consumption.
### Average credits per managed request
`settled credits / settled managed requests`. Local and BYOK events consume no
server credits and are excluded.
## Monetization
### 7-day and 30-day free-to-paid conversion
The percentage of newly registered accounts with a first credited StoreKit
purchase no later than 7 or 30 days after registration. Cohorts whose conversion
window has not elapsed are reported separately from mature cohorts.
### Paying users
Distinct accounts with at least one credited StoreKit purchase in the period.
### Repeat purchase rate
The percentage of paying accounts with at least two credited StoreKit purchases
across their lifetime.
StoreKit transaction count and granted credits are operational proxies. Net
revenue, App Store commission and refunds require App Store financial data and
are outside this service's first version.
## Referral funnel
The ordered growth funnel is:
1. `REFERRAL_SHARED` distinct sharing installations.
2. Invitation opens: accepted `INVITE_OPENED` client events plus anonymous
first-party invitation page views. Page views are aggregate requests rather
than distinct people and must be interpreted as a directional funnel signal.
3. Referral-bound accounts.
4. Referral-bound accounts that reach their first value event.
5. Rewarded referral bindings.
Pending and ineligible bindings are parallel status counts, not sequential
funnel steps.
## Experience guardrails
- AI success rate: successful client AI completions divided by starts with a
terminal success or failure event.
- Managed request failure rate: terminal non-settled `provider_requests` divided
by terminal managed requests.
- P50/P95 latency: client duration bucket distribution for all modes; exact
server duration percentiles may be added later.
- Credit-blocked users: distinct installations reporting
`INSUFFICIENT_CREDITS` during the period.
Guardrails are diagnostic and never count as value-active events.
+9 -2
View File
@@ -26,6 +26,10 @@ It does not replace the app's local ASR, BYOK provider access, or iCloud sync.
- `integrity` evaluates DeviceCheck and App Attest evidence.
- `gateway` proxies Volcengine ASR and DeepSeek text requests.
- `inviteweb` serves the first-party invitation landing page.
- `analytics` accepts a strict allowlist of product event metadata and links a
pseudonymous installation to an account only when a valid session is present.
- `admin/stats` combines accepted product events with authoritative billing,
StoreKit, account, and referral aggregates for the internal dashboard.
Modules communicate through narrow ports. Provider clients, Apple clients,
integrity clients, clocks, token generators, and repositories are replaceable in
@@ -60,7 +64,10 @@ tests.
claims required for replay protection.
- Usage metadata contains feature, model, metering units, latency, status, and
rate-card version.
- Product events contain enums, release versions, timestamps, and duration
buckets only. Installation UUIDs are stored as SHA-256 digests.
- Operational logs use request IDs and internal opaque IDs, never Apple subjects
or bearer credentials.
- Account deletion revokes credentials and removes user-linked records. Only
non-identifying aggregate service metrics may remain.
- Account deletion revokes credentials and cascades through linked product
installations and events. Only non-identifying aggregate service metrics may
remain.
+191
View File
@@ -0,0 +1,191 @@
# Product analytics
OSGKeyboard analytics records product behavior metadata only. It is separate
from billing, provider execution and immutable credit ledgers.
## Privacy boundary
Never send or persist:
- audio or audio-derived content;
- keyboard input, prompts, context, transcripts or model output;
- Apple subjects, email addresses, names, tokens, API keys or provider
credentials;
- arbitrary property names or free-form text.
The service stores a SHA-256 digest of the random installation identifier, not
the identifier supplied by the client. Event DTO string representations are
redacted. Product analytics rows linked to an account are deleted with that
account. Anonymous installations that never link to an account are retained for
90 days and are eligible for scheduled deletion in a later operational job.
## Ingestion API
`POST /v1/analytics/events` accepts one batch of 1 to 50 events.
- Authentication is optional so first-open and pre-login events can be
measured. Invalid bearer credentials are rejected.
- `installationId` must be a client-generated UUID stored in the containing app
and shared with the keyboard extension through the App Group.
- When a valid account session is present, the installation is linked to that
account. An installation cannot later be linked to a different account.
- Every `clientEventId` is a client-generated UUID. The pair
`(installation, clientEventId)` is unique.
- Replaying an identical event is accepted and reported as replayed.
- Reusing an event ID with different values returns `409 conflict`.
- The whole batch is validated before persistence and committed atomically.
- `occurredAt` must be no more than 35 days old and no more than five minutes in
the future. UTC ISO-8601 timestamps are required.
The successful response reports `accepted` and `replayed` event counts. It does
not return account or installation identifiers.
## Event catalog
### Lifecycle events
`FIRST_OPEN`
- Required: `surface=APP`, `acquisitionChannel`.
- Optional: `appVersion`, `osVersion`.
- Must be emitted once per installation. Server idempotency protects retries.
`SESSION_STARTED`
- Required: `surface` (`APP` or `KEYBOARD`).
- Optional: `appVersion`, `osVersion`.
`KEYBOARD_ACTIVATED`
- Required: `surface=KEYBOARD`.
- Emit when the custom keyboard becomes active, not for every keystroke.
### AI events
`AI_FEATURE_STARTED`
- Required: `feature`, `executionMode`, `surface`.
- Optional: `durationBucket` is not allowed.
`AI_FEATURE_SUCCEEDED`
- Required: `feature`, `executionMode`, `surface`, `durationBucket`.
- This is the client-side core value event.
- Managed totals remain server-authoritative; the event provides feature and
surface detail.
`AI_FEATURE_FAILED`
- Required: `feature`, `executionMode`, `surface`, `failureCategory`.
- Optional: `durationBucket`.
- Never include provider response bodies or user content.
### Purchase events
`PURCHASE_VIEWED`
- Required: `surface=APP`.
`PURCHASE_STARTED`
- Required: `surface=APP`.
`PURCHASE_CANCELLED`
- Required: `surface=APP`, `failureCategory=CANCELLED`.
Successful purchases are derived from verified StoreKit transactions and must
not be duplicated as client success events.
### Referral events
`REFERRAL_SHARED`
- Required: `surface=APP`.
`INVITE_OPENED`
- Required: `surface=INVITE_WEB`, `acquisitionChannel=REFERRAL`.
- The app may emit this event after handling an invitation Universal Link.
- The first-party web route separately increments a UTC daily aggregate after
validating the referral code. It stores no IP address, user agent, cookie,
installation ID, referral code, or other request metadata.
- No third-party analytics script is permitted.
Referral binding, value qualification and reward are derived from server data.
## Allowed dimensions
`surface`
- `APP`
- `KEYBOARD`
- `INVITE_WEB`
`acquisitionChannel`
- `APP_STORE_ORGANIC`
- `REFERRAL`
- `SOCIAL_CONTENT`
- `UNKNOWN`
`feature`
- `TRANSCRIPTION`
- `POLISH`
- `AI_ASSISTANT`
- `AGENT`
- `HOTWORD`
- `OTHER`
`executionMode`
- `MANAGED`
- `LOCAL`
- `BYOK`
`failureCategory`
- `NETWORK`
- `PROVIDER`
- `TIMEOUT`
- `CANCELLED`
- `INSUFFICIENT_CREDITS`
- `VALIDATION`
- `UNKNOWN`
`durationBucket`
- `LT_1S`
- `S1_TO_3`
- `S3_TO_10`
- `S10_TO_30`
- `GTE_30S`
`appVersion` and `osVersion` are optional ASCII release identifiers of at most
32 characters. They may not contain spaces, user-generated values or device
names.
## Client delivery guidance
- Persist pending events in an append-only local queue.
- Retry with exponential backoff after network failures and HTTP 5xx.
- Drop events rejected with HTTP 400/422 after recording a local diagnostic
counter; do not retry malformed events indefinitely.
- Reuse the same `clientEventId` for every retry.
- Send at most 50 events per batch and remove events only after a successful
response.
- Do not block the AI interaction or purchase flow on analytics delivery.
## Reporting
The admin analytics endpoints combine:
- accepted product events for acquisition, local/BYOK usage and client funnel
detail;
- settled credit usage for managed AI counts and credit consumption;
- accounts, StoreKit and referrals for registration, monetization and referral
outcomes.
Metric formulas are defined in
[`ANALYTICS_METRICS_DICTIONARY.md`](ANALYTICS_METRICS_DICTIONARY.md).
+8
View File
@@ -40,6 +40,9 @@ GRANT SELECT ON osg_account.admin_sessions TO 'osg_account_runtime'@'10.20.%';
GRANT SELECT ON osg_account.admin_audit_log TO 'osg_account_runtime'@'10.20.%';
GRANT SELECT ON osg_account.admin_credit_grants TO 'osg_account_runtime'@'10.20.%';
GRANT SELECT ON osg_account.storekit_credit_purchases TO 'osg_account_runtime'@'10.20.%';
GRANT SELECT ON osg_account.product_analytics_installations TO 'osg_account_runtime'@'10.20.%';
GRANT SELECT ON osg_account.product_analytics_events TO 'osg_account_runtime'@'10.20.%';
GRANT SELECT ON osg_account.product_analytics_daily_counters TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT, UPDATE, DELETE ON osg_account.accounts TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT, UPDATE ON osg_account.apple_credentials TO 'osg_account_runtime'@'10.20.%';
@@ -70,6 +73,11 @@ GRANT INSERT, UPDATE, DELETE ON osg_account.admin_sessions TO 'osg_account_runti
GRANT INSERT ON osg_account.admin_audit_log TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT ON osg_account.admin_credit_grants TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT ON osg_account.storekit_credit_purchases TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT, UPDATE, DELETE ON osg_account.product_analytics_installations
TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT ON osg_account.product_analytics_events TO 'osg_account_runtime'@'10.20.%';
GRANT INSERT, UPDATE ON osg_account.product_analytics_daily_counters
TO 'osg_account_runtime'@'10.20.%';
-- Deliberately absent: global privileges, GRANT OPTION, FILE, PROCESS, SUPER,
-- CREATE USER, and UPDATE/DELETE on immutable ledger or usage-history tables.
+258 -1
View File
@@ -73,6 +73,33 @@ paths:
responses:
"204": { description: Session revoked }
default: { $ref: "#/components/responses/Error" }
/v1/analytics/events:
post:
security:
- {}
- bearerAuth: []
summary: Idempotently accept privacy-minimized product events
description: |
Accepts pre-login or authenticated client events. The random
installation UUID is stored only as a digest. When a valid bearer
session is supplied, the installation is linked to the account and is
deleted with that account. Audio, user text, prompts, transcripts,
model output, credentials and arbitrary properties are never accepted.
requestBody:
required: true
content:
application/json:
schema: { $ref: "#/components/schemas/ProductAnalyticsBatchRequest" }
responses:
"200":
description: Atomic batch acceptance and replay counts
content:
application/json:
schema: { $ref: "#/components/schemas/ProductAnalyticsBatchResponse" }
"400": { $ref: "#/components/responses/Error" }
"409": { $ref: "#/components/responses/Error" }
"422": { $ref: "#/components/responses/Error" }
default: { $ref: "#/components/responses/Error" }
/v1/account:
get:
summary: Return the account profile
@@ -571,6 +598,22 @@ paths:
schema: { $ref: "#/components/schemas/AdminReferralOverview" }
"400": { description: Range is invalid }
"401": { description: Session is invalid }
/v1/admin/analytics:
get:
security:
- adminMtls: []
adminSession: []
summary: Return product growth, retention, usage and monetization analytics
parameters:
- $ref: "#/components/parameters/AdminRange"
responses:
"200":
description: Privacy-minimized product analytics aggregates
content:
application/json:
schema: { $ref: "#/components/schemas/AdminProductAnalytics" }
"400": { description: Range is invalid }
"401": { description: Session is invalid }
/v1/admin/users:
get:
security:
@@ -904,6 +947,69 @@ components:
application/json:
schema: { $ref: "#/components/schemas/GatewayError" }
schemas:
ProductAnalyticsEvent:
type: object
additionalProperties: false
required: [clientEventId, eventType, occurredAt, surface]
properties:
clientEventId: { type: string, format: uuid }
eventType:
type: string
enum:
- FIRST_OPEN
- SESSION_STARTED
- KEYBOARD_ACTIVATED
- AI_FEATURE_STARTED
- AI_FEATURE_SUCCEEDED
- AI_FEATURE_FAILED
- PURCHASE_VIEWED
- PURCHASE_STARTED
- PURCHASE_CANCELLED
- REFERRAL_SHARED
- INVITE_OPENED
occurredAt: { type: string, format: date-time }
surface: { type: string, enum: [APP, KEYBOARD, INVITE_WEB] }
acquisitionChannel:
type: string
enum: [APP_STORE_ORGANIC, REFERRAL, SOCIAL_CONTENT, UNKNOWN]
feature:
type: string
enum: [TRANSCRIPTION, POLISH, AI_ASSISTANT, AGENT, HOTWORD, OTHER]
executionMode: { type: string, enum: [MANAGED, LOCAL, BYOK] }
failureCategory:
type: string
enum: [NETWORK, PROVIDER, TIMEOUT, CANCELLED, INSUFFICIENT_CREDITS, VALIDATION, UNKNOWN]
durationBucket:
type: string
enum: [LT_1S, S1_TO_3, S3_TO_10, S10_TO_30, GTE_30S]
appVersion:
type: string
minLength: 1
maxLength: 32
pattern: "^[A-Za-z0-9._+-]+$"
osVersion:
type: string
minLength: 1
maxLength: 32
pattern: "^[A-Za-z0-9._+-]+$"
ProductAnalyticsBatchRequest:
type: object
additionalProperties: false
required: [installationId, events]
properties:
installationId: { type: string, format: uuid }
events:
type: array
minItems: 1
maxItems: 50
items: { $ref: "#/components/schemas/ProductAnalyticsEvent" }
ProductAnalyticsBatchResponse:
type: object
additionalProperties: false
required: [accepted, replayed]
properties:
accepted: { type: integer, minimum: 0, maximum: 50 }
replayed: { type: integer, minimum: 0, maximum: 50 }
AdminSessionState:
type: object
additionalProperties: false
@@ -978,7 +1084,7 @@ components:
properties:
label:
type: string
enum: [邀请码创建, 成功绑定, 有效使用并奖励, 待资格确认, 不符合奖励条件]
enum: [邀请码创建, 成功绑定, 有效使用并奖励]
count: { type: integer, format: int64, minimum: 0 }
AdminReferralRank:
type: object
@@ -1002,6 +1108,157 @@ components:
ranking:
type: array
items: { $ref: "#/components/schemas/AdminReferralRank" }
AdminAnalyticsRate:
type: object
additionalProperties: false
required: [numerator, denominator]
properties:
numerator: { type: integer, format: int64, minimum: 0 }
denominator: { type: integer, format: int64, minimum: 0 }
percent: { type: ["number", "null"], minimum: 0, maximum: 100 }
AdminAnalyticsFunnelStep:
type: object
additionalProperties: false
required: [label, count]
properties:
label: { type: string, maxLength: 64 }
count: { type: integer, format: int64, minimum: 0 }
AdminAnalyticsChannel:
type: object
additionalProperties: false
required: [channel, installations, activated, activationRate]
properties:
channel:
type: string
enum: [APP_STORE_ORGANIC, REFERRAL, SOCIAL_CONTENT, UNKNOWN]
installations: { type: integer, format: int64, minimum: 0 }
activated: { type: integer, format: int64, minimum: 0 }
activationRate: { $ref: "#/components/schemas/AdminAnalyticsRate" }
AdminAnalyticsCohort:
type: object
additionalProperties: false
required: [cohortDate, size]
properties:
cohortDate: { type: string, format: date }
size: { type: integer, format: int64, minimum: 0 }
d1:
anyOf:
- { $ref: "#/components/schemas/AdminAnalyticsRate" }
- { type: "null" }
d7:
anyOf:
- { $ref: "#/components/schemas/AdminAnalyticsRate" }
- { type: "null" }
d30:
anyOf:
- { $ref: "#/components/schemas/AdminAnalyticsRate" }
- { type: "null" }
AdminAnalyticsFeatureUsage:
type: object
additionalProperties: false
required: [feature, executionMode, users, successes]
properties:
feature:
type: string
enum: [TRANSCRIPTION, POLISH, AI_ASSISTANT, AGENT, HOTWORD, OTHER]
executionMode: { type: string, enum: [MANAGED, LOCAL, BYOK] }
users: { type: integer, format: int64, minimum: 0 }
successes: { type: integer, format: int64, minimum: 0 }
AdminProductAnalytics:
type: object
additionalProperties: false
required:
- period
- northStar
- growth
- activity
- consumption
- monetization
- growthFunnel
- retention
- aiFeatures
- referralFunnel
- guardrails
properties:
period:
type: object
additionalProperties: false
required: [from, until]
properties:
from: { type: string, format: date-time }
until: { type: string, format: date-time }
northStar:
type: object
additionalProperties: false
required: [weeklyAiActiveUsers, previousWeeklyAiActiveUsers]
properties:
weeklyAiActiveUsers: { type: integer, format: int64, minimum: 0 }
previousWeeklyAiActiveUsers: { type: integer, format: int64, minimum: 0 }
weekOverWeekPercent: { type: ["number", "null"] }
growth:
type: object
additionalProperties: false
required: [newInstallations, newAccounts, activation24h, channels]
properties:
newInstallations: { type: integer, format: int64, minimum: 0 }
newAccounts: { type: integer, format: int64, minimum: 0 }
activation24h: { $ref: "#/components/schemas/AdminAnalyticsRate" }
medianTimeToValueMinutes: { type: ["number", "null"], minimum: 0 }
channels:
type: array
items: { $ref: "#/components/schemas/AdminAnalyticsChannel" }
activity:
type: object
additionalProperties: false
required: [dau, wau, mau, successfulAiRequests]
properties:
dau: { type: integer, format: int64, minimum: 0 }
wau: { type: integer, format: int64, minimum: 0 }
mau: { type: integer, format: int64, minimum: 0 }
stickinessPercent: { type: ["number", "null"], minimum: 0, maximum: 100 }
successfulAiRequests: { type: integer, format: int64, minimum: 0 }
successfulRequestsPerActiveUser: { type: ["number", "null"], minimum: 0 }
consumption:
type: object
additionalProperties: false
required: [totalCredits]
properties:
totalCredits: { type: integer, format: int64, minimum: 0 }
averageDailyCreditsPerActiveUser: { type: ["number", "null"], minimum: 0 }
medianUserDailyCredits: { type: ["number", "null"], minimum: 0 }
averageCreditsPerManagedRequest: { type: ["number", "null"], minimum: 0 }
monetization:
type: object
additionalProperties: false
required:
[payingUsers, purchases, creditsPurchased, conversion7d, conversion30d, repeatPurchaseRate]
properties:
payingUsers: { type: integer, format: int64, minimum: 0 }
purchases: { type: integer, format: int64, minimum: 0 }
creditsPurchased: { type: integer, format: int64, minimum: 0 }
conversion7d: { $ref: "#/components/schemas/AdminAnalyticsRate" }
conversion30d: { $ref: "#/components/schemas/AdminAnalyticsRate" }
repeatPurchaseRate: { $ref: "#/components/schemas/AdminAnalyticsRate" }
growthFunnel:
type: array
items: { $ref: "#/components/schemas/AdminAnalyticsFunnelStep" }
retention:
type: array
items: { $ref: "#/components/schemas/AdminAnalyticsCohort" }
aiFeatures:
type: array
items: { $ref: "#/components/schemas/AdminAnalyticsFeatureUsage" }
referralFunnel:
type: array
items: { $ref: "#/components/schemas/AdminAnalyticsFunnelStep" }
guardrails:
type: object
additionalProperties: false
required: [clientAiSuccessRate, managedSuccessRate, creditBlockedUsers]
properties:
clientAiSuccessRate: { $ref: "#/components/schemas/AdminAnalyticsRate" }
managedSuccessRate: { $ref: "#/components/schemas/AdminAnalyticsRate" }
creditBlockedUsers: { type: integer, format: int64, minimum: 0 }
AdminUserSummary:
type: object
additionalProperties: false