# OSG Account Server Kotlin 2.4.10 / Ktor 3.5.2 managed AI gateway and invitation page. ## Included - Controlled `polish`, `ai`, and `agent` requests through DeepSeek's OpenAI-compatible API. - Volcengine SAUC v3 ASR over WebSocket with a strict binary frame codec. - Credit reservation, settlement, release, identity, grant, and metadata persistence ports. - Apple DeviceCheck trial enforcement and App Attest attestation/assertion validation. - Bilingual invitation page at `GET /i/{code}` with no analytics, tracking, or fingerprinting. - Flyway migrations for provider metadata, metered usage, gateway grants, and integrity state. Prompts, audio, transcripts, and provider response bodies are never sent to the usage persistence port. Application logging must also keep request/response body logging disabled. ## HTTP API - `GET /v1/gateway/catalog` - `POST /v1/gateway/grants`, `POST /v1/gateway/grants/refresh`, `DELETE /v1/gateway/grants/{grantId}` - `POST /v1/gateway/llm/{polish|ai|agent}` - `POST /v1/gateway/asr` - `POST /v1/gateway/asr/sessions`, `GET (WebSocket) /v1/gateway/asr/sessions/{sessionId}/stream` - `POST /v1/integrity/challenges` - `POST /v1/integrity/attest` - `POST /v1/integrity/assert` - `POST /v1/auth/apple` - `POST /v1/auth/refresh`, `POST /v1/auth/logout` - `GET/DELETE /v1/account` - `POST /v1/apple/events` - `GET /v1/credits/{balance|ledger|rates}` - `GET/POST /v1/referrals...` - `GET /health`, `GET /health/live`, `GET /health/ready` - `GET /i/{code}` - `GET /.well-known/apple-app-site-association` - `GET /apple-app-site-association` See `docs/openapi.yaml` for request limits and response formats. ## Local development JDK 21 is required. Start a local MySQL 8.4 database, copy `.env.example` to an untracked `.env`, replace every required placeholder, then export it into the shell before starting Ktor: ```bash set -a source .env set +a ./gradlew test ./gradlew run curl --fail http://127.0.0.1:8080/health/live ``` Development may set `APP_ENV=development`, `APPLE_INTEGRITY_ENVIRONMENT=development`, and both integrity enforcement flags to `false`. Production refuses to start with those values. Never commit `.env`, `.p8` files, API keys, tokens, database dumps, or generated response data. ## Application wiring `Application.kt` is the composition root. It installs Ktor security, serialization, authentication, rate limiting, WebSockets, error handling and Koin; initializes Flyway/Exposed; and mounts auth, account, Apple event, credit, referral, gateway, invitation and health routes. The first-party invitation page uses the registered read-only `ReferralLookupPort` adapter. It preserves case and checks code existence plus campaign validity; the composition root mounts exactly one `/i/{code}` route. Gateway identity uses the signed session JWT and verifies the account plus session family in MySQL on every authenticated request. Logout, refresh-token replay response, Apple events, and account deletion therefore invalidate access tokens immediately. Gateway grants, provider metadata and metered usage are persisted in MySQL. Managed calls reserve, settle or release credits through the same immutable credit ledger exposed read-only by the public credit API. Clients create scope-limited, time-limited gateway grants explicitly. Signup trial credits are granted only after DeviceCheck bit0 is conservatively marked. ## Apple integrity DeviceCheck uses Apple's official `query_two_bits` and `update_two_bits` endpoints. ES256 bearer JWTs are created from `APPLE_TEAM_ID`, `APPLE_KEY_ID`, and `APPLE_PRIVATE_KEY_PEM`; select the matching development or production endpoint with `APPLE_INTEGRITY_ENVIRONMENT`. Bit0 means the device has consumed its signup trial, while bit1 is preserved. The plaintext device token is never persisted or logged. Because DeviceCheck tokens are ephemeral, a MySQL named lock serializes the complete query-update-confirm window across all server instances. Apple is marked and confirmed before the idempotent credit entry; an interruption may conservatively forfeit a trial but cannot duplicate one. App Attest is bound to the configured Apple team, bundle, and environment. Cryptographic attestation, receipt, assertion, and certificate-chain validation use the official Apple App Attestation Root CA bundled from Apple Certificate Authority. The server stores the validated public key, receipt, and strictly increasing assertion counter. Request an `attestation` challenge after `generateKey`, then call `/attest` with the resulting CBOR object. For login assertions, request an `assertion` challenge and generate the assertion over SHA-256 of the canonical UTF-8 payload documented in `docs/openapi.yaml`. Challenges are single-use and expire after `APP_ATTEST_CHALLENGE_TTL_SECONDS`. `ENFORCE_DEVICE_CHECK` and `ENFORCE_APP_ATTEST` fail closed for missing or unavailable evidence. `MONITOR` permits login during rollout or Apple outages, but unavailable DeviceCheck evidence never qualifies for trial credits. Cryptographically rejected evidence is never allowed by either mode. Production refuses to start unless both policies are `ENFORCE` and the production Apple environment is selected. ## Account deletion and anti-abuse retention Account deletion requires a fresh matching Sign in with Apple identity token, authorization code, and nonce. After reauthentication succeeds, local deletion is committed before the Apple revocation call. Sessions, referral state, credit balances and reservations, gateway grants, and provider metadata are removed through database cascades. The newly issued Apple refresh token moves encrypted into a durable outbox; a successful revoke clears it immediately, while failures retry with bounded exponential backoff. The immutable credit ledger is retained with only the now-unmapped random account UUID for financial auditability. To prevent deletion/recreation trial and self-referral abuse, the server retains only an HMAC-SHA-256 identity tombstone—never the Apple subject—for `IDENTITY_TOMBSTONE_RETENTION_DAYS` (365 days by default). `IDENTITY_HMAC_KEY` is a separate secret and must not reuse JWT or field-encryption keys. Tombstoned identities may sign in but cannot receive another signup trial or participate in referrals during the retention window. Apple server events accept both the documented `account-delete` name and the previously observed `account-deleted` variant. Gateway execution and settlement rules: - `X-Request-ID` is atomically claimed within the authenticated account and can never invoke a provider twice. Replays return `409`, including claimed, started, pending, settled, and released calls. - LLM requests reserve a UTF-8 byte upper bound plus maximum output tokens, then require internally consistent provider input, output, and total token usage. - Raw PCM/WAV reservations are derived from audio bytes. Compressed audio reserves the full accepted ten-minute boundary because its duration cannot be proven from byte length alone. - Only failures before provider completion release a reservation. - Provider success is durably moved to `SETTLEMENT_PENDING` before settlement. Settlement failures keep credits frozen and the background reconciliation job retries idempotently. - A successful settlement is returned to the client even if usage metadata needs a later retry. - Settlement retries must repeat the same actual usage; parameter drift returns a conflict. - Public idempotency keys are SHA-256 namespaced before storage and cannot collide with internal referral, signup-trial, or gateway ledger keys. ## Provider configuration Existing `application.yaml` values can be mapped into: - `DeepSeekConfig(endpoint, apiKey, model, reasoningModel)` - `VolcengineAsrConfig(endpoint, resourceId, appId, accessToken)` - `InviteWebConfig(appStoreUrl, appleAppId, universalLinkBaseUrl)` `VolcengineAsrConfig` also supports the newer single `apiKey` credential when the application configuration exposes it. Missing credentials, non-TLS provider URLs, missing resource IDs, malformed frames, provider error frames, missing final frames, and missing metered duration all fail closed. Production additionally requires the exact provider hosts `api.deepseek.com` and `openspeech.bytedance.com` on the default TLS port. Configuration ownership: - Apple: `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_CLIENT_ID`, `APPLE_PRIVATE_KEY_PEM`, `APPLE_INTEGRITY_ENVIRONMENT`, plus the two integrity enforcement flags. - Volcengine: prefer `VOLCENGINE_API_KEY`; set the SAUC v3 `VOLCENGINE_RESOURCE_ID` and WSS `VOLCENGINE_ASR_ENDPOINT`. The legacy app ID/access token pair remains optional. - DeepSeek: set `DEEPSEEK_API_KEY`, the provisioned low-latency `DEEPSEEK_MODEL`, and HTTPS `DEEPSEEK_ENDPOINT`. `DEEPSEEK_REASONING_MODEL` is optional and falls back to `DEEPSEEK_MODEL`. Gateway text requests may include the optional stable `taskKind` values documented in `docs/openapi.yaml`. The server maps `capability + taskKind` to a deterministic execution policy; it never infers task type from user content. Polish and transform tasks explicitly disable DeepSeek thinking and do not retry an empty buffered result. AI questions and agent planning explicitly use high-effort thinking. Search and tools remain disabled for every task because no safe, billable implementation is configured. Store production values in 1Panel's secret/environment facility. The Compose environment receives them at runtime because this application does not read Docker `/run/secrets/*` files directly. ## SAUC v3 protocol boundary The endpoint uses the SAUC v3 API, while the documented binary header version nibble remains `1`. The codec implements: - 4-byte base header and big-endian integer fields. - Full-client JSON request (`0x1`) with gzip. - Audio-only request (`0x2`) with gzip and final-packet flag `0x2`. - Full-server response (`0x9`) with a required strictly increasing sequence and negative final sequence. - Error response (`0xF`) with error-code field. - Strict declared-size, payload-limit, gzip, message-type, and final-frame validation. The client supports `pcm`, `wav`, `ogg`, and `mp3`; 16 kHz, 16-bit, one or two channels; and a maximum duration of ten minutes. Only the final frame can supply billable `audio_info.duration`. Clients may use the buffered HTTP endpoint or create a one-shot downstream WebSocket session. Both paths durably claim the request ID before contacting Volcengine and share the same reservation and settlement state machine. ## Database The application runs Flyway against the existing MySQL 8.4 instance before opening the runtime pool. V1-V7 remain append-only migration history; the invitation web feature requires no new table and does not modify V4. `V5__account_security_hardening.sql` adds account foreign keys, identity tombstones, and the encrypted Apple revocation outbox. `V6__gateway_execution_state.sql` scopes request IDs by account, binds provider requests to credit reservations, and stores only metering metadata needed for settlement reconciliation. No table has a request-body, prompt, audio, transcript, or response-body column. ## Container deployment The production Compose file starts only `account-server`; it does not create a second MySQL. The application port is bound to `127.0.0.1:18080`, and OpenResty is the only public entry point. Local build: ```bash ./gradlew clean build docker build -t osg-account-server:local . ``` Deployment: ```bash # Create this only when the existing MySQL is containerized and the network is absent. docker network create --internal account-backend # Authenticate once because the GHCR package is private. echo "$GHCR_TOKEN" | docker login ghcr.io -u hkgood --password-stdin # Put deployment values in an uncommitted .env or 1Panel secret/environment store. docker compose config --quiet docker compose pull docker compose up -d docker compose ps curl --fail http://127.0.0.1:18080/health/ready ``` Every successful `main` CI run publishes `ghcr.io/hkgood/osg-account-server:main` plus an immutable `sha-` tag. Production should pin a tested immutable tag in `IMAGE_TAG`; use `main` only for initial staging. `GHCR_TOKEN` needs package-read permission and must not be stored in `.env`. If MySQL runs directly on the host or another private server, keep the external network declaration but set `DATABASE_URL` to a private hostname reachable from that network. The application also joins the separate egress network for Apple and provider HTTPS calls. Never publish MySQL port 3306 to the Internet. ### Required environment - Public URLs: `PUBLIC_BASE_URL`, `INVITE_BASE_URL`, `APP_STORE_URL`. - Existing MySQL: `DATABASE_URL`, runtime `DATABASE_USER` / `DATABASE_PASSWORD`, migration-only `DATABASE_MIGRATION_USER` / `DATABASE_MIGRATION_PASSWORD`, and `DATABASE_POOL_SIZE`. Keep `connectionTimeZone=UTC&forceConnectionTimeZoneToSession=true` in the JDBC URL. - Session and encryption secrets: `JWT_SECRET`, `FIELD_ENCRYPTION_KEY`, `IDENTITY_HMAC_KEY`. The two Base64 keys must be distinct; the field key decodes to exactly 32 bytes. - Apple: `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_CLIENT_ID`, `APPLE_PRIVATE_KEY_PEM`, `APPLE_INTEGRITY_ENVIRONMENT`. - Providers: `VOLCENGINE_API_KEY`, `VOLCENGINE_RESOURCE_ID`, `DEEPSEEK_API_KEY`, `DEEPSEEK_MODEL`; optionally `DEEPSEEK_REASONING_MODEL`. - Production controls: `APP_ENV=production`, `ENFORCE_DEVICE_CHECK=true`, `ENFORCE_APP_ATTEST=true`. - Optional tuning: token lifetimes, gateway grant days, credit values, binding window and pool size; defaults are visible in `compose.yaml`. All values in the Compose file are placeholders or environment substitutions. Keep real credentials out of Git, Compose YAML, 1Panel screenshots and shell history. The Apple ES256 `.p8` material is supplied as `APPLE_PRIVATE_KEY_PEM` with literal `\n` line breaks; the DeviceCheck JWS adapter reuses that configured Team ID, Key ID and private key. Keep Volcengine and DeepSeek API keys in separate 1Panel secret fields and rotate each provider independently. ### Existing MySQL minimum privileges The server uses a Flyway-only account for migrations and a separately pooled runtime account. Apply `docs/mysql-minimum-privileges.sql` as an administrator after replacing its private host pattern and generated passwords. The runtime account can append to ledger and usage history but cannot update/delete that history or alter rate versions or schemas. The migration account is scoped to `osg_account.*` and has no global privileges. Replace the example `10.20.%` host pattern with the actual container subnet. Do not grant either account `FILE`, `PROCESS`, `SUPER`, `CREATE USER`, `GRANT OPTION`, or access from `%`. ### 1Panel and firewall 1. Create HTTPS sites for `account.osglab.com` and `osglab.com` in 1Panel. 2. Apply `deploy/openresty-account.conf`, adjusting certificate/root paths when needed. 3. Validate with `openresty -t`; AASA is proxied to Ktor at both extensionless paths. 4. Point both DNS records to the server and reload OpenResty after certificate issuance. 5. Allow inbound TCP 80/443 only; allow SSH only from an administrator IP. Do not expose 18080, 8080 or 3306. Permit required outbound HTTPS for Apple, DeepSeek and Volcengine. See `docs/DEPLOYMENT.md` for rollback, verification and Universal Link checks. ### Invitation link architecture Invitation links use the first-party Universal Link `https://osglab.com/i/{code}`. The page contains no third-party JavaScript, analytics, tracking pixels or external fonts. Firebase Dynamic Links and Branch are deliberately not used. ## Tests Run: ```bash ./gradlew test ``` Gateway tests cover account-scoped replay exclusion, cross-account isolation, provider failure release, settlement pending behavior, reconciliation, whole-call timeout, forged audio duration, malformed SAUC sequences, and final-frame-only duration. The MySQL Testcontainers suite validates all migrations, account-deletion cascades, stale access-token rejection, real InnoDB concurrent balance locking, and exactly-once referral rewards. It is automatically skipped when Docker is unavailable; CI requires Docker and runs it on every build. ## Staged acceptance 1. Local: tests pass, malformed/unknown/referral-lookup-failure paths fail closed, and the page makes no third-party requests. 2. Container: UID/GID 10001, read-only root filesystem, dropped capabilities, and both health endpoints pass. 3. Pre-production: migrations use only the migrator; runtime uses only its table grants; HTTP and WSS reverse proxy tests pass. 4. Apple: AASA returns JSON without redirects; a physical device opens the Universal Link; an uninstalled device receives the responsive bilingual page. 5. Production: only 80/443 are public, invitation paths are excluded from access logs, provider credentials work, and an encrypted backup has been restored successfully in isolation. See `docs/DEPLOYMENT.md` for exact rollout commands and `docs/BACKUP.md` for backup and recovery.