OSGAccountServer/README.md

# OSG Account Server

Kotlin 2.4.10 / Ktor 3.5.2 managed AI gateway and invitation page.

## Included

- Controlled `polish`, `ai`, and `agent` requests through DeepSeek's OpenAI-compatible API.
- Volcengine SAUC v3 ASR over WebSocket with a strict binary frame codec.
- Credit reservation, settlement, release, identity, grant, and metadata persistence ports.
- Apple DeviceCheck trial enforcement and App Attest attestation/assertion validation.
- Bilingual invitation page at `GET /i/{code}` with no third-party analytics,
  tracking scripts, or fingerprinting.
- Privacy-minimized, idempotent product event ingestion and an internal growth,
  retention, AI usage, monetization, and referral dashboard.
- Flyway migrations for provider metadata, metered usage, gateway grants, and integrity state.

Prompts, audio, transcripts, and provider response bodies are never sent to the usage persistence port.
Application logging must also keep request/response body logging disabled.

## HTTP API

- `GET /v1/gateway/catalog`
- `POST /v1/gateway/grants`, `POST /v1/gateway/grants/refresh`, `DELETE /v1/gateway/grants/{grantId}`
- `POST /v1/gateway/llm/{polish|ai|agent}`
- `POST /v1/gateway/asr`
- `POST /v1/gateway/asr/sessions`, `GET (WebSocket) /v1/gateway/asr/sessions/{sessionId}/stream`
- `POST /v1/integrity/challenges`
- `POST /v1/integrity/attest`
- `POST /v1/integrity/assert`
- `POST /v1/auth/apple`
- `POST /v1/auth/refresh`, `POST /v1/auth/logout`
- `POST /v1/analytics/events`
- `GET/DELETE /v1/account`
- `POST /v1/apple/events`
- `GET /v1/credits/{balance|ledger|rates}`
- `GET/POST /v1/referrals...`
- `GET /health`, `GET /health/live`, `GET /health/ready`
- `GET /i/{code}`
- `GET /.well-known/apple-app-site-association`
- `GET /apple-app-site-association`
- `GET /v1/admin/analytics` through the private mTLS-protected admin edge

See `docs/openapi.yaml` for request limits and response formats.

## Local development

JDK 21 is required. Start a local MySQL 8.4 database, copy `.env.example` to an untracked `.env`,
replace every required placeholder, then export it into the shell before starting Ktor:

```bash
set -a
source .env
set +a
./gradlew test
./gradlew run
curl --fail http://127.0.0.1:8080/health/live
```

Development may set `APP_ENV=development`, `APPLE_INTEGRITY_ENVIRONMENT=development`, and both
integrity enforcement flags to `false`. Production refuses to start with those values. Never commit
`.env`, `.p8` files, API keys, tokens, database dumps, or generated response data.

## Application wiring

`Application.kt` is the composition root. It installs Ktor security, serialization, authentication,
rate limiting, WebSockets, error handling and Koin; initializes Flyway/Exposed; and mounts auth,
account, Apple event, credit, referral, gateway, invitation and health routes.

The first-party invitation page uses the registered read-only `ReferralLookupPort` adapter. It
preserves case and checks code existence plus campaign validity; the composition root mounts exactly
one `/i/{code}` route.

Gateway identity uses the signed session JWT and verifies the account plus session family in MySQL on
every authenticated request. Logout, refresh-token replay response, Apple events, and account deletion
therefore invalidate access tokens immediately. Gateway grants, provider metadata and metered usage are
persisted in MySQL. Managed calls reserve, settle or release credits through the same immutable credit
ledger exposed read-only by the public credit API. Clients create scope-limited, time-limited gateway
grants explicitly. Signup trial credits are granted only after DeviceCheck bit0 is conservatively marked.

## Apple integrity

DeviceCheck uses Apple's official `query_two_bits` and `update_two_bits` endpoints. ES256 bearer JWTs
are created from `APPLE_TEAM_ID`, `APPLE_KEY_ID`, and `APPLE_PRIVATE_KEY_PEM`; select the matching
development or production endpoint with `APPLE_INTEGRITY_ENVIRONMENT`. Bit0 means the device has
consumed its signup trial, while bit1 is preserved. The plaintext device token is never persisted or
logged. Because DeviceCheck tokens are ephemeral, a MySQL named lock serializes the complete
query-update-confirm window across all server instances. Apple is marked and confirmed before the
idempotent credit entry; an interruption may conservatively forfeit a trial but cannot duplicate one.

App Attest is bound to the configured Apple team, bundle, and environment. Cryptographic attestation,
receipt, assertion, and certificate-chain validation use the
official Apple App Attestation Root CA bundled from Apple Certificate Authority. The server stores the
validated public key, receipt, and strictly increasing assertion counter.

Request an `attestation` challenge after `generateKey`, then call `/attest` with the resulting CBOR
object. For login assertions, request an `assertion` challenge and generate the assertion over SHA-256
of the canonical UTF-8 payload documented in `docs/openapi.yaml`. Challenges are single-use and expire
after `APP_ATTEST_CHALLENGE_TTL_SECONDS`.

`ENFORCE_DEVICE_CHECK` and `ENFORCE_APP_ATTEST` fail closed for missing or unavailable evidence.
`MONITOR` permits login during rollout or Apple outages, but unavailable DeviceCheck evidence never
qualifies for trial credits. Cryptographically rejected evidence is never allowed by either mode.
Production refuses to start unless both policies are `ENFORCE` and the production Apple environment is
selected.

## Account deletion and anti-abuse retention

Account deletion requires a fresh matching Sign in with Apple identity token, authorization code, and
nonce. After reauthentication succeeds, local deletion is committed before the Apple revocation call.
Sessions, referral state, credit balances and reservations, gateway grants, and provider metadata are
removed through database cascades. The newly issued Apple refresh token moves encrypted into a durable
outbox; a successful revoke clears it immediately, while failures retry with bounded exponential backoff.

The immutable credit ledger is retained with only the now-unmapped random account UUID for financial
auditability. To prevent deletion/recreation trial and self-referral abuse, the server retains only an
HMAC-SHA-256 identity tombstone—never the Apple subject—for
`IDENTITY_TOMBSTONE_RETENTION_DAYS` (365 days by default). `IDENTITY_HMAC_KEY` is a separate secret and
must not reuse JWT or field-encryption keys. Tombstoned identities may sign in but cannot receive another
signup trial or participate in referrals during the retention window.

Apple server events accept both the documented `account-delete` name and the previously observed
`account-deleted` variant.

Gateway execution and settlement rules:

- `X-Request-ID` is atomically claimed within the authenticated account and can never invoke a
  provider twice. Replays return `409`, including claimed, started, pending, settled, and released calls.
- LLM requests reserve a UTF-8 byte upper bound plus maximum output tokens, then require internally
  consistent provider input, output, and total token usage.
- Raw PCM/WAV reservations are derived from audio bytes. Compressed audio reserves the full accepted
  ten-minute boundary because its duration cannot be proven from byte length alone.
- Only failures before provider completion release a reservation.
- Provider success is durably moved to `SETTLEMENT_PENDING` before settlement. Settlement failures keep
  credits frozen and the background reconciliation job retries idempotently.
- A successful settlement is returned to the client even if usage metadata needs a later retry.
- Settlement retries must repeat the same actual usage; parameter drift returns a conflict.
- Public idempotency keys are SHA-256 namespaced before storage and cannot collide with internal
  referral, signup-trial, or gateway ledger keys.

## Provider configuration

Existing `application.yaml` values can be mapped into:

- `DeepSeekConfig(endpoint, apiKey, model, reasoningModel)`
- `VolcengineAsrConfig(endpoint, resourceId, appId, accessToken)`
- `InviteWebConfig(appStoreUrl, appleAppId, universalLinkBaseUrl)`

`VolcengineAsrConfig` also supports the newer single `apiKey` credential when the application
configuration exposes it. Missing credentials, non-TLS provider URLs, missing resource IDs, malformed
frames, provider error frames, missing final frames, and missing metered duration all fail closed.
Production additionally requires the exact provider hosts `api.deepseek.com` and
`openspeech.bytedance.com` on the default TLS port.

Configuration ownership:

- Apple: `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_CLIENT_ID`, `APPLE_PRIVATE_KEY_PEM`,
  `APPLE_INTEGRITY_ENVIRONMENT`, plus the two integrity enforcement flags.
- Volcengine: prefer `VOLCENGINE_API_KEY`; set the SAUC v3 `VOLCENGINE_RESOURCE_ID` and WSS
  `VOLCENGINE_ASR_ENDPOINT`. The legacy app ID/access token pair remains optional.
- DeepSeek: set `DEEPSEEK_API_KEY`, the provisioned low-latency `DEEPSEEK_MODEL`,
  and HTTPS `DEEPSEEK_ENDPOINT`. `DEEPSEEK_REASONING_MODEL` is optional and
  falls back to `DEEPSEEK_MODEL`.

Gateway text requests may include the optional stable `taskKind` values documented in
`docs/openapi.yaml`. The server maps `capability + taskKind` to a deterministic execution policy;
it never infers task type from user content. Polish and transform tasks explicitly disable DeepSeek
thinking and do not retry an empty buffered result. AI questions and agent planning explicitly use
high-effort thinking. Search and tools remain disabled for every task because no safe, billable
implementation is configured.

Store production values in 1Panel's secret/environment facility. The Compose environment receives
them at runtime because this application does not read Docker `/run/secrets/*` files directly.

## SAUC v3 protocol boundary

The endpoint uses the SAUC v3 API, while the documented binary header version nibble remains `1`.
The codec implements:

- 4-byte base header and big-endian integer fields.
- Full-client JSON request (`0x1`) with gzip.
- Audio-only request (`0x2`) with gzip and final-packet flag `0x2`.
- Full-server response (`0x9`) with a required strictly increasing sequence and negative final sequence.
- Error response (`0xF`) with error-code field.
- Strict declared-size, payload-limit, gzip, message-type, and final-frame validation.

The client supports `pcm`, `wav`, `ogg`, and `mp3`; 16 kHz, 16-bit, one or two channels; and a
maximum duration of ten minutes. Only the final frame can supply billable `audio_info.duration`.
Clients may use the buffered HTTP endpoint or create a one-shot downstream WebSocket session. Both
paths durably claim the request ID before contacting Volcengine and share the same reservation and
settlement state machine.

## Database

The application runs Flyway against the existing MySQL 8.4 instance before opening the runtime pool.
V1-V7 remain append-only migration history; the invitation web feature requires no new table and does
not modify V4. `V5__account_security_hardening.sql` adds account
foreign keys, identity tombstones, and the encrypted Apple revocation outbox.
`V6__gateway_execution_state.sql` scopes request IDs by account, binds provider requests to credit
reservations, and stores only metering metadata needed for settlement reconciliation. No table has a
request-body, prompt, audio, transcript, or response-body column.

## Container deployment

The production Compose file starts only `account-server`; it does not create a second MySQL.
The application port is bound to `127.0.0.1:18080`, and OpenResty is the only public entry point.

Local build:

```bash
./gradlew clean build
docker build -t osg-account-server:local .
```

Deployment:

```bash
# Create this only when the existing MySQL is containerized and the network is absent.
docker network create --internal account-backend

# Authenticate once because the GHCR package is private.
echo "$GHCR_TOKEN" | docker login ghcr.io -u hkgood --password-stdin

# Put deployment values in an uncommitted .env or 1Panel secret/environment store.
docker compose config --quiet
docker compose pull
docker compose up -d
docker compose ps
curl --fail http://127.0.0.1:18080/health/ready
```

Every successful `main` CI run publishes `ghcr.io/hkgood/osg-account-server:main` plus an immutable
`sha-<commit>` tag. Production should pin a tested immutable tag in `IMAGE_TAG`; use `main` only for
initial staging. `GHCR_TOKEN` needs package-read permission and must not be stored in `.env`.

If MySQL runs directly on the host or another private server, keep the external network declaration
but set `DATABASE_URL` to a private hostname reachable from that network. The application also joins
the separate egress network for Apple and provider HTTPS calls. Never publish MySQL port 3306 to the
Internet.

### Required environment

- Public URLs: `PUBLIC_BASE_URL`, `INVITE_BASE_URL`, `APP_STORE_URL`.
- Existing MySQL: `DATABASE_URL`, runtime `DATABASE_USER` / `DATABASE_PASSWORD`,
  migration-only `DATABASE_MIGRATION_USER` / `DATABASE_MIGRATION_PASSWORD`, and
  `DATABASE_POOL_SIZE`.
  Keep `connectionTimeZone=UTC&forceConnectionTimeZoneToSession=true` in the JDBC URL.
- Session and encryption secrets: `JWT_SECRET`, `FIELD_ENCRYPTION_KEY`, `IDENTITY_HMAC_KEY`.
  The two Base64 keys must be distinct; the field key decodes to exactly 32 bytes.
- Apple: `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_CLIENT_ID`, `APPLE_PRIVATE_KEY_PEM`,
  `APPLE_INTEGRITY_ENVIRONMENT`.
- Providers: `VOLCENGINE_API_KEY`, `VOLCENGINE_RESOURCE_ID`, `DEEPSEEK_API_KEY`,
  `DEEPSEEK_MODEL`; optionally `DEEPSEEK_REASONING_MODEL`.
- Production controls: `APP_ENV=production`, `ENFORCE_DEVICE_CHECK=true`,
  `ENFORCE_APP_ATTEST=true`.
- Optional tuning: token lifetimes, gateway grant days, credit values, binding window and pool size;
  defaults are visible in `compose.yaml`.

All values in the Compose file are placeholders or environment substitutions. Keep real credentials
out of Git, Compose YAML, 1Panel screenshots and shell history.

The Apple ES256 `.p8` material is supplied as `APPLE_PRIVATE_KEY_PEM` with literal `\n` line breaks;
the DeviceCheck JWS adapter reuses that configured Team ID, Key ID and private key. Keep Volcengine and
DeepSeek API keys in separate 1Panel secret fields and rotate each provider independently.

### Existing MySQL minimum privileges

The server uses a Flyway-only account for migrations and a separately pooled runtime account. Apply
`docs/mysql-minimum-privileges.sql` as an administrator after replacing its private host pattern and
generated passwords.
The runtime account can append to ledger and usage history but cannot update/delete that history or
alter rate versions or schemas. The migration account is scoped to `osg_account.*` and has no global
privileges.

Replace the example `10.20.%` host pattern with the actual container subnet. Do not grant either account `FILE`, `PROCESS`,
`SUPER`, `CREATE USER`, `GRANT OPTION`, or access from `%`.

### 1Panel and firewall

1. Create HTTPS sites for `account.osglab.com` and `osglab.com` in 1Panel.
2. Apply `deploy/openresty-account.conf`, adjusting certificate/root paths when needed.
3. Validate with `openresty -t`; AASA is proxied to Ktor at both extensionless paths.
4. Point both DNS records to the server and reload OpenResty after certificate issuance.
5. Allow inbound TCP 80/443 only; allow SSH only from an administrator IP. Do not expose
   18080, 8080 or 3306. Permit required outbound HTTPS for Apple, DeepSeek and Volcengine.

See `docs/DEPLOYMENT.md` for rollback, verification and Universal Link checks.

### Invitation link architecture

Invitation links use the first-party Universal Link `https://osglab.com/i/{code}`. The page contains
no third-party JavaScript, analytics, tracking pixels or external fonts. Firebase Dynamic Links and
Branch are deliberately not used.

## Tests

Run:

```bash
./gradlew test
```

Gateway tests cover account-scoped replay exclusion, cross-account isolation, provider failure release,
settlement pending behavior, reconciliation, whole-call timeout, forged audio duration, malformed SAUC
sequences, and final-frame-only duration.

The MySQL Testcontainers suite validates all migrations, account-deletion
cascades (including linked analytics events), stale access-token rejection, real
InnoDB concurrent balance locking, and exactly-once referral rewards. Analytics
tests additionally cover event validation, idempotent replay, conflict handling,
offline timestamps, and privacy-minimized responses. It is automatically
skipped when Docker is unavailable; CI requires Docker and runs it on every
build.

## Staged acceptance

1. Local: tests pass, malformed/unknown/referral-lookup-failure paths fail closed, and the page makes no
   third-party requests.
2. Container: UID/GID 10001, read-only root filesystem, dropped capabilities, and both health endpoints
   pass.
3. Pre-production: migrations use only the migrator; runtime uses only its table grants; HTTP and WSS
   reverse proxy tests pass.
4. Apple: AASA returns JSON without redirects; a physical device opens the Universal Link; an
   uninstalled device receives the responsive bilingual page.
5. Production: only 80/443 are public, invitation paths are excluded from access logs, provider
   credentials work, and an encrypted backup has been restored successfully in isolation.

See `docs/DEPLOYMENT.md` for exact rollout commands and `docs/BACKUP.md` for backup and recovery.