Harden admin deployment and local acceptance

Enforce mTLS and least-privilege runtime boundaries while adding repeatable MySQL 8.4 and Docker smoke checks that require no production secrets.
This commit is contained in:
Rocky
2026-08-17 15:20:46 +08:00
parent 1a9c518f96
commit 405a2cfc0f
17 changed files with 1480 additions and 37 deletions
+86
View File
@@ -0,0 +1,86 @@
# 管理端 mTLS 部署
`account.osglab.com` 在同一个 TLS `server` 中同时承载移动端 API 和管理端。
由于 TLS 握手发生在 HTTP 路径匹配之前,配置必须使用 server 级
`ssl_verify_client optional`:普通客户端不提供证书时仍可正常访问,只有
`/admin``/admin/``/v1/admin` 和其子路径要求验证成功。
## CA 与证书
1. 为管理客户端创建独立私有 CA,不要复用公网服务端证书 CA 或其他内部 CA。
2. CA 私钥离线保存;不要放入仓库、OpenResty 主机或容器镜像。
3. 管理客户端证书使用短有效期和唯一密钥,并限制为 TLS Client Authentication
(`clientAuth`) 用途。
4. 仅将 CA 证书链(不含任何私钥)部署到:
`/www/server/openresty/conf/mtls/admin-client-ca.pem`
5. CA 文件由 OpenResty 运行用户只读,目录不可由应用进程或非特权用户写入。
6. 更新 CA 文件后先运行 `openresty -t`,成功后再平滑重载。若需要立即吊销证书,
应另外配置并维护 `ssl_crl`;当前配置只依据证书链和有效期验证。
不要把客户端证书、客户端私钥、CA 私钥或生产证书标识提交到仓库。
计划轮换与紧急处置步骤见 [ROTATION.md](ROTATION.md)。
## 上游信任边界
OpenResty 仅在管理路径且 `$ssl_client_verify = SUCCESS` 时向 Ktor 设置固定头:
```text
X-OSG-mTLS-Verified: SUCCESS
```
客户端传入的同名头会被覆盖;其他路径会删除该头。Ktor 只能把这个头作为“边缘已验证”
信号,不能信任客户端提供的证书相关头,也不能用 DN、CN 或证书正文做隐式授权。
后端端口必须继续只监听 `127.0.0.1:18080`,否则攻击者可绕过边缘伪造该头。
mTLS 只证明客户端持有受信证书,管理接口仍应执行应用层身份认证、授权和审计。
## 一次性管理员 Bootstrap
首次部署前运行 `./gradlew generateAdminCredentials`,将生成的 runtime 文件仅临时写入
1Panel/Compose 环境,并同时设置:
```text
ADMIN_ENABLED=true
ADMIN_BOOTSTRAP_ENABLED=true
```
确认初始管理员已创建且可以登录后,必须将 `ADMIN_BOOTSTRAP_ENABLED` 改回 `false`
并从 1Panel、Compose 环境和部署文件中永久删除
`ADMIN_BOOTSTRAP_OPERATOR_ID``ADMIN_BOOTSTRAP_USERNAME`
`ADMIN_BOOTSTRAP_PASSWORD_HASH``ADMIN_BOOTSTRAP_TOTP_SECRET_BASE32`
日常运行只保留 `ADMIN_ENABLED=true`。重启后再次验证登录,确保服务不再依赖 Bootstrap
秘密。
## 验证
将测试域名解析到目标边缘后执行:
```sh
# 无证书:管理路径必须是 404。
curl -i https://account.osglab.com/admin
curl -i https://account.osglab.com/v1/admin
# 有效管理证书:请求应到达 Ktor,状态码由管理接口决定。
curl -i --cert admin-client.pem --key admin-client-key.pem \
https://account.osglab.com/v1/admin
# 无证书的普通移动端 API:响应应与变更前一致。
curl -i https://account.osglab.com/health
# 即使客户端伪造信任头,无证书访问管理路径仍必须是 404。
curl -i -H 'X-OSG-mTLS-Verified: SUCCESS' \
https://account.osglab.com/v1/admin
```
还应使用由非管理 CA 签发或已过期的客户端证书确认返回 404,并在 Ktor 测试端点确认:
管理请求只收到固定值 `SUCCESS`,普通 API 不收到 `X-OSG-mTLS-Verified`
部署后可在受信设备运行不含登录凭据的自动验收:
```sh
ADMIN_CLIENT_CERT=/secure/path/admin-client.pem \
ADMIN_CLIENT_KEY=/secure/path/admin-client-key.pem \
bash deploy/verify-admin.sh
```
脚本验证公开健康检查、无证书隐藏、伪造边缘头拦截、有效证书访问,以及 HSTS/CSP
安全响应头;它不会读取或传输管理员密码和 TOTP 密钥。
+44
View File
@@ -0,0 +1,44 @@
# 管理端 mTLS 证书轮换
管理端客户端证书应至少每 180 天轮换一次;设备丢失、人员离职或私钥疑似泄露时立即轮换。
当前配置未启用 CRL,因此单个客户端证书无法被可靠撤销。出现紧急事件时必须轮换整个管理端
CA,不能只删除本地 `.p12` 文件。
## 零中断轮换
1. 在离线受信设备上生成一套新的 CA 和客户端证书,输出到带日期的新目录。不要覆盖现有目录。
2. 校验新证书的有效期、用途和私钥匹配关系:
```shell
openssl verify -CAfile admin-client-ca.pem admin-client.pem
openssl x509 -in admin-client.pem -noout -dates -subject -issuer
openssl pkcs12 -in admin-client.p12 -info -noout
```
3. 将旧 CA 与新 CA 的公开证书按顺序合并为临时信任包。信任包不得包含任何 CA 私钥:
```shell
cp current-admin-client-ca.pem admin-client-ca-transition.pem
printf '\n' >> admin-client-ca-transition.pem
openssl x509 -in new-admin-client-ca.pem -outform PEM >> admin-client-ca-transition.pem
```
4. 通过 1Panel 将临时信任包替换到 OpenResty 配置引用的
`/www/server/openresty/conf/mtls/admin-client-ca.pem`,先执行 `openresty -t`,成功后再平滑重载。
5. 分别使用旧客户端证书和新客户端证书验证 `/admin/` 与 `/v1/admin/auth/session`。
6. 通过受控的点对点渠道交付新的 `.p12` 与独立密码。不得使用工单附件、群聊、Git 或公开网盘。
7. 确认所有获准设备完成安装后,将服务器信任包替换为仅包含新 CA 的证书,再次执行
`nginx -t` 和平滑重载。
8. 验证新证书成功、旧证书返回 404,然后安全销毁旧客户端私钥和旧 CA 私钥。
## 验收记录
每次轮换必须在安全审计记录中保存以下非敏感信息:
- 轮换时间、执行人与复核人
- 新 CA 和客户端证书的 SHA-256 指纹与失效时间
- 新旧证书切换验证结果
- 旧证书停止信任的准确时间
- 触发原因:计划轮换、设备丢失、人员变更或疑似泄露
不得记录 `.p12` 密码、TOTP 密钥、私钥内容或管理员密码。
+160
View File
@@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""Generate a dedicated admin client CA and one short-lived client certificate."""
from __future__ import annotations
import argparse
import base64
import datetime as dt
import os
from pathlib import Path
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives.serialization import pkcs12
from cryptography.x509.oid import ExtendedKeyUsageOID, NameOID
def private_write(path: Path, value: bytes) -> None:
path.write_bytes(value)
path.chmod(0o600)
def public_write(path: Path, value: bytes) -> None:
path.write_bytes(value)
path.chmod(0o644)
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("output_directory", type=Path)
parser.add_argument("--client-name", default="osg-admin-owner")
args = parser.parse_args()
output = args.output_directory.expanduser().resolve()
if output.exists():
raise SystemExit("output_directory_already_exists")
output.mkdir(mode=0o700, parents=False)
now = dt.datetime.now(dt.timezone.utc)
ca_key = ec.generate_private_key(ec.SECP384R1())
ca_subject = x509.Name(
[
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "OSG Admin"),
x509.NameAttribute(NameOID.COMMON_NAME, "OSG Admin Client CA"),
]
)
ca_certificate = (
x509.CertificateBuilder()
.subject_name(ca_subject)
.issuer_name(ca_subject)
.public_key(ca_key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - dt.timedelta(minutes=5))
.not_valid_after(now + dt.timedelta(days=3650))
.add_extension(x509.BasicConstraints(ca=True, path_length=0), critical=True)
.add_extension(
x509.KeyUsage(
digital_signature=False,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=True,
crl_sign=True,
encipher_only=False,
decipher_only=False,
),
critical=True,
)
.add_extension(x509.SubjectKeyIdentifier.from_public_key(ca_key.public_key()), False)
.sign(ca_key, hashes.SHA384())
)
client_key = ec.generate_private_key(ec.SECP256R1())
client_subject = x509.Name(
[
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "OSG Admin"),
x509.NameAttribute(NameOID.COMMON_NAME, args.client_name),
]
)
client_certificate = (
x509.CertificateBuilder()
.subject_name(client_subject)
.issuer_name(ca_certificate.subject)
.public_key(client_key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - dt.timedelta(minutes=5))
.not_valid_after(now + dt.timedelta(days=180))
.add_extension(x509.BasicConstraints(ca=False, path_length=None), critical=True)
.add_extension(
x509.KeyUsage(
digital_signature=True,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=False,
crl_sign=False,
encipher_only=False,
decipher_only=False,
),
critical=True,
)
.add_extension(
x509.ExtendedKeyUsage([ExtendedKeyUsageOID.CLIENT_AUTH]),
critical=True,
)
.add_extension(
x509.SubjectKeyIdentifier.from_public_key(client_key.public_key()),
critical=False,
)
.add_extension(
x509.AuthorityKeyIdentifier.from_issuer_public_key(ca_key.public_key()),
critical=False,
)
.sign(ca_key, hashes.SHA256())
)
password = base64.urlsafe_b64encode(os.urandom(24)).rstrip(b"=")
private_write(
output / "admin-client-ca-key.pem",
ca_key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.PKCS8,
serialization.NoEncryption(),
),
)
public_write(
output / "admin-client-ca.pem",
ca_certificate.public_bytes(serialization.Encoding.PEM),
)
private_write(
output / "admin-client-key.pem",
client_key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.PKCS8,
serialization.NoEncryption(),
),
)
public_write(
output / "admin-client.pem",
client_certificate.public_bytes(serialization.Encoding.PEM),
)
private_write(
output / "admin-client.p12",
pkcs12.serialize_key_and_certificates(
args.client_name.encode(),
client_key,
client_certificate,
[ca_certificate],
serialization.BestAvailableEncryption(password),
),
)
private_write(output / "admin-client-p12-password.txt", password + b"\n")
print("certificates_created=true")
if __name__ == "__main__":
main()
+52 -2
View File
@@ -31,6 +31,14 @@ server {
ssl_protocols TLSv1.2 TLSv1.3;
ssl_session_timeout 1d;
ssl_session_cache shared:account_tls:10m;
# Request client certificates at the shared TLS edge without requiring them
# for mobile APIs. Only the dedicated admin CA is trusted for verification.
ssl_client_certificate /www/server/openresty/conf/mtls/admin-client-ca.pem;
ssl_verify_client optional;
ssl_verify_depth 2;
# An invalid certificate is rejected before location processing; normalize
# that TLS verification failure so it does not reveal the protected surface.
error_page 495 =404 @client_certificate_not_found;
client_max_body_size 21m;
server_tokens off;
@@ -39,8 +47,48 @@ server {
add_header X-Frame-Options "DENY" always;
add_header Referrer-Policy "no-referrer" always;
# Never publish operational or administrative paths through this vhost.
location ~ ^/(?:admin|internal|v1/admin)(?:/|$) {
# Internal-only endpoints remain unavailable through this public vhost.
location ~ ^/internal(?:/|$) {
return 404;
}
# Administrative endpoints are indistinguishable from missing routes unless
# OpenResty verified a certificate issued by the dedicated admin client CA.
location ~ ^/(?:admin|v1/admin)(?:/|$) {
if ($ssl_client_verify != SUCCESS) {
return 404;
}
client_max_body_size 32k;
limit_req zone=account_api burst=40 nodelay;
# Defining a location-level header disables inheritance from the server
# block, so repeat the security headers explicitly.
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-Frame-Options "DENY" always;
add_header Referrer-Policy "no-referrer" always;
add_header Content-Security-Policy "default-src 'self'; base-uri 'none'; frame-ancestors 'none'; form-action 'self'; object-src 'none'; script-src 'self'; style-src 'self'; connect-src 'self'" always;
add_header Cache-Control "no-store" always;
proxy_pass http://osg_account_server;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Request-ID $request_id;
# Overwrite any client-supplied value; Ktor must trust only this header.
proxy_set_header X-OSG-mTLS-Verified "SUCCESS";
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_hide_header Server;
proxy_request_buffering off;
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 360s;
proxy_send_timeout 360s;
}
location @client_certificate_not_found {
return 404;
}
@@ -53,6 +101,8 @@ server {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Request-ID $request_id;
# Suppress spoofed trust signals on every non-admin request.
proxy_set_header X-OSG-mTLS-Verified "";
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_hide_header Server;
+564
View File
@@ -0,0 +1,564 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
COMPOSE_FILE="$ROOT_DIR/compose.smoke.yaml"
SMOKE_DIR="$ROOT_DIR/deploy/smoke"
WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/osg-account-smoke.XXXXXX")"
SECRET_DIR="$WORK_DIR/secrets"
DIAGNOSTICS_FILE="$WORK_DIR/diagnostics.log"
COMPOSE_ENV="$SECRET_DIR/compose.env"
RUNTIME_ENV="$SECRET_DIR/runtime.env"
ADMIN_ENV="$SECRET_DIR/admin.env"
ADMIN_GENERATED_ENV="$SECRET_DIR/admin.generated.env"
ADMIN_HANDOFF="$SECRET_DIR/admin-handoff.txt"
SUPER_COOKIE_JAR="$SECRET_DIR/super.cookies"
ANALYST_COOKIE_JAR="$SECRET_DIR/analyst.cookies"
RESPONSE_BODY="$WORK_DIR/response.json"
ACTUAL_PRIVILEGES="$WORK_DIR/runtime-privileges.tsv"
RUN_ID="$(date -u +%Y%m%d%H%M%S)-$$"
PROJECT_NAME="osg-account-smoke-$RUN_ID"
SMOKE_ACCOUNT_ID="10000000-0000-0000-0000-000000000001"
FAILURES=0
mkdir -m 700 "$SECRET_DIR"
touch "$ADMIN_ENV"
chmod 600 "$ADMIN_ENV"
log() {
printf '[smoke] %s\n' "$*"
}
fail() {
printf '[smoke] FAIL: %s\n' "$*" >&2
return 1
}
require_command() {
command -v "$1" >/dev/null 2>&1 || {
printf '[smoke] missing required command: %s\n' "$1" >&2
exit 1
}
}
random_hex() {
openssl rand -hex "$1"
}
random_base64_key() {
openssl rand -base64 32 | tr -d '\n'
}
free_loopback_port() {
python3 - <<'PY'
import socket
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
print(sock.getsockname()[1])
PY
}
compose() {
docker compose \
--project-name "$PROJECT_NAME" \
--env-file "$COMPOSE_ENV" \
--file "$COMPOSE_FILE" \
"$@"
}
mysql_root() {
compose exec -T \
-e "MYSQL_PWD=$MYSQL_ROOT_PASSWORD" \
mysql mysql --protocol=tcp --host=127.0.0.1 --user=root "$@"
}
mysql_runtime() {
compose exec -T \
-e "MYSQL_PWD=$MYSQL_RUNTIME_PASSWORD" \
mysql mysql --protocol=tcp --host=127.0.0.1 --user=osg_smoke_runtime "$@"
}
capture_diagnostics() {
{
printf '%s\n' '=== compose ps ==='
compose --profile setup ps --all || true
printf '%s\n' '=== mysql/app logs (last 200 lines) ==='
compose --profile setup logs --no-color --tail 200 \
mysql schema-migrator account-server || true
} >"$DIAGNOSTICS_FILE" 2>&1
chmod 600 "$DIAGNOSTICS_FILE"
}
cleanup() {
local status=$?
trap - EXIT INT TERM
if ((status != 0)); then
capture_diagnostics
fi
compose --profile setup down --volumes --remove-orphans --rmi local >/dev/null 2>&1 || true
rm -f \
"$COMPOSE_ENV" \
"$RUNTIME_ENV" \
"$ADMIN_ENV" \
"$ADMIN_GENERATED_ENV" \
"$ADMIN_HANDOFF" \
"$SUPER_COOKIE_JAR" \
"$ANALYST_COOKIE_JAR" \
"$RESPONSE_BODY" \
"$ACTUAL_PRIVILEGES" \
"$WORK_DIR/denied.err"
rmdir "$SECRET_DIR" 2>/dev/null || true
if ((status == 0)); then
rm -f "$DIAGNOSTICS_FILE"
rmdir "$WORK_DIR" 2>/dev/null || true
log "PASS: containers, volume, image, and temporary credentials removed"
else
printf '[smoke] non-sensitive diagnostics kept at %s\n' "$DIAGNOSTICS_FILE" >&2
fi
exit "$status"
}
trap cleanup EXIT INT TERM
wait_for_service_http() {
local service=$1
local url=$2
local attempts=${3:-90}
local attempt
for ((attempt = 1; attempt <= attempts; attempt++)); do
if compose exec -T "$service" wget -q -O /dev/null "$url" 2>/dev/null; then
return 0
fi
if [[ "$(compose ps --all --format json "$service" 2>/dev/null || true)" == *'"State":"exited"'* ]]; then
return 1
fi
sleep 2
done
return 1
}
wait_for_host_http() {
local path=$1
local attempts=${2:-30}
local attempt status
for ((attempt = 1; attempt <= attempts; attempt++)); do
status="$(curl --silent --output /dev/null --write-out '%{http_code}' \
"http://127.0.0.1:$APP_PORT$path" 2>/dev/null || true)"
if [[ "$status" == "200" ]]; then
return 0
fi
sleep 1
done
return 1
}
http_request() {
local expected=$1
local method=$2
local path=$3
local cookie_jar=$4
local body=$5
shift 5
local -a command=(
curl --silent --show-error
--output "$RESPONSE_BODY"
--write-out '%{http_code}'
--request "$method"
"http://127.0.0.1:$APP_PORT$path"
)
local header
if [[ "$cookie_jar" != "-" ]]; then
command+=(--cookie "$cookie_jar" --cookie-jar "$cookie_jar")
fi
if [[ -n "$body" ]]; then
command+=(--header 'Content-Type: application/json' --data "$body")
fi
for header in "$@"; do
command+=(--header "$header")
done
local status
status="$("${command[@]}")"
[[ "$status" == "$expected" ]] || fail "$method $path returned $status, expected $expected"
}
json_value() {
local file=$1
shift
python3 - "$file" "$@" <<'PY'
import json
import sys
value = json.load(open(sys.argv[1], encoding="utf-8"))
for key in sys.argv[2:]:
if key.isdigit():
value = value[int(key)]
else:
value = value[key]
if value is None:
print("")
elif isinstance(value, bool):
print(str(value).lower())
else:
print(value)
PY
}
totp_code() {
local secret=$1
python3 - "$secret" <<'PY'
import base64
import hashlib
import hmac
import struct
import sys
import time
secret = base64.b32decode(sys.argv[1], casefold=True)
counter = int(time.time()) // 30
digest = hmac.new(secret, struct.pack(">Q", counter), hashlib.sha1).digest()
offset = digest[-1] & 0x0F
number = struct.unpack(">I", digest[offset:offset + 4])[0] & 0x7FFFFFFF
print(f"{number % 1_000_000:06d}")
PY
}
url_encode() {
python3 - "$1" <<'PY'
import sys
import urllib.parse
print(urllib.parse.quote(sys.argv[1], safe=""))
PY
}
assert_json_value() {
local expected=$1
shift
local actual
actual="$(json_value "$RESPONSE_BODY" "$@")"
[[ "$actual" == "$expected" ]] || fail "JSON value '$actual' did not equal '$expected'"
}
verify_privilege_matrix() {
mysql_root --batch --skip-column-names information_schema >"$ACTUAL_PRIVILEGES" <<'SQL'
SELECT TABLE_NAME, PRIVILEGE_TYPE
FROM TABLE_PRIVILEGES
WHERE GRANTEE = '''osg_smoke_runtime''@''%'''
AND TABLE_SCHEMA = 'osg_account_smoke'
ORDER BY TABLE_NAME, PRIVILEGE_TYPE;
SQL
python3 - "$SMOKE_DIR/runtime-grants.sql" "$ACTUAL_PRIVILEGES" <<'PY'
import re
import sys
grant_pattern = re.compile(
r"^GRANT\s+(.+?)\s+ON\s+osg_account_smoke\.([a-z0-9_]+)\s+TO\s+'osg_smoke_runtime'@'%';$",
re.IGNORECASE,
)
expected = set()
for raw_line in open(sys.argv[1], encoding="utf-8"):
match = grant_pattern.match(raw_line.strip())
if match:
for privilege in match.group(1).split(","):
expected.add((match.group(2).lower(), privilege.strip().upper()))
actual = set()
for raw_line in open(sys.argv[2], encoding="utf-8"):
table, privilege = raw_line.rstrip("\n").split("\t")
actual.add((table.lower(), privilege.upper()))
missing = sorted(expected - actual)
unexpected = sorted(actual - expected)
if missing or unexpected:
print(f"missing privileges: {missing}", file=sys.stderr)
print(f"unexpected privileges: {unexpected}", file=sys.stderr)
raise SystemExit(1)
PY
local elevated_count
elevated_count="$(mysql_root --batch --skip-column-names information_schema <<'SQL'
SELECT
(SELECT COUNT(*) FROM USER_PRIVILEGES
WHERE GRANTEE = '''osg_smoke_runtime''@''%''' AND PRIVILEGE_TYPE <> 'USAGE')
+ (SELECT COUNT(*) FROM SCHEMA_PRIVILEGES
WHERE GRANTEE = '''osg_smoke_runtime''@''%''')
+ (SELECT COUNT(*) FROM mysql.procs_priv
WHERE User = 'osg_smoke_runtime' AND Host = '%');
SQL
)"
[[ "$elevated_count" == "0" ]] || fail "runtime user received global, schema, or routine privileges"
}
expect_runtime_denied() {
local description=$1
local statement=$2
if mysql_runtime osg_account_smoke --execute "$statement" \
>/dev/null 2>"$WORK_DIR/denied.err"; then
fail "$description unexpectedly succeeded"
fi
}
verify_immutable_history_denials() {
expect_runtime_denied \
"credit ledger UPDATE" \
"UPDATE credit_ledger SET amount_delta = amount_delta WHERE 1 = 0"
expect_runtime_denied \
"credit ledger DELETE" \
"DELETE FROM credit_ledger WHERE 1 = 0"
expect_runtime_denied \
"admin audit UPDATE" \
"UPDATE admin_audit_log SET outcome = outcome WHERE 1 = 0"
expect_runtime_denied \
"admin audit DELETE" \
"DELETE FROM admin_audit_log WHERE 1 = 0"
expect_runtime_denied \
"admin grant UPDATE" \
"UPDATE admin_credit_grants SET amount = amount WHERE 1 = 0"
expect_runtime_denied \
"admin grant DELETE" \
"DELETE FROM admin_credit_grants WHERE 1 = 0"
expect_runtime_denied \
"Flyway metadata read" \
"SELECT version FROM flyway_schema_history LIMIT 1"
}
verify_session_cleanup_permission() {
mysql_runtime osg_account_smoke \
--execute "DELETE FROM admin_sessions WHERE expires_at < UTC_TIMESTAMP() AND 1 = 0"
}
verify_ledger_pagination() {
local cursor first_id second_id encoded_cursor
http_request 200 GET "/v1/admin/users/$SMOKE_ACCOUNT_ID/ledger" "$SUPER_COOKIE_JAR" "" \
'X-OSG-mTLS-Verified: SUCCESS'
[[ "$(json_value "$RESPONSE_BODY" items | tr -d '\n')" != "" ]] || fail "ledger page was empty"
[[ "$(python3 - "$RESPONSE_BODY" <<'PY'
import json
import sys
print(len(json.load(open(sys.argv[1], encoding="utf-8"))["items"]))
PY
)" == "100" ]] || fail "first ledger page did not contain 100 entries"
first_id="$(json_value "$RESPONSE_BODY" items 0 entryId)"
cursor="$(json_value "$RESPONSE_BODY" nextCursor)"
[[ -n "$cursor" ]] || fail "first ledger page omitted nextCursor with more than 100 entries"
encoded_cursor="$(url_encode "$cursor")"
http_request 200 GET \
"/v1/admin/users/$SMOKE_ACCOUNT_ID/ledger?cursor=$encoded_cursor" \
"$SUPER_COOKIE_JAR" "" \
'X-OSG-mTLS-Verified: SUCCESS'
second_id="$(json_value "$RESPONSE_BODY" items 0 entryId)"
[[ -n "$second_id" && "$second_id" != "$first_id" ]] || fail "ledger cursor repeated the first page"
}
require_command curl
require_command docker
require_command openssl
require_command python3
APP_PORT="$(free_loopback_port)"
MYSQL_ROOT_PASSWORD="$(random_hex 24)"
MYSQL_RUNTIME_PASSWORD="$(random_hex 24)"
MYSQL_MIGRATION_PASSWORD="$(random_hex 24)"
JWT_SECRET="$(random_hex 32)"
FIELD_ENCRYPTION_KEY="$(random_base64_key)"
IDENTITY_HMAC_KEY="$(random_base64_key)"
cat >"$COMPOSE_ENV" <<EOF
SMOKE_RUN_ID=$RUN_ID
SMOKE_APP_PORT=$APP_PORT
SMOKE_MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD
SMOKE_MYSQL_MIGRATION_PASSWORD=$MYSQL_MIGRATION_PASSWORD
SMOKE_RUNTIME_ENV=$RUNTIME_ENV
SMOKE_ADMIN_ENV=$ADMIN_ENV
SMOKE_SECRET_DIR=$SECRET_DIR
SMOKE_HOST_UID=$(id -u)
SMOKE_HOST_GID=$(id -g)
EOF
cat >"$RUNTIME_ENV" <<EOF
APP_ENV=test
PORT=8080
PUBLIC_BASE_URL=http://127.0.0.1:$APP_PORT
INVITE_BASE_URL=https://osglab.com/i
APP_STORE_URL=https://apps.apple.com/us/app/smoke/id1
DATABASE_URL=jdbc:mysql://mysql:3306/osg_account_smoke?useUnicode=true&characterEncoding=utf8&connectionTimeZone=UTC&forceConnectionTimeZoneToSession=true
DATABASE_USER=osg_smoke_runtime
DATABASE_PASSWORD=$MYSQL_RUNTIME_PASSWORD
DATABASE_POOL_SIZE=4
DATABASE_MIGRATION_USER=osg_smoke_migrator
DATABASE_MIGRATION_PASSWORD=$MYSQL_MIGRATION_PASSWORD
JWT_ISSUER=http://127.0.0.1:$APP_PORT
JWT_AUDIENCE=osg-smoke
JWT_SECRET=$JWT_SECRET
ACCESS_TOKEN_MINUTES=15
REFRESH_TOKEN_DAYS=1
GATEWAY_GRANT_DAYS=1
FIELD_ENCRYPTION_KEY=$FIELD_ENCRYPTION_KEY
IDENTITY_HMAC_KEY=$IDENTITY_HMAC_KEY
IDENTITY_TOMBSTONE_RETENTION_DAYS=1
APPLE_TEAM_ID=
APPLE_KEY_ID=
APPLE_CLIENT_ID=com.osgkeyboard.smoke
APPLE_PRIVATE_KEY_PEM=
APPLE_JWKS_URL=http://127.0.0.1:9/apple/jwks
APPLE_TOKEN_URL=http://127.0.0.1:9/apple/token
APPLE_REVOKE_URL=http://127.0.0.1:9/apple/revoke
APPLE_INTEGRITY_ENVIRONMENT=development
ENFORCE_DEVICE_CHECK=false
ENFORCE_APP_ATTEST=false
APP_ATTEST_CHALLENGE_TTL_SECONDS=30
VOLCENGINE_API_KEY=
VOLCENGINE_APP_ID=
VOLCENGINE_ACCESS_TOKEN=
VOLCENGINE_RESOURCE_ID=smoke-disabled
VOLCENGINE_ASR_ENDPOINT=ws://127.0.0.1:9/volcengine
DEEPSEEK_API_KEY=
DEEPSEEK_MODEL=smoke-disabled
DEEPSEEK_ENDPOINT=http://127.0.0.1:9/deepseek
SIGNUP_TRIAL_CREDITS=100
REFERRAL_INVITER_CREDITS=100
REFERRAL_INVITEE_CREDITS=100
REFERRAL_BINDING_DAYS=1
ADMIN_SESSION_HOURS=1
ADMIN_MAXIMUM_MANUAL_GRANT=1000
EOF
chmod 600 "$COMPOSE_ENV" "$RUNTIME_ENV"
log "validating isolated Compose model"
compose --profile setup config --quiet
log "building local application image"
compose --profile setup build credential-generator
log "generating one-time administrator credentials"
compose --profile setup run --rm --no-deps credential-generator >/dev/null
mv "$ADMIN_GENERATED_ENV" "$ADMIN_ENV"
chmod 600 "$ADMIN_ENV" "$ADMIN_HANDOFF"
ADMIN_PASSWORD="$(awk -F '' '/^密码:/{print $2}' "$ADMIN_HANDOFF")"
ADMIN_TOTP_SECRET="$(awk -F '' '/^TOTP 密钥:/{print $2}' "$ADMIN_HANDOFF")"
[[ -n "$ADMIN_PASSWORD" && -n "$ADMIN_TOTP_SECRET" ]] || fail "administrator credential generation failed"
log "starting disposable MySQL 8.4"
compose up --detach --wait --wait-timeout 120 mysql
log "creating isolated migration and runtime users"
mysql_root <<SQL
CREATE USER 'osg_smoke_migrator'@'%' IDENTIFIED BY '$MYSQL_MIGRATION_PASSWORD';
CREATE USER 'osg_smoke_runtime'@'%' IDENTIFIED BY '$MYSQL_RUNTIME_PASSWORD';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, REFERENCES, TRIGGER
ON osg_account_smoke.* TO 'osg_smoke_migrator'@'%';
SQL
log "applying Flyway migrations with the dedicated migrator"
compose --profile setup up --detach schema-migrator
wait_for_service_http schema-migrator 'http://127.0.0.1:8080/health/ready' ||
fail "schema migrator did not become ready"
MIGRATIONS="$(mysql_root --batch --skip-column-names osg_account_smoke <<'SQL'
SELECT CONCAT(version, ':', success)
FROM flyway_schema_history
WHERE version IS NOT NULL
ORDER BY installed_rank;
SQL
)"
EXPECTED_MIGRATIONS=$'1:1\n2:1\n3:1\n4:1\n5:1\n6:1\n7:1\n8:1'
[[ "$MIGRATIONS" == "$EXPECTED_MIGRATIONS" ]] ||
fail "Flyway history was not exactly successful V1-V8"
compose --profile setup stop schema-migrator >/dev/null
log "installing exact runtime grants and disposable fixture"
mysql_root osg_account_smoke <"$SMOKE_DIR/runtime-grants.sql"
mysql_root osg_account_smoke <"$SMOKE_DIR/fixture.sql"
verify_privilege_matrix
log "starting application with runtime-only database access"
compose up --detach account-server
wait_for_service_http account-server 'http://127.0.0.1:8080/health/ready' ||
fail "account server did not become ready"
wait_for_host_http '/health/ready' ||
fail "account server loopback port did not become ready"
http_request 200 GET '/health/ready' - ''
assert_json_value UP status
log "verifying hidden admin edge and authenticated session"
http_request 404 GET '/v1/admin/auth/session' - ''
http_request 200 GET '/v1/admin/auth/session' - '' 'X-OSG-mTLS-Verified: SUCCESS'
assert_json_value false authenticated
ADMIN_TOTP_CODE="$(totp_code "$ADMIN_TOTP_SECRET")"
http_request 200 POST '/v1/admin/auth/login' "$SUPER_COOKIE_JAR" \
"{\"username\":\"smoke-admin\",\"password\":\"$ADMIN_PASSWORD\",\"totpCode\":\"$ADMIN_TOTP_CODE\"}" \
'X-OSG-mTLS-Verified: SUCCESS' \
"Origin: http://127.0.0.1:$APP_PORT"
assert_json_value SUPER_ADMIN role
CSRF_TOKEN="$(json_value "$RESPONSE_BODY" csrfToken)"
log "verifying CSRF, RBAC, core reads, and append-only grant path"
http_request 403 POST '/v1/admin/credits/grants' "$SUPER_COOKIE_JAR" \
"{\"userId\":\"$SMOKE_ACCOUNT_ID\",\"amount\":7,\"reason\":\"smoke grant\"}" \
'X-OSG-mTLS-Verified: SUCCESS' \
"Origin: http://127.0.0.1:$APP_PORT" \
'Idempotency-Key: smoke-grant-no-csrf'
assert_json_value CSRF_INVALID code
http_request 200 POST '/v1/admin/credits/grants' "$SUPER_COOKIE_JAR" \
"{\"userId\":\"$SMOKE_ACCOUNT_ID\",\"amount\":7,\"reason\":\"smoke grant\"}" \
'X-OSG-mTLS-Verified: SUCCESS' \
"Origin: http://127.0.0.1:$APP_PORT" \
"X-CSRF-Token: $CSRF_TOKEN" \
'Idempotency-Key: smoke-grant-success'
assert_json_value 108 balanceAfter
for path in \
'/v1/admin/overview?range=7d' \
'/v1/admin/referrals?range=7d' \
'/v1/admin/users' \
"/v1/admin/users/$SMOKE_ACCOUNT_ID" \
'/v1/admin/operators/summary' \
'/v1/admin/operators' \
'/v1/admin/audit'; do
http_request 200 GET "$path" "$SUPER_COOKIE_JAR" '' 'X-OSG-mTLS-Verified: SUCCESS'
done
http_request 201 POST '/v1/admin/operators' "$SUPER_COOKIE_JAR" \
'{"username":"smoke-analyst","password":"smoke-analyst-password","role":"ANALYST"}' \
'X-OSG-mTLS-Verified: SUCCESS' \
"Origin: http://127.0.0.1:$APP_PORT" \
"X-CSRF-Token: $CSRF_TOKEN"
ANALYST_TOTP_SECRET="$(json_value "$RESPONSE_BODY" totpSecret)"
ANALYST_TOTP_CODE="$(totp_code "$ANALYST_TOTP_SECRET")"
http_request 200 POST '/v1/admin/auth/login' "$ANALYST_COOKIE_JAR" \
"{\"username\":\"smoke-analyst\",\"password\":\"smoke-analyst-password\",\"totpCode\":\"$ANALYST_TOTP_CODE\"}" \
'X-OSG-mTLS-Verified: SUCCESS' \
"Origin: http://127.0.0.1:$APP_PORT"
http_request 403 GET '/v1/admin/users' "$ANALYST_COOKIE_JAR" '' \
'X-OSG-mTLS-Verified: SUCCESS'
assert_json_value INSUFFICIENT_PERMISSION code
if ! verify_ledger_pagination; then
FAILURES=$((FAILURES + 1))
printf '[smoke] ledger pagination verification failed\n' >&2
fi
verify_immutable_history_denials
verify_session_cleanup_permission
log "verifying logout and session revocation"
http_request 204 POST '/v1/admin/auth/logout' "$SUPER_COOKIE_JAR" '' \
'X-OSG-mTLS-Verified: SUCCESS' \
"Origin: http://127.0.0.1:$APP_PORT" \
"X-CSRF-Token: $CSRF_TOKEN"
ACTIVE_SUPER_SESSIONS="$(mysql_runtime --batch --skip-column-names osg_account_smoke <<'SQL'
SELECT COUNT(*)
FROM admin_sessions AS session
JOIN admin_operators AS operator ON operator.id = session.operator_id
WHERE operator.username = 'smoke-admin'
AND session.revoked_at IS NULL;
SQL
)"
[[ "$ACTIVE_SUPER_SESSIONS" == "0" ]] || fail "logout did not revoke the super-administrator session"
((FAILURES == 0)) || fail "$FAILURES smoke verification(s) failed"
+28
View File
@@ -0,0 +1,28 @@
# Local Docker smoke verification
Run from any directory:
```bash
/Users/rocky/Documents/OSGAccountServer/deploy/smoke-local.sh
```
Requirements: Docker with Compose v2, `curl`, `openssl`, and Python 3.
The runner builds the current checkout, creates an isolated MySQL 8.4 project,
generates one-time keys and administrator credentials, and uses an internal
Docker network. Apple, DeepSeek, and Volcengine endpoints point to the
container's closed loopback port, so no provider request can leave the project.
The checks cover:
- successful Flyway V1-V8 history and `/health/ready`;
- hidden admin routes without the trusted edge header;
- TOTP login, cookies, CSRF, role denial, core reads, grants, and logout;
- a 101-row ledger fixture that requires a second cursor page;
- an exact per-table runtime grant matrix;
- denied `UPDATE` and `DELETE` on ledger, audit, and grant history.
Containers, the named volume, the local smoke image, cookie jars, generated
keys, and administrator credentials are removed on exit. On failure, only
recent container status and logs remain in the printed temporary diagnostics
path.
+47
View File
@@ -0,0 +1,47 @@
-- A disposable account with more than one admin-ledger page.
INSERT INTO accounts (
id,
apple_sub,
created_at,
updated_at,
identity_fingerprint,
anti_abuse_restricted
) VALUES (
'10000000-0000-0000-0000-000000000001',
'smoke-only-apple-subject',
UTC_TIMESTAMP(6),
UTC_TIMESTAMP(6),
REPEAT('1', 64),
FALSE
);
INSERT INTO credit_accounts (user_id, balance, updated_at)
VALUES ('10000000-0000-0000-0000-000000000001', 101, UTC_TIMESTAMP(6));
INSERT INTO credit_ledger (
id,
user_id,
entry_type,
amount_delta,
balance_after,
idempotency_key,
reference_id,
created_at
)
WITH RECURSIVE sequence_number AS (
SELECT 1 AS value
UNION ALL
SELECT value + 1
FROM sequence_number
WHERE value < 101
)
SELECT
CONCAT('20000000-0000-0000-0000-', LPAD(value, 12, '0')),
'10000000-0000-0000-0000-000000000001',
'SIGNUP_TRIAL',
1,
value,
CONCAT('smoke-ledger-', LPAD(value, 4, '0')),
NULL,
TIMESTAMPADD(MICROSECOND, value, '2026-01-01 00:00:00.000000')
FROM sequence_number;
+54
View File
@@ -0,0 +1,54 @@
-- Smoke-only grants. The user and schema are disposable and never point at production.
GRANT SELECT ON osg_account_smoke.accounts TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.apple_credentials TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.sessions TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.apple_event_receipts TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.credit_accounts TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.credit_rate_versions TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.credit_reservations TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.referral_campaigns TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.referral_campaign_budgets TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.referral_codes TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.referral_bindings TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.credit_usage_records TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.credit_ledger TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.provider_requests TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.usage_records TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.gateway_grants TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.gateway_grant_scopes TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.gateway_refresh_tokens TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.devicecheck_trial_claims TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.app_attest_challenges TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.app_attest_keys TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.account_identity_tombstones TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.apple_revocation_outbox TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.admin_operators TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.admin_sessions TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.admin_audit_log TO 'osg_smoke_runtime'@'%';
GRANT SELECT ON osg_account_smoke.admin_credit_grants TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE, DELETE ON osg_account_smoke.accounts TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.apple_credentials TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.sessions TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.apple_event_receipts TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.credit_accounts TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.credit_reservations TO 'osg_smoke_runtime'@'%';
GRANT UPDATE ON osg_account_smoke.referral_campaign_budgets TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.referral_codes TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.referral_bindings TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.credit_usage_records TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.credit_ledger TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.provider_requests TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.usage_records TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.gateway_grants TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.gateway_grant_scopes TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.gateway_refresh_tokens TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.devicecheck_trial_claims TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.app_attest_challenges TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.app_attest_keys TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.account_identity_tombstones TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.apple_revocation_outbox TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE ON osg_account_smoke.admin_operators TO 'osg_smoke_runtime'@'%';
GRANT INSERT, UPDATE, DELETE ON osg_account_smoke.admin_sessions TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.admin_audit_log TO 'osg_smoke_runtime'@'%';
GRANT INSERT ON osg_account_smoke.admin_credit_grants TO 'osg_smoke_runtime'@'%';
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env bash
set -euo pipefail
BASE_URL="${BASE_URL:-https://account.osglab.com}"
ADMIN_CLIENT_CERT="${ADMIN_CLIENT_CERT:-}"
ADMIN_CLIENT_KEY="${ADMIN_CLIENT_KEY:-}"
if [[ -z "$ADMIN_CLIENT_CERT" || -z "$ADMIN_CLIENT_KEY" ]]; then
echo "Set ADMIN_CLIENT_CERT and ADMIN_CLIENT_KEY to local PEM paths." >&2
exit 2
fi
work_directory="$(mktemp -d)"
trap 'rm -rf "$work_directory"' EXIT
request_status() {
local name="$1"
local expected="$2"
shift 2
local status
status="$(curl --silent --show-error \
--output "$work_directory/$name.body" \
--dump-header "$work_directory/$name.headers" \
--write-out '%{http_code}' \
"$@")"
if [[ "$status" != "$expected" ]]; then
echo "$name expected HTTP $expected but received $status" >&2
exit 1
fi
}
request_status "health" "200" "$BASE_URL/health"
request_status "admin-without-certificate" "404" "$BASE_URL/admin/"
request_status "forged-edge-header" "404" \
--header "X-OSG-mTLS-Verified: SUCCESS" \
"$BASE_URL/v1/admin/auth/session"
request_status "admin-with-certificate" "200" \
--cert "$ADMIN_CLIENT_CERT" \
--key "$ADMIN_CLIENT_KEY" \
"$BASE_URL/admin/"
request_status "session-with-certificate" "200" \
--cert "$ADMIN_CLIENT_CERT" \
--key "$ADMIN_CLIENT_KEY" \
"$BASE_URL/v1/admin/auth/session"
if ! grep -Fq "OSG 运营后台" "$work_directory/admin-with-certificate.body"; then
echo "Admin HTML marker was not found." >&2
exit 1
fi
if ! grep -Eiq '^strict-transport-security: .*max-age=31536000' \
"$work_directory/admin-with-certificate.headers"; then
echo "Strict-Transport-Security header is missing." >&2
exit 1
fi
if ! grep -Eiq "^content-security-policy: .*script-src 'self'; style-src 'self'" \
"$work_directory/admin-with-certificate.headers"; then
echo "The expected strict Content-Security-Policy header is missing." >&2
exit 1
fi
if grep -Fiq "unsafe-inline" "$work_directory/admin-with-certificate.headers"; then
echo "Content-Security-Policy unexpectedly permits unsafe-inline." >&2
exit 1
fi
echo "Admin edge acceptance checks passed."