Enforce deterministic gateway task policies
Make the server authoritative for thinking, model, search, tools, retry, and output budgets while preserving legacy client behavior.
This commit is contained in:
@@ -61,6 +61,8 @@ VOLCENGINE_ASR_ENDPOINT=wss://openspeech.bytedance.com/api/v3/sauc/bigmodel
|
|||||||
|
|
||||||
DEEPSEEK_API_KEY=replace-with-deepseek-api-key
|
DEEPSEEK_API_KEY=replace-with-deepseek-api-key
|
||||||
DEEPSEEK_MODEL=deepseek-v4-flash
|
DEEPSEEK_MODEL=deepseek-v4-flash
|
||||||
|
# Optional; defaults to DEEPSEEK_MODEL when omitted.
|
||||||
|
DEEPSEEK_REASONING_MODEL=
|
||||||
DEEPSEEK_ENDPOINT=https://api.deepseek.com/v1
|
DEEPSEEK_ENDPOINT=https://api.deepseek.com/v1
|
||||||
|
|
||||||
SIGNUP_TRIAL_CREDITS=1000
|
SIGNUP_TRIAL_CREDITS=1000
|
||||||
|
|||||||
@@ -136,7 +136,7 @@ Gateway execution and settlement rules:
|
|||||||
|
|
||||||
Existing `application.yaml` values can be mapped into:
|
Existing `application.yaml` values can be mapped into:
|
||||||
|
|
||||||
- `DeepSeekConfig(endpoint, apiKey, model)`
|
- `DeepSeekConfig(endpoint, apiKey, model, reasoningModel)`
|
||||||
- `VolcengineAsrConfig(endpoint, resourceId, appId, accessToken)`
|
- `VolcengineAsrConfig(endpoint, resourceId, appId, accessToken)`
|
||||||
- `InviteWebConfig(appStoreUrl, appleAppId, universalLinkBaseUrl)`
|
- `InviteWebConfig(appStoreUrl, appleAppId, universalLinkBaseUrl)`
|
||||||
|
|
||||||
@@ -152,8 +152,16 @@ Configuration ownership:
|
|||||||
`APPLE_INTEGRITY_ENVIRONMENT`, plus the two integrity enforcement flags.
|
`APPLE_INTEGRITY_ENVIRONMENT`, plus the two integrity enforcement flags.
|
||||||
- Volcengine: prefer `VOLCENGINE_API_KEY`; set the SAUC v3 `VOLCENGINE_RESOURCE_ID` and WSS
|
- Volcengine: prefer `VOLCENGINE_API_KEY`; set the SAUC v3 `VOLCENGINE_RESOURCE_ID` and WSS
|
||||||
`VOLCENGINE_ASR_ENDPOINT`. The legacy app ID/access token pair remains optional.
|
`VOLCENGINE_ASR_ENDPOINT`. The legacy app ID/access token pair remains optional.
|
||||||
- DeepSeek: set `DEEPSEEK_API_KEY`, the provisioned `DEEPSEEK_MODEL`, and HTTPS
|
- DeepSeek: set `DEEPSEEK_API_KEY`, the provisioned low-latency `DEEPSEEK_MODEL`,
|
||||||
`DEEPSEEK_ENDPOINT`.
|
and HTTPS `DEEPSEEK_ENDPOINT`. `DEEPSEEK_REASONING_MODEL` is optional and
|
||||||
|
falls back to `DEEPSEEK_MODEL`.
|
||||||
|
|
||||||
|
Gateway text requests may include the optional stable `taskKind` values documented in
|
||||||
|
`docs/openapi.yaml`. The server maps `capability + taskKind` to a deterministic execution policy;
|
||||||
|
it never infers task type from user content. Polish and transform tasks explicitly disable DeepSeek
|
||||||
|
thinking and do not retry an empty buffered result. AI questions and agent planning explicitly use
|
||||||
|
high-effort thinking. Search and tools remain disabled for every task because no safe, billable
|
||||||
|
implementation is configured.
|
||||||
|
|
||||||
Store production values in 1Panel's secret/environment facility. The Compose environment receives
|
Store production values in 1Panel's secret/environment facility. The Compose environment receives
|
||||||
them at runtime because this application does not read Docker `/run/secrets/*` files directly.
|
them at runtime because this application does not read Docker `/run/secrets/*` files directly.
|
||||||
@@ -236,7 +244,7 @@ Internet.
|
|||||||
- Apple: `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_CLIENT_ID`, `APPLE_PRIVATE_KEY_PEM`,
|
- Apple: `APPLE_TEAM_ID`, `APPLE_KEY_ID`, `APPLE_CLIENT_ID`, `APPLE_PRIVATE_KEY_PEM`,
|
||||||
`APPLE_INTEGRITY_ENVIRONMENT`.
|
`APPLE_INTEGRITY_ENVIRONMENT`.
|
||||||
- Providers: `VOLCENGINE_API_KEY`, `VOLCENGINE_RESOURCE_ID`, `DEEPSEEK_API_KEY`,
|
- Providers: `VOLCENGINE_API_KEY`, `VOLCENGINE_RESOURCE_ID`, `DEEPSEEK_API_KEY`,
|
||||||
`DEEPSEEK_MODEL`.
|
`DEEPSEEK_MODEL`; optionally `DEEPSEEK_REASONING_MODEL`.
|
||||||
- Production controls: `APP_ENV=production`, `ENFORCE_DEVICE_CHECK=true`,
|
- Production controls: `APP_ENV=production`, `ENFORCE_DEVICE_CHECK=true`,
|
||||||
`ENFORCE_APP_ATTEST=true`.
|
`ENFORCE_APP_ATTEST=true`.
|
||||||
- Optional tuning: token lifetimes, gateway grant days, credit values, binding window and pool size;
|
- Optional tuning: token lifetimes, gateway grant days, credit values, binding window and pool size;
|
||||||
|
|||||||
@@ -54,6 +54,7 @@ services:
|
|||||||
VOLCENGINE_ASR_ENDPOINT: ${VOLCENGINE_ASR_ENDPOINT:-wss://openspeech.bytedance.com/api/v3/sauc/bigmodel}
|
VOLCENGINE_ASR_ENDPOINT: ${VOLCENGINE_ASR_ENDPOINT:-wss://openspeech.bytedance.com/api/v3/sauc/bigmodel}
|
||||||
DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY:?set DeepSeek API key}
|
DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY:?set DeepSeek API key}
|
||||||
DEEPSEEK_MODEL: ${DEEPSEEK_MODEL:-deepseek-v4-flash}
|
DEEPSEEK_MODEL: ${DEEPSEEK_MODEL:-deepseek-v4-flash}
|
||||||
|
DEEPSEEK_REASONING_MODEL: ${DEEPSEEK_REASONING_MODEL:-}
|
||||||
DEEPSEEK_ENDPOINT: ${DEEPSEEK_ENDPOINT:-https://api.deepseek.com/v1}
|
DEEPSEEK_ENDPOINT: ${DEEPSEEK_ENDPOINT:-https://api.deepseek.com/v1}
|
||||||
|
|
||||||
SIGNUP_TRIAL_CREDITS: ${SIGNUP_TRIAL_CREDITS:-1000}
|
SIGNUP_TRIAL_CREDITS: ${SIGNUP_TRIAL_CREDITS:-1000}
|
||||||
|
|||||||
+32
-1
@@ -356,6 +356,11 @@ paths:
|
|||||||
/v1/gateway/llm/{capability}:
|
/v1/gateway/llm/{capability}:
|
||||||
post:
|
post:
|
||||||
summary: Run a metered polish, AI, or agent request
|
summary: Run a metered polish, AI, or agent request
|
||||||
|
description: |
|
||||||
|
The server deterministically selects model, thinking, search, tools, retry,
|
||||||
|
and output-budget policy from `capability` plus optional `taskKind`. It
|
||||||
|
never infers task type from `input` or `context`, and clients cannot
|
||||||
|
supply provider parameters. Search and tools are currently disabled.
|
||||||
parameters:
|
parameters:
|
||||||
- $ref: "#/components/parameters/RequestId"
|
- $ref: "#/components/parameters/RequestId"
|
||||||
- name: capability
|
- name: capability
|
||||||
@@ -1374,9 +1379,35 @@ components:
|
|||||||
properties:
|
properties:
|
||||||
input: { type: string, minLength: 1, maxLength: 32000 }
|
input: { type: string, minLength: 1, maxLength: 32000 }
|
||||||
context: { type: ["string", "null"], maxLength: 32000 }
|
context: { type: ["string", "null"], maxLength: 32000 }
|
||||||
maxOutputTokens: { type: integer, minimum: 1, maximum: 4096, default: 512 }
|
maxOutputTokens:
|
||||||
|
type: integer
|
||||||
|
minimum: 1
|
||||||
|
maximum: 4096
|
||||||
|
default: 512
|
||||||
|
description: |
|
||||||
|
Requested output budget. The server clamps dictation polish and
|
||||||
|
edit-last-input to 512 tokens; translation, clipboard transform,
|
||||||
|
and custom skill to 2,048; and reasoning tasks to 4,096.
|
||||||
temperature: { type: number, minimum: 0, maximum: 1, default: 0.2 }
|
temperature: { type: number, minimum: 0, maximum: 1, default: 0.2 }
|
||||||
stream: { type: boolean, default: false }
|
stream: { type: boolean, default: false }
|
||||||
|
taskKind:
|
||||||
|
type: ["string", "null"]
|
||||||
|
enum:
|
||||||
|
- dictation_polish
|
||||||
|
- translation
|
||||||
|
- edit_last_input
|
||||||
|
- ai_question
|
||||||
|
- clipboard_transform
|
||||||
|
- custom_skill
|
||||||
|
- agent_planning
|
||||||
|
- null
|
||||||
|
description: |
|
||||||
|
Optional deterministic task selector. Allowed combinations are:
|
||||||
|
`polish` with `dictation_polish`, `translation`, or `edit_last_input`;
|
||||||
|
`ai` with `ai_question`, `clipboard_transform`, or `custom_skill`;
|
||||||
|
and `agent` with `agent_planning`. Omission defaults respectively to
|
||||||
|
`dictation_polish`, `ai_question`, and `agent_planning`. A mismatch
|
||||||
|
returns `400 invalid_request`.
|
||||||
requestSource:
|
requestSource:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
enum: [hotword, null]
|
enum: [hotword, null]
|
||||||
|
|||||||
@@ -609,6 +609,7 @@ private fun configuredProviders(config: AppConfig, client: HttpClient): List<Gat
|
|||||||
endpoint = config.providers.deepSeek.endpoint,
|
endpoint = config.providers.deepSeek.endpoint,
|
||||||
apiKey = apiKey,
|
apiKey = apiKey,
|
||||||
model = config.providers.deepSeek.model,
|
model = config.providers.deepSeek.model,
|
||||||
|
reasoningModel = config.providers.deepSeek.reasoningModel,
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -104,6 +104,10 @@ data class AppConfig(
|
|||||||
"app.storeKit.products is required when StoreKit is enabled"
|
"app.storeKit.products is required when StoreKit is enabled"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
val deepSeekModel = config.valueOrDefault(
|
||||||
|
"app.providers.deepseek.model",
|
||||||
|
"deepseek-v4-flash",
|
||||||
|
)
|
||||||
val providers = ProvidersConfig(
|
val providers = ProvidersConfig(
|
||||||
volcengine = VolcengineConfig(
|
volcengine = VolcengineConfig(
|
||||||
endpoint = config.valueOrDefault(
|
endpoint = config.valueOrDefault(
|
||||||
@@ -124,7 +128,9 @@ data class AppConfig(
|
|||||||
"https://api.deepseek.com/v1",
|
"https://api.deepseek.com/v1",
|
||||||
),
|
),
|
||||||
apiKey = config.optionalValue("app.providers.deepseek.apiKey"),
|
apiKey = config.optionalValue("app.providers.deepseek.apiKey"),
|
||||||
model = config.valueOrDefault("app.providers.deepseek.model", "deepseek-v4-flash"),
|
model = deepSeekModel,
|
||||||
|
reasoningModel = config.optionalValue("app.providers.deepseek.reasoningModel")
|
||||||
|
?: deepSeekModel,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
val integrity = IntegrityConfig(
|
val integrity = IntegrityConfig(
|
||||||
@@ -411,6 +417,7 @@ data class DeepSeekConfig(
|
|||||||
val endpoint: String,
|
val endpoint: String,
|
||||||
val apiKey: String?,
|
val apiKey: String?,
|
||||||
val model: String,
|
val model: String,
|
||||||
|
val reasoningModel: String = model,
|
||||||
) {
|
) {
|
||||||
val credentialsAvailable: Boolean
|
val credentialsAvailable: Boolean
|
||||||
get() = !apiKey.isNullOrBlank()
|
get() = !apiKey.isNullOrBlank()
|
||||||
|
|||||||
@@ -60,8 +60,84 @@ data class TextGatewayRequest(
|
|||||||
val temperature: Double = 0.2,
|
val temperature: Double = 0.2,
|
||||||
val stream: Boolean = false,
|
val stream: Boolean = false,
|
||||||
val requestSource: GatewayRequestSource? = null,
|
val requestSource: GatewayRequestSource? = null,
|
||||||
|
val taskKind: GatewayTaskKind? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
enum class GatewayTaskKind {
|
||||||
|
@SerialName("dictation_polish")
|
||||||
|
DICTATION_POLISH,
|
||||||
|
|
||||||
|
@SerialName("translation")
|
||||||
|
TRANSLATION,
|
||||||
|
|
||||||
|
@SerialName("edit_last_input")
|
||||||
|
EDIT_LAST_INPUT,
|
||||||
|
|
||||||
|
@SerialName("ai_question")
|
||||||
|
AI_QUESTION,
|
||||||
|
|
||||||
|
@SerialName("clipboard_transform")
|
||||||
|
CLIPBOARD_TRANSFORM,
|
||||||
|
|
||||||
|
@SerialName("custom_skill")
|
||||||
|
CUSTOM_SKILL,
|
||||||
|
|
||||||
|
@SerialName("agent_planning")
|
||||||
|
AGENT_PLANNING,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class GatewayThinkingMode {
|
||||||
|
DISABLED,
|
||||||
|
ENABLED,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class GatewayReasoningEffort {
|
||||||
|
LOW,
|
||||||
|
HIGH,
|
||||||
|
MAX,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class GatewayWebSearchMode {
|
||||||
|
DISABLED,
|
||||||
|
ALLOWED,
|
||||||
|
REQUIRED,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class GatewayToolsMode {
|
||||||
|
DISABLED,
|
||||||
|
ALLOWED,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class GatewayModelProfile {
|
||||||
|
LOW_LATENCY,
|
||||||
|
REASONING,
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Provider-independent policy selected exclusively from trusted server rules.
|
||||||
|
* Provider-specific request fields must be derived from this value.
|
||||||
|
*/
|
||||||
|
data class GatewayTaskExecutionPolicy(
|
||||||
|
val taskKind: GatewayTaskKind,
|
||||||
|
val modelProfile: GatewayModelProfile,
|
||||||
|
val thinking: GatewayThinkingMode,
|
||||||
|
val reasoningEffort: GatewayReasoningEffort?,
|
||||||
|
val webSearch: GatewayWebSearchMode,
|
||||||
|
val tools: GatewayToolsMode,
|
||||||
|
val allowEmptyContentRetry: Boolean,
|
||||||
|
val maxOutputTokens: Int,
|
||||||
|
) {
|
||||||
|
init {
|
||||||
|
require(maxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS)
|
||||||
|
require(
|
||||||
|
(thinking == GatewayThinkingMode.ENABLED) == (reasoningEffort != null),
|
||||||
|
) {
|
||||||
|
"reasoning effort must be explicit exactly when thinking is enabled"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@Serializable
|
@Serializable
|
||||||
data class AsrGatewayOptions(
|
data class AsrGatewayOptions(
|
||||||
val format: String = "pcm",
|
val format: String = "pcm",
|
||||||
@@ -83,6 +159,7 @@ sealed interface ProviderRequest {
|
|||||||
data class TextProviderRequest(
|
data class TextProviderRequest(
|
||||||
override val requestId: String,
|
override val requestId: String,
|
||||||
override val capability: GatewayCapability,
|
override val capability: GatewayCapability,
|
||||||
|
val executionPolicy: GatewayTaskExecutionPolicy,
|
||||||
val input: String,
|
val input: String,
|
||||||
val context: String?,
|
val context: String?,
|
||||||
val maxOutputTokens: Int,
|
val maxOutputTokens: Int,
|
||||||
|
|||||||
+74
-2
@@ -3,6 +3,11 @@ package com.osglab.account.features.gateway.providers.deepseek
|
|||||||
import com.osglab.account.features.gateway.agent.AgentPlan
|
import com.osglab.account.features.gateway.agent.AgentPlan
|
||||||
import com.osglab.account.features.gateway.models.GatewayCapability
|
import com.osglab.account.features.gateway.models.GatewayCapability
|
||||||
import com.osglab.account.features.gateway.models.GatewayLimits
|
import com.osglab.account.features.gateway.models.GatewayLimits
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayModelProfile
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayReasoningEffort
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayThinkingMode
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayToolsMode
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayWebSearchMode
|
||||||
import com.osglab.account.features.gateway.models.ProviderDescriptor
|
import com.osglab.account.features.gateway.models.ProviderDescriptor
|
||||||
import com.osglab.account.features.gateway.models.ProviderOutput
|
import com.osglab.account.features.gateway.models.ProviderOutput
|
||||||
import com.osglab.account.features.gateway.models.ProviderRequest
|
import com.osglab.account.features.gateway.models.ProviderRequest
|
||||||
@@ -40,6 +45,7 @@ data class DeepSeekConfig(
|
|||||||
val endpoint: String,
|
val endpoint: String,
|
||||||
val apiKey: String,
|
val apiKey: String,
|
||||||
val model: String,
|
val model: String,
|
||||||
|
val reasoningModel: String = model,
|
||||||
) {
|
) {
|
||||||
init {
|
init {
|
||||||
val url = runCatching { Url(endpoint) }
|
val url = runCatching { Url(endpoint) }
|
||||||
@@ -53,6 +59,15 @@ data class DeepSeekConfig(
|
|||||||
if (model.isBlank()) {
|
if (model.isBlank()) {
|
||||||
throw DeepSeekConfigurationException("DEEPSEEK_MODEL must not be blank")
|
throw DeepSeekConfigurationException("DEEPSEEK_MODEL must not be blank")
|
||||||
}
|
}
|
||||||
|
if (reasoningModel.isBlank()) {
|
||||||
|
throw DeepSeekConfigurationException("DEEPSEEK_REASONING_MODEL must not be blank")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun modelFor(profile: GatewayModelProfile): String =
|
||||||
|
when (profile) {
|
||||||
|
GatewayModelProfile.LOW_LATENCY -> model
|
||||||
|
GatewayModelProfile.REASONING -> reasoningModel
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -107,7 +122,12 @@ class DeepSeekProvider(
|
|||||||
failure.reasoningContentPresent,
|
failure.reasoningContentPresent,
|
||||||
failure.usagePresent,
|
failure.usagePresent,
|
||||||
)
|
)
|
||||||
if (request.stream || attempt >= MAX_BUFFERED_ATTEMPTS) throw failure
|
if (request.stream ||
|
||||||
|
!request.executionPolicy.allowEmptyContentRetry ||
|
||||||
|
attempt >= MAX_BUFFERED_ATTEMPTS
|
||||||
|
) {
|
||||||
|
throw failure
|
||||||
|
}
|
||||||
attempt += 1
|
attempt += 1
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -122,6 +142,15 @@ class DeepSeekProvider(
|
|||||||
require(request.maxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS) {
|
require(request.maxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS) {
|
||||||
"maxOutputTokens is out of range"
|
"maxOutputTokens is out of range"
|
||||||
}
|
}
|
||||||
|
require(request.maxOutputTokens == request.executionPolicy.maxOutputTokens) {
|
||||||
|
"maxOutputTokens must match the server execution policy"
|
||||||
|
}
|
||||||
|
require(request.executionPolicy.webSearch == GatewayWebSearchMode.DISABLED) {
|
||||||
|
"DeepSeek web search is not configured"
|
||||||
|
}
|
||||||
|
require(request.executionPolicy.tools == GatewayToolsMode.DISABLED) {
|
||||||
|
"DeepSeek tools are not configured"
|
||||||
|
}
|
||||||
require(request.temperature in 0.0..1.0 && request.temperature.isFinite()) {
|
require(request.temperature in 0.0..1.0 && request.temperature.isFinite()) {
|
||||||
"temperature is out of range"
|
"temperature is out of range"
|
||||||
}
|
}
|
||||||
@@ -153,11 +182,18 @@ class KtorDeepSeekClient(
|
|||||||
output: ProviderOutput,
|
output: ProviderOutput,
|
||||||
): ProviderUsage {
|
): ProviderUsage {
|
||||||
val payload = DeepSeekChatRequest(
|
val payload = DeepSeekChatRequest(
|
||||||
model = config.model,
|
model = config.modelFor(request.executionPolicy.modelProfile),
|
||||||
messages = controlledMessages(request),
|
messages = controlledMessages(request),
|
||||||
maxTokens = request.maxOutputTokens,
|
maxTokens = request.maxOutputTokens,
|
||||||
temperature = request.temperature,
|
temperature = request.temperature,
|
||||||
stream = request.stream,
|
stream = request.stream,
|
||||||
|
thinking = DeepSeekThinking(
|
||||||
|
type = when (request.executionPolicy.thinking) {
|
||||||
|
GatewayThinkingMode.DISABLED -> DeepSeekThinkingType.DISABLED
|
||||||
|
GatewayThinkingMode.ENABLED -> DeepSeekThinkingType.ENABLED
|
||||||
|
},
|
||||||
|
),
|
||||||
|
reasoningEffort = request.executionPolicy.reasoningEffort?.toDeepSeekReasoningEffort(),
|
||||||
streamOptions = if (request.stream) StreamOptions(includeUsage = true) else null,
|
streamOptions = if (request.stream) StreamOptions(includeUsage = true) else null,
|
||||||
responseFormat = if (request.capability == GatewayCapability.AGENT) {
|
responseFormat = if (request.capability == GatewayCapability.AGENT) {
|
||||||
ResponseFormat(type = "json_object")
|
ResponseFormat(type = "json_object")
|
||||||
@@ -484,6 +520,13 @@ class KtorDeepSeekClient(
|
|||||||
return listOf(ChatMessage("system", system), ChatMessage("user", userText))
|
return listOf(ChatMessage("system", system), ChatMessage("user", userText))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private fun GatewayReasoningEffort.toDeepSeekReasoningEffort(): DeepSeekReasoningEffort =
|
||||||
|
when (this) {
|
||||||
|
GatewayReasoningEffort.LOW -> DeepSeekReasoningEffort.LOW
|
||||||
|
GatewayReasoningEffort.HIGH -> DeepSeekReasoningEffort.HIGH
|
||||||
|
GatewayReasoningEffort.MAX -> DeepSeekReasoningEffort.MAX
|
||||||
|
}
|
||||||
|
|
||||||
private companion object {
|
private companion object {
|
||||||
const val MAX_AGENT_ID_CHARS = 128
|
const val MAX_AGENT_ID_CHARS = 128
|
||||||
const val MAX_AGENT_FIELD_CHARS = 4_096
|
const val MAX_AGENT_FIELD_CHARS = 4_096
|
||||||
@@ -504,12 +547,41 @@ private data class DeepSeekChatRequest(
|
|||||||
val maxTokens: Int,
|
val maxTokens: Int,
|
||||||
val temperature: Double,
|
val temperature: Double,
|
||||||
val stream: Boolean,
|
val stream: Boolean,
|
||||||
|
val thinking: DeepSeekThinking,
|
||||||
|
@SerialName("reasoning_effort")
|
||||||
|
val reasoningEffort: DeepSeekReasoningEffort?,
|
||||||
@SerialName("stream_options")
|
@SerialName("stream_options")
|
||||||
val streamOptions: StreamOptions?,
|
val streamOptions: StreamOptions?,
|
||||||
@SerialName("response_format")
|
@SerialName("response_format")
|
||||||
val responseFormat: ResponseFormat?,
|
val responseFormat: ResponseFormat?,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
private data class DeepSeekThinking(
|
||||||
|
val type: DeepSeekThinkingType,
|
||||||
|
)
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
private enum class DeepSeekThinkingType {
|
||||||
|
@SerialName("disabled")
|
||||||
|
DISABLED,
|
||||||
|
|
||||||
|
@SerialName("enabled")
|
||||||
|
ENABLED,
|
||||||
|
}
|
||||||
|
|
||||||
|
@Serializable
|
||||||
|
private enum class DeepSeekReasoningEffort {
|
||||||
|
@SerialName("low")
|
||||||
|
LOW,
|
||||||
|
|
||||||
|
@SerialName("high")
|
||||||
|
HIGH,
|
||||||
|
|
||||||
|
@SerialName("max")
|
||||||
|
MAX,
|
||||||
|
}
|
||||||
|
|
||||||
@Serializable
|
@Serializable
|
||||||
private data class ChatMessage(
|
private data class ChatMessage(
|
||||||
val role: String,
|
val role: String,
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ import com.osglab.account.features.gateway.services.GatewayGrantService
|
|||||||
import com.osglab.account.features.gateway.services.GatewayRefreshTokenInvalidException
|
import com.osglab.account.features.gateway.services.GatewayRefreshTokenInvalidException
|
||||||
import com.osglab.account.features.gateway.services.GatewayRefreshTokenReuseException
|
import com.osglab.account.features.gateway.services.GatewayRefreshTokenReuseException
|
||||||
import com.osglab.account.features.gateway.services.GatewayService
|
import com.osglab.account.features.gateway.services.GatewayService
|
||||||
|
import com.osglab.account.features.gateway.services.GatewayTaskPolicyResolver
|
||||||
import io.ktor.http.ContentType
|
import io.ktor.http.ContentType
|
||||||
import io.ktor.http.HttpHeaders
|
import io.ktor.http.HttpHeaders
|
||||||
import io.ktor.http.HttpStatusCode
|
import io.ktor.http.HttpStatusCode
|
||||||
@@ -67,6 +68,7 @@ fun Route.configureGatewayRoutes(
|
|||||||
gatewayIdentity: GatewayAccessTokenPort,
|
gatewayIdentity: GatewayAccessTokenPort,
|
||||||
grantService: GatewayGrantService? = null,
|
grantService: GatewayGrantService? = null,
|
||||||
asrStreaming: AsrStreamingService? = null,
|
asrStreaming: AsrStreamingService? = null,
|
||||||
|
taskPolicyResolver: GatewayTaskPolicyResolver = GatewayTaskPolicyResolver(),
|
||||||
) {
|
) {
|
||||||
route("/v1/gateway") {
|
route("/v1/gateway") {
|
||||||
if (grantService != null) {
|
if (grantService != null) {
|
||||||
@@ -240,10 +242,16 @@ fun Route.configureGatewayRoutes(
|
|||||||
"Only polish, ai and agent are supported",
|
"Only polish, ai and agent are supported",
|
||||||
requestId,
|
requestId,
|
||||||
)
|
)
|
||||||
val body = runCatching {
|
val (body, executionPolicy) = runCatching {
|
||||||
ROUTE_JSON.decodeFromString<TextGatewayRequest>(
|
val request = ROUTE_JSON.decodeFromString<TextGatewayRequest>(
|
||||||
call.receiveBounded(GatewayLimits.MAX_JSON_BODY_BYTES).decodeToString(),
|
call.receiveBounded(GatewayLimits.MAX_JSON_BODY_BYTES).decodeToString(),
|
||||||
).also { TextRequestPolicy.validate(it, capability) }
|
)
|
||||||
|
TextRequestPolicy.validate(request, capability)
|
||||||
|
request to taskPolicyResolver.resolve(
|
||||||
|
capability = capability,
|
||||||
|
requestedTaskKind = request.taskKind,
|
||||||
|
requestedMaxOutputTokens = request.maxOutputTokens,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
.getOrElse {
|
.getOrElse {
|
||||||
if (it is GatewayBodyTooLargeException || it is GatewayRequestTimeoutException) {
|
if (it is GatewayBodyTooLargeException || it is GatewayRequestTimeoutException) {
|
||||||
@@ -259,9 +267,10 @@ fun Route.configureGatewayRoutes(
|
|||||||
val providerRequest = TextProviderRequest(
|
val providerRequest = TextProviderRequest(
|
||||||
requestId = requestId,
|
requestId = requestId,
|
||||||
capability = capability,
|
capability = capability,
|
||||||
|
executionPolicy = executionPolicy,
|
||||||
input = body.input,
|
input = body.input,
|
||||||
context = body.context,
|
context = body.context,
|
||||||
maxOutputTokens = body.maxOutputTokens,
|
maxOutputTokens = executionPolicy.maxOutputTokens,
|
||||||
temperature = body.temperature,
|
temperature = body.temperature,
|
||||||
stream = body.stream,
|
stream = body.stream,
|
||||||
requestSource = body.requestSource,
|
requestSource = body.requestSource,
|
||||||
|
|||||||
+133
@@ -0,0 +1,133 @@
|
|||||||
|
package com.osglab.account.features.gateway.services
|
||||||
|
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayCapability
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayLimits
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayModelProfile
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayReasoningEffort
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayTaskExecutionPolicy
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayTaskKind
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayThinkingMode
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayToolsMode
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayWebSearchMode
|
||||||
|
|
||||||
|
data class GatewayTaskPolicyConfig(
|
||||||
|
val polishMaxOutputTokens: Int = 512,
|
||||||
|
val transformMaxOutputTokens: Int = 2_048,
|
||||||
|
val reasoningMaxOutputTokens: Int = GatewayLimits.MAX_OUTPUT_TOKENS,
|
||||||
|
val aiReasoningEffort: GatewayReasoningEffort = GatewayReasoningEffort.HIGH,
|
||||||
|
val agentReasoningEffort: GatewayReasoningEffort = GatewayReasoningEffort.HIGH,
|
||||||
|
) {
|
||||||
|
init {
|
||||||
|
require(polishMaxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS)
|
||||||
|
require(transformMaxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS)
|
||||||
|
require(reasoningMaxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Deterministic server-side task policy. It never inspects user content and
|
||||||
|
* never accepts provider parameters from the client.
|
||||||
|
*/
|
||||||
|
class GatewayTaskPolicyResolver(
|
||||||
|
private val config: GatewayTaskPolicyConfig = GatewayTaskPolicyConfig(),
|
||||||
|
) {
|
||||||
|
fun resolve(
|
||||||
|
capability: GatewayCapability,
|
||||||
|
requestedTaskKind: GatewayTaskKind?,
|
||||||
|
requestedMaxOutputTokens: Int,
|
||||||
|
): GatewayTaskExecutionPolicy {
|
||||||
|
require(requestedMaxOutputTokens in 1..GatewayLimits.MAX_OUTPUT_TOKENS) {
|
||||||
|
"maxOutputTokens is out of range"
|
||||||
|
}
|
||||||
|
val taskKind = requestedTaskKind ?: defaultTaskKind(capability)
|
||||||
|
require(taskKind in allowedTaskKinds(capability)) {
|
||||||
|
"taskKind is not supported for this capability"
|
||||||
|
}
|
||||||
|
|
||||||
|
return when (taskKind) {
|
||||||
|
GatewayTaskKind.DICTATION_POLISH,
|
||||||
|
GatewayTaskKind.EDIT_LAST_INPUT -> nonThinkingPolicy(
|
||||||
|
taskKind = taskKind,
|
||||||
|
maxOutputTokens = minOf(requestedMaxOutputTokens, config.polishMaxOutputTokens),
|
||||||
|
)
|
||||||
|
|
||||||
|
GatewayTaskKind.TRANSLATION,
|
||||||
|
GatewayTaskKind.CLIPBOARD_TRANSFORM,
|
||||||
|
GatewayTaskKind.CUSTOM_SKILL -> nonThinkingPolicy(
|
||||||
|
taskKind = taskKind,
|
||||||
|
maxOutputTokens = minOf(requestedMaxOutputTokens, config.transformMaxOutputTokens),
|
||||||
|
)
|
||||||
|
|
||||||
|
GatewayTaskKind.AI_QUESTION -> reasoningPolicy(
|
||||||
|
taskKind = taskKind,
|
||||||
|
effort = config.aiReasoningEffort,
|
||||||
|
maxOutputTokens = minOf(requestedMaxOutputTokens, config.reasoningMaxOutputTokens),
|
||||||
|
)
|
||||||
|
|
||||||
|
GatewayTaskKind.AGENT_PLANNING -> reasoningPolicy(
|
||||||
|
taskKind = taskKind,
|
||||||
|
effort = config.agentReasoningEffort,
|
||||||
|
maxOutputTokens = minOf(requestedMaxOutputTokens, config.reasoningMaxOutputTokens),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun nonThinkingPolicy(
|
||||||
|
taskKind: GatewayTaskKind,
|
||||||
|
maxOutputTokens: Int,
|
||||||
|
) = GatewayTaskExecutionPolicy(
|
||||||
|
taskKind = taskKind,
|
||||||
|
modelProfile = GatewayModelProfile.LOW_LATENCY,
|
||||||
|
thinking = GatewayThinkingMode.DISABLED,
|
||||||
|
reasoningEffort = null,
|
||||||
|
webSearch = GatewayWebSearchMode.DISABLED,
|
||||||
|
tools = GatewayToolsMode.DISABLED,
|
||||||
|
allowEmptyContentRetry = false,
|
||||||
|
maxOutputTokens = maxOutputTokens,
|
||||||
|
)
|
||||||
|
|
||||||
|
private fun reasoningPolicy(
|
||||||
|
taskKind: GatewayTaskKind,
|
||||||
|
effort: GatewayReasoningEffort,
|
||||||
|
maxOutputTokens: Int,
|
||||||
|
) = GatewayTaskExecutionPolicy(
|
||||||
|
taskKind = taskKind,
|
||||||
|
modelProfile = GatewayModelProfile.REASONING,
|
||||||
|
thinking = GatewayThinkingMode.ENABLED,
|
||||||
|
reasoningEffort = effort,
|
||||||
|
webSearch = GatewayWebSearchMode.DISABLED,
|
||||||
|
tools = GatewayToolsMode.DISABLED,
|
||||||
|
allowEmptyContentRetry = true,
|
||||||
|
maxOutputTokens = maxOutputTokens,
|
||||||
|
)
|
||||||
|
|
||||||
|
private fun defaultTaskKind(capability: GatewayCapability): GatewayTaskKind =
|
||||||
|
when (capability) {
|
||||||
|
GatewayCapability.POLISH -> GatewayTaskKind.DICTATION_POLISH
|
||||||
|
GatewayCapability.AI -> GatewayTaskKind.AI_QUESTION
|
||||||
|
GatewayCapability.AGENT -> GatewayTaskKind.AGENT_PLANNING
|
||||||
|
GatewayCapability.ASR -> throw IllegalArgumentException("ASR does not support text tasks")
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun allowedTaskKinds(capability: GatewayCapability): Set<GatewayTaskKind> =
|
||||||
|
when (capability) {
|
||||||
|
GatewayCapability.POLISH -> POLISH_TASKS
|
||||||
|
GatewayCapability.AI -> AI_TASKS
|
||||||
|
GatewayCapability.AGENT -> AGENT_TASKS
|
||||||
|
GatewayCapability.ASR -> emptySet()
|
||||||
|
}
|
||||||
|
|
||||||
|
private companion object {
|
||||||
|
val POLISH_TASKS = setOf(
|
||||||
|
GatewayTaskKind.DICTATION_POLISH,
|
||||||
|
GatewayTaskKind.TRANSLATION,
|
||||||
|
GatewayTaskKind.EDIT_LAST_INPUT,
|
||||||
|
)
|
||||||
|
val AI_TASKS = setOf(
|
||||||
|
GatewayTaskKind.AI_QUESTION,
|
||||||
|
GatewayTaskKind.CLIPBOARD_TRANSFORM,
|
||||||
|
GatewayTaskKind.CUSTOM_SKILL,
|
||||||
|
)
|
||||||
|
val AGENT_TASKS = setOf(GatewayTaskKind.AGENT_PLANNING)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -68,6 +68,7 @@ app:
|
|||||||
endpoint: "$DEEPSEEK_ENDPOINT:https://api.deepseek.com/v1"
|
endpoint: "$DEEPSEEK_ENDPOINT:https://api.deepseek.com/v1"
|
||||||
apiKey: "$DEEPSEEK_API_KEY:"
|
apiKey: "$DEEPSEEK_API_KEY:"
|
||||||
model: "$DEEPSEEK_MODEL:deepseek-v4-flash"
|
model: "$DEEPSEEK_MODEL:deepseek-v4-flash"
|
||||||
|
reasoningModel: "$DEEPSEEK_REASONING_MODEL:"
|
||||||
integrity:
|
integrity:
|
||||||
enforceDeviceCheck: "$ENFORCE_DEVICE_CHECK:false"
|
enforceDeviceCheck: "$ENFORCE_DEVICE_CHECK:false"
|
||||||
enforceAppAttest: "$ENFORCE_APP_ATTEST:false"
|
enforceAppAttest: "$ENFORCE_APP_ATTEST:false"
|
||||||
|
|||||||
@@ -2,6 +2,8 @@ package com.osglab.account.features.gateway.models
|
|||||||
|
|
||||||
import io.kotest.assertions.throwables.shouldThrow
|
import io.kotest.assertions.throwables.shouldThrow
|
||||||
import io.kotest.core.spec.style.StringSpec
|
import io.kotest.core.spec.style.StringSpec
|
||||||
|
import io.kotest.matchers.shouldBe
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
|
||||||
class TextRequestPolicyTest : StringSpec({
|
class TextRequestPolicyTest : StringSpec({
|
||||||
"rejects blank and oversized input" {
|
"rejects blank and oversized input" {
|
||||||
@@ -54,4 +56,20 @@ class TextRequestPolicyTest : StringSpec({
|
|||||||
TextRequestPolicy.validate(request, GatewayCapability.POLISH)
|
TextRequestPolicy.validate(request, GatewayCapability.POLISH)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
"keeps taskKind wire values stable" {
|
||||||
|
val values = mapOf(
|
||||||
|
"dictation_polish" to GatewayTaskKind.DICTATION_POLISH,
|
||||||
|
"translation" to GatewayTaskKind.TRANSLATION,
|
||||||
|
"edit_last_input" to GatewayTaskKind.EDIT_LAST_INPUT,
|
||||||
|
"ai_question" to GatewayTaskKind.AI_QUESTION,
|
||||||
|
"clipboard_transform" to GatewayTaskKind.CLIPBOARD_TRANSFORM,
|
||||||
|
"custom_skill" to GatewayTaskKind.CUSTOM_SKILL,
|
||||||
|
"agent_planning" to GatewayTaskKind.AGENT_PLANNING,
|
||||||
|
)
|
||||||
|
|
||||||
|
values.forEach { (wireValue, taskKind) ->
|
||||||
|
Json.decodeFromString<GatewayTaskKind>("\"$wireValue\"") shouldBe taskKind
|
||||||
|
}
|
||||||
|
}
|
||||||
})
|
})
|
||||||
|
|||||||
+86
-5
@@ -1,14 +1,17 @@
|
|||||||
package com.osglab.account.features.gateway.providers.deepseek
|
package com.osglab.account.features.gateway.providers.deepseek
|
||||||
|
|
||||||
import com.osglab.account.features.gateway.models.GatewayCapability
|
import com.osglab.account.features.gateway.models.GatewayCapability
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayTaskKind
|
||||||
import com.osglab.account.features.gateway.models.ProviderOutput
|
import com.osglab.account.features.gateway.models.ProviderOutput
|
||||||
import com.osglab.account.features.gateway.models.TextProviderRequest
|
import com.osglab.account.features.gateway.models.TextProviderRequest
|
||||||
|
import com.osglab.account.features.gateway.services.GatewayTaskPolicyResolver
|
||||||
import io.kotest.assertions.throwables.shouldThrow
|
import io.kotest.assertions.throwables.shouldThrow
|
||||||
import io.kotest.core.spec.style.StringSpec
|
import io.kotest.core.spec.style.StringSpec
|
||||||
import io.kotest.matchers.shouldBe
|
import io.kotest.matchers.shouldBe
|
||||||
import io.ktor.client.HttpClient
|
import io.ktor.client.HttpClient
|
||||||
import io.ktor.client.engine.mock.MockEngine
|
import io.ktor.client.engine.mock.MockEngine
|
||||||
import io.ktor.client.engine.mock.respond
|
import io.ktor.client.engine.mock.respond
|
||||||
|
import io.ktor.client.engine.mock.toByteArray
|
||||||
import io.ktor.client.plugins.contentnegotiation.ContentNegotiation
|
import io.ktor.client.plugins.contentnegotiation.ContentNegotiation
|
||||||
import io.ktor.http.ContentType
|
import io.ktor.http.ContentType
|
||||||
import io.ktor.http.HttpHeaders
|
import io.ktor.http.HttpHeaders
|
||||||
@@ -16,6 +19,8 @@ import io.ktor.http.HttpStatusCode
|
|||||||
import io.ktor.http.headersOf
|
import io.ktor.http.headersOf
|
||||||
import io.ktor.serialization.kotlinx.json.json
|
import io.ktor.serialization.kotlinx.json.json
|
||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
|
|
||||||
class DeepSeekClientTest : StringSpec({
|
class DeepSeekClientTest : StringSpec({
|
||||||
"prefers provider token usage" {
|
"prefers provider token usage" {
|
||||||
@@ -79,6 +84,55 @@ class DeepSeekClientTest : StringSpec({
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
"serializes disabled thinking for polish without search or tools" {
|
||||||
|
var requestBody = ""
|
||||||
|
val client = client(
|
||||||
|
"""{"choices":[{"message":{"content":"ok"}}],"usage":{"prompt_tokens":10,"completion_tokens":3,"total_tokens":13}}""",
|
||||||
|
onRequest = { requestBody = it },
|
||||||
|
)
|
||||||
|
try {
|
||||||
|
KtorDeepSeekClient(client, CONFIG).complete(
|
||||||
|
request(capability = GatewayCapability.POLISH),
|
||||||
|
DISCARD_OUTPUT,
|
||||||
|
)
|
||||||
|
|
||||||
|
val payload = Json.parseToJsonElement(requestBody).jsonObject
|
||||||
|
payload.getValue("model").jsonPrimitive.content shouldBe "configured-fast-model"
|
||||||
|
payload.getValue("thinking").jsonObject.getValue("type").jsonPrimitive.content shouldBe
|
||||||
|
"disabled"
|
||||||
|
payload["reasoning_effort"] shouldBe null
|
||||||
|
payload["web_search"] shouldBe null
|
||||||
|
payload["enable_search"] shouldBe null
|
||||||
|
payload["tools"] shouldBe null
|
||||||
|
payload["tool_choice"] shouldBe null
|
||||||
|
} finally {
|
||||||
|
client.close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
"serializes explicit high-effort thinking and the reasoning model for AI" {
|
||||||
|
var requestBody = ""
|
||||||
|
val client = client(
|
||||||
|
"""{"choices":[{"message":{"content":"ok"}}],"usage":{"prompt_tokens":10,"completion_tokens":3,"total_tokens":13}}""",
|
||||||
|
onRequest = { requestBody = it },
|
||||||
|
)
|
||||||
|
try {
|
||||||
|
KtorDeepSeekClient(client, CONFIG).complete(request(), DISCARD_OUTPUT)
|
||||||
|
|
||||||
|
val payload = Json.parseToJsonElement(requestBody).jsonObject
|
||||||
|
payload.getValue("model").jsonPrimitive.content shouldBe "configured-reasoning-model"
|
||||||
|
payload.getValue("thinking").jsonObject.getValue("type").jsonPrimitive.content shouldBe
|
||||||
|
"enabled"
|
||||||
|
payload.getValue("reasoning_effort").jsonPrimitive.content shouldBe "high"
|
||||||
|
payload["web_search"] shouldBe null
|
||||||
|
payload["enable_search"] shouldBe null
|
||||||
|
payload["tools"] shouldBe null
|
||||||
|
payload["tool_choice"] shouldBe null
|
||||||
|
} finally {
|
||||||
|
client.close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
"retries one buffered empty result and returns the successful retry" {
|
"retries one buffered empty result and returns the successful retry" {
|
||||||
var attempts = 0
|
var attempts = 0
|
||||||
val provider = DeepSeekProvider(
|
val provider = DeepSeekProvider(
|
||||||
@@ -117,6 +171,22 @@ class DeepSeekClientTest : StringSpec({
|
|||||||
attempts shouldBe 2
|
attempts shouldBe 2
|
||||||
}
|
}
|
||||||
|
|
||||||
|
"does not retry an empty buffered polish result" {
|
||||||
|
var attempts = 0
|
||||||
|
val provider = DeepSeekProvider(
|
||||||
|
DeepSeekClient { _, _ ->
|
||||||
|
attempts += 1
|
||||||
|
throw DeepSeekEmptyResultException()
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
shouldThrow<DeepSeekEmptyResultException> {
|
||||||
|
provider.execute(request(capability = GatewayCapability.POLISH), DISCARD_OUTPUT)
|
||||||
|
}
|
||||||
|
|
||||||
|
attempts shouldBe 1
|
||||||
|
}
|
||||||
|
|
||||||
"does not retry an empty streaming result after output may have started" {
|
"does not retry an empty streaming result after output may have started" {
|
||||||
var attempts = 0
|
var attempts = 0
|
||||||
val provider = DeepSeekProvider(
|
val provider = DeepSeekProvider(
|
||||||
@@ -235,8 +305,10 @@ class DeepSeekClientTest : StringSpec({
|
|||||||
private fun client(
|
private fun client(
|
||||||
responseBody: String,
|
responseBody: String,
|
||||||
contentType: ContentType = ContentType.Application.Json,
|
contentType: ContentType = ContentType.Application.Json,
|
||||||
|
onRequest: suspend (String) -> Unit = {},
|
||||||
) = HttpClient(
|
) = HttpClient(
|
||||||
MockEngine {
|
MockEngine { request ->
|
||||||
|
onRequest(request.body.toByteArray().decodeToString())
|
||||||
respond(
|
respond(
|
||||||
content = responseBody,
|
content = responseBody,
|
||||||
status = HttpStatusCode.OK,
|
status = HttpStatusCode.OK,
|
||||||
@@ -249,21 +321,30 @@ private fun client(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun request(capability: GatewayCapability = GatewayCapability.AI) = TextProviderRequest(
|
private fun request(
|
||||||
|
capability: GatewayCapability = GatewayCapability.AI,
|
||||||
|
taskKind: GatewayTaskKind? = null,
|
||||||
|
): TextProviderRequest {
|
||||||
|
val executionPolicy = TASK_POLICY.resolve(capability, taskKind, 32)
|
||||||
|
return TextProviderRequest(
|
||||||
requestId = "deepseek-request",
|
requestId = "deepseek-request",
|
||||||
capability = capability,
|
capability = capability,
|
||||||
|
executionPolicy = executionPolicy,
|
||||||
input = "hello",
|
input = "hello",
|
||||||
context = null,
|
context = null,
|
||||||
maxOutputTokens = 32,
|
maxOutputTokens = executionPolicy.maxOutputTokens,
|
||||||
temperature = 0.2,
|
temperature = 0.2,
|
||||||
stream = false,
|
stream = false,
|
||||||
)
|
)
|
||||||
|
}
|
||||||
|
|
||||||
private val CONFIG = DeepSeekConfig(
|
private val CONFIG = DeepSeekConfig(
|
||||||
endpoint = "https://api.deepseek.com/v1",
|
endpoint = "https://api.deepseek.com/v1",
|
||||||
apiKey = "test-key",
|
apiKey = "test-key",
|
||||||
model = "configured-model",
|
model = "configured-fast-model",
|
||||||
|
reasoningModel = "configured-reasoning-model",
|
||||||
)
|
)
|
||||||
|
private val TASK_POLICY = GatewayTaskPolicyResolver()
|
||||||
private val SUCCESS_USAGE = com.osglab.account.features.gateway.models.ProviderUsage(
|
private val SUCCESS_USAGE = com.osglab.account.features.gateway.models.ProviderUsage(
|
||||||
meter = com.osglab.account.features.gateway.models.UsageMeter.LLM_TOKEN,
|
meter = com.osglab.account.features.gateway.models.UsageMeter.LLM_TOKEN,
|
||||||
units = 13,
|
units = 13,
|
||||||
|
|||||||
@@ -3,10 +3,13 @@ package com.osglab.account.features.gateway.routes
|
|||||||
import com.osglab.account.features.gateway.models.GatewayCapability
|
import com.osglab.account.features.gateway.models.GatewayCapability
|
||||||
import com.osglab.account.features.gateway.models.GatewayPrincipal
|
import com.osglab.account.features.gateway.models.GatewayPrincipal
|
||||||
import com.osglab.account.features.gateway.models.GatewayRequestSource
|
import com.osglab.account.features.gateway.models.GatewayRequestSource
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayTaskKind
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayThinkingMode
|
||||||
import com.osglab.account.features.gateway.models.ProviderDescriptor
|
import com.osglab.account.features.gateway.models.ProviderDescriptor
|
||||||
import com.osglab.account.features.gateway.models.ProviderOutput
|
import com.osglab.account.features.gateway.models.ProviderOutput
|
||||||
import com.osglab.account.features.gateway.models.ProviderRequest
|
import com.osglab.account.features.gateway.models.ProviderRequest
|
||||||
import com.osglab.account.features.gateway.models.ProviderUsage
|
import com.osglab.account.features.gateway.models.ProviderUsage
|
||||||
|
import com.osglab.account.features.gateway.models.TextProviderRequest
|
||||||
import com.osglab.account.features.gateway.models.UsageMeter
|
import com.osglab.account.features.gateway.models.UsageMeter
|
||||||
import com.osglab.account.features.gateway.ports.CreditReservation
|
import com.osglab.account.features.gateway.ports.CreditReservation
|
||||||
import com.osglab.account.features.gateway.ports.CreditReservationPort
|
import com.osglab.account.features.gateway.ports.CreditReservationPort
|
||||||
@@ -17,7 +20,9 @@ import com.osglab.account.features.gateway.providers.GatewayProvider
|
|||||||
import com.osglab.account.features.gateway.providers.ProviderCatalog
|
import com.osglab.account.features.gateway.providers.ProviderCatalog
|
||||||
import com.osglab.account.features.gateway.services.GatewayService
|
import com.osglab.account.features.gateway.services.GatewayService
|
||||||
import io.kotest.core.spec.style.StringSpec
|
import io.kotest.core.spec.style.StringSpec
|
||||||
|
import io.kotest.matchers.string.shouldContain
|
||||||
import io.kotest.matchers.shouldBe
|
import io.kotest.matchers.shouldBe
|
||||||
|
import io.ktor.client.statement.bodyAsText
|
||||||
import io.ktor.client.request.header
|
import io.ktor.client.request.header
|
||||||
import io.ktor.client.request.post
|
import io.ktor.client.request.post
|
||||||
import io.ktor.client.request.setBody
|
import io.ktor.client.request.setBody
|
||||||
@@ -71,6 +76,65 @@ class GatewayRequestIdTest : StringSpec({
|
|||||||
provider.lastRequestSource shouldBe GatewayRequestSource.HOTWORD
|
provider.lastRequestSource shouldBe GatewayRequestSource.HOTWORD
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
"defaults legacy polish requests to the server-controlled low-latency policy" {
|
||||||
|
val provider = RequestIdProvider()
|
||||||
|
|
||||||
|
testApplication {
|
||||||
|
application { gatewayTestApplication(provider) }
|
||||||
|
|
||||||
|
val response = client.post("/v1/gateway/llm/polish") {
|
||||||
|
header("X-Request-ID", "legacy-polish-123")
|
||||||
|
contentType(ContentType.Application.Json)
|
||||||
|
setBody("""{"input":"hello","maxOutputTokens":4096}""")
|
||||||
|
}
|
||||||
|
|
||||||
|
response.status shouldBe HttpStatusCode.OK
|
||||||
|
provider.lastTextRequest?.executionPolicy?.taskKind shouldBe
|
||||||
|
GatewayTaskKind.DICTATION_POLISH
|
||||||
|
provider.lastTextRequest?.executionPolicy?.thinking shouldBe
|
||||||
|
GatewayThinkingMode.DISABLED
|
||||||
|
provider.lastTextRequest?.maxOutputTokens shouldBe 512
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
"rejects a capability and taskKind mismatch before billing or provider execution" {
|
||||||
|
val provider = RequestIdProvider()
|
||||||
|
|
||||||
|
testApplication {
|
||||||
|
application { gatewayTestApplication(provider) }
|
||||||
|
|
||||||
|
val response = client.post("/v1/gateway/llm/polish") {
|
||||||
|
header("X-Request-ID", "invalid-task-123")
|
||||||
|
contentType(ContentType.Application.Json)
|
||||||
|
setBody("""{"input":"hello","taskKind":"agent_planning"}""")
|
||||||
|
}
|
||||||
|
|
||||||
|
response.status shouldBe HttpStatusCode.BadRequest
|
||||||
|
response.bodyAsText() shouldContain """"code":"invalid_request""""
|
||||||
|
provider.calls shouldBe 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
"applies an explicit AI transform task without allowing thinking" {
|
||||||
|
val provider = RequestIdProvider()
|
||||||
|
|
||||||
|
testApplication {
|
||||||
|
application { gatewayTestApplication(provider) }
|
||||||
|
|
||||||
|
val response = client.post("/v1/gateway/llm/ai") {
|
||||||
|
header("X-Request-ID", "clipboard-task-123")
|
||||||
|
contentType(ContentType.Application.Json)
|
||||||
|
setBody("""{"input":"hello","taskKind":"clipboard_transform"}""")
|
||||||
|
}
|
||||||
|
|
||||||
|
response.status shouldBe HttpStatusCode.OK
|
||||||
|
provider.lastTextRequest?.executionPolicy?.taskKind shouldBe
|
||||||
|
GatewayTaskKind.CLIPBOARD_TRANSFORM
|
||||||
|
provider.lastTextRequest?.executionPolicy?.thinking shouldBe
|
||||||
|
GatewayThinkingMode.DISABLED
|
||||||
|
}
|
||||||
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
private fun io.ktor.server.application.Application.gatewayTestApplication(provider: RequestIdProvider) {
|
private fun io.ktor.server.application.Application.gatewayTestApplication(provider: RequestIdProvider) {
|
||||||
@@ -96,10 +160,15 @@ private class RequestIdProvider : GatewayProvider {
|
|||||||
var calls = 0
|
var calls = 0
|
||||||
var lastRequestId: String? = null
|
var lastRequestId: String? = null
|
||||||
var lastRequestSource: GatewayRequestSource? = null
|
var lastRequestSource: GatewayRequestSource? = null
|
||||||
|
var lastTextRequest: TextProviderRequest? = null
|
||||||
|
|
||||||
override val descriptor = ProviderDescriptor(
|
override val descriptor = ProviderDescriptor(
|
||||||
id = "request-id-provider",
|
id = "request-id-provider",
|
||||||
capabilities = setOf(GatewayCapability.AI),
|
capabilities = setOf(
|
||||||
|
GatewayCapability.POLISH,
|
||||||
|
GatewayCapability.AI,
|
||||||
|
GatewayCapability.AGENT,
|
||||||
|
),
|
||||||
streaming = true,
|
streaming = true,
|
||||||
usageMeter = UsageMeter.LLM_TOKEN,
|
usageMeter = UsageMeter.LLM_TOKEN,
|
||||||
)
|
)
|
||||||
@@ -108,6 +177,7 @@ private class RequestIdProvider : GatewayProvider {
|
|||||||
calls += 1
|
calls += 1
|
||||||
lastRequestId = request.requestId
|
lastRequestId = request.requestId
|
||||||
lastRequestSource = request.requestSource
|
lastRequestSource = request.requestSource
|
||||||
|
lastTextRequest = request as? TextProviderRequest
|
||||||
output.emit("""{"result":"ok"}""".encodeToByteArray())
|
output.emit("""{"result":"ok"}""".encodeToByteArray())
|
||||||
return ProviderUsage(
|
return ProviderUsage(
|
||||||
meter = UsageMeter.LLM_TOKEN,
|
meter = UsageMeter.LLM_TOKEN,
|
||||||
@@ -158,5 +228,9 @@ private object RequestIdUsage : GatewayUsagePort {
|
|||||||
private val REQUEST_ID_PRINCIPAL = GatewayPrincipal(
|
private val REQUEST_ID_PRINCIPAL = GatewayPrincipal(
|
||||||
userId = "00000000-0000-0000-0000-000000000001",
|
userId = "00000000-0000-0000-0000-000000000001",
|
||||||
grantId = "00000000-0000-0000-0000-000000000003",
|
grantId = "00000000-0000-0000-0000-000000000003",
|
||||||
scopes = setOf(GatewayCapability.AI),
|
scopes = setOf(
|
||||||
|
GatewayCapability.POLISH,
|
||||||
|
GatewayCapability.AI,
|
||||||
|
GatewayCapability.AGENT,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|||||||
+11
-3
@@ -114,15 +114,23 @@ private fun replayService(
|
|||||||
asrProviderTimeoutMillis = timeoutMillis,
|
asrProviderTimeoutMillis = timeoutMillis,
|
||||||
)
|
)
|
||||||
|
|
||||||
private fun replayRequest() = TextProviderRequest(
|
private fun replayRequest(): TextProviderRequest {
|
||||||
|
val executionPolicy = GatewayTaskPolicyResolver().resolve(
|
||||||
|
GatewayCapability.AI,
|
||||||
|
requestedTaskKind = null,
|
||||||
|
requestedMaxOutputTokens = 32,
|
||||||
|
)
|
||||||
|
return TextProviderRequest(
|
||||||
requestId = REPLAY_ID,
|
requestId = REPLAY_ID,
|
||||||
capability = GatewayCapability.AI,
|
capability = GatewayCapability.AI,
|
||||||
|
executionPolicy = executionPolicy,
|
||||||
input = "hello",
|
input = "hello",
|
||||||
context = null,
|
context = null,
|
||||||
maxOutputTokens = 32,
|
maxOutputTokens = executionPolicy.maxOutputTokens,
|
||||||
temperature = 0.2,
|
temperature = 0.2,
|
||||||
stream = false,
|
stream = false,
|
||||||
)
|
)
|
||||||
|
}
|
||||||
|
|
||||||
private class ReplayCredits(
|
private class ReplayCredits(
|
||||||
private val failRelease: Boolean = false,
|
private val failRelease: Boolean = false,
|
||||||
|
|||||||
+11
-3
@@ -217,16 +217,24 @@ private fun service(
|
|||||||
usageRecords = usageRecords,
|
usageRecords = usageRecords,
|
||||||
)
|
)
|
||||||
|
|
||||||
private fun request(requestSource: GatewayRequestSource? = null) = TextProviderRequest(
|
private fun request(requestSource: GatewayRequestSource? = null): TextProviderRequest {
|
||||||
|
val executionPolicy = GatewayTaskPolicyResolver().resolve(
|
||||||
|
GatewayCapability.AI,
|
||||||
|
requestedTaskKind = null,
|
||||||
|
requestedMaxOutputTokens = 32,
|
||||||
|
)
|
||||||
|
return TextProviderRequest(
|
||||||
requestId = "request-123",
|
requestId = "request-123",
|
||||||
capability = GatewayCapability.AI,
|
capability = GatewayCapability.AI,
|
||||||
|
executionPolicy = executionPolicy,
|
||||||
input = "hello",
|
input = "hello",
|
||||||
context = null,
|
context = null,
|
||||||
maxOutputTokens = 32,
|
maxOutputTokens = executionPolicy.maxOutputTokens,
|
||||||
temperature = 0.2,
|
temperature = 0.2,
|
||||||
stream = false,
|
stream = false,
|
||||||
requestSource = requestSource,
|
requestSource = requestSource,
|
||||||
)
|
)
|
||||||
|
}
|
||||||
|
|
||||||
private class FakeCredits(
|
private class FakeCredits(
|
||||||
private val failSettle: Boolean = false,
|
private val failSettle: Boolean = false,
|
||||||
|
|||||||
+92
@@ -0,0 +1,92 @@
|
|||||||
|
package com.osglab.account.features.gateway.services
|
||||||
|
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayCapability
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayModelProfile
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayReasoningEffort
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayTaskKind
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayThinkingMode
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayToolsMode
|
||||||
|
import com.osglab.account.features.gateway.models.GatewayWebSearchMode
|
||||||
|
import io.kotest.assertions.throwables.shouldThrow
|
||||||
|
import io.kotest.core.spec.style.StringSpec
|
||||||
|
import io.kotest.matchers.shouldBe
|
||||||
|
|
||||||
|
class GatewayTaskPolicyResolverTest : StringSpec({
|
||||||
|
val resolver = GatewayTaskPolicyResolver()
|
||||||
|
|
||||||
|
"uses backward-compatible task defaults" {
|
||||||
|
resolver.resolve(GatewayCapability.POLISH, null, 512).taskKind shouldBe
|
||||||
|
GatewayTaskKind.DICTATION_POLISH
|
||||||
|
resolver.resolve(GatewayCapability.AI, null, 512).taskKind shouldBe
|
||||||
|
GatewayTaskKind.AI_QUESTION
|
||||||
|
resolver.resolve(GatewayCapability.AGENT, null, 512).taskKind shouldBe
|
||||||
|
GatewayTaskKind.AGENT_PLANNING
|
||||||
|
}
|
||||||
|
|
||||||
|
"forces all low-latency transform tasks to disable costly capabilities" {
|
||||||
|
val tasks = listOf(
|
||||||
|
GatewayCapability.POLISH to GatewayTaskKind.DICTATION_POLISH,
|
||||||
|
GatewayCapability.POLISH to GatewayTaskKind.TRANSLATION,
|
||||||
|
GatewayCapability.POLISH to GatewayTaskKind.EDIT_LAST_INPUT,
|
||||||
|
GatewayCapability.AI to GatewayTaskKind.CLIPBOARD_TRANSFORM,
|
||||||
|
GatewayCapability.AI to GatewayTaskKind.CUSTOM_SKILL,
|
||||||
|
)
|
||||||
|
|
||||||
|
tasks.forEach { (capability, taskKind) ->
|
||||||
|
val policy = resolver.resolve(capability, taskKind, 512)
|
||||||
|
|
||||||
|
policy.modelProfile shouldBe GatewayModelProfile.LOW_LATENCY
|
||||||
|
policy.thinking shouldBe GatewayThinkingMode.DISABLED
|
||||||
|
policy.reasoningEffort shouldBe null
|
||||||
|
policy.webSearch shouldBe GatewayWebSearchMode.DISABLED
|
||||||
|
policy.tools shouldBe GatewayToolsMode.DISABLED
|
||||||
|
policy.allowEmptyContentRetry shouldBe false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
"enables explicit high-effort reasoning only for question and agent tasks" {
|
||||||
|
listOf(
|
||||||
|
GatewayCapability.AI to GatewayTaskKind.AI_QUESTION,
|
||||||
|
GatewayCapability.AGENT to GatewayTaskKind.AGENT_PLANNING,
|
||||||
|
).forEach { (capability, taskKind) ->
|
||||||
|
val policy = resolver.resolve(capability, taskKind, 512)
|
||||||
|
|
||||||
|
policy.modelProfile shouldBe GatewayModelProfile.REASONING
|
||||||
|
policy.thinking shouldBe GatewayThinkingMode.ENABLED
|
||||||
|
policy.reasoningEffort shouldBe GatewayReasoningEffort.HIGH
|
||||||
|
policy.webSearch shouldBe GatewayWebSearchMode.DISABLED
|
||||||
|
policy.tools shouldBe GatewayToolsMode.DISABLED
|
||||||
|
policy.allowEmptyContentRetry shouldBe true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
"rejects capability and task mismatches without inspecting content" {
|
||||||
|
shouldThrow<IllegalArgumentException> {
|
||||||
|
resolver.resolve(
|
||||||
|
GatewayCapability.POLISH,
|
||||||
|
GatewayTaskKind.AGENT_PLANNING,
|
||||||
|
512,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
shouldThrow<IllegalArgumentException> {
|
||||||
|
resolver.resolve(
|
||||||
|
GatewayCapability.AI,
|
||||||
|
GatewayTaskKind.TRANSLATION,
|
||||||
|
512,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
"applies server output budgets" {
|
||||||
|
resolver.resolve(
|
||||||
|
GatewayCapability.POLISH,
|
||||||
|
GatewayTaskKind.DICTATION_POLISH,
|
||||||
|
4_096,
|
||||||
|
).maxOutputTokens shouldBe 512
|
||||||
|
resolver.resolve(
|
||||||
|
GatewayCapability.AI,
|
||||||
|
GatewayTaskKind.CLIPBOARD_TRANSFORM,
|
||||||
|
4_096,
|
||||||
|
).maxOutputTokens shouldBe 2_048
|
||||||
|
}
|
||||||
|
})
|
||||||
Reference in New Issue
Block a user