chore(semantics): add v6 release gate pipeline

- Add reproducible v6 boundary, blessing, and consensus-adjudication corpora, plus the tiny-transformer trainer and v6 release-gate evaluator that gate every candidate on the deployed baselines. - Wire consensus-label merging, product-policy anchor evaluation, and sealed blessing benchmark review with their pytest coverage. - Refresh open-training corpus generation, iterative retraining runner, and random-holdout evaluation so v6 candidates can be benchmarked end-to-end.
2026-08-29 11:51:42 +08:00
parent b275b6b0d9
commit aa37067f79
50 changed files with 12107 additions and 197 deletions
@@ -0,0 +1,40 @@
+# Clipboard semantic evidence adjudication v1
+
+Review only the fields listed in `unresolvedFields`. Judge from the text itself;
+do not inspect previous model votes, source labels, or another adjudicator.
+
+Return one JSON object per input record:
+
+```json
+{
+  "id": "same id",
+  "resolutions": {
+    "task": "true",
+    "ambiguous": "false"
+  },
+  "confidence": {
+    "task": 0.97,
+    "ambiguous": 0.94
+  },
+  "evidence": {
+    "task": "send the report",
+    "ambiguous": "by Friday"
+  }
+}
+```
+
+Requirements:
+
+- `resolutions`, `confidence`, and `evidence` must contain exactly the fields in
+  `unresolvedFields`.
+- Intent and flag values are `true`, `false`, or `unknown`.
+- Sentiment values are `positive`, `neutral`, `negative`, or `unknown`.
+- Evidence must be a short exact quote copied from the input text.
+- Use `unknown` when the text alone does not justify a decision.
+- Confidence is per field and must be between 0 and 1.
+- Do not output explanations, markdown, or additional fields.
+
+Use the boundaries from `labeling-instructions-v2.md`. In particular, distinguish
+requests from personal plans, genuine information questions from request-shaped
+commands, direct messages from terminal notices, and expressed wishes from
+quoted, future, sarcastic, or received blessings.
@@ -0,0 +1,56 @@
+# Clipboard semantic evidence adjudication v2
+
+Review only the fields listed in `unresolvedFields`. Judge from the text itself;
+do not inspect previous model votes, source labels, or another adjudicator.
+Apply the product-approved boundaries from `labeling-instructions-v3.md`.
+
+Return one JSON object per input record:
+
+```json
+{
+  "id": "same id",
+  "recordDisposition": "keep",
+  "dispositionConfidence": 0.98,
+  "dispositionEvidence": "send the report",
+  "resolutions": {
+    "task": "true",
+    "ambiguous": "false"
+  },
+  "confidence": {
+    "task": 0.97,
+    "ambiguous": 0.94
+  },
+  "evidence": {
+    "task": "send the report",
+    "ambiguous": "by Friday"
+  }
+}
+```
+
+Requirements:
+
+- `recordDisposition` must be `keep` or `exclude-device-command`.
+- Use `exclude-device-command` only when the text is clearly addressed to a
+  device, app, search engine, or virtual assistant. Do not use it for an
+  ordinary request to another person.
+- `dispositionConfidence` must be between 0 and 1.
+- `dispositionEvidence` must be a short exact quote copied from the text.
+- `resolutions`, `confidence`, and `evidence` must contain exactly the fields in
+  `unresolvedFields`, even when the record is marked for exclusion.
+- Intent and flag values are `true`, `false`, or `unknown`.
+- Sentiment values are `positive`, `neutral`, `negative`, or `unknown`.
+- Field evidence must be a short exact quote copied from the input text.
+- Use `unknown` when the text alone does not justify a decision.
+- Confidence is per field and must be between 0 and 1.
+- Do not output explanations, markdown, or additional fields.
+
+Critical product decisions:
+
+- An invitation question is `invitation=true`, `question=false`, and normally
+  `replyableMessage=true`.
+- An explicit self-reminder is `followUpReminder=true`, `task=false`.
+- A first-person need implying action is `task=true`.
+- A problem is not a complaint unless dissatisfaction, criticism, or objection
+  is explicitly expressed.
+- Generic encouragement or happiness is not a blessing; require an explicit
+  wish, prayer, congratulation, or conventional blessing.
@@ -0,0 +1,23 @@
+# Clipboard semantic evidence adjudication v3
+
+Review only the fields listed in `unresolvedFields`. Judge from the text itself;
+do not inspect previous model votes, source labels, or another adjudicator.
+Apply `labeling-instructions-v4.md`.
+
+Return one JSON object per input record using the exact schema defined in
+`adjudication-instructions-v2.md`.
+
+For compatibility, `recordDisposition` remains `keep` or
+`exclude-device-command`. In v3, `exclude-device-command` also covers generic
+search, system/account queries, and other clearly virtual-assistant-only text.
+It does not cover real-world service requests, named-recipient communication
+with content, or private/shared-context interpersonal questions.
+
+Requirements:
+
+- Evidence must be a short exact quote copied from the input text.
+- `resolutions`, `confidence`, and `evidence` must contain exactly the fields in
+  `unresolvedFields`.
+- Use `unknown` rather than inventing context for low-information fragments.
+- Confidence is per field and must be between 0 and 1.
+- Do not output explanations, markdown, or additional fields.
@@ -0,0 +1,17 @@
+# Clipboard semantic evidence adjudication v4
+
+Review only the fields listed in `unresolvedFields`. Judge from the text itself;
+do not inspect human anchor labels, previous model votes, source labels, or
+another adjudicator.
+
+Apply `labeling-instructions-v5.md` and every inherited rule from
+`labeling-instructions-v4.md`. Return one JSON object per input record using the
+exact schema from `adjudication-instructions-v2.md`.
+
+For compatibility, `recordDisposition` remains `keep` or
+`exclude-device-command`; the excluded state also covers generic search,
+system/account queries, and clearly virtual-assistant-only commands.
+
+Evidence must be an exact text quote. Use `unknown` for every affected intent
+when a short fragment lacks enough context. Do not output explanations,
+markdown, or additional fields.
@@ -0,0 +1,79 @@
+# Clipboard semantic evidence adjudication v5
+
+Review only the fields listed in `unresolvedFields`. Judge from the text itself;
+do not inspect source labels, provenance, previous model votes, human anchors,
+or another adjudicator.
+
+Apply `labeling-instructions-v6.md`. Return one JSON object per input record
+using the evidence schema from `adjudication-instructions-v2.md`:
+
+```json
+{
+  "id": "same id",
+  "recordDisposition": "keep",
+  "dispositionConfidence": 0.98,
+  "dispositionEvidence": "play my workout playlist",
+  "resolutions": {
+    "assistantCommand": "true",
+    "domain": "media"
+  },
+  "confidence": {
+    "assistantCommand": 0.97,
+    "domain": 0.95
+  },
+  "evidence": {
+    "assistantCommand": "play",
+    "domain": "workout playlist"
+  }
+}
+```
+
+Requirements:
+
+- `recordDisposition` remains `keep` or `exclude-device-command` for file-format
+  compatibility. Under v6, assistant commands, information queries, and system
+  notifications are taxonomy records and must be `keep`.
+- Use `exclude-device-command` only when replaying a queue explicitly frozen
+  under v4/v5 exclusion policy. Do not use it in a new v6 queue.
+- `dispositionConfidence` and every per-field confidence must be between 0 and
+  1.
+- `dispositionEvidence` and field evidence must be short exact quotes copied
+  from the input text.
+- `resolutions`, `confidence`, and `evidence` must contain exactly the fields in
+  `unresolvedFields`, even if a legacy replay record is excluded.
+- Intent and flag values are `true`, `false`, or `unknown`.
+- `domain` is `finance`, `travel`, `calendar`, `communication`, `media`,
+  `smartHome`, `shopping`, `dining`, `health`, `weather`, `accountService`,
+  `generalKnowledge`, or `unknown`.
+- `sentiment` is `positive`, `neutral`, `negative`, or `unknown`.
+- Use `unknown` when the text alone does not justify a stable decision. A
+  low-information fragment must not be forced to `false` or into
+  `generalKnowledge`.
+- Do not output explanations, markdown, comments, or additional fields.
+
+Critical boundary checks:
+
+- Resolve clearly device-, app-, or assistant-directed direct digital
+  operations as `assistantCommand=true`; resolve generic lookups as
+  `informationQuery=true`; resolve machine-authored alerts and status messages
+  as `systemNotification=true`.
+- A request for a real-world service that naturally needs confirmation, such as
+  booking a taxi, restaurant, hotel, or ticket, remains `task=true` rather than
+  `assistantCommand=true`.
+- Do not infer `task`, `question`, or `replyableMessage` from those records
+  unless the text independently contains an interpersonal act.
+- A named-recipient communication request with actual content is an
+  interpersonal replyable task. A bare “call Mark” remains unknown for affected
+  fields when the addressee is unclear.
+- Invitation questions are `invitation=true`, `question=false`; explicit
+  self-reminders are `followUpReminder=true`, `task=false`; first-person needs
+  implying personal action remain tasks.
+- Complaints require explicit dissatisfaction, and blessings require an
+  explicit wish, prayer, congratulation, or conventional blessing.
+- Choose one domain from the operation target. If no primary target can be
+  established, resolve `domain` as `unknown`.
+
+`knownLabels` is corpus metadata and must not appear in adjudicator output.
+Downstream merging may add an adjudicated field to `knownLabels` only when both
+the field value and its evidence pass the configured acceptance gate and the
+value is not `unknown`.
@@ -0,0 +1,70 @@
+# Clipboard semantic consensus labeling v2
+
+Prompt version: `clipboard-consensus-v2`
+
+Label each record independently using only its text. Do not infer missing
+conversation history, and do not inspect source labels or other model outputs.
+
+Return exactly one JSON object per input record:
+
+```json
+{
+  "id": "same id",
+  "labels": {
+    "task": "false",
+    "question": "false",
+    "invitation": "false",
+    "complaint": "false",
+    "scheduleNegotiation": "false",
+    "confirmationDecision": "false",
+    "followUpReminder": "false",
+    "blessing": "false",
+    "replyableMessage": "true"
+  },
+  "sentiment": "neutral",
+  "ambiguous": false,
+  "quotedOrMeta": false,
+  "confidence": 0.96
+}
+```
+
+Every intent value must be `true`, `false`, or `unknown`. Use `unknown` when
+the text alone does not contain enough evidence. Absence of evidence is not
+automatically evidence of a negative label.
+
+## Intent boundaries
+
+- `task`: another person is explicitly requested or assigned to perform an
+  action. A personal plan is not a task.
+- `question`: a genuine request for information. Rhetorical, quoted, search,
+  and documentation examples are not questions.
+- `invitation`: an invitation to join an event, meeting, visit, meal, or social
+  activity.
+- `complaint`: present dissatisfaction, malfunction, bad service, or an
+  unresolved problem. Negative sentiment alone is insufficient.
+- `scheduleNegotiation`: proposing, changing, comparing, or choosing between
+  times. A fixed appointment or deadline alone is insufficient.
+- `confirmationDecision`: explicit approval, rejection, commitment, or
+  selection of an option. Acknowledgment alone is insufficient.
+- `followUpReminder`: a request to remind, check back, or follow up later or
+  after a trigger. An ordinary task with a deadline is insufficient.
+- `blessing`: the author directly expresses a good wish, congratulation,
+  prayer, or hope for any recipient, including self or third parties.
+- `replyableMessage`: a direct conversational message that naturally invites
+  a response. Terminal acknowledgments, personal notes, quoted examples, and
+  factual notices are negative.
+
+Multi-label combinations are valid. For example, “Could you send the report?”
+is `task + question + replyableMessage`.
+
+## Special cases
+
+- Set `quotedOrMeta = true` when intent-bearing language is quoted, reported,
+  searched, documented, requested as a writing example, or discussed rather
+  than performed.
+- Set `ambiguous = true` when material context is missing or multiple
+  interpretations remain equally plausible.
+- Sarcasm, negation, hypothetical future intent, and received thanks must be
+  interpreted semantically rather than by keyword matching.
+- `sentiment` must be `positive`, `neutral`, `negative`, or `unknown`.
+- Do not output reasoning, markdown, comments, or additional fields.
@@ -0,0 +1,46 @@
+# Clipboard semantic consensus labeling v3
+
+Prompt version: `clipboard-consensus-v3`
+
+Label each record independently using only its text. Do not infer missing
+conversation history, and do not inspect source labels or other model outputs.
+
+Return exactly one JSON object per input record using the schema from
+`labeling-instructions-v2.md`.
+
+## Product-approved boundaries
+
+These rules override the corresponding v2 boundaries:
+
+- Exclude commands that are clearly addressed to a device, app, search engine,
+  or virtual assistant rather than another person. Examples include opening an
+  inbox, playing music, changing device volume, or showing an account value.
+  Ordinary requests sent to another person remain in scope.
+- `task`: a first-person need that implies an action is a task even when the
+  recipient is not explicit. A self-reminder is not a task. A device or virtual
+  assistant command is excluded before intent labeling.
+- `question`: an invitation phrased as a question is not a `question`.
+  Request-shaped commands are also not information questions.
+- `invitation`: an invitation phrased as a question is
+  `invitation=true`, `question=false`, and normally
+  `replyableMessage=true`.
+- `complaint`: require an explicit expression of dissatisfaction, criticism, or
+  objection. A loss, theft, malfunction, or unresolved problem without
+  expressed dissatisfaction is not a complaint.
+- `followUpReminder`: an explicit self-reminder is
+  `followUpReminder=true` and `task=false`.
+- `blessing`: require an explicit wish, prayer, congratulation, or conventional
+  blessing. Generic encouragement, happiness for someone, optimism, or “good
+  luck”-free motivational language is not sufficient. Conventional expressions
+  such as “生日快乐”, “一路顺风”, “恭喜晋升”, “happy birthday”, and
+  “congratulations on the promotion” are explicit.
+
+## Unchanged requirements
+
+- Every intent value is `true`, `false`, or `unknown`.
+- `sentiment` is `positive`, `neutral`, `negative`, or `unknown`.
+- Set `quotedOrMeta=true` for quoted, reported, searched, documented, or
+  example-only intent language.
+- Set `ambiguous=true` only when missing context materially prevents a stable
+  product label.
+- Do not output reasoning, markdown, comments, or additional fields.
@@ -0,0 +1,64 @@
+# Clipboard semantic consensus labeling v4
+
+Prompt version: `clipboard-consensus-v4`
+
+Label from the text alone. Do not inspect source labels, model votes, or hidden
+conversation history. The product owner decisions below override all earlier
+versions.
+
+## Record scope
+
+Exclude text that is clearly a generic search, device control, app/account
+query, alarm/calendar operation, or other virtual-assistant-only command.
+
+Keep these in scope:
+
+- a request to perform a real-world service that naturally needs confirmation,
+  such as booking a taxi;
+- a request to communicate with a named recipient when the content to convey is
+  present;
+- a private or shared-context information question that could naturally be sent
+  to another person, such as asking for a relative's email address.
+
+If a short fragment does not contain enough evidence to distinguish a human
+message from a query or command, keep it unresolved with `ambiguous=true` and
+the affected intents set to `unknown`. Do not force it into the excluded or
+negative class.
+
+## Product intent boundaries
+
+- `replyableMessage=true` when an in-scope interpersonal message naturally
+  supports a response. Questions, assignments, ongoing decisions, emotional
+  updates, and outcome sharing can be replyable.
+- Terminal acknowledgments and thanks such as “知道了，谢谢” are not
+  replyable. A passive factual notice that creates no conversational next step
+  is also not replyable.
+- `task=true` for an assigned action, an explicit first-person commitment, or a
+  first-person need that implies a personal action. A pure status question is
+  not a task.
+- A request to email, text, or otherwise contact a named recipient is a
+  replyable task when the message content or purpose is included. A bare
+  command such as “call Mark” is ambiguous without more context.
+- `question=true` for any genuine request for information, including an
+  imperative such as “tell me her email address”.
+- A polite interrogative action request such as “Can you send the report?” is
+  both `task=true` and `question=true`. A question about when an existing task
+  will happen is `question=true`, `task=false`.
+- A request for a recommendation is excluded when it is clearly a generic
+  assistant/search query rather than an interpersonal request.
+- An invitation phrased as a question remains `invitation=true`,
+  `question=false`, and normally `replyableMessage=true`.
+- An explicit self-reminder remains `followUpReminder=true`, `task=false`.
+- A complaint still requires explicit dissatisfaction, criticism, or objection.
+- A blessing still requires an explicit wish, prayer, congratulation, or
+  conventional blessing.
+
+## Output states
+
+- Intent values are `true`, `false`, or `unknown`.
+- `sentiment` is `positive`, `neutral`, `negative`, or `unknown`.
+- Use `ambiguous=true` only when missing context materially prevents a stable
+  product label.
+- Use `quotedOrMeta=true` for quoted, reported, searched, documented, or
+  example-only intent language.
+- Do not output explanations, markdown, comments, or additional fields.
@@ -0,0 +1,39 @@
+# Clipboard semantic consensus labeling v5
+
+Prompt version: `clipboard-consensus-v5`
+
+Apply every rule in `labeling-instructions-v4.md`, with the following
+product-owner clarifications taking precedence.
+
+## Replyable message clarifications
+
+- “take your time” is an interpersonal supportive message:
+  `replyableMessage=true`, `task=false`.
+- “知道了，谢谢” is terminal and not replyable.
+- “活动规则按当前方案通过” is a passive decision notice and not replyable.
+- Sharing a personal outcome such as “事情总算处理完了，结果居然成了” is
+  replyable even without a direct question.
+- A private first-person need such as “I need to set up a new PIN” is a task
+  but not replyable unless it is addressed to another person.
+
+## Task clarifications
+
+- A first-person decision followed by an impersonal consequence is not
+  automatically an assignment. “我拍板先发布基础版，其他候选停止评估” is
+  replyable but not a task because it does not directly assign the recipient.
+- A decision that explicitly hands off a next action is a task. “我批准退款流程
+  的最终版本，可以签字” is replyable and a task.
+- Named-recipient communication with actual content is a replyable task:
+  “text Sarah that I'll be late” and “send an email to Julie that I can meet
+  Saturday” are both `replyableMessage=true`, `task=true`.
+
+## Scope and ambiguity clarifications
+
+- “tell me what's new” and “my claim status” are generic assistant/system
+  queries and must be excluded.
+- A bare fragment such as “call Mark” does not reveal whether it is an
+  interpersonal assignment or an assistant command. Keep it unresolved:
+  `ambiguous=true`, with `replyableMessage`, `task`, and `question` all
+  `unknown`.
+- Apply the same unknown treatment to other low-information fragments rather
+  than converting unspecified fields to `false`.
@@ -0,0 +1,201 @@
+# Clipboard semantic consensus labeling v6
+
+Prompt version: `clipboard-consensus-v6`
+
+Label each record independently from its text. Do not inspect source labels,
+model votes, provenance, or hidden conversation history. This version keeps the
+nine product intents from v5 and adds three routing intents plus one domain
+field. Its definitions override earlier instructions when they conflict.
+
+## Output schema
+
+Return exactly one JSON object per input record:
+
+```json
+{
+  "id": "same id",
+  "labels": {
+    "task": "false",
+    "question": "false",
+    "invitation": "false",
+    "complaint": "false",
+    "scheduleNegotiation": "false",
+    "confirmationDecision": "false",
+    "followUpReminder": "false",
+    "blessing": "false",
+    "replyableMessage": "true",
+    "assistantCommand": "false",
+    "informationQuery": "false",
+    "systemNotification": "false"
+  },
+  "domain": "communication",
+  "sentiment": "neutral",
+  "ambiguous": false,
+  "quotedOrMeta": false,
+  "confidence": 0.96
+}
+```
+
+Every intent value is `true`, `false`, or `unknown`. `domain` is one of the
+twelve values below or `unknown`. `sentiment` is `positive`, `neutral`,
+`negative`, or `unknown`. Do not output reasoning, comments, markdown, or
+additional fields.
+
+## Nine product intents
+
+- `task`: a person is assigned or asked to perform an action, or the author
+  states an explicit personal commitment or need that implies action. A
+  request for a real-world service that naturally needs confirmation, such as
+  booking a taxi, restaurant, hotel, or ticket, is also a task. A status
+  question, self-reminder, passive decision notice, and pure device/app
+  operation are not tasks.
+- `question`: a genuine interpersonal request for information. It includes
+  imperative requests such as “tell me her email address” when they could
+  naturally be sent to a person. Invitation questions and clearly generic
+  assistant/search/account queries are not questions.
+- `invitation`: an invitation to join an event, meeting, visit, meal, or social
+  activity. A question-shaped invitation is normally also
+  `replyableMessage=true`, but `question=false`.
+- `complaint`: explicit present dissatisfaction, criticism, objection, bad
+  service, or an unresolved problem framed as a complaint. A loss,
+  malfunction, negative fact, or negative sentiment without expressed
+  dissatisfaction is insufficient.
+- `scheduleNegotiation`: proposing, changing, comparing, or choosing between
+  times. A fixed appointment, reminder time, or deadline alone is insufficient.
+- `confirmationDecision`: explicit approval, rejection, commitment, or
+  selection of an option. Acknowledgment, receipt confirmation, and passive
+  status notice alone are insufficient.
+- `followUpReminder`: an explicit request to remind, check back, or follow up
+  later or after a trigger, including a self-reminder. An ordinary task with a
+  deadline is insufficient; a self-reminder is not a `task`.
+- `blessing`: the author directly expresses an explicit wish, prayer,
+  congratulation, or conventional blessing. Generic encouragement, optimism,
+  happiness for someone, quoted wishes, and requests to write a blessing are
+  insufficient.
+- `replyableMessage`: an interpersonal message that naturally supports a
+  response. Questions, assignments, invitations, ongoing decisions, emotional
+  updates, and outcome sharing may qualify. Terminal acknowledgments or thanks,
+  private notes, passive factual notices, generic assistant interactions, and
+  machine notifications do not.
+
+The v5 product-owner examples remain authoritative: “take your time” is
+replyable but not a task; “知道了，谢谢” is terminal; personal outcome sharing
+may be replyable; a private first-person need may be a task without being
+replyable; and named-recipient communication with actual content is a replyable
+task.
+
+## Three routing intents
+
+- `assistantCommand`: an instruction to a device, app, service, search engine,
+  or virtual assistant to perform a direct digital or device operation. This
+  includes opening or changing app state, alarms and calendar operations, media
+  playback, smart-home control, and immediate account/app settings. A
+  real-world service request that naturally needs confirmation remains a
+  `task`, even when submitted through an assistant.
+- `informationQuery`: a generic assistant, search, reference, weather, account,
+  or service-status lookup that asks for information rather than asking a
+  person. “tell me what's new”, “my claim status”, and generic recommendation
+  searches qualify.
+- `systemNotification`: machine- or service-generated status, alert, receipt,
+  security warning, delivery update, or other notification presented to the
+  user rather than authored as an interpersonal message.
+
+These three labels replace the old blanket exclusion of assistant-only text.
+Keep such records and label them explicitly. They are normally mutually
+exclusive, and their clearly assistant/system-scoped records must not become
+`task`, `question`, or `replyableMessage` merely because similar words could
+occur in human conversation. Real-world bookings remain tasks. A request to
+contact a named person with message content is interpersonal, not an
+`assistantCommand`; a bare fragment such as “call Mark” remains ambiguous when
+addressee and interaction mode cannot be determined.
+
+## Domains
+
+Choose the single primary subject or operation target:
+
+- `finance`: banking, payments, cards, transfers, investments, insurance, or
+  claims.
+- `travel`: transport, routes, tickets, hotels, trips, or reservations other
+  than restaurant bookings.
+- `calendar`: dates, events, meetings, availability, alarms, reminders, or
+  scheduling.
+- `communication`: calls, contacts, messages, email, social communication, or
+  interpersonal conversation.
+- `media`: music, podcasts, radio, video, photos, news playback, or media
+  discovery.
+- `smartHome`: lights, appliances, climate, locks, cameras, or other connected
+  home devices.
+- `shopping`: products, orders, retail delivery, returns, refunds, or
+  marketplace activity.
+- `dining`: restaurants, food, menus, takeaway, restaurant reservations, or
+  dining service.
+- `health`: symptoms, care, medicine, fitness, wellbeing, or medical
+  appointments.
+- `weather`: current conditions, forecasts, temperature, or weather alerts.
+- `accountService`: login, identity, profile, PIN/password, subscription,
+  membership, entitlement, or general service support not better covered above.
+- `generalKnowledge`: general facts, definitions, recommendations, and
+  non-specialized content that does not fit another domain.
+
+Use the action target to resolve a cross-domain record: “text Sam about the
+flight” is `communication`, while “is my flight delayed?” is `travel`. Use
+`unknown`, not `generalKnowledge`, when missing context prevents a stable
+choice.
+
+## Unknown, ambiguity, and metadata
+
+- Use `unknown` only when the text lacks enough evidence for that field. Do not
+  turn missing annotation or missing context into `false`.
+- Use `false` when the field is in scope and the text provides enough evidence
+  that the intent is absent.
+- Set `ambiguous=true` when missing context materially prevents a stable product
+  label. Set each affected intent and `domain` to `unknown`; unaffected fields
+  may still be resolved.
+- Set `quotedOrMeta=true` when intent-bearing language is quoted, reported,
+  searched, documented, requested as a writing example, or discussed rather
+  than performed.
+- Multi-label product combinations remain valid, such as
+  `task + question + replyableMessage` for an interpersonal “Could you send the
+  report?”
+
+## `knownLabels` contract for corpus records
+
+`knownLabels` is ingestion metadata, not part of labeler output. It lists only
+the fields a source genuinely annotates after an audited deterministic mapping.
+Allowed names are the twelve intent names, `domain`, and `sentiment`.
+
+- A field in `knownLabels` may train from its resolved value, including an
+  explicit `false`.
+- A field absent from `knownLabels` is `unknown` for training and contributes no
+  positive or negative loss.
+- Source intent names, topic names, or missing columns must never be expanded
+  into negative labels for the rest of the taxonomy.
+- A mapped source label may make only its audited target fields known.
+  Synthetic data must not claim all labels known merely because the generator
+  omitted them.
+- Consensus or human review may add a field to `knownLabels` only after that
+  field receives a non-`unknown` decision under this taxonomy.
+
+## Training-data boundary
+
+- External data may enter candidate generation only when its commercial-use
+  rights and required notices are recorded, its immutable revision is pinned,
+  and it comes from the upstream official `train` split. Upstream validation,
+  development, test, challenge, and hidden-evaluation records never train.
+- When an upstream source publishes only one split explicitly named `train`, it
+  may supply training candidates but may not supply OSGKeyboard calibration or
+  evaluation truth. If no official train designation exists, the source waits
+  in audit and is not locally re-split into eligibility.
+- Exact and normalized near-duplicate overlap with any frozen local holdout is a
+  fatal exclusion. Privacy, credentials, direct contact data, unsafe content,
+  and unsupported language variants are filtered before labeling.
+- Synthetic records are training-only, carry explicit synthetic provenance,
+  use sample weight at most `0.35`, and may know only the fields guaranteed by
+  their generation contract. They cannot enter calibration, evaluation, human
+  gold, or policy-anchor sets; cannot override a conflicting human or licensed
+  non-synthetic example; and cannot by themselves authorize a new boundary or
+  deployment threshold.
+- Dataset admission means eligibility for the audited candidate queue, not
+  automatic inclusion in commercial training. Every generated artifact still
+  requires pinned license evidence, attribution, mapping review, deduplication,
+  and acceptance gates.
@@ -0,0 +1,182 @@
+[
+  {
+    "id": "human-anchor-001",
+    "text": "下一班离开纽约市的火车几点出发",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-002",
+    "text": "告诉我露丝姑妈的电子邮件地址",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "false", "question": "true", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-003",
+    "text": "顺便说一下，这件事我会亲自处理：预约会议室，计划周五之前开始。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "true", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-004",
+    "text": "在我上班的路上有什么事故吗",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-005",
+    "text": "take your time",
+    "language": "en",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "false", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-006",
+    "text": "tell me what's new",
+    "language": "en",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-007",
+    "text": "知道了，这边谢谢。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "false", "task": "false", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-008",
+    "text": "另外，活动规则按当前方案通过。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "false", "task": "false", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-009",
+    "text": "我拍板用先发布基础版，其他候选停止评估。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "false", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-010",
+    "text": "还有一件事，我批准退款流程的最终版本，可以签字。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "true", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-011",
+    "text": "事情总算处理完了，结果居然成了。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "false", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-012",
+    "text": "好的，谢谢您的回答。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "false", "task": "false", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-013",
+    "text": "你能把报告发给我吗？",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "true", "question": "true", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-014",
+    "text": "报告什么时候发给我？",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "false", "question": "true", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-015",
+    "text": "请推荐一家附近评分 4.5 以上的餐馆。",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-016",
+    "text": "I need to set up a new PIN.",
+    "language": "en",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "false", "task": "true", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-017",
+    "text": "my claim status",
+    "language": "en",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-018",
+    "text": "send an email to Julie that I can meet her Saturday at 1 p.m.",
+    "language": "en",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "true", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-019",
+    "text": "call Mark",
+    "language": "en",
+    "expected": {"replyableMessage": "unknown", "task": "unknown", "question": "unknown", "ambiguous": "true"}
+  },
+  {
+    "id": "human-anchor-020",
+    "text": "text Sarah that I'll be late",
+    "language": "en",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "true", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-021",
+    "text": "open my inbox",
+    "language": "en",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-022",
+    "text": "show me my account balance",
+    "language": "en",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-023",
+    "text": "book me a taxi to the airport",
+    "language": "en",
+    "expected": {"recordDisposition": "keep", "replyableMessage": "true", "task": "true", "question": "false", "ambiguous": "false"}
+  },
+  {
+    "id": "human-anchor-024",
+    "text": "set an alarm for 4 a.m.",
+    "language": "en",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  },
+  {
+    "id": "human-anchor-025",
+    "text": "internet",
+    "language": "en",
+    "expected": {"replyableMessage": "unknown", "task": "unknown", "question": "unknown", "ambiguous": "true"}
+  },
+  {
+    "id": "human-anchor-026",
+    "text": "chennai 45879",
+    "language": "en",
+    "expected": {"replyableMessage": "unknown", "task": "unknown", "question": "unknown", "ambiguous": "true"}
+  },
+  {
+    "id": "human-anchor-027",
+    "text": "seat assignment",
+    "language": "en",
+    "expected": {"replyableMessage": "unknown", "task": "unknown", "question": "unknown", "ambiguous": "true"}
+  },
+  {
+    "id": "human-anchor-028",
+    "text": "maximum kilometer to delivery",
+    "language": "en",
+    "expected": {"replyableMessage": "unknown", "task": "unknown", "question": "unknown", "ambiguous": "true"}
+  },
+  {
+    "id": "human-anchor-029",
+    "text": "事件重复的整个月",
+    "language": "zh-Hans",
+    "expected": {"replyableMessage": "unknown", "task": "unknown", "question": "unknown", "ambiguous": "true"}
+  },
+  {
+    "id": "human-anchor-030",
+    "text": "意大利面的食谱",
+    "language": "zh-Hans",
+    "expected": {"recordDisposition": "exclude-device-command"}
+  }
+]