Files
OSGKeyboard/ModelTraining/ClipboardSemantics/Consensus/adjudication-instructions-v5.md
Rocky aa37067f79 chore(semantics): add v6 release gate pipeline
- Add reproducible v6 boundary, blessing, and consensus-adjudication
  corpora, plus the tiny-transformer trainer and v6 release-gate
  evaluator that gate every candidate on the deployed baselines.
- Wire consensus-label merging, product-policy anchor evaluation, and
  sealed blessing benchmark review with their pytest coverage.
- Refresh open-training corpus generation, iterative retraining runner,
  and random-holdout evaluation so v6 candidates can be benchmarked
  end-to-end.
2026-08-29 11:51:42 +08:00

3.4 KiB

Clipboard semantic evidence adjudication v5

Review only the fields listed in unresolvedFields. Judge from the text itself; do not inspect source labels, provenance, previous model votes, human anchors, or another adjudicator.

Apply labeling-instructions-v6.md. Return one JSON object per input record using the evidence schema from adjudication-instructions-v2.md:

{
  "id": "same id",
  "recordDisposition": "keep",
  "dispositionConfidence": 0.98,
  "dispositionEvidence": "play my workout playlist",
  "resolutions": {
    "assistantCommand": "true",
    "domain": "media"
  },
  "confidence": {
    "assistantCommand": 0.97,
    "domain": 0.95
  },
  "evidence": {
    "assistantCommand": "play",
    "domain": "workout playlist"
  }
}

Requirements:

  • recordDisposition remains keep or exclude-device-command for file-format compatibility. Under v6, assistant commands, information queries, and system notifications are taxonomy records and must be keep.
  • Use exclude-device-command only when replaying a queue explicitly frozen under v4/v5 exclusion policy. Do not use it in a new v6 queue.
  • dispositionConfidence and every per-field confidence must be between 0 and 1.
  • dispositionEvidence and field evidence must be short exact quotes copied from the input text.
  • resolutions, confidence, and evidence must contain exactly the fields in unresolvedFields, even if a legacy replay record is excluded.
  • Intent and flag values are true, false, or unknown.
  • domain is finance, travel, calendar, communication, media, smartHome, shopping, dining, health, weather, accountService, generalKnowledge, or unknown.
  • sentiment is positive, neutral, negative, or unknown.
  • Use unknown when the text alone does not justify a stable decision. A low-information fragment must not be forced to false or into generalKnowledge.
  • Do not output explanations, markdown, comments, or additional fields.

Critical boundary checks:

  • Resolve clearly device-, app-, or assistant-directed direct digital operations as assistantCommand=true; resolve generic lookups as informationQuery=true; resolve machine-authored alerts and status messages as systemNotification=true.
  • A request for a real-world service that naturally needs confirmation, such as booking a taxi, restaurant, hotel, or ticket, remains task=true rather than assistantCommand=true.
  • Do not infer task, question, or replyableMessage from those records unless the text independently contains an interpersonal act.
  • A named-recipient communication request with actual content is an interpersonal replyable task. A bare “call Mark” remains unknown for affected fields when the addressee is unclear.
  • Invitation questions are invitation=true, question=false; explicit self-reminders are followUpReminder=true, task=false; first-person needs implying personal action remain tasks.
  • Complaints require explicit dissatisfaction, and blessings require an explicit wish, prayer, congratulation, or conventional blessing.
  • Choose one domain from the operation target. If no primary target can be established, resolve domain as unknown.

knownLabels is corpus metadata and must not appear in adjudicator output. Downstream merging may add an adjudicated field to knownLabels only when both the field value and its evidence pass the configured acceptance gate and the value is not unknown.