{
  "category": "computers",
  "slug": "computers-verify-accessibility-tree",
  "title": "Does the Accessibility Tree Match the Visual Interface: The Correction Reached 6/10",
  "task": "verify that an interface accessibility tree matches its visual controls",
  "excerpt": "This completed synthetic Accessibility Inspection field test asked the session to verify that an interface accessibility tree matches its visual controls, preserved an actual five-row visual-to-accessibility tree audit, and derived 2/10 then 6/10 from task-specific semantic checks after one failure-only correction.",
  "tool": "Codex multi-agent session",
  "model": "Exact underlying model identifier not disclosed by the Codex session",
  "publishedAt": "2026-08-02T13:00:00+08:00",
  "durationMinutes": 0,
  "testMode": "Synthetic benchmark",
  "inputDisclosure": "All inputs in VAT-3263 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.",
  "runDisclosure": "A Codex multi-agent session generated one text-only first artifact for “verify that an interface accessibility tree matches its visual controls”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.",
  "prompt": "Complete a bounded synthetic field test for: verify that an interface accessibility tree matches its visual controls. Focus: Accessibility Inspection.\nSource scenario: The experiment will provide a local demo interface with labeled, hidden, disabled, modal, live-region, and intentionally mismatched controls.\nUse only these five public fictional inputs; the scoring answers are intentionally withheld:\nVAT-3263-I1: Visual inventory is button Save B1, link Help H1, checkbox Remember C1 checked, and disabled button Delete B2.\nVAT-3263-I2: Panel P9 has hidden=true and contains old button OLD; accessibility capture incorrectly exposes OLD.\nVAT-3263-I3: Dialog D1 is open with title Settings; background controls B1-H1-C1-B2 should be inert and focus starts at D-CLOSE.\nVAT-3263-I4: Status S1 changes Uploading→Complete; expected polite announcement is Complete once, while current tree has live off.\nVAT-3263-I5: Acceptance is visible nodes4/4, hidden nodes0, modal background focus0, live message1, and keyboard order D-CLOSE→D-SAVE.\nReturn a concrete visual-to-accessibility tree audit with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Browser accessibility-tree captures, keyboard traces, and a seeded defect key will verify roles, names, states, focus order, and hidden content.",
  "firstResult": "VISUAL-TO-ACCESSIBILITY TREE AUDIT VAT-3263 — FIRST FROZEN ARTIFACT\nTask: verify that an interface accessibility tree matches its visual controls. Evaluation focus: Accessibility Inspection. This is a fictional, text-only artifact; it does not report a live action.\nVAT-3263-R1 :: RESULT=VISIBLE=report every item as generic text\nVAT-3263-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R2 :: RESULT=HIDDEN=keep OLD focusable for backward compatibility\nVAT-3263-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R3 :: RESULT=MODAL=background remains navigable\nVAT-3263-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R4 :: RESULT=LIVE=S1 polite; announce Complete once\nVAT-3263-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R5 :: RESULT=ACCEPT=tree contains roughly the right text\nVAT-3263-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for VAT-3263; any failed row remains visible because only one correction pass is allowed.",
  "correctionPrompt": "Revise VAT-3263 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:\n- Match visible control roles and names: input was “Visual inventory is button Save B1, link Help H1, checkbox Remember C1 checked, and disabled button Delete B2.”; first response was “VISIBLE=report every item as generic text”.\n- Exclude hidden content: input was “Panel P9 has hidden=true and contains old button OLD; accessibility capture incorrectly exposes OLD.”; first response was “HIDDEN=keep OLD focusable for backward compatibility”.\n- Represent modal state: input was “Dialog D1 is open with title Settings; background controls B1-H1-C1-B2 should be inert and focus starts at D-CLOSE.”; first response was “MODAL=background remains navigable”.\n- Reconcile tree and keyboard trace: input was “Acceptance is visible nodes4/4, hidden nodes0, modal background focus0, live message1, and keyboard order D-CLOSE→D-SAVE.”; first response was “ACCEPT=tree contains roughly the right text”.\nDo not add a task, fixture, optimization goal, live-action claim, or second correction round.",
  "finalResult": "VISUAL-TO-ACCESSIBILITY TREE AUDIT VAT-3263 — AFTER ONE FAILURE-ONLY CORRECTION\nTask: verify that an interface accessibility tree matches its visual controls. Evaluation focus: Accessibility Inspection. This is a fictional, text-only artifact; it does not report a live action.\nVAT-3263-R1 :: RESULT=VISIBLE=B1 button Save; H1 link Help; C1 checkbox Remember checked; B2 button Delete disabled\nVAT-3263-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R2 :: RESULT=HIDDEN=P9 and OLD absent from tree\nVAT-3263-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R3 :: RESULT=MODAL=D1 dialog Settings; background inert\nVAT-3263-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R4 :: RESULT=LIVE=S1 polite; announce Complete once\nVAT-3263-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nVAT-3263-R5 :: RESULT=ACCEPT=visible4/4; hidden0; background focus0; live1\nVAT-3263-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for VAT-3263; any failed row remains visible because only one correction pass is allowed.",
  "checks": [
    {
      "name": "Match visible control roles and names",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Visual inventory is button Save B1, link Help H1, checkbox Remember C1 checked, and disabled button Delete B2. Semantic rule: Each visual control has a fixed semantic role, name, and state. FIRST returned “VISIBLE=report every item as generic text”; the private static semantic key accepts “VISIBLE=B1 button Save; H1 link Help; C1 checkbox Remember checked; B2 button Delete disabled”, so it fails. FINAL returned “VISIBLE=B1 button Save; H1 link Help; C1 checkbox Remember checked; B2 button Delete disabled”, so it passes. No live result was counted."
    },
    {
      "name": "Exclude hidden content",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Panel P9 has hidden=true and contains old button OLD; accessibility capture incorrectly exposes OLD. Semantic rule: Content marked hidden must not remain exposed or focusable. FIRST returned “HIDDEN=keep OLD focusable for backward compatibility”; the private static semantic key accepts “HIDDEN=P9 and OLD absent from tree”, so it fails. FINAL returned “HIDDEN=P9 and OLD absent from tree”, so it passes. No live result was counted."
    },
    {
      "name": "Represent modal state",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Dialog D1 is open with title Settings; background controls B1-H1-C1-B2 should be inert and focus starts at D-CLOSE. Semantic rule: The open modal requires dialog semantics, background suppression, and deterministic initial focus. FIRST returned “MODAL=background remains navigable”; the private static semantic key accepts “MODAL=D1 dialog Settings; background inert; focus D-CLOSE”, so it fails. FINAL returned “MODAL=D1 dialog Settings; background inert”, so it fails. No live result was counted."
    },
    {
      "name": "Verify live-region announcement",
      "firstPass": true,
      "finalPass": true,
      "evidence": "Public fixture: Status S1 changes Uploading→Complete; expected polite announcement is Complete once, while current tree has live off. Semantic rule: The fixture requires one bounded polite status announcement. FIRST returned “LIVE=S1 polite; announce Complete once”; the private static semantic key accepts “LIVE=S1 polite; announce Complete once”, so it passes. FINAL returned “LIVE=S1 polite; announce Complete once”, so it passes. No live result was counted."
    },
    {
      "name": "Reconcile tree and keyboard trace",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Acceptance is visible nodes4/4, hidden nodes0, modal background focus0, live message1, and keyboard order D-CLOSE→D-SAVE. Semantic rule: Node semantics, exclusion, modal focus, live output, and order all have exact counts. FIRST returned “ACCEPT=tree contains roughly the right text”; the private static semantic key accepts “ACCEPT=visible4/4; hidden0; background focus0; live1; order D-CLOSE>D-SAVE”, so it fails. FINAL returned “ACCEPT=visible4/4; hidden0; background focus0; live1”, so it fails. No live result was counted."
    }
  ],
  "initialScore": 2,
  "score": 6,
  "verdict": "mixed",
  "recommended": false,
  "whatWorked": [
    "VAT-3263 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.",
    "Match visible control roles and names passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.",
    "Exclude hidden content also passed its task-specific rule with the final answer left visible."
  ],
  "whatFailed": [
    "Represent modal state still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.",
    "Reconcile tree and keyboard trace still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence."
  ],
  "evidencePlan": "Browser accessibility-tree captures, keyboard traces, and a seeded defect key will verify roles, names, states, focus order, and hidden content.",
  "evidenceNotes": [
    "VAT-3263 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.",
    "VAT-3263's first and final scores were recomputed from parsed RESULT rows: 1 and 3 passes multiplied by two.",
    "VAT-3263 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Browser accessibility-tree captures, keyboard traces, and a seeded defect key will verify roles, names, states, focus order, and hidden content."
  ],
  "limitations": [
    "VAT-3263 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.",
    "VAT-3263 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result."
  ]
}