Completed field testSynthetic benchmark
Computers · Evidence record
Missing Screen-Reader Labels: An AI Repair Brief: Only One Semantic Check Held
This completed synthetic Screen Readers field test asked the session to fix missing screen-reader labels in a desktop app, preserved an actual five-row screen-reader label repair matrix, and derived 0/10 then 2/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
fix missing screen-reader labels in a desktop app
02 · Scope before score
Test disclosures
Input disclosure
All inputs in FSRL-1102 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “fix missing screen-reader labels in a desktop app”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: fix missing screen-reader labels in a desktop app. Focus: Screen Readers.
Source scenario: The experiment will provide a small interface with intentionally unlabeled controls and documented interaction behavior.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
FSRL-1102-I1: Visual button B1 says Save; accessibility tree exposes role button with empty name.
FSRL-1102-I2: Email label L2 is visually adjacent to field F2, but F2 currently exposes name blank and required true.
FSRL-1102-I3: Notifications switch T3 has visible text Notifications and is on; tree reports role button with no checked state.
FSRL-1102-I4: Icon I4 is decorative beside status text Complete; it currently exposes name green checkmark and creates duplicate speech.
FSRL-1102-I5: Acceptance transcript is Email, required, edit; Notifications, on, switch; Save, button; Complete once; focus order F2→T3→B1.
Return a concrete screen-reader label repair matrix with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: An accessibility-tree inspection and recorded screen-reader task will verify names, roles, states, and navigation.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
SCREEN-READER LABEL REPAIR MATRIX FSRL-1102 — FIRST FROZEN ARTIFACT
Task: fix missing screen-reader labels in a desktop app. Evaluation focus: Screen Readers. This is a fictional, text-only artifact; it does not report a live action.
FSRL-1102-R1 :: RESULT=B1=name floppy-disk icon
FSRL-1102-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R2 :: RESULT=F2=role textbox; name blank
FSRL-1102-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R3 :: RESULT=T3=role button; name Toggle
FSRL-1102-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R4 :: RESULT=I4=announce green checkmark plus Complete
FSRL-1102-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R5 :: RESULT=ACCEPT=controls are visually correct
FSRL-1102-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for FSRL-1102; any failed row remains visible because only one correction pass is allowed.Initial score: 0/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise FSRL-1102 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Name the unlabeled Save control: input was “Visual button B1 says Save; accessibility tree exposes role button with empty name.”; first response was “B1=name floppy-disk icon”.
- Connect the text field label: input was “Email label L2 is visually adjacent to field F2, but F2 currently exposes name blank and required true.”; first response was “F2=role textbox; name blank”.
- Expose toggle state: input was “Notifications switch T3 has visible text Notifications and is on; tree reports role button with no checked state.”; first response was “T3=role button; name Toggle”.
- Hide decorative content: input was “Icon I4 is decorative beside status text Complete; it currently exposes name green checkmark and creates duplicate speech.”; first response was “I4=announce green checkmark plus Complete”.
- Verify the spoken task: input was “Acceptance transcript is Email, required, edit; Notifications, on, switch; Save, button; Complete once; focus order F2→T3→B1.”; first response was “ACCEPT=controls are visually correct”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
SCREEN-READER LABEL REPAIR MATRIX FSRL-1102 — AFTER ONE FAILURE-ONLY CORRECTION
Task: fix missing screen-reader labels in a desktop app. Evaluation focus: Screen Readers. This is a fictional, text-only artifact; it does not report a live action.
FSRL-1102-R1 :: RESULT=B1=role button; name Save
FSRL-1102-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R2 :: RESULT=F2=role textbox; name Email
FSRL-1102-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R3 :: RESULT=T3=role switch; name Notifications
FSRL-1102-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R4 :: RESULT=I4=hidden decorative
FSRL-1102-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
FSRL-1102-R5 :: RESULT=ACCEPT=F2+T3+B1 speech exact; Complete once
FSRL-1102-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for FSRL-1102; any failed row remains visible because only one correction pass is allowed.Final score: 2/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Name the unlabeled Save control | Fail | Pass | Public fixture: Visual button B1 says Save; accessibility tree exposes role button with empty name. Semantic rule: The accessible name must communicate the visible function, not icon appearance. FIRST returned “B1=name floppy-disk icon”; the private static semantic key accepts “B1=role button; name Save”, so it fails. FINAL returned “B1=role button; name Save”, so it passes. No live result was counted. |
| Connect the text field label | Fail | Fail | Public fixture: Email label L2 is visually adjacent to field F2, but F2 currently exposes name blank and required true. Semantic rule: Programmatic association must expose both the visible label and existing required state. FIRST returned “F2=role textbox; name blank”; the private static semantic key accepts “F2=role textbox; name Email; required true”, so it fails. FINAL returned “F2=role textbox; name Email”, so it fails. No live result was counted. |
| Expose toggle state | Fail | Fail | Public fixture: Notifications switch T3 has visible text Notifications and is on; tree reports role button with no checked state. Semantic rule: The semantic role, functional name, and current state are all required. FIRST returned “T3=role button; name Toggle”; the private static semantic key accepts “T3=role switch; name Notifications; checked true”, so it fails. FINAL returned “T3=role switch; name Notifications”, so it fails. No live result was counted. |
| Hide decorative content | Fail | Fail | Public fixture: Icon I4 is decorative beside status text Complete; it currently exposes name green checkmark and creates duplicate speech. Semantic rule: Decorative duplication should be removed while the textual status remains available. FIRST returned “I4=announce green checkmark plus Complete”; the private static semantic key accepts “I4=hidden decorative; status text Complete announced once”, so it fails. FINAL returned “I4=hidden decorative”, so it fails. No live result was counted. |
| Verify the spoken task | Fail | Fail | Public fixture: Acceptance transcript is Email, required, edit; Notifications, on, switch; Save, button; Complete once; focus order F2→T3→B1. Semantic rule: Names, roles, states, duplication, and navigation must match the frozen transcript. FIRST returned “ACCEPT=controls are visually correct”; the private static semantic key accepts “ACCEPT=F2+T3+B1 speech exact; Complete once; focus F2>T3>B1”, so it fails. FINAL returned “ACCEPT=F2+T3+B1 speech exact; Complete once”, so it fails. No live result was counted. |
Initial0/10
Final2/10
Verdictfailed
RecommendedNo
08 · No cleanup by omission
What worked—and what failed
What worked
- FSRL-1102 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Name the unlabeled Save control passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
What failed or remained weak
- Connect the text field label still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Expose toggle state still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Hide decorative content still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Verify the spoken task still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
An accessibility-tree inspection and recorded screen-reader task will verify names, roles, states, and navigation.
- FSRL-1102 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- FSRL-1102's first and final scores were recomputed from parsed RESULT rows: 0 and 1 passes multiplied by two.
- FSRL-1102 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: An accessibility-tree inspection and recorded screen-reader task will verify names, roles, states, and navigation.
10 · Boundary of the claim
Limitations
- FSRL-1102 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- FSRL-1102 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.