Completed field testSynthetic benchmark
Computers · Evidence record
Where Should AI Start With Intermittent Wi-Fi Drops: One Verified Gap Remained
This completed synthetic Wi-Fi field test asked the session to diagnose intermittent wi-fi drops, preserved an actual five-row wi-fi drop diagnostic record, and derived 4/10 then 8/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
diagnose intermittent wi-fi drops
02 · Scope before score
Test disclosures
Input disclosure
All inputs in DWD-9161 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “diagnose intermittent wi-fi drops”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: diagnose intermittent wi-fi drops. Focus: Wi-Fi.
Source scenario: The experiment will present connection logs and network conditions from a controlled intermittent wireless fault.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
DWD-9161-I1: Client WD-PC uses 192.0.2.41; access point AP-6 is on 5 GHz channel 44; gateway is 192.0.2.1.
DWD-9161-I2: Drops occur at 10:14, 10:29, and 10:44 exactly when AP-6 changes channel 44→149; RSSI stays -51 dBm.
DWD-9161-I3: During each drop, association resets before any DNS query; wired control resolves the same names with zero failures.
DWD-9161-I4: Test policy allows disabling automatic channel selection and pinning channel 44; baseline export NET-WD hash a81d44c0 is frozen.
DWD-9161-I5: Retest requires 12 association cycles, zero drops, RSSI at least -60 dBm, latency below 20 ms, and baseline services unchanged.
Return a concrete wi-fi drop diagnostic record with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Router logs and repeated connection tests will verify whether the proposed diagnosis identifies the seeded fault.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
WI-FI DROP DIAGNOSTIC RECORD DWD-9161 — FIRST FROZEN ARTIFACT
Task: diagnose intermittent wi-fi drops. Evaluation focus: Wi-Fi. This is a fictional, text-only artifact; it does not report a live action.
DWD-9161-R1 :: RESULT=IDENTITY=WD-PC 192.0.2.41; AP-6 channel44; gateway192.0.2.1
DWD-9161-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R2 :: RESULT=CAUSE=weak signal caused the drops
DWD-9161-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R3 :: RESULT=DNS=not primary; association resets first; wired control 0 failures
DWD-9161-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R4 :: RESULT=CHANGE=factory-reset the router
DWD-9161-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R5 :: RESULT=RETEST=one web page loads
DWD-9161-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for DWD-9161; any failed row remains visible because only one correction pass is allowed.Initial score: 4/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise DWD-9161 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Rank the correlated fault: input was “Drops occur at 10:14, 10:29, and 10:44 exactly when AP-6 changes channel 44→149; RSSI stays -51 dBm.”; first response was “CAUSE=weak signal caused the drops”.
- Propose one reversible change: input was “Test policy allows disabling automatic channel selection and pinning channel 44; baseline export NET-WD hash a81d44c0 is frozen.”; first response was “CHANGE=factory-reset the router”.
- Specify a repeatable retest: input was “Retest requires 12 association cycles, zero drops, RSSI at least -60 dBm, latency below 20 ms, and baseline services unchanged.”; first response was “RETEST=one web page loads”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
WI-FI DROP DIAGNOSTIC RECORD DWD-9161 — AFTER ONE FAILURE-ONLY CORRECTION
Task: diagnose intermittent wi-fi drops. Evaluation focus: Wi-Fi. This is a fictional, text-only artifact; it does not report a live action.
DWD-9161-R1 :: RESULT=IDENTITY=WD-PC 192.0.2.41; AP-6 channel44; gateway192.0.2.1
DWD-9161-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R2 :: RESULT=CAUSE=rank automatic channel switch above weak signal; 3/3 drops coincide; RSSI -51dBm stable
DWD-9161-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R3 :: RESULT=DNS=not primary; association resets first; wired control 0 failures
DWD-9161-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R4 :: RESULT=CHANGE=freeze NET-WD a81d44c0; pin channel44; no other router changes
DWD-9161-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
DWD-9161-R5 :: RESULT=RETEST=12 cycles; 0 drops; RSSI>=-60dBm; latency<20ms
DWD-9161-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for DWD-9161; any failed row remains visible because only one correction pass is allowed.Final score: 8/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Use the bounded client and access point | Pass | Pass | Public fixture: Client WD-PC uses 192.0.2.41; access point AP-6 is on 5 GHz channel 44; gateway is 192.0.2.1. Semantic rule: The diagnosis must stay inside the disclosed client, AP, band, channel, and gateway. FIRST returned “IDENTITY=WD-PC 192.0.2.41; AP-6 channel44; gateway192.0.2.1”; the private static semantic key accepts “IDENTITY=WD-PC 192.0.2.41; AP-6 channel44; gateway192.0.2.1”, so it passes. FINAL returned “IDENTITY=WD-PC 192.0.2.41; AP-6 channel44; gateway192.0.2.1”, so it passes. No live result was counted. |
| Rank the correlated fault | Fail | Pass | Public fixture: Drops occur at 10:14, 10:29, and 10:44 exactly when AP-6 changes channel 44→149; RSSI stays -51 dBm. Semantic rule: Stable strong signal plus perfect event correlation supports the channel-switch hypothesis. FIRST returned “CAUSE=weak signal caused the drops”; the private static semantic key accepts “CAUSE=rank automatic channel switch above weak signal; 3/3 drops coincide; RSSI -51dBm stable”, so it fails. FINAL returned “CAUSE=rank automatic channel switch above weak signal; 3/3 drops coincide; RSSI -51dBm stable”, so it passes. No live result was counted. |
| Separate DNS from link loss | Pass | Pass | Public fixture: During each drop, association resets before any DNS query; wired control resolves the same names with zero failures. Semantic rule: The event order and clean wired control do not support DNS as the initiating fault. FIRST returned “DNS=not primary; association resets first; wired control 0 failures”; the private static semantic key accepts “DNS=not primary; association resets first; wired control 0 failures”, so it passes. FINAL returned “DNS=not primary; association resets first; wired control 0 failures”, so it passes. No live result was counted. |
| Propose one reversible change | Fail | Pass | Public fixture: Test policy allows disabling automatic channel selection and pinning channel 44; baseline export NET-WD hash a81d44c0 is frozen. Semantic rule: The only approved intervention directly controls the correlated variable and retains rollback. FIRST returned “CHANGE=factory-reset the router”; the private static semantic key accepts “CHANGE=freeze NET-WD a81d44c0; pin channel44; no other router changes”, so it fails. FINAL returned “CHANGE=freeze NET-WD a81d44c0; pin channel44; no other router changes”, so it passes. No live result was counted. |
| Specify a repeatable retest | Fail | Fail | Public fixture: Retest requires 12 association cycles, zero drops, RSSI at least -60 dBm, latency below 20 ms, and baseline services unchanged. Semantic rule: The retest must cover recurrence, signal, latency, and regression boundaries. FIRST returned “RETEST=one web page loads”; the private static semantic key accepts “RETEST=12 cycles; 0 drops; RSSI>=-60dBm; latency<20ms; services unchanged”, so it fails. FINAL returned “RETEST=12 cycles; 0 drops; RSSI>=-60dBm; latency<20ms”, so it fails. No live result was counted. |
Initial4/10
Final8/10
Verdictworked
RecommendedYes, for this scope
08 · No cleanup by omission
What worked—and what failed
What worked
- DWD-9161 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Use the bounded client and access point passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Rank the correlated fault also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- Specify a repeatable retest still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
Router logs and repeated connection tests will verify whether the proposed diagnosis identifies the seeded fault.
- DWD-9161 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- DWD-9161's first and final scores were recomputed from parsed RESULT rows: 2 and 4 passes multiplied by two.
- DWD-9161 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Router logs and repeated connection tests will verify whether the proposed diagnosis identifies the seeded fault.
10 · Boundary of the claim
Limitations
- DWD-9161 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- DWD-9161 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.