Completed field testSynthetic benchmark
Computers · Evidence record
Planning a Clean OS Reinstall: All Five Semantic Checks Passed
This completed synthetic OS Installation field test asked the session to plan a clean operating system reinstall, preserved an actual five-row clean os reinstall runbook, and derived 6/10 then 10/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
plan a clean operating system reinstall
02 · Scope before score
Test disclosures
Input disclosure
All inputs in PCOR-8049 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “plan a clean operating system reinstall”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: plan a clean operating system reinstall. Focus: OS Installation.
Source scenario: The experiment will ask AI to prepare a reinstall sequence that preserves user files, settings, and required drivers.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
PCOR-8049-I1: Backup BK-OS-17 contains 286 files, 42 settings exports, SHA-256 6f3a9c12, and a successful sample restore.
PCOR-8049-I2: Disk 0 has EFI 512 MB, OS 180 GB, Recovery 2 GB, and 42 GB free inside the OS partition.
PCOR-8049-I3: Current build is 24H2-26100; approved installer is signed build 24H2-26120; no later build is in the fixture.
PCOR-8049-I4: Network package NET-6.1.18 and storage package STOR-4.8.2 have valid signatures and are required before first sign-in.
PCOR-8049-I5: Acceptance requires two clean boots, 286/286 file hashes, 42/42 settings imports, network link, and no unknown devices.
Return a concrete clean os reinstall runbook with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: A disposable test computer and a post-install checklist will verify data preservation, driver availability, and boot success.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
CLEAN OS REINSTALL RUNBOOK PCOR-8049 — FIRST FROZEN ARTIFACT
Task: plan a clean operating system reinstall. Evaluation focus: OS Installation. This is a fictional, text-only artifact; it does not report a live action.
PCOR-8049-R1 :: RESULT=BACKUP=freeze BK-OS-17; require hash 6f3a9c12; sample restore already passed
PCOR-8049-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R2 :: RESULT=PARTITIONS=retain EFI 512MB; retain Recovery 2GB; reinstall only inside OS 180GB
PCOR-8049-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R3 :: RESULT=BUILD=install signed 24H2-26120; do not infer a newer release
PCOR-8049-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R4 :: RESULT=DRIVERS=depend on the network after reinstall
PCOR-8049-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R5 :: RESULT=ACCEPT=desktop appears once
PCOR-8049-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for PCOR-8049; any failed row remains visible because only one correction pass is allowed.Initial score: 6/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise PCOR-8049 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Stage the required offline drivers: input was “Network package NET-6.1.18 and storage package STOR-4.8.2 have valid signatures and are required before first sign-in.”; first response was “DRIVERS=depend on the network after reinstall”.
- Define post-install acceptance: input was “Acceptance requires two clean boots, 286/286 file hashes, 42/42 settings imports, network link, and no unknown devices.”; first response was “ACCEPT=desktop appears once”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
CLEAN OS REINSTALL RUNBOOK PCOR-8049 — AFTER ONE FAILURE-ONLY CORRECTION
Task: plan a clean operating system reinstall. Evaluation focus: OS Installation. This is a fictional, text-only artifact; it does not report a live action.
PCOR-8049-R1 :: RESULT=BACKUP=freeze BK-OS-17; require hash 6f3a9c12; sample restore already passed
PCOR-8049-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R2 :: RESULT=PARTITIONS=retain EFI 512MB; retain Recovery 2GB; reinstall only inside OS 180GB
PCOR-8049-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R3 :: RESULT=BUILD=install signed 24H2-26120; do not infer a newer release
PCOR-8049-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R4 :: RESULT=DRIVERS=stage NET-6.1.18 and STOR-4.8.2; verify both signatures
PCOR-8049-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PCOR-8049-R5 :: RESULT=ACCEPT=2 boots; 286/286 hashes; 42/42 settings; network linked; 0 unknown devices
PCOR-8049-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for PCOR-8049; any failed row remains visible because only one correction pass is allowed.Final score: 10/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Freeze the verified user-data backup | Pass | Pass | Public fixture: Backup BK-OS-17 contains 286 files, 42 settings exports, SHA-256 6f3a9c12, and a successful sample restore. Semantic rule: A clean reinstall may start only after the named backup hash and restore sample both pass. FIRST returned “BACKUP=freeze BK-OS-17; require hash 6f3a9c12; sample restore already passed”; the private static semantic key accepts “BACKUP=freeze BK-OS-17; require hash 6f3a9c12; sample restore already passed”, so it passes. FINAL returned “BACKUP=freeze BK-OS-17; require hash 6f3a9c12; sample restore already passed”, so it passes. No live result was counted. |
| Preserve the recovery partitions | Pass | Pass | Public fixture: Disk 0 has EFI 512 MB, OS 180 GB, Recovery 2 GB, and 42 GB free inside the OS partition. Semantic rule: Only the OS partition is in scope; EFI and Recovery are fixed recovery assets. FIRST returned “PARTITIONS=retain EFI 512MB; retain Recovery 2GB; reinstall only inside OS 180GB”; the private static semantic key accepts “PARTITIONS=retain EFI 512MB; retain Recovery 2GB; reinstall only inside OS 180GB”, so it passes. FINAL returned “PARTITIONS=retain EFI 512MB; retain Recovery 2GB; reinstall only inside OS 180GB”, so it passes. No live result was counted. |
| Pin the approved operating-system build | Pass | Pass | Public fixture: Current build is 24H2-26100; approved installer is signed build 24H2-26120; no later build is in the fixture. Semantic rule: The target must equal the disclosed signed build, not an unobserved latest version. FIRST returned “BUILD=install signed 24H2-26120; do not infer a newer release”; the private static semantic key accepts “BUILD=install signed 24H2-26120; do not infer a newer release”, so it passes. FINAL returned “BUILD=install signed 24H2-26120; do not infer a newer release”, so it passes. No live result was counted. |
| Stage the required offline drivers | Fail | Pass | Public fixture: Network package NET-6.1.18 and storage package STOR-4.8.2 have valid signatures and are required before first sign-in. Semantic rule: Both exact packages must be available offline and signature-checked before reinstall. FIRST returned “DRIVERS=depend on the network after reinstall”; the private static semantic key accepts “DRIVERS=stage NET-6.1.18 and STOR-4.8.2; verify both signatures”, so it fails. FINAL returned “DRIVERS=stage NET-6.1.18 and STOR-4.8.2; verify both signatures”, so it passes. No live result was counted. |
| Define post-install acceptance | Fail | Pass | Public fixture: Acceptance requires two clean boots, 286/286 file hashes, 42/42 settings imports, network link, and no unknown devices. Semantic rule: The reinstall passes only when every declared boot, data, setting, network, and device check passes. FIRST returned “ACCEPT=desktop appears once”; the private static semantic key accepts “ACCEPT=2 boots; 286/286 hashes; 42/42 settings; network linked; 0 unknown devices”, so it fails. FINAL returned “ACCEPT=2 boots; 286/286 hashes; 42/42 settings; network linked; 0 unknown devices”, so it passes. No live result was counted. |
Initial6/10
Final10/10
Verdictworked
RecommendedYes, for this scope
08 · No cleanup by omission
What worked—and what failed
What worked
- PCOR-8049 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Freeze the verified user-data backup passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Preserve the recovery partitions also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- The first artifact failed Stage the required offline drivers; the one permitted correction resolved it, but the initial defect remains published.
09 · Inspectable record
Evidence notes
A disposable test computer and a post-install checklist will verify data preservation, driver availability, and boot success.
- PCOR-8049 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- PCOR-8049's first and final scores were recomputed from parsed RESULT rows: 3 and 5 passes multiplied by two.
- PCOR-8049 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: A disposable test computer and a post-install checklist will verify data preservation, driver availability, and boot success.
10 · Boundary of the claim
Limitations
- PCOR-8049 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- PCOR-8049 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.