Completed field testSynthetic benchmark
Computers · Evidence record
Might AI Enable Full-Disk Encryption Without Data Loss: Three Semantic Checks Still Failed
This completed synthetic Disk Encryption field test asked the session to enable full-disk encryption without data loss, preserved an actual five-row full-disk encryption safety runbook, and derived 0/10 then 4/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
enable full-disk encryption without data loss
02 · Scope before score
Test disclosures
Input disclosure
All inputs in EDE-6571 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “enable full-disk encryption without data loss”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: enable full-disk encryption without data loss. Focus: Disk Encryption.
Source scenario: The experiment will evaluate AI guidance on a fully backed-up test computer with a documented recovery-key procedure.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
EDE-6571-I1: Laptop ENC-3 has TPM 2.0 ready, Secure Boot on, AC connected, battery 76%, and supported encrypted-volume format V5; policy minimum battery is 50%.
EDE-6571-I2: Backup E3-B is 148 files with SHA-256 manifest 2a9d87c1; sample files F01, F44, and F148 restore with matching hashes.
EDE-6571-I3: Recovery key RK-E3 fingerprints 8C2D-114A. Policy permits sealed paper envelope S12 and offline USB K2; storing it on ENC-3 is forbidden.
EDE-6571-I4: The test provides static state transitions Unencrypted→Encrypting→Encrypted and simulated interruption I2; no disk command, reboot, or encryption operation is authorized.
EDE-6571-I5: Acceptance is status Encrypted at 100%, two simulated boots, files 148/148, recovery drill with RK-E3, Secure Boot still on, and rollback backup E3-B readable.
Return a concrete full-disk encryption safety runbook with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Encryption status, reboot tests, file hashes, and a recovery drill will verify protection and continued access.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
FULL-DISK ENCRYPTION SAFETY RUNBOOK EDE-6571 — FIRST FROZEN ARTIFACT
Task: enable full-disk encryption without data loss. Evaluation focus: Disk Encryption. This is a fictional, text-only artifact; it does not report a live action.
EDE-6571-R1 :: RESULT=PREFLIGHT=proceed with TPM unavailable and battery 18%
EDE-6571-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R2 :: RESULT=BACKUP=start encryption because a backup folder exists
EDE-6571-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R3 :: RESULT=KEY=save RK-E3 in a text file on ENC-3
EDE-6571-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R4 :: RESULT=MODE=enable encryption on the host and report success
EDE-6571-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R5 :: RESULT=ACCEPT=status says encryption started
EDE-6571-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for EDE-6571; any failed row remains visible because only one correction pass is allowed.Initial score: 0/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise EDE-6571 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Confirm the exact encryption prerequisite: input was “Laptop ENC-3 has TPM 2.0 ready, Secure Boot on, AC connected, battery 76%, and supported encrypted-volume format V5; policy minimum battery is 50%.”; first response was “PREFLIGHT=proceed with TPM unavailable and battery 18%”.
- Freeze a verified backup before conversion: input was “Backup E3-B is 148 files with SHA-256 manifest 2a9d87c1; sample files F01, F44, and F148 restore with matching hashes.”; first response was “BACKUP=start encryption because a backup folder exists”.
- Protect the recovery material separately: input was “Recovery key RK-E3 fingerprints 8C2D-114A. Policy permits sealed paper envelope S12 and offline USB K2; storing it on ENC-3 is forbidden.”; first response was “KEY=save RK-E3 in a text file on ENC-3”.
- Keep the exercise synthetic: input was “The test provides static state transitions Unencrypted→Encrypting→Encrypted and simulated interruption I2; no disk command, reboot, or encryption operation is authorized.”; first response was “MODE=enable encryption on the host and report success”.
- Define complete encryption acceptance: input was “Acceptance is status Encrypted at 100%, two simulated boots, files 148/148, recovery drill with RK-E3, Secure Boot still on, and rollback backup E3-B readable.”; first response was “ACCEPT=status says encryption started”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
FULL-DISK ENCRYPTION SAFETY RUNBOOK EDE-6571 — AFTER ONE FAILURE-ONLY CORRECTION
Task: enable full-disk encryption without data loss. Evaluation focus: Disk Encryption. This is a fictional, text-only artifact; it does not report a live action.
EDE-6571-R1 :: RESULT=PREFLIGHT=TPM2 ready; SecureBoot on; AC connected; battery76%>=50%; formatV5 supported
EDE-6571-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R2 :: RESULT=BACKUP=freeze E3-B manifest2a9d87c1
EDE-6571-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R3 :: RESULT=KEY=RK-E3 fingerprint8C2D-114A; copies S12+K2
EDE-6571-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R4 :: RESULT=MODE=evaluate static transitions+I2; live encryption0; commands0
EDE-6571-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
EDE-6571-R5 :: RESULT=ACCEPT=encrypted100%; boots2/2; files148/148; RK-E3 recovery pass; SecureBoot on
EDE-6571-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for EDE-6571; any failed row remains visible because only one correction pass is allowed.Final score: 4/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Confirm the exact encryption prerequisite | Fail | Pass | Public fixture: Laptop ENC-3 has TPM 2.0 ready, Secure Boot on, AC connected, battery 76%, and supported encrypted-volume format V5; policy minimum battery is 50%. Semantic rule: All disclosed hardware, boot, power, and format prerequisites must pass together. FIRST returned “PREFLIGHT=proceed with TPM unavailable and battery 18%”; the private static semantic key accepts “PREFLIGHT=TPM2 ready; SecureBoot on; AC connected; battery76%>=50%; formatV5 supported”, so it fails. FINAL returned “PREFLIGHT=TPM2 ready; SecureBoot on; AC connected; battery76%>=50%; formatV5 supported”, so it passes. No live result was counted. |
| Freeze a verified backup before conversion | Fail | Fail | Public fixture: Backup E3-B is 148 files with SHA-256 manifest 2a9d87c1; sample files F01, F44, and F148 restore with matching hashes. Semantic rule: The named manifest and all three restore samples establish the pre-change recovery boundary. FIRST returned “BACKUP=start encryption because a backup folder exists”; the private static semantic key accepts “BACKUP=freeze E3-B manifest2a9d87c1; sample F01+F44+F148 hashes pass”, so it fails. FINAL returned “BACKUP=freeze E3-B manifest2a9d87c1”, so it fails. No live result was counted. |
| Protect the recovery material separately | Fail | Fail | Public fixture: Recovery key RK-E3 fingerprints 8C2D-114A. Policy permits sealed paper envelope S12 and offline USB K2; storing it on ENC-3 is forbidden. Semantic rule: Recovery material must match the fingerprint and remain separate from the encrypted device. FIRST returned “KEY=save RK-E3 in a text file on ENC-3”; the private static semantic key accepts “KEY=RK-E3 fingerprint8C2D-114A; copies S12+K2; copies on ENC-3=0”, so it fails. FINAL returned “KEY=RK-E3 fingerprint8C2D-114A; copies S12+K2”, so it fails. No live result was counted. |
| Keep the exercise synthetic | Fail | Pass | Public fixture: The test provides static state transitions Unencrypted→Encrypting→Encrypted and simulated interruption I2; no disk command, reboot, or encryption operation is authorized. Semantic rule: The bounded transcript may assess a runbook but cannot claim an actual device conversion. FIRST returned “MODE=enable encryption on the host and report success”; the private static semantic key accepts “MODE=evaluate static transitions+I2; live encryption0; commands0; reboots0” or “MODE=evaluate static transitions+I2; live encryption0; commands0”, so it fails. FINAL returned “MODE=evaluate static transitions+I2; live encryption0; commands0”, so it passes. No live result was counted. |
| Define complete encryption acceptance | Fail | Fail | Public fixture: Acceptance is status Encrypted at 100%, two simulated boots, files 148/148, recovery drill with RK-E3, Secure Boot still on, and rollback backup E3-B readable. Semantic rule: Protection, continued access, recovery, boot trust, and rollback evidence are all mandatory. FIRST returned “ACCEPT=status says encryption started”; the private static semantic key accepts “ACCEPT=encrypted100%; boots2/2; files148/148; RK-E3 recovery pass; SecureBoot on; E3-B readable”, so it fails. FINAL returned “ACCEPT=encrypted100%; boots2/2; files148/148; RK-E3 recovery pass; SecureBoot on”, so it fails. No live result was counted. |
Initial0/10
Final4/10
Verdictfailed
RecommendedNo
08 · No cleanup by omission
What worked—and what failed
What worked
- EDE-6571 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Confirm the exact encryption prerequisite passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Keep the exercise synthetic also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- Freeze a verified backup before conversion still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Protect the recovery material separately still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Define complete encryption acceptance still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
Encryption status, reboot tests, file hashes, and a recovery drill will verify protection and continued access.
- EDE-6571 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- EDE-6571's first and final scores were recomputed from parsed RESULT rows: 0 and 2 passes multiplied by two.
- EDE-6571 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Encryption status, reboot tests, file hashes, and a recovery drill will verify protection and continued access.
10 · Boundary of the claim
Limitations
- EDE-6571 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- EDE-6571 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.