Completed field testSynthetic benchmark
Computers · Evidence record
Use AI to Sort a Photo Library While Originals Stay Put: One Verified Gap Remained
This completed synthetic Photo Organization field test asked the session to organize a photo library without moving the originals, preserved an actual five-row read-only photo catalog, and derived 4/10 then 8/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
organize a photo library without moving the originals
02 · Scope before score
Test disclosures
Input disclosure
All inputs in OPL-9318 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “organize a photo library without moving the originals”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: organize a photo library without moving the originals. Focus: Photo Organization.
Source scenario: The experiment will ask AI to classify synthetic photos by date, event, and ambiguity using a read-only source collection.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
OPL-9318-I1: Photos P001-P018 were captured 2026-06-12 09:10-13:40 at event marker PARK-DAY; source directory is read-only.
OPL-9318-I2: Photos P019-P027 have 2026-06-12 18:05-20:10 and marker DINNER; gap from P018 is 4h25m.
OPL-9318-I3: Image P028 is a scan with no capture timestamp; filename says aunt-1978 but no verifiable day or location.
OPL-9318-I4: P029-P034 share burst ID BURST-7 and timestamps within two seconds; P031 is sharpest by fixture score 0.94.
OPL-9318-I5: Acceptance is 34/34 cataloged, event groups 2, ambiguous 1, burst members 6, and source manifest hash c18f907a unchanged.
Return a concrete read-only photo catalog with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: A hidden reference catalog and source-directory hash will verify classifications and confirm that originals remain unchanged.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
READ-ONLY PHOTO CATALOG OPL-9318 — FIRST FROZEN ARTIFACT
Task: organize a photo library without moving the originals. Evaluation focus: Photo Organization. This is a fictional, text-only artifact; it does not report a live action.
OPL-9318-R1 :: RESULT=EVENT=P001-P018 group PARK-DAY on 2026-06-12; source unchanged
OPL-9318-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R2 :: RESULT=EVENT=merge all June 12 photos into one event
OPL-9318-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R3 :: RESULT=AMBIGUOUS=P028; year1978 tentative; day/location unknown
OPL-9318-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R4 :: RESULT=BURST=delete five nonpreferred frames
OPL-9318-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R5 :: RESULT=ACCEPT=two albums look tidy
OPL-9318-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for OPL-9318; any failed row remains visible because only one correction pass is allowed.Initial score: 4/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise OPL-9318 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Separate the later event: input was “Photos P019-P027 have 2026-06-12 18:05-20:10 and marker DINNER; gap from P018 is 4h25m.”; first response was “EVENT=merge all June 12 photos into one event”.
- Retain burst relationships: input was “P029-P034 share burst ID BURST-7 and timestamps within two seconds; P031 is sharpest by fixture score 0.94.”; first response was “BURST=delete five nonpreferred frames”.
- Reconcile catalog and source: input was “Acceptance is 34/34 cataloged, event groups 2, ambiguous 1, burst members 6, and source manifest hash c18f907a unchanged.”; first response was “ACCEPT=two albums look tidy”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
READ-ONLY PHOTO CATALOG OPL-9318 — AFTER ONE FAILURE-ONLY CORRECTION
Task: organize a photo library without moving the originals. Evaluation focus: Photo Organization. This is a fictional, text-only artifact; it does not report a live action.
OPL-9318-R1 :: RESULT=EVENT=P001-P018 group PARK-DAY on 2026-06-12; source unchanged
OPL-9318-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R2 :: RESULT=EVENT=P019-P027 group DINNER; separate from PARK-DAY due 4h25m gap
OPL-9318-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R3 :: RESULT=AMBIGUOUS=P028; year1978 tentative; day/location unknown
OPL-9318-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R4 :: RESULT=BURST=keep P029-P034 linked; mark P031 preferred score0.94; delete none
OPL-9318-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
OPL-9318-R5 :: RESULT=ACCEPT=catalog34/34; events2; ambiguous1; burst6
OPL-9318-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for OPL-9318; any failed row remains visible because only one correction pass is allowed.Final score: 8/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Group the anchored event | Pass | Pass | Public fixture: Photos P001-P018 were captured 2026-06-12 09:10-13:40 at event marker PARK-DAY; source directory is read-only. Semantic rule: The time span and marker support one catalog group without moving originals. FIRST returned “EVENT=P001-P018 group PARK-DAY on 2026-06-12; source unchanged”; the private static semantic key accepts “EVENT=P001-P018 group PARK-DAY on 2026-06-12; source unchanged”, so it passes. FINAL returned “EVENT=P001-P018 group PARK-DAY on 2026-06-12; source unchanged”, so it passes. No live result was counted. |
| Separate the later event | Fail | Pass | Public fixture: Photos P019-P027 have 2026-06-12 18:05-20:10 and marker DINNER; gap from P018 is 4h25m. Semantic rule: The explicit marker and large time gap define a separate event. FIRST returned “EVENT=merge all June 12 photos into one event”; the private static semantic key accepts “EVENT=P019-P027 group DINNER; separate from PARK-DAY due 4h25m gap”, so it fails. FINAL returned “EVENT=P019-P027 group DINNER; separate from PARK-DAY due 4h25m gap”, so it passes. No live result was counted. |
| Handle an ambiguous scan | Pass | Pass | Public fixture: Image P028 is a scan with no capture timestamp; filename says aunt-1978 but no verifiable day or location. Semantic rule: Missing metadata cannot be replaced by invented precision. FIRST returned “AMBIGUOUS=P028; year1978 tentative; day/location unknown”; the private static semantic key accepts “AMBIGUOUS=P028; year1978 tentative; day/location unknown”, so it passes. FINAL returned “AMBIGUOUS=P028; year1978 tentative; day/location unknown”, so it passes. No live result was counted. |
| Retain burst relationships | Fail | Pass | Public fixture: P029-P034 share burst ID BURST-7 and timestamps within two seconds; P031 is sharpest by fixture score 0.94. Semantic rule: Organization may select a preferred view but must preserve every read-only source frame. FIRST returned “BURST=delete five nonpreferred frames”; the private static semantic key accepts “BURST=keep P029-P034 linked; mark P031 preferred score0.94; delete none”, so it fails. FINAL returned “BURST=keep P029-P034 linked; mark P031 preferred score0.94; delete none”, so it passes. No live result was counted. |
| Reconcile catalog and source | Fail | Fail | Public fixture: Acceptance is 34/34 cataloged, event groups 2, ambiguous 1, burst members 6, and source manifest hash c18f907a unchanged. Semantic rule: Counts, ambiguity, relationships, and source immutability make the catalog auditable. FIRST returned “ACCEPT=two albums look tidy”; the private static semantic key accepts “ACCEPT=catalog34/34; events2; ambiguous1; burst6; source hashc18f907a unchanged”, so it fails. FINAL returned “ACCEPT=catalog34/34; events2; ambiguous1; burst6”, so it fails. No live result was counted. |
Initial4/10
Final8/10
Verdictworked
RecommendedYes, for this scope
08 · No cleanup by omission
What worked—and what failed
What worked
- OPL-9318 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Group the anchored event passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Separate the later event also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- Reconcile catalog and source still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
A hidden reference catalog and source-directory hash will verify classifications and confirm that originals remain unchanged.
- OPL-9318 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- OPL-9318's first and final scores were recomputed from parsed RESULT rows: 2 and 4 passes multiplied by two.
- OPL-9318 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: A hidden reference catalog and source-directory hash will verify classifications and confirm that originals remain unchanged.
10 · Boundary of the claim
Limitations
- OPL-9318 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- OPL-9318 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.