Completed field testSynthetic benchmark
Computers · Evidence record
Planning a Zero-Downtime Database Migration: The Correction Reached 6/10
This completed synthetic Database Migration field test asked the session to plan a zero-downtime database migration with a safe rollback, preserved an actual five-row zero-downtime migration traffic and rollback plan, and derived 0/10 then 6/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
plan a zero-downtime database migration with a safe rollback
02 · Scope before score
Test disclosures
Input disclosure
All inputs in PZDM-1039 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “plan a zero-downtime database migration with a safe rollback”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: plan a zero-downtime database migration with a safe rollback. Focus: Database Migration.
Source scenario: The experiment will specify a synthetic schema change, traffic profile, replication behavior, compatibility window, and recovery objective.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
PZDM-1039-I1: Traffic replay is 800 requests/s: 70% old readers and 30% new writers. New nullable region_code may be added; old region must remain through this release.
PZDM-1039-I2: Replica rehearsal measures metadata-only add-column lock at 85 ms; policy ceiling is 200 ms. A NOT NULL rewrite rehearsal holds the lock 4.8 s.
PZDM-1039-I3: Source has 240,000 rows. Backfill batches 5,000 rows and pauses above 2 s replica lag; batch 7 reaches 2.8 s while prior batches stay below 1.4 s.
PZDM-1039-I4: Traffic replay is 800 requests/s for two minutes with 30% writes, producing 28,800 writes. Acceptance expects source and destination at 240,000 rows, dual-write mismatches zero, and checksum 5f0a2c11 on both.
PZDM-1039-I5: Recovery objective is RTO at most 5 minutes and RPO zero. Synthetic rollback switches readers to old region while dual-write remains; rehearsal completes in 3m40s with zero lost writes.
Return a concrete zero-downtime migration traffic and rollback plan with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: A disposable replica and scripted traffic replay will verify compatibility, lock time, data consistency, cutover order, and rollback.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
ZERO-DOWNTIME MIGRATION TRAFFIC AND ROLLBACK PLAN PZDM-1039 — FIRST FROZEN ARTIFACT
Task: plan a zero-downtime database migration with a safe rollback. Evaluation focus: Database Migration. This is a fictional, text-only artifact; it does not report a live action.
PZDM-1039-R1 :: RESULT=ORDER=drop old region before deploying new writers
PZDM-1039-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R2 :: RESULT=LOCK=use 4.8s rewrite during peak traffic
PZDM-1039-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R3 :: RESULT=BACKFILL=update all240000 rows in one transaction
PZDM-1039-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R4 :: RESULT=CUTOVER=switch after the application starts without reconciliation
PZDM-1039-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R5 :: RESULT=ROLLBACK=delete region_code values and accept ten minutes downtime
PZDM-1039-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for PZDM-1039; any failed row remains visible because only one correction pass is allowed.Initial score: 0/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise PZDM-1039 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Use expand-before-contract under mixed traffic: input was “Traffic replay is 800 requests/s: 70% old readers and 30% new writers. New nullable region_code may be added; old region must remain through this release.”; first response was “ORDER=drop old region before deploying new writers”.
- Keep schema lock time below the gate: input was “Replica rehearsal measures metadata-only add-column lock at 85 ms; policy ceiling is 200 ms. A NOT NULL rewrite rehearsal holds the lock 4.8 s.”; first response was “LOCK=use 4.8s rewrite during peak traffic”.
- Bound backfill by replication lag: input was “Source has 240,000 rows. Backfill batches 5,000 rows and pauses above 2 s replica lag; batch 7 reaches 2.8 s while prior batches stay below 1.4 s.”; first response was “BACKFILL=update all240000 rows in one transaction”.
- Reconcile dual-write traffic before cutover: input was “Traffic replay is 800 requests/s for two minutes with 30% writes, producing 28,800 writes. Acceptance expects source and destination at 240,000 rows, dual-write mismatches zero, and checksum 5f0a2c11 on both.”; first response was “CUTOVER=switch after the application starts without reconciliation”.
- Meet the recovery objective with a reversible rollback: input was “Recovery objective is RTO at most 5 minutes and RPO zero. Synthetic rollback switches readers to old region while dual-write remains; rehearsal completes in 3m40s with zero lost writes.”; first response was “ROLLBACK=delete region_code values and accept ten minutes downtime”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
ZERO-DOWNTIME MIGRATION TRAFFIC AND ROLLBACK PLAN PZDM-1039 — AFTER ONE FAILURE-ONLY CORRECTION
Task: plan a zero-downtime database migration with a safe rollback. Evaluation focus: Database Migration. This is a fictional, text-only artifact; it does not report a live action.
PZDM-1039-R1 :: RESULT=ORDER=add nullable region_code; keep old region; old readers70%+new writers30% compatible at800rps
PZDM-1039-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R2 :: RESULT=LOCK=choose metadata add at85ms<=200ms; reject rewrite4.8s
PZDM-1039-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R3 :: RESULT=BACKFILL=5000/batch; pause batch7 at lag2.8s>2s
PZDM-1039-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R4 :: RESULT=CUTOVER=traffic800rps/2min; writes28800; rows240000/240000; mismatches0
PZDM-1039-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
PZDM-1039-R5 :: RESULT=ROLLBACK=read old region; keep dual-write; RTO3m40s<=5m; RPO0
PZDM-1039-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for PZDM-1039; any failed row remains visible because only one correction pass is allowed.Final score: 6/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Use expand-before-contract under mixed traffic | Fail | Pass | Public fixture: Traffic replay is 800 requests/s: 70% old readers and 30% new writers. New nullable region_code may be added; old region must remain through this release. Semantic rule: Compatibility must hold for the disclosed old/new client mix at the frozen traffic rate. FIRST returned “ORDER=drop old region before deploying new writers”; the private static semantic key accepts “ORDER=add nullable region_code; keep old region; old readers70%+new writers30% compatible at800rps”, so it fails. FINAL returned “ORDER=add nullable region_code; keep old region; old readers70%+new writers30% compatible at800rps”, so it passes. No live result was counted. |
| Keep schema lock time below the gate | Fail | Pass | Public fixture: Replica rehearsal measures metadata-only add-column lock at 85 ms; policy ceiling is 200 ms. A NOT NULL rewrite rehearsal holds the lock 4.8 s. Semantic rule: The only candidate within the explicit lock-time objective is the metadata-only expansion. FIRST returned “LOCK=use 4.8s rewrite during peak traffic”; the private static semantic key accepts “LOCK=choose metadata add at85ms<=200ms; reject rewrite4.8s”, so it fails. FINAL returned “LOCK=choose metadata add at85ms<=200ms; reject rewrite4.8s”, so it passes. No live result was counted. |
| Bound backfill by replication lag | Fail | Fail | Public fixture: Source has 240,000 rows. Backfill batches 5,000 rows and pauses above 2 s replica lag; batch 7 reaches 2.8 s while prior batches stay below 1.4 s. Semantic rule: The declared lag stop condition must govern the exact batch where the threshold is crossed. FIRST returned “BACKFILL=update all240000 rows in one transaction”; the private static semantic key accepts “BACKFILL=5000/batch; pause batch7 at lag2.8s>2s; resume only below2s”, so it fails. FINAL returned “BACKFILL=5000/batch; pause batch7 at lag2.8s>2s”, so it fails. No live result was counted. |
| Reconcile dual-write traffic before cutover | Fail | Fail | Public fixture: Traffic replay is 800 requests/s for two minutes with 30% writes, producing 28,800 writes. Acceptance expects source and destination at 240,000 rows, dual-write mismatches zero, and checksum 5f0a2c11 on both. Semantic rule: The workload, writer share, write count, row counts, mismatch count, and hashes are the fixed consistency gate. FIRST returned “CUTOVER=switch after the application starts without reconciliation”; the private static semantic key accepts “CUTOVER=traffic800rps/2min; writes28800; rows240000/240000; mismatches0; hashes5f0a2c11”, so it fails. FINAL returned “CUTOVER=traffic800rps/2min; writes28800; rows240000/240000; mismatches0”, so it fails. No live result was counted. |
| Meet the recovery objective with a reversible rollback | Fail | Pass | Public fixture: Recovery objective is RTO at most 5 minutes and RPO zero. Synthetic rollback switches readers to old region while dual-write remains; rehearsal completes in 3m40s with zero lost writes. Semantic rule: The rollback must preserve both schemas and meet the explicit time and zero-data-loss objectives. FIRST returned “ROLLBACK=delete region_code values and accept ten minutes downtime”; the private static semantic key accepts “ROLLBACK=read old region; keep dual-write; RTO3m40s<=5m; RPO0; lost writes0” or “ROLLBACK=read old region; keep dual-write; RTO3m40s<=5m; RPO0”, so it fails. FINAL returned “ROLLBACK=read old region; keep dual-write; RTO3m40s<=5m; RPO0”, so it passes. No live result was counted. |
Initial0/10
Final6/10
Verdictmixed
RecommendedNo
08 · No cleanup by omission
What worked—and what failed
What worked
- PZDM-1039 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Use expand-before-contract under mixed traffic passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Keep schema lock time below the gate also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- Bound backfill by replication lag still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Reconcile dual-write traffic before cutover still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
A disposable replica and scripted traffic replay will verify compatibility, lock time, data consistency, cutover order, and rollback.
- PZDM-1039 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- PZDM-1039's first and final scores were recomputed from parsed RESULT rows: 0 and 3 passes multiplied by two.
- PZDM-1039 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: A disposable replica and scripted traffic replay will verify compatibility, lock time, data consistency, cutover order, and rollback.
10 · Boundary of the claim
Limitations
- PZDM-1039 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- PZDM-1039 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.