{
  "category": "computers",
  "slug": "computers-port-shell-automation",
  "title": "From One OS to Another: Porting Shell Automation: The Correction Reached 6/10",
  "task": "port a shell automation between operating systems",
  "excerpt": "This completed synthetic Portability field test asked the session to port a shell automation between operating systems, preserved an actual five-row cross-platform shell portability matrix, and derived 0/10 then 6/10 from task-specific semantic checks after one failure-only correction.",
  "tool": "Codex multi-agent session",
  "model": "Exact underlying model identifier not disclosed by the Codex session",
  "publishedAt": "2026-05-24T16:00:00+08:00",
  "durationMinutes": 0,
  "testMode": "Synthetic benchmark",
  "inputDisclosure": "All inputs in PSA-0632 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.",
  "runDisclosure": "A Codex multi-agent session generated one text-only first artifact for “port a shell automation between operating systems”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.",
  "prompt": "Complete a bounded synthetic field test for: port a shell automation between operating systems. Focus: Portability.\nSource scenario: The experiment will ask AI to adapt a file-processing automation for a second operating system under equivalent requirements.\nUse only these five public fictional inputs; the scoring answers are intentionally withheld:\nPSA-0632-I1: Script PORT-6 calls readlink -f on Linux; the target macOS runner lacks that option. Python 3.12 is installed on both systems and symlink fixture L1 must resolve to /tmp/Port Case/input.csv.\nPSA-0632-I2: Linux GNU sed accepts sed -i, while macOS BSD sed requires an explicit backup suffix. Fixture C2 changes mode=dev to mode=test and must leave no backup file.\nPSA-0632-I3: Input directory is /tmp/Port Case with files alpha one.csv and beta.csv; the current unquoted loop splits alpha one.csv into two arguments.\nPSA-0632-I4: Worker W3 exits 23 after creating /tmp/port-6.stage; caller currently returns 0. Policy requires original nonzero status and removal of the stage file.\nPSA-0632-I5: Frozen fixtures expect hashes a11c for F1, b22d for F2, and exit 23 for F3 on Ubuntu 24.04 and macOS 15; shellcheck findings must be zero.\nReturn a concrete cross-platform shell portability matrix with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Identical fixtures and expected-output comparisons on both systems will verify compatible behavior and error handling.",
  "firstResult": "CROSS-PLATFORM SHELL PORTABILITY MATRIX PSA-0632 — FIRST FROZEN ARTIFACT\nTask: port a shell automation between operating systems. Evaluation focus: Portability. This is a fictional, text-only artifact; it does not report a live action.\nPSA-0632-R1 :: RESULT=PATH=retain readlink -f and assume macOS supports it\nPSA-0632-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R2 :: RESULT=EDIT=run sed -i identically on both systems\nPSA-0632-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R3 :: RESULT=QUOTING=escape only the directory name and leave loop variables unquoted\nPSA-0632-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R4 :: RESULT=FAILURE=return0 after printing a warning\nPSA-0632-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R5 :: RESULT=ACCEPT=one happy-path run on Ubuntu\nPSA-0632-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for PSA-0632; any failed row remains visible because only one correction pass is allowed.",
  "correctionPrompt": "Revise PSA-0632 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:\n- Replace the nonportable path resolver: input was “Script PORT-6 calls readlink -f on Linux; the target macOS runner lacks that option. Python 3.12 is installed on both systems and symlink fixture L1 must resolve to /tmp/Port Case/input.csv.”; first response was “PATH=retain readlink -f and assume macOS supports it”.\n- Handle in-place editing on both sed variants: input was “Linux GNU sed accepts sed -i, while macOS BSD sed requires an explicit backup suffix. Fixture C2 changes mode=dev to mode=test and must leave no backup file.”; first response was “EDIT=run sed -i identically on both systems”.\n- Preserve paths containing spaces: input was “Input directory is /tmp/Port Case with files alpha one.csv and beta.csv; the current unquoted loop splits alpha one.csv into two arguments.”; first response was “QUOTING=escape only the directory name and leave loop variables unquoted”.\n- Propagate failure and clean temporary state: input was “Worker W3 exits 23 after creating /tmp/port-6.stage; caller currently returns 0. Policy requires original nonzero status and removal of the stage file.”; first response was “FAILURE=return0 after printing a warning”.\n- Define a two-system acceptance matrix: input was “Frozen fixtures expect hashes a11c for F1, b22d for F2, and exit 23 for F3 on Ubuntu 24.04 and macOS 15; shellcheck findings must be zero.”; first response was “ACCEPT=one happy-path run on Ubuntu”.\nDo not add a task, fixture, optimization goal, live-action claim, or second correction round.",
  "finalResult": "CROSS-PLATFORM SHELL PORTABILITY MATRIX PSA-0632 — AFTER ONE FAILURE-ONLY CORRECTION\nTask: port a shell automation between operating systems. Evaluation focus: Portability. This is a fictional, text-only artifact; it does not report a live action.\nPSA-0632-R1 :: RESULT=PATH=use python3.12 realpath equivalent; L1 resolves /tmp/Port Case/input.csv; Linux+macOS same\nPSA-0632-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R2 :: RESULT=EDIT=use portable temp-file then atomic replace; C2 mode=test; backup files0\nPSA-0632-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R3 :: RESULT=QUOTING=iterate two files; alpha one.csv remains one argument\nPSA-0632-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R4 :: RESULT=FAILURE=return23; remove /tmp/port-6.stage\nPSA-0632-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPSA-0632-R5 :: RESULT=ACCEPT=Ubuntu+macOS F1=a11c F2=b22d F3=exit23; shellcheck0\nPSA-0632-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for PSA-0632; any failed row remains visible because only one correction pass is allowed.",
  "checks": [
    {
      "name": "Replace the nonportable path resolver",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Script PORT-6 calls readlink -f on Linux; the target macOS runner lacks that option. Python 3.12 is installed on both systems and symlink fixture L1 must resolve to /tmp/Port Case/input.csv. Semantic rule: The replacement must use a disclosed common runtime and preserve the exact symlink resolution on both targets. FIRST returned “PATH=retain readlink -f and assume macOS supports it”; the private static semantic key accepts “PATH=use python3.12 realpath equivalent; L1 resolves /tmp/Port Case/input.csv; Linux+macOS same”, so it fails. FINAL returned “PATH=use python3.12 realpath equivalent; L1 resolves /tmp/Port Case/input.csv; Linux+macOS same”, so it passes. No live result was counted."
    },
    {
      "name": "Handle in-place editing on both sed variants",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Linux GNU sed accepts sed -i, while macOS BSD sed requires an explicit backup suffix. Fixture C2 changes mode=dev to mode=test and must leave no backup file. Semantic rule: The plan must avoid variant-specific flags while producing the same content and cleanup state. FIRST returned “EDIT=run sed -i identically on both systems”; the private static semantic key accepts “EDIT=use portable temp-file then atomic replace; C2 mode=test; backup files0”, so it fails. FINAL returned “EDIT=use portable temp-file then atomic replace; C2 mode=test; backup files0”, so it passes. No live result was counted."
    },
    {
      "name": "Preserve paths containing spaces",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Input directory is /tmp/Port Case with files alpha one.csv and beta.csv; the current unquoted loop splits alpha one.csv into two arguments. Semantic rule: Both the directory and each expanded filename require boundaries that survive embedded spaces. FIRST returned “QUOTING=escape only the directory name and leave loop variables unquoted”; the private static semantic key accepts “QUOTING=iterate two files; alpha one.csv remains one argument; word-splitting errors0” or “QUOTING=iterate two files; alpha one.csv remains one argument”, so it fails. FINAL returned “QUOTING=iterate two files; alpha one.csv remains one argument”, so it passes. No live result was counted."
    },
    {
      "name": "Propagate failure and clean temporary state",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Worker W3 exits 23 after creating /tmp/port-6.stage; caller currently returns 0. Policy requires original nonzero status and removal of the stage file. Semantic rule: The port must preserve the worker status, execute cleanup, and stop subsequent work. FIRST returned “FAILURE=return0 after printing a warning”; the private static semantic key accepts “FAILURE=return23; remove /tmp/port-6.stage; do not continue later workers”, so it fails. FINAL returned “FAILURE=return23; remove /tmp/port-6.stage”, so it fails. No live result was counted."
    },
    {
      "name": "Define a two-system acceptance matrix",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Frozen fixtures expect hashes a11c for F1, b22d for F2, and exit 23 for F3 on Ubuntu 24.04 and macOS 15; shellcheck findings must be zero. Semantic rule: Acceptance requires both operating systems, all three fixtures, identical hashes, the error path, and the static check. FIRST returned “ACCEPT=one happy-path run on Ubuntu”; the private static semantic key accepts “ACCEPT=Ubuntu+macOS F1=a11c F2=b22d F3=exit23; shellcheck0; outputs identical”, so it fails. FINAL returned “ACCEPT=Ubuntu+macOS F1=a11c F2=b22d F3=exit23; shellcheck0”, so it fails. No live result was counted."
    }
  ],
  "initialScore": 0,
  "score": 6,
  "verdict": "mixed",
  "recommended": false,
  "whatWorked": [
    "PSA-0632 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.",
    "Replace the nonportable path resolver passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.",
    "Handle in-place editing on both sed variants also passed its task-specific rule with the final answer left visible."
  ],
  "whatFailed": [
    "Propagate failure and clean temporary state still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.",
    "Define a two-system acceptance matrix still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence."
  ],
  "evidencePlan": "Identical fixtures and expected-output comparisons on both systems will verify compatible behavior and error handling.",
  "evidenceNotes": [
    "PSA-0632 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.",
    "PSA-0632's first and final scores were recomputed from parsed RESULT rows: 0 and 3 passes multiplied by two.",
    "PSA-0632 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Identical fixtures and expected-output comparisons on both systems will verify compatible behavior and error handling."
  ],
  "limitations": [
    "PSA-0632 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.",
    "PSA-0632 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result."
  ]
}