{
  "category": "computers",
  "slug": "computers-package-command-line-app",
  "title": "Build a Cleanly Installable CLI Package: Three Semantic Checks Still Failed",
  "task": "package a command-line application for clean installation",
  "excerpt": "This completed synthetic App Packaging field test asked the session to package a command-line application for clean installation, preserved an actual five-row cli package lifecycle record, and derived 0/10 then 4/10 from task-specific semantic checks after one failure-only correction.",
  "tool": "Codex multi-agent session",
  "model": "Exact underlying model identifier not disclosed by the Codex session",
  "publishedAt": "2026-02-24T16:00:00+08:00",
  "durationMinutes": 0,
  "testMode": "Synthetic benchmark",
  "inputDisclosure": "All inputs in PCLA-5811 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.",
  "runDisclosure": "A Codex multi-agent session generated one text-only first artifact for “package a command-line application for clean installation”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.",
  "prompt": "Complete a bounded synthetic field test for: package a command-line application for clean installation. Focus: App Packaging.\nSource scenario: The experiment will ask AI to package a small application with versioned metadata, dependencies, and uninstall behavior.\nUse only these five public fictional inputs; the scoring answers are intentionally withheld:\nPCLA-5811-I1: Application tidy version 1.4.2 exposes command tidy through bin/tidy.js; metadata license is MIT and runtime is Node >=22.\nPCLA-5811-I2: Package needs bin/tidy.js, lib/a.js, lib/b.js, README, and LICENSE; tests/, fixtures/, and .env are excluded.\nPCLA-5811-I3: Runtime requires parse 3.2.1 and color 5.0.0; dev-only testlib 9.1.0 must not install for users.\nPCLA-5811-I4: Upgrade fixture moves 1.4.1→1.4.2 while retaining config CFG-USER and replacing executable hash old11 with new42.\nPCLA-5811-I5: Acceptance: clean install, tidy --version prints 1.4.2, sample output hash 6cf1, upgrade passes, uninstall removes command but retains CFG-USER.\nReturn a concrete cli package lifecycle record with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Fresh install, upgrade, invocation, and uninstall checks will verify the package lifecycle on a clean system.",
  "firstResult": "CLI PACKAGE LIFECYCLE RECORD PCLA-5811 — FIRST FROZEN ARTIFACT\nTask: package a command-line application for clean installation. Evaluation focus: App Packaging. This is a fictional, text-only artifact; it does not report a live action.\nPCLA-5811-R1 :: RESULT=METADATA=omit version and entry point\nPCLA-5811-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R2 :: RESULT=CONTENTS=publish the whole repository including .env\nPCLA-5811-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R3 :: RESULT=DEPENDENCIES=ship testlib and floating versions\nPCLA-5811-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R4 :: RESULT=UPGRADE=delete CFG-USER during upgrade\nPCLA-5811-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R5 :: RESULT=ACCEPT=archive file exists\nPCLA-5811-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for PCLA-5811; any failed row remains visible because only one correction pass is allowed.",
  "correctionPrompt": "Revise PCLA-5811 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:\n- Declare package identity and entry point: input was “Application tidy version 1.4.2 exposes command tidy through bin/tidy.js; metadata license is MIT and runtime is Node >=22.”; first response was “METADATA=omit version and entry point”.\n- Include only runtime files: input was “Package needs bin/tidy.js, lib/a.js, lib/b.js, README, and LICENSE; tests/, fixtures/, and .env are excluded.”; first response was “CONTENTS=publish the whole repository including .env”.\n- Pin production dependencies: input was “Runtime requires parse 3.2.1 and color 5.0.0; dev-only testlib 9.1.0 must not install for users.”; first response was “DEPENDENCIES=ship testlib and floating versions”.\n- Preserve upgrade behavior: input was “Upgrade fixture moves 1.4.1→1.4.2 while retaining config CFG-USER and replacing executable hash old11 with new42.”; first response was “UPGRADE=delete CFG-USER during upgrade”.\n- Verify install, invocation, and uninstall: input was “Acceptance: clean install, tidy --version prints 1.4.2, sample output hash 6cf1, upgrade passes, uninstall removes command but retains CFG-USER.”; first response was “ACCEPT=archive file exists”.\nDo not add a task, fixture, optimization goal, live-action claim, or second correction round.",
  "finalResult": "CLI PACKAGE LIFECYCLE RECORD PCLA-5811 — AFTER ONE FAILURE-ONLY CORRECTION\nTask: package a command-line application for clean installation. Evaluation focus: App Packaging. This is a fictional, text-only artifact; it does not report a live action.\nPCLA-5811-R1 :: RESULT=METADATA=tidy1.4.2; bin tidy->bin/tidy.js; MIT; Node>=22\nPCLA-5811-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R2 :: RESULT=CONTENTS=include5 declared paths; exclude tests+fixtures+.env\nPCLA-5811-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R3 :: RESULT=DEPENDENCIES=parse3.2.1+color5.0.0 runtime\nPCLA-5811-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R4 :: RESULT=UPGRADE=1.4.1>1.4.2; retain CFG-USER\nPCLA-5811-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nPCLA-5811-R5 :: RESULT=ACCEPT=install pass; version1.4.2; sample6cf1; upgrade pass; command removed\nPCLA-5811-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for PCLA-5811; any failed row remains visible because only one correction pass is allowed.",
  "checks": [
    {
      "name": "Declare package identity and entry point",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Application tidy version 1.4.2 exposes command tidy through bin/tidy.js; metadata license is MIT and runtime is Node >=22. Semantic rule: A clean installer needs the exact identity, executable mapping, license, and runtime constraint. FIRST returned “METADATA=omit version and entry point”; the private static semantic key accepts “METADATA=tidy1.4.2; bin tidy->bin/tidy.js; MIT; Node>=22”, so it fails. FINAL returned “METADATA=tidy1.4.2; bin tidy->bin/tidy.js; MIT; Node>=22”, so it passes. No live result was counted."
    },
    {
      "name": "Include only runtime files",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Package needs bin/tidy.js, lib/a.js, lib/b.js, README, and LICENSE; tests/, fixtures/, and .env are excluded. Semantic rule: The frozen allowlist and sensitive-file denylist determine package contents. FIRST returned “CONTENTS=publish the whole repository including .env”; the private static semantic key accepts “CONTENTS=include5 declared paths; exclude tests+fixtures+.env”, so it fails. FINAL returned “CONTENTS=include5 declared paths; exclude tests+fixtures+.env”, so it passes. No live result was counted."
    },
    {
      "name": "Pin production dependencies",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Runtime requires parse 3.2.1 and color 5.0.0; dev-only testlib 9.1.0 must not install for users. Semantic rule: User installation should contain exact production dependencies only. FIRST returned “DEPENDENCIES=ship testlib and floating versions”; the private static semantic key accepts “DEPENDENCIES=parse3.2.1+color5.0.0 runtime; exclude testlib9.1.0”, so it fails. FINAL returned “DEPENDENCIES=parse3.2.1+color5.0.0 runtime”, so it fails. No live result was counted."
    },
    {
      "name": "Preserve upgrade behavior",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Upgrade fixture moves 1.4.1→1.4.2 while retaining config CFG-USER and replacing executable hash old11 with new42. Semantic rule: Version replacement must not destroy the declared user-owned configuration. FIRST returned “UPGRADE=delete CFG-USER during upgrade”; the private static semantic key accepts “UPGRADE=1.4.1>1.4.2; retain CFG-USER; executable hashnew42”, so it fails. FINAL returned “UPGRADE=1.4.1>1.4.2; retain CFG-USER”, so it fails. No live result was counted."
    },
    {
      "name": "Verify install, invocation, and uninstall",
      "firstPass": false,
      "finalPass": false,
      "evidence": "Public fixture: Acceptance: clean install, tidy --version prints 1.4.2, sample output hash 6cf1, upgrade passes, uninstall removes command but retains CFG-USER. Semantic rule: The full package lifecycle includes installation, behavior, upgrade, and scoped removal. FIRST returned “ACCEPT=archive file exists”; the private static semantic key accepts “ACCEPT=install pass; version1.4.2; sample6cf1; upgrade pass; command removed; CFG-USER retained”, so it fails. FINAL returned “ACCEPT=install pass; version1.4.2; sample6cf1; upgrade pass; command removed”, so it fails. No live result was counted."
    }
  ],
  "initialScore": 0,
  "score": 4,
  "verdict": "failed",
  "recommended": false,
  "whatWorked": [
    "PCLA-5811 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.",
    "Declare package identity and entry point passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.",
    "Include only runtime files also passed its task-specific rule with the final answer left visible."
  ],
  "whatFailed": [
    "Pin production dependencies still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.",
    "Preserve upgrade behavior still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.",
    "Verify install, invocation, and uninstall still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence."
  ],
  "evidencePlan": "Fresh install, upgrade, invocation, and uninstall checks will verify the package lifecycle on a clean system.",
  "evidenceNotes": [
    "PCLA-5811 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.",
    "PCLA-5811's first and final scores were recomputed from parsed RESULT rows: 0 and 2 passes multiplied by two.",
    "PCLA-5811 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Fresh install, upgrade, invocation, and uninstall checks will verify the package lifecycle on a clean system."
  ],
  "limitations": [
    "PCLA-5811 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.",
    "PCLA-5811 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result."
  ]
}