{
  "category": "computers",
  "slug": "computers-inventory-software-components",
  "title": "Mapping a Project's Software Components: All Five Semantic Checks Passed",
  "task": "inventory a project's software components",
  "excerpt": "This completed synthetic Supply Chain field test asked the session to inventory a project's software components, preserved an actual five-row software bill of materials audit, and derived 4/10 then 10/10 from task-specific semantic checks after one failure-only correction.",
  "tool": "Codex multi-agent session",
  "model": "Exact underlying model identifier not disclosed by the Codex session",
  "publishedAt": "2026-02-27T11:00:00+08:00",
  "durationMinutes": 0,
  "testMode": "Synthetic benchmark",
  "inputDisclosure": "All inputs in ISC-0349 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.",
  "runDisclosure": "A Codex multi-agent session generated one text-only first artifact for “inventory a project's software components”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.",
  "prompt": "Complete a bounded synthetic field test for: inventory a project's software components. Focus: Supply Chain.\nSource scenario: The experiment will ask AI to derive a component inventory from a small application with direct and transitive dependencies.\nUse only these five public fictional inputs; the scoring answers are intentionally withheld:\nISC-0349-I1: Fixture project SBOM-17 contains application atlas-ui 3.4.1, direct packages react 19.1.0 and zod 3.25.8, transitive packages scheduler 0.26.0 and picomatch 4.0.2, plus vendored parser vp-2.6.\nISC-0349-I2: Manifest edges are atlas-ui→react, atlas-ui→zod, react→scheduler, and zod→picomatch; vp-2.6 has no package-manager parent.\nISC-0349-I3: package-lock pins picomatch 4.0.2 while the deployment notice lists picomatch 4.0.1; runtime snapshot reports 4.0.2.\nISC-0349-I4: License records: react MIT, zod MIT, scheduler MIT, picomatch MIT, vp-2.6 GPL-3.0-only. Policy permits MIT and sends GPL-3.0-only vendored code to review.\nISC-0349-I5: The approved vp-2.6 archive hash is 8f31c0aa, but the fixture copy hashes to 8f31c0ab; all package-manager integrity fields match.\nReturn a concrete software bill of materials audit with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: A maintainer-authored dependency manifest will verify component names, versions, relationships, and omissions.",
  "firstResult": "SOFTWARE BILL OF MATERIALS AUDIT ISC-0349 — FIRST FROZEN ARTIFACT\nTask: inventory a project's software components. Evaluation focus: Supply Chain. This is a fictional, text-only artifact; it does not report a live action.\nISC-0349-R1 :: RESULT=INVENTORY=atlas-ui3.4.1; direct react19.1.0+zod3.25.8; transitive scheduler0.26.0+picomatch4.0.2; vendored vp-2.6; components6\nISC-0349-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R2 :: RESULT=EDGES=treat every package as a direct dependency\nISC-0349-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R3 :: RESULT=VERSION_EXCEPTION=picomatch deployment-notice4.0.1 versus lock+runtime4.0.2; investigate notice\nISC-0349-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R4 :: RESULT=LICENSE=all six components are MIT and cleared\nISC-0349-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R5 :: RESULT=INTEGRITY=pass because the archive filenames match\nISC-0349-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for ISC-0349; any failed row remains visible because only one correction pass is allowed.",
  "correctionPrompt": "Revise ISC-0349 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:\n- Map declared dependency relationships: input was “Manifest edges are atlas-ui→react, atlas-ui→zod, react→scheduler, and zod→picomatch; vp-2.6 has no package-manager parent.”; first response was “EDGES=treat every package as a direct dependency”.\n- Apply the supplied license policy: input was “License records: react MIT, zod MIT, scheduler MIT, picomatch MIT, vp-2.6 GPL-3.0-only. Policy permits MIT and sends GPL-3.0-only vendored code to review.”; first response was “LICENSE=all six components are MIT and cleared”.\n- Verify the vendored artifact identity: input was “The approved vp-2.6 archive hash is 8f31c0aa, but the fixture copy hashes to 8f31c0ab; all package-manager integrity fields match.”; first response was “INTEGRITY=pass because the archive filenames match”.\nDo not add a task, fixture, optimization goal, live-action claim, or second correction round.",
  "finalResult": "SOFTWARE BILL OF MATERIALS AUDIT ISC-0349 — AFTER ONE FAILURE-ONLY CORRECTION\nTask: inventory a project's software components. Evaluation focus: Supply Chain. This is a fictional, text-only artifact; it does not report a live action.\nISC-0349-R1 :: RESULT=INVENTORY=atlas-ui3.4.1; direct react19.1.0+zod3.25.8; transitive scheduler0.26.0+picomatch4.0.2; vendored vp-2.6; components6\nISC-0349-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R2 :: RESULT=EDGES=atlas-ui>react,zod; react>scheduler; zod>picomatch; vp-2.6=vendored-root\nISC-0349-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R3 :: RESULT=VERSION_EXCEPTION=picomatch deployment-notice4.0.1 versus lock+runtime4.0.2; investigate notice\nISC-0349-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R4 :: RESULT=LICENSE=MIT packages4 allowed; vp-2.6 GPL-3.0-only review; do not mark cleared\nISC-0349-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nISC-0349-R5 :: RESULT=INTEGRITY=managed packages pass; vp-2.6 mismatch 8f31c0ab!=8f31c0aa\nISC-0349-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.\nArtifact boundary: exactly five scored rows were frozen for ISC-0349; any failed row remains visible because only one correction pass is allowed.",
  "checks": [
    {
      "name": "Reconcile the component inventory",
      "firstPass": true,
      "finalPass": true,
      "evidence": "Public fixture: Fixture project SBOM-17 contains application atlas-ui 3.4.1, direct packages react 19.1.0 and zod 3.25.8, transitive packages scheduler 0.26.0 and picomatch 4.0.2, plus vendored parser vp-2.6. Semantic rule: The inventory must retain the application, both dependency levels, the vendored component, exact versions, and total count. FIRST returned “INVENTORY=atlas-ui3.4.1; direct react19.1.0+zod3.25.8; transitive scheduler0.26.0+picomatch4.0.2; vendored vp-2.6; components6”; the private static semantic key accepts “INVENTORY=atlas-ui3.4.1; direct react19.1.0+zod3.25.8; transitive scheduler0.26.0+picomatch4.0.2; vendored vp-2.6; components6”, so it passes. FINAL returned “INVENTORY=atlas-ui3.4.1; direct react19.1.0+zod3.25.8; transitive scheduler0.26.0+picomatch4.0.2; vendored vp-2.6; components6”, so it passes. No live result was counted."
    },
    {
      "name": "Map declared dependency relationships",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: Manifest edges are atlas-ui→react, atlas-ui→zod, react→scheduler, and zod→picomatch; vp-2.6 has no package-manager parent. Semantic rule: The relationship map must distinguish manifest edges from the independently vendored root. FIRST returned “EDGES=treat every package as a direct dependency”; the private static semantic key accepts “EDGES=atlas-ui>react,zod; react>scheduler; zod>picomatch; vp-2.6=vendored-root”, so it fails. FINAL returned “EDGES=atlas-ui>react,zod; react>scheduler; zod>picomatch; vp-2.6=vendored-root”, so it passes. No live result was counted."
    },
    {
      "name": "Flag the seeded version discrepancy",
      "firstPass": true,
      "finalPass": true,
      "evidence": "Public fixture: package-lock pins picomatch 4.0.2 while the deployment notice lists picomatch 4.0.1; runtime snapshot reports 4.0.2. Semantic rule: The two agreeing machine records do not erase the conflicting deployment notice; the discrepancy must remain visible. FIRST returned “VERSION_EXCEPTION=picomatch deployment-notice4.0.1 versus lock+runtime4.0.2; investigate notice”; the private static semantic key accepts “VERSION_EXCEPTION=picomatch deployment-notice4.0.1 versus lock+runtime4.0.2; investigate notice”, so it passes. FINAL returned “VERSION_EXCEPTION=picomatch deployment-notice4.0.1 versus lock+runtime4.0.2; investigate notice”, so it passes. No live result was counted."
    },
    {
      "name": "Apply the supplied license policy",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: License records: react MIT, zod MIT, scheduler MIT, picomatch MIT, vp-2.6 GPL-3.0-only. Policy permits MIT and sends GPL-3.0-only vendored code to review. Semantic rule: The decision must use each disclosed license and the special review rule for vendored GPL-only code. FIRST returned “LICENSE=all six components are MIT and cleared”; the private static semantic key accepts “LICENSE=MIT packages4 allowed; vp-2.6 GPL-3.0-only review; do not mark cleared”, so it fails. FINAL returned “LICENSE=MIT packages4 allowed; vp-2.6 GPL-3.0-only review; do not mark cleared”, so it passes. No live result was counted."
    },
    {
      "name": "Verify the vendored artifact identity",
      "firstPass": false,
      "finalPass": true,
      "evidence": "Public fixture: The approved vp-2.6 archive hash is 8f31c0aa, but the fixture copy hashes to 8f31c0ab; all package-manager integrity fields match. Semantic rule: A one-character checksum difference is an integrity failure and blocks a complete inventory status. FIRST returned “INTEGRITY=pass because the archive filenames match”; the private static semantic key accepts “INTEGRITY=managed packages pass; vp-2.6 mismatch 8f31c0ab!=8f31c0aa; inventory status incomplete” or “INTEGRITY=managed packages pass; vp-2.6 mismatch 8f31c0ab!=8f31c0aa”, so it fails. FINAL returned “INTEGRITY=managed packages pass; vp-2.6 mismatch 8f31c0ab!=8f31c0aa”, so it passes. No live result was counted."
    }
  ],
  "initialScore": 4,
  "score": 10,
  "verdict": "worked",
  "recommended": true,
  "whatWorked": [
    "ISC-0349 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.",
    "Reconcile the component inventory passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.",
    "Map declared dependency relationships also passed its task-specific rule with the final answer left visible."
  ],
  "whatFailed": [
    "The first artifact failed Map declared dependency relationships; the one permitted correction resolved it, but the initial defect remains published."
  ],
  "evidencePlan": "A maintainer-authored dependency manifest will verify component names, versions, relationships, and omissions.",
  "evidenceNotes": [
    "ISC-0349 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.",
    "ISC-0349's first and final scores were recomputed from parsed RESULT rows: 2 and 5 passes multiplied by two.",
    "ISC-0349 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: A maintainer-authored dependency manifest will verify component names, versions, relationships, and omissions."
  ],
  "limitations": [
    "ISC-0349 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.",
    "ISC-0349 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result."
  ]
}