Completed field testSynthetic benchmark
Computers · Evidence record
Least-Privilege Folder Sharing Through AI Guidance: One Verified Gap Remained
This completed synthetic File Permissions field test asked the session to set least-privilege shared-folder permissions, preserved an actual five-row shared-folder least-privilege matrix, and derived 4/10 then 8/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
set least-privilege shared-folder permissions
02 · Scope before score
Test disclosures
Input disclosure
All inputs in SSFP-5116 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “set least-privilege shared-folder permissions”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: set least-privilege shared-folder permissions. Focus: File Permissions.
Source scenario: The experiment will define several user roles and ask AI to map them to a controlled shared-folder hierarchy.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
SSFP-5116-I1: Share TEAM-6 has group Editors={Ava,Bo}, Reviewers={Cy}, and user Dex with no project role. Owner is Admin-O.
SSFP-5116-I2: Editors may list, read, create, modify, and rename inside TEAM-6, but may not change ACLs, ownership, or share settings.
SSFP-5116-I3: Cy must list and read every project file, cannot create or modify, and can add comments only through app comment store CMT-6 outside the filesystem.
SSFP-5116-I4: Parent folder LAB grants Everyone write, but TEAM-6 policy disables that inherited ACE while retaining Admin-O full control and system backup read.
SSFP-5116-I5: Acceptance expects Ava create/rename pass, Cy read pass and write denial, Dex list denial, Admin-O ACL edit pass, with ACL export hash 92c0d6fe.
Return a concrete shared-folder least-privilege matrix with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: An access matrix and positive and negative permission tests will verify every role boundary.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
SHARED-FOLDER LEAST-PRIVILEGE MATRIX SSFP-5116 — FIRST FROZEN ARTIFACT
Task: set least-privilege shared-folder permissions. Evaluation focus: File Permissions. This is a fictional, text-only artifact; it does not report a live action.
SSFP-5116-R1 :: RESULT=ROLES=Admin-O owner; Ava+Bo Editors; Cy Reviewer; Dex none
SSFP-5116-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R2 :: RESULT=EDITORS=full control including ownership
SSFP-5116-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R3 :: RESULT=REVIEWER=list+read; filesystem create+modify denied; comments via CMT-6 only
SSFP-5116-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R4 :: RESULT=INHERITANCE=keep parent Everyone-write
SSFP-5116-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R5 :: RESULT=ACCEPT=Ava can open one file
SSFP-5116-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for SSFP-5116; any failed row remains visible because only one correction pass is allowed.Initial score: 4/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise SSFP-5116 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Grant editors bounded content rights: input was “Editors may list, read, create, modify, and rename inside TEAM-6, but may not change ACLs, ownership, or share settings.”; first response was “EDITORS=full control including ownership”.
- Prevent inherited privilege expansion: input was “Parent folder LAB grants Everyone write, but TEAM-6 policy disables that inherited ACE while retaining Admin-O full control and system backup read.”; first response was “INHERITANCE=keep parent Everyone-write”.
- Run positive and negative matrix checks: input was “Acceptance expects Ava create/rename pass, Cy read pass and write denial, Dex list denial, Admin-O ACL edit pass, with ACL export hash 92c0d6fe.”; first response was “ACCEPT=Ava can open one file”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
SHARED-FOLDER LEAST-PRIVILEGE MATRIX SSFP-5116 — AFTER ONE FAILURE-ONLY CORRECTION
Task: set least-privilege shared-folder permissions. Evaluation focus: File Permissions. This is a fictional, text-only artifact; it does not report a live action.
SSFP-5116-R1 :: RESULT=ROLES=Admin-O owner; Ava+Bo Editors; Cy Reviewer; Dex none
SSFP-5116-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R2 :: RESULT=EDITORS=list+read+create+modify+rename; ACL+owner+share changes denied
SSFP-5116-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R3 :: RESULT=REVIEWER=list+read; filesystem create+modify denied; comments via CMT-6 only
SSFP-5116-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R4 :: RESULT=INHERITANCE=remove Everyone-write on TEAM-6; retain Admin-O full+backup read
SSFP-5116-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
SSFP-5116-R5 :: RESULT=ACCEPT=Ava create+rename pass; Cy read pass/write denied; Dex list denied; Admin-O ACL pass
SSFP-5116-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for SSFP-5116; any failed row remains visible because only one correction pass is allowed.Final score: 8/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Apply the declared group roles | Pass | Pass | Public fixture: Share TEAM-6 has group Editors={Ava,Bo}, Reviewers={Cy}, and user Dex with no project role. Owner is Admin-O. Semantic rule: The access map must reproduce the explicit group membership and unassigned user. FIRST returned “ROLES=Admin-O owner; Ava+Bo Editors; Cy Reviewer; Dex none”; the private static semantic key accepts “ROLES=Admin-O owner; Ava+Bo Editors; Cy Reviewer; Dex none”, so it passes. FINAL returned “ROLES=Admin-O owner; Ava+Bo Editors; Cy Reviewer; Dex none”, so it passes. No live result was counted. |
| Grant editors bounded content rights | Fail | Pass | Public fixture: Editors may list, read, create, modify, and rename inside TEAM-6, but may not change ACLs, ownership, or share settings. Semantic rule: Content collaboration does not imply permission or ownership administration. FIRST returned “EDITORS=full control including ownership”; the private static semantic key accepts “EDITORS=list+read+create+modify+rename; ACL+owner+share changes denied”, so it fails. FINAL returned “EDITORS=list+read+create+modify+rename; ACL+owner+share changes denied”, so it passes. No live result was counted. |
| Keep reviewer access read-only | Pass | Pass | Public fixture: Cy must list and read every project file, cannot create or modify, and can add comments only through app comment store CMT-6 outside the filesystem. Semantic rule: The reviewer path separates read-only share rights from the external comment facility. FIRST returned “REVIEWER=list+read; filesystem create+modify denied; comments via CMT-6 only”; the private static semantic key accepts “REVIEWER=list+read; filesystem create+modify denied; comments via CMT-6 only”, so it passes. FINAL returned “REVIEWER=list+read; filesystem create+modify denied; comments via CMT-6 only”, so it passes. No live result was counted. |
| Prevent inherited privilege expansion | Fail | Pass | Public fixture: Parent folder LAB grants Everyone write, but TEAM-6 policy disables that inherited ACE while retaining Admin-O full control and system backup read. Semantic rule: The child ACL must block the explicitly unsafe inherited entry without removing required administration or backup access. FIRST returned “INHERITANCE=keep parent Everyone-write”; the private static semantic key accepts “INHERITANCE=remove Everyone-write on TEAM-6; retain Admin-O full+backup read”, so it fails. FINAL returned “INHERITANCE=remove Everyone-write on TEAM-6; retain Admin-O full+backup read”, so it passes. No live result was counted. |
| Run positive and negative matrix checks | Fail | Fail | Public fixture: Acceptance expects Ava create/rename pass, Cy read pass and write denial, Dex list denial, Admin-O ACL edit pass, with ACL export hash 92c0d6fe. Semantic rule: Every role boundary and the resulting ACL identity must match the frozen matrix. FIRST returned “ACCEPT=Ava can open one file”; the private static semantic key accepts “ACCEPT=Ava create+rename pass; Cy read pass/write denied; Dex list denied; Admin-O ACL pass; hash92c0d6fe”, so it fails. FINAL returned “ACCEPT=Ava create+rename pass; Cy read pass/write denied; Dex list denied; Admin-O ACL pass”, so it fails. No live result was counted. |
Initial4/10
Final8/10
Verdictworked
RecommendedYes, for this scope
08 · No cleanup by omission
What worked—and what failed
What worked
- SSFP-5116 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Apply the declared group roles passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Grant editors bounded content rights also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- Run positive and negative matrix checks still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
An access matrix and positive and negative permission tests will verify every role boundary.
- SSFP-5116 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- SSFP-5116's first and final scores were recomputed from parsed RESULT rows: 2 and 4 passes multiplied by two.
- SSFP-5116 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: An access matrix and positive and negative permission tests will verify every role boundary.
10 · Boundary of the claim
Limitations
- SSFP-5116 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- SSFP-5116 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.