Completed field testSynthetic benchmark
Computers · Evidence record
Camera and Microphone Permissions: An AI Lockdown Brief: All Five Semantic Checks Passed
This completed synthetic Device Privacy field test asked the session to lock down camera and microphone access, preserved an actual five-row camera and microphone permission matrix, and derived 6/10 then 10/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
lock down camera and microphone access
02 · Scope before score
Test disclosures
Input disclosure
All inputs in LDCM-2709 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “lock down camera and microphone access”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: lock down camera and microphone access. Focus: Device Privacy.
Source scenario: The experiment will ask AI to configure per-application privacy permissions while retaining approved calling functions.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
LDCM-2709-I1: App matrix permits MeetBox camera and microphone while a call is active; MeetBox background access must be off. Its signed app ID is mbx-2.
LDCM-2709-I2: NoteNest requests microphone despite its declared features being text notes and local search; it requests no camera access.
LDCM-2709-I3: Browser profile B1 permits camera and microphone for call.example.test only; news.example.test is denied and the default for new sites is Ask.
LDCM-2709-I4: System microphone M1 is the only input device and must remain enabled at OS level so MeetBox can work; lockdown is through app permissions, not disabling M1.
LDCM-2709-I5: Acceptance trials are MeetBox active call camera/mic pass, MeetBox background both denied, NoteNest mic denied, call site pass, news site denied, and sensor indicator appears for allowed trials.
Return a concrete camera and microphone permission matrix with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Permission-state captures and allowed and denied access tests will verify every requested boundary.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
CAMERA AND MICROPHONE PERMISSION MATRIX LDCM-2709 — FIRST FROZEN ARTIFACT
Task: lock down camera and microphone access. Evaluation focus: Device Privacy. This is a fictional, text-only artifact; it does not report a live action.
LDCM-2709-R1 :: RESULT=MEETBOX=mbx-2 camera+mic during active call; background denied
LDCM-2709-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R2 :: RESULT=NOTENEST=microphone denied; camera denied; text+search unaffected
LDCM-2709-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R3 :: RESULT=BROWSER=call.example.test allow camera+mic; news.example.test deny; default Ask
LDCM-2709-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R4 :: RESULT=DEVICE=disable M1 globally
LDCM-2709-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R5 :: RESULT=ACCEPT=camera light appears once
LDCM-2709-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for LDCM-2709; any failed row remains visible because only one correction pass is allowed.Initial score: 6/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise LDCM-2709 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Preserve the approved MeetBox input capability: input was “System microphone M1 is the only input device and must remain enabled at OS level so MeetBox can work; lockdown is through app permissions, not disabling M1.”; first response was “DEVICE=disable M1 globally”.
- Verify allowed and denied sensor states: input was “Acceptance trials are MeetBox active call camera/mic pass, MeetBox background both denied, NoteNest mic denied, call site pass, news site denied, and sensor indicator appears for allowed trials.”; first response was “ACCEPT=camera light appears once”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
CAMERA AND MICROPHONE PERMISSION MATRIX LDCM-2709 — AFTER ONE FAILURE-ONLY CORRECTION
Task: lock down camera and microphone access. Evaluation focus: Device Privacy. This is a fictional, text-only artifact; it does not report a live action.
LDCM-2709-R1 :: RESULT=MEETBOX=mbx-2 camera+mic during active call; background denied
LDCM-2709-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R2 :: RESULT=NOTENEST=microphone denied; camera denied; text+search unaffected
LDCM-2709-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R3 :: RESULT=BROWSER=call.example.test allow camera+mic; news.example.test deny; default Ask
LDCM-2709-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R4 :: RESULT=DEVICE=M1 OS enabled; enforce app/site permissions; MeetBox call remains possible
LDCM-2709-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
LDCM-2709-R5 :: RESULT=ACCEPT=MeetBox foreground2 pass/background2 deny; NoteNest mic deny; call site2 pass; news2 deny; indicator on allowed
LDCM-2709-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for LDCM-2709; any failed row remains visible because only one correction pass is allowed.Final score: 10/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Retain the approved video-call boundary | Pass | Pass | Public fixture: App matrix permits MeetBox camera and microphone while a call is active; MeetBox background access must be off. Its signed app ID is mbx-2. Semantic rule: The app needs both sensors only in the declared foreground call context. FIRST returned “MEETBOX=mbx-2 camera+mic during active call; background denied”; the private static semantic key accepts “MEETBOX=mbx-2 camera+mic during active call; background denied”, so it passes. FINAL returned “MEETBOX=mbx-2 camera+mic during active call; background denied”, so it passes. No live result was counted. |
| Deny the unneeded note-app permissions | Pass | Pass | Public fixture: NoteNest requests microphone despite its declared features being text notes and local search; it requests no camera access. Semantic rule: Requested access is not necessary when no disclosed feature uses either sensor. FIRST returned “NOTENEST=microphone denied; camera denied; text+search unaffected”; the private static semantic key accepts “NOTENEST=microphone denied; camera denied; text+search unaffected”, so it passes. FINAL returned “NOTENEST=microphone denied; camera denied; text+search unaffected”, so it passes. No live result was counted. |
| Constrain browser site permissions | Pass | Pass | Public fixture: Browser profile B1 permits camera and microphone for call.example.test only; news.example.test is denied and the default for new sites is Ask. Semantic rule: Per-site exceptions and the new-site default must reproduce the stated origin boundaries. FIRST returned “BROWSER=call.example.test allow camera+mic; news.example.test deny; default Ask”; the private static semantic key accepts “BROWSER=call.example.test allow camera+mic; news.example.test deny; default Ask”, so it passes. FINAL returned “BROWSER=call.example.test allow camera+mic; news.example.test deny; default Ask”, so it passes. No live result was counted. |
| Preserve the approved MeetBox input capability | Fail | Pass | Public fixture: System microphone M1 is the only input device and must remain enabled at OS level so MeetBox can work; lockdown is through app permissions, not disabling M1. Semantic rule: Least access here requires application scoping while retaining the sole approved-call device. FIRST returned “DEVICE=disable M1 globally”; the private static semantic key accepts “DEVICE=M1 OS enabled; enforce app/site permissions; MeetBox call remains possible”, so it fails. FINAL returned “DEVICE=M1 OS enabled; enforce app/site permissions; MeetBox call remains possible”, so it passes. No live result was counted. |
| Verify allowed and denied sensor states | Fail | Pass | Public fixture: Acceptance trials are MeetBox active call camera/mic pass, MeetBox background both denied, NoteNest mic denied, call site pass, news site denied, and sensor indicator appears for allowed trials. Semantic rule: The full positive and negative matrix plus visible use indicator establish the requested boundary. FIRST returned “ACCEPT=camera light appears once”; the private static semantic key accepts “ACCEPT=MeetBox foreground2 pass/background2 deny; NoteNest mic deny; call site2 pass; news2 deny; indicator on allowed”, so it fails. FINAL returned “ACCEPT=MeetBox foreground2 pass/background2 deny; NoteNest mic deny; call site2 pass; news2 deny; indicator on allowed”, so it passes. No live result was counted. |
Initial6/10
Final10/10
Verdictworked
RecommendedYes, for this scope
08 · No cleanup by omission
What worked—and what failed
What worked
- LDCM-2709 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Retain the approved video-call boundary passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Deny the unneeded note-app permissions also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- The first artifact failed Preserve the approved MeetBox input capability; the one permitted correction resolved it, but the initial defect remains published.
09 · Inspectable record
Evidence notes
Permission-state captures and allowed and denied access tests will verify every requested boundary.
- LDCM-2709 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- LDCM-2709's first and final scores were recomputed from parsed RESULT rows: 3 and 5 passes multiplied by two.
- LDCM-2709 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Permission-state captures and allowed and denied access tests will verify every requested boundary.
10 · Boundary of the claim
Limitations
- LDCM-2709 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- LDCM-2709 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.