Completed field testSynthetic benchmark
Computers · Evidence record
Browser Notification Spam: A Reversible Cleanup Task for AI: The Correction Reached 6/10
This completed synthetic Browser Hygiene field test asked the session to remove unwanted browser notification spam, preserved an actual five-row browser notification permission cleanup, and derived 2/10 then 6/10 from task-specific semantic checks after one failure-only correction.
- Exact prompts and outputs
- One correction only
- Synthetic inputs disclosed
01 · The assignment
The task
remove unwanted browser notification spam
02 · Scope before score
Test disclosures
Input disclosure
All inputs in RNS-4546 are fictional and appear verbatim in the exact prompt. Hidden scoring answers were not shown to the response generator. No personal, production, customer, learner, or device data was used. Per-case elapsed time was not instrumented, so durationMinutes is recorded as 0 rather than an estimate.
Run disclosure
A Codex multi-agent session generated one text-only first artifact for “remove unwanted browser notification spam”. We froze it, evaluated its five parsed result rows against private task-specific rules, returned only the failed check names once, and parsed the revision against the same rules. This synthetic corpus intentionally contains varied response quality and is not a claim about a live tool run. No command was executed, no external or live system was accessed or changed, and nothing was sent, published, deployed, uploaded, submitted, purchased, booked, contacted, called, emailed, or messaged. No external, live, or production action occurred. Per-case elapsed time was not instrumented during the batch session.
- Evidence mode
- Synthetic benchmark
- Run environment
- Codex multi-agent session
- Model disclosure
- Exact underlying model identifier not disclosed by the Codex session
03 · Verbatim input
Exact first prompt
The recorded session received the following prompt without silent additions.
Complete a bounded synthetic field test for: remove unwanted browser notification spam. Focus: Browser Hygiene.
Source scenario: The experiment will reproduce unwanted notification permissions in a disposable profile and request a reversible cleanup plan.
Use only these five public fictional inputs; the scoring answers are intentionally withheld:
RNS-4546-I1: Browser profile NP-3 lists alerts.weather.example Allow, coupons-spam.invalid Allow, calendar.example Ask, and default Ask. Spam samples S1-S14 all originate from coupons-spam.invalid.
RNS-4546-I2: coupons-spam.invalid has service worker SW-C9 and notification permission Allow; clearing page history alone leaves both records active.
RNS-4546-I3: Weather control W2 sends one severe-weather test alert through worker SW-W1; this origin is explicitly approved and its storage hash is c81a.
RNS-4546-I4: Baseline export NP3-B records four permission rows and workers SW-C9 and SW-W1 with hash 0ab311c2; no live browser changes are authorized.
RNS-4546-I5: Synthetic post-state expects coupons permission Block and SW-C9 absent, spam events 0/14 delivered, W2 delivered once, calendar still Ask, and default Ask.
Return a concrete browser notification permission cleanup with exactly five result rows, assumptions visible, and no claim that a command, message, booking, transaction, teaching session, or live-system change occurred. Evidence target: Permission audits and controlled notification tests will verify removal while preserving approved site settings.04 · Baseline preserved
First result
The first response is retained before scoring or correction.
BROWSER NOTIFICATION PERMISSION CLEANUP RNS-4546 — FIRST FROZEN ARTIFACT
Task: remove unwanted browser notification spam. Evaluation focus: Browser Hygiene. This is a fictional, text-only artifact; it does not report a live action.
RNS-4546-R1 :: RESULT=ORIGIN=disable notifications for every origin
RNS-4546-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R2 :: RESULT=CLEANUP=clear browsing history and leave SW-C9 registered
RNS-4546-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R3 :: RESULT=PRESERVE=remove SW-W1 with every other worker
RNS-4546-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R4 :: RESULT=ROLLBACK=freeze NP3-B rows4+workers2 hash0ab311c2; live changes0
RNS-4546-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R5 :: RESULT=ACCEPT=no pop-up appears for five minutes
RNS-4546-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for RNS-4546; any failed row remains visible because only one correction pass is allowed.Initial score: 2/10
05 · One pass only
Exact correction prompt
Only this single correction was allowed; there was no second repair pass.
Revise RNS-4546 once. Preserve every row that passed. Reconsider only the failed semantic checks below using the original fictional inputs; the private expected answer is not supplied:
- Identify the unwanted origin: input was “Browser profile NP-3 lists alerts.weather.example Allow, coupons-spam.invalid Allow, calendar.example Ask, and default Ask. Spam samples S1-S14 all originate from coupons-spam.invalid.”; first response was “ORIGIN=disable notifications for every origin”.
- Account for the service worker: input was “coupons-spam.invalid has service worker SW-C9 and notification permission Allow; clearing page history alone leaves both records active.”; first response was “CLEANUP=clear browsing history and leave SW-C9 registered”.
- Preserve the approved weather alerts: input was “Weather control W2 sends one severe-weather test alert through worker SW-W1; this origin is explicitly approved and its storage hash is c81a.”; first response was “PRESERVE=remove SW-W1 with every other worker”.
- Specify positive and negative retests: input was “Synthetic post-state expects coupons permission Block and SW-C9 absent, spam events 0/14 delivered, W2 delivered once, calendar still Ask, and default Ask.”; first response was “ACCEPT=no pop-up appears for five minutes”.
Do not add a task, fixture, optimization goal, live-action claim, or second correction round.06 · Corrected output
Corrected final result
BROWSER NOTIFICATION PERMISSION CLEANUP RNS-4546 — AFTER ONE FAILURE-ONLY CORRECTION
Task: remove unwanted browser notification spam. Evaluation focus: Browser Hygiene. This is a fictional, text-only artifact; it does not report a live action.
RNS-4546-R1 :: RESULT=ORIGIN=coupons-spam.invalid causes14/14 spam samples; weather remains Allow; calendar+default remain Ask
RNS-4546-R1-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R2 :: RESULT=CLEANUP=propose revoke coupons permission+remove SW-C9 registration; history-only insufficient
RNS-4546-R2-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R3 :: RESULT=PRESERVE=alerts.weather.example Allow+SW-W1; W2 remains eligible
RNS-4546-R3-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R4 :: RESULT=ROLLBACK=freeze NP3-B rows4+workers2 hash0ab311c2; live changes0
RNS-4546-R4-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
RNS-4546-R5 :: RESULT=ACCEPT=coupons Block; SW-C9 absent; spam0/14; W2 1/1; calendar Ask
RNS-4546-R5-NOTE :: The proposed technical step is static and bounded; no command output or successful device change is invented.
Artifact boundary: exactly five scored rows were frozen for RNS-4546; any failed row remains visible because only one correction pass is allowed.Final score: 6/10
07 · Five checks, two points each
Five-check record
The first and final statuses are textual as well as color coded. Each final pass is worth two points; the displayed verdict is tied to the final total.
| Check | First | Final | Evidence |
|---|---|---|---|
| Identify the unwanted origin | Fail | Pass | Public fixture: Browser profile NP-3 lists alerts.weather.example Allow, coupons-spam.invalid Allow, calendar.example Ask, and default Ask. Spam samples S1-S14 all originate from coupons-spam.invalid. Semantic rule: The evidence isolates one unwanted origin while defining three settings that must remain unchanged. FIRST returned “ORIGIN=disable notifications for every origin”; the private static semantic key accepts “ORIGIN=coupons-spam.invalid causes14/14 spam samples; weather remains Allow; calendar+default remain Ask”, so it fails. FINAL returned “ORIGIN=coupons-spam.invalid causes14/14 spam samples; weather remains Allow; calendar+default remain Ask”, so it passes. No live result was counted. |
| Account for the service worker | Fail | Pass | Public fixture: coupons-spam.invalid has service worker SW-C9 and notification permission Allow; clearing page history alone leaves both records active. Semantic rule: A complete reversible plan addresses both the permission grant and persistent worker named in the fixture. FIRST returned “CLEANUP=clear browsing history and leave SW-C9 registered”; the private static semantic key accepts “CLEANUP=propose revoke coupons permission+remove SW-C9 registration; history-only insufficient”, so it fails. FINAL returned “CLEANUP=propose revoke coupons permission+remove SW-C9 registration; history-only insufficient”, so it passes. No live result was counted. |
| Preserve the approved weather alerts | Fail | Fail | Public fixture: Weather control W2 sends one severe-weather test alert through worker SW-W1; this origin is explicitly approved and its storage hash is c81a. Semantic rule: Removing spam must not disturb the independently approved alert channel or its state. FIRST returned “PRESERVE=remove SW-W1 with every other worker”; the private static semantic key accepts “PRESERVE=alerts.weather.example Allow+SW-W1; W2 remains eligible; storagec81a unchanged”, so it fails. FINAL returned “PRESERVE=alerts.weather.example Allow+SW-W1; W2 remains eligible”, so it fails. No live result was counted. |
| Keep a rollback record | Pass | Pass | Public fixture: Baseline export NP3-B records four permission rows and workers SW-C9 and SW-W1 with hash 0ab311c2; no live browser changes are authorized. Semantic rule: A proposed cleanup remains auditable only with the exact pre-change permission and worker inventory. FIRST returned “ROLLBACK=freeze NP3-B rows4+workers2 hash0ab311c2; live changes0”; the private static semantic key accepts “ROLLBACK=freeze NP3-B rows4+workers2 hash0ab311c2; live changes0”, so it passes. FINAL returned “ROLLBACK=freeze NP3-B rows4+workers2 hash0ab311c2; live changes0”, so it passes. No live result was counted. |
| Specify positive and negative retests | Fail | Fail | Public fixture: Synthetic post-state expects coupons permission Block and SW-C9 absent, spam events 0/14 delivered, W2 delivered once, calendar still Ask, and default Ask. Semantic rule: The result must test the blocked origin while proving the wanted and undecided settings remain functional. FIRST returned “ACCEPT=no pop-up appears for five minutes”; the private static semantic key accepts “ACCEPT=coupons Block; SW-C9 absent; spam0/14; W2 1/1; calendar Ask; default Ask”, so it fails. FINAL returned “ACCEPT=coupons Block; SW-C9 absent; spam0/14; W2 1/1; calendar Ask”, so it fails. No live result was counted. |
Initial2/10
Final6/10
Verdictmixed
RecommendedNo
08 · No cleanup by omission
What worked—and what failed
What worked
- RNS-4546 preserved the exact public prompt, first artifact, failure-only correction, final artifact, and independently derived semantic check results.
- Identify the unwanted origin passed because the parsed final answer matched the private fixture rule rather than merely repeating an input identifier.
- Account for the service worker also passed its task-specific rule with the final answer left visible.
What failed or remained weak
- Preserve the approved weather alerts still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
- Specify positive and negative retests still failed after the only permitted correction; its final value and expected semantic rule remain quoted in the evidence.
09 · Inspectable record
Evidence notes
Permission audits and controlled notification tests will verify removal while preserving approved site settings.
- RNS-4546 stores the public five-input fixture separately from the private semantic answer strings quoted only after evaluation.
- RNS-4546's first and final scores were recomputed from parsed RESULT rows: 1 and 3 passes multiplied by two.
- RNS-4546 preserves every unresolved final mismatch; the source evidence plan remains unexecuted because this is a static synthetic benchmark: Permission audits and controlled notification tests will verify removal while preserving approved site settings.
10 · Boundary of the claim
Limitations
- RNS-4546 is a static synthetic response benchmark, not evidence that the task succeeded with a real person, organization, device, account, service, or environment.
- RNS-4546 uses one Codex multi-agent transcript and a private deterministic fixture key; another prompt, model, evaluator, or real-world input could produce a different result.