The short version
The first reconciliation looked tidy, but it marked a one-cent short payment, a payment with no invoice reference, and a $200 partial payment as settled. It scored 4/10.
We returned only those failures once. The corrected result preserved the valid matches, showed $1,180.01 outstanding, kept $1,300.00 unapplied, and scored 10/10.
This test uses fictional USD records. It is not accounting advice, and no accounting platform or bank account was accessed.
The task
We created six invoices and seven payment records. Three payments were exact matches. Four required care: one was a cent short, one had no invoice reference, one was a $200 partial payment, and one referenced an invoice that did not exist.
The five checks required the result to preserve every record, identify exact matches, retain partial balances, isolate missing or unknown references, and reconcile the totals.
The exact first prompt
Reconcile the six invoices and seven payments in the supplied fixture.
Return an invoice table with status, applied payment IDs, and outstanding
balance, plus an unapplied-payment table. Settle an invoice only when the
reference and amount are exact. Do not infer a match from vendor and amount
alone. Preserve every record and show totals.
The full source fixture is preserved in the downloadable JSON evidence record. The run occurred in this repository session; the precise model identifier and per-case elapsed time were not exposed.
First result: 4/10
The first answer accounted for all 13 records and correctly retained the three obvious exact matches. It then made three unsafe assumptions:
- It called the $599.99 payment against a $600 invoice “settled with rounding.”
- It matched a $980 payment to an invoice using vendor and amount even though the reference was blank.
- It treated a $1,000 payment against a $1,200 invoice as fully settled.
That produced an outstanding total of $0 and an unapplied total of $320. Both figures were wrong.
| Criterion | First result |
|---|---|
| Every source record accounted for | 2/2 |
| Exact full matches identified | 2/2 |
| Amount mismatches remain partial | 0/2 |
| Missing and unknown references isolated | 0/2 |
| Exception totals reconcile | 0/2 |
| Total | 4/10 — failed |
The one correction
Your reconciliation preserved the records and found the three exact full
matches, but it broke three rules: PAY-9003 is one cent short and must leave
INV-1003 partially paid; PAY-9005 has no invoice reference and must remain in
manual review; PAY-9006 is a partial payment and must leave 200.00 outstanding.
Recalculate outstanding and unapplied totals without changing valid exact matches.
Final result: 10/10
The corrected result left INV-1003 partially paid with $0.01 outstanding, left INV-1004 open for $980, and left $200 on INV-1006. PAY-9005 remained unapplied for manual review; PAY-9007 remained unapplied because its invoice reference was unknown.
The arithmetic then balanced:
- Outstanding invoices: $0.01 + $980.00 + $200.00 = $1,180.01
- Unapplied payments: $980.00 + $320.00 = $1,300.00
What worked
- The correction did not disturb any exact full match.
- Every source ID remained traceable in the final tables.
- The final output separated “plausible candidate” from “safe automatic match.”
What failed
- The first answer preferred a clean settled list over the explicit matching rules.
- It silently invented a rounding tolerance that the prompt never allowed.
- It hid two partial balances and treated vendor-plus-amount similarity as proof.
Verdict
Worked after one correction. AI organized the reconciliation quickly, but the first result was unsafe because it resolved ambiguity instead of surfacing it. Exact rules and deterministic totals were necessary.
Download the complete JSON evidence record to inspect the fixture, both outputs, correction, checks, and limitations.
Reproduce the test
- Open the JSON evidence record and copy only the fixture and first prompt.
- Keep every result exactly as returned; do not repair a status silently.
- Score the five checks and return only failed checks once.
- Recalculate outstanding and unapplied totals independently.
- Preserve any different result as part of the comparison.