All projects
Document AI
LedgerLens
LLM invoice extraction with human review
Extracts invoices into validated structured data, auto-approves only clean invoices and routes every anomaly to a reviewer dashboard.
PythonGeminiFastAPIPostgreSQLHTMXOpenTelemetryDockerpytest
0
false auto-approvals
held-out
11/11
planted anomalies caught
1.00
line-item F1
$0.0019
cost per invoice
The problem
Auto-approving invoices saves time only if it never approves a wrong one. A single false approval costs more than all the time saved.
Architecture
- Invoice
- LLM extraction
- Schema validation
- Business rules
- Auto-approve or review
- Reviewer UI
How it works
- The model fills a flat schema that code converts and validates.
- Arithmetic and business checks (line totals, duplicates, caps) decide approval, not the model.
- Reviewer UI shows the reasons and lets a human approve or reject.
Safety & reliability
- Hard gate: zero false auto-approvals before any release.
- Frozen held-out test set, run once and logged.
Results
Held-out test (50 invoices, run once)
Percent of invoices; higher is better
Valid structured output100%
Header field accuracy100%
Anomalies caught11/11
Clean invoices auto-approved92.1%
What I learned
An early 20-invoice run produced one false auto-approval. It drove stricter validation and a new prompt version before the held-out run.