A document extraction demo usually runs on a clean, digital PDF. Production runs on a photo taken at an angle, a supplier invoice with a rubber stamp over the total, a scanned purchase order in Bangla and English, and a bank statement exported with merged cells. The model is rarely the weak point. The workflow around it is.
This guide works through the problems that show up after go-live, in the order they usually appear, and pairs each with a cause and a fix that keeps a human in control of anything that affects money or compliance.
Symptom 1: fields are wrong on scanned or photographed documents
Cause: the capture step is uncontrolled. Images arrive through WhatsApp, which compresses them, or through a phone camera held at an angle under poor light. Skew, shadow and low resolution remove detail before the model sees it.
Fix: set a capture standard and enforce it in the intake channel. Ask for the original PDF where one exists, require a minimum resolution, detect blur and skew before extraction, and reject documents that fail rather than processing them badly. If suppliers send by messaging app, provide a form or email address that accepts files without recompression.
Symptom 2: line items extract but the totals do not match
Cause: the extraction step is treated as the whole workflow. No arithmetic or business rule checks the output.
Fix: add a validation layer that does not depend on the model. Sum the line items, apply the tax rule, compare with the stated total, and flag any difference beyond a defined tolerance. Validate dates against the document period, check that the purchase order exists, and compare the supplier against the approved vendor list. Extraction produces candidates; validation produces records.
Symptom 3: confidence scores are ignored
Cause: the output schema drops the model's confidence, or the workflow sends everything straight through because the demo did.
Fix: carry a confidence value per field, not just per document, and set thresholds by field. An invoice number may tolerate a lower threshold than a bank account number or a total. Route low-confidence fields to a review queue, and record the reviewer's correction so the threshold can be tuned with evidence.
Symptom 4: the exception queue grows every week
Cause: exceptions are queued without priority, ownership or a service target. Nobody is accountable for clearing them, so the queue becomes a second backlog beside the manual process it was meant to replace.
Fix: triage by value and risk. High-value invoices and anything touching payment details go first. Set a daily clearing window, assign a named owner, and measure the age of the oldest item. If the queue consistently exceeds capacity, the answer is not more reviewers; it is fixing the document types that generate most exceptions.
Symptom 5: the same document is processed twice
Cause: retries, duplicate uploads and multiple intake channels. The workflow has no document identity.
Fix: compute a fingerprint from the file content and check it before extraction. Use an idempotency key on the downstream write so a retry cannot create a second invoice or payment. Reconcile counts between the intake channel and the accounting system daily.
Symptom 6: mixed-language, stamped and handwritten documents fail
Cause: a general-purpose model was chosen for documents that need specialised handling. Stamps overlap text, handwriting varies, and Bangla and English appear in the same table.
Fix: classify the document type and language before extraction, then use the appropriate model or template. For repeat suppliers, a fixed template is often more accurate and cheaper than a general model. Keep a human review step for handwritten amounts and anything obscured by a stamp, and never let a low-confidence handwritten total post automatically.
Symptom 7: there is no audit trail
Cause: only the extracted record is stored. The original file, model version, prompt or template, confidence values and reviewer are lost.
Fix: store the original document, the raw extraction output, the validation result, the model or template version, the reviewer identity and the final posted record. Link them with one document ID. When a supplier disputes an amount six months later, this is the difference between a five-minute answer and a rebuild.
Symptom 8: cost and lock-in creep
Cause: pricing is per page, per field or per API call, and volume grows quietly. Output is stored in a vendor-specific format that nothing else can read.
Fix: measure pages per month by document type and track cost per successfully posted document, not cost per API call. Store a normalised output in your own database. Keep the raw response for audit, but never let downstream systems depend on the vendor's schema.
Diagnostic sequence
- Sample fifty recent documents across suppliers, channels and languages.
- Score field-level accuracy against a human-verified answer key.
- Group errors by document type and capture channel to find the pattern.
- Check validation coverage: which rules would have caught each error?
- Measure queue age, rework rate and cost per posted document.
- Fix the largest error group, then re-run the same sample and compare.
Metrics
- Field-level accuracy by document type and by field.
- Straight-through rate, meaning documents posted without human touch.
- Exception rate and age of the oldest exception.
- Duplicate rate on downstream writes.
- Cost per successfully posted document.
- Percentage of documents with a complete audit trail.
Local conditions
Bangladeshi document workflows mix supplier invoices, delivery challans, purchase orders, bank statements, mobile money statements and trade licences. Many arrive as photos through messaging apps, and a large share are bilingual. Design for that reality: accept the channel people already use, convert it into a controlled intake, and keep a review step for the fields that carry financial or legal weight.
Where this fits in our work
We design document workflows with extraction, validation and human review through Business Automation, with scope and pricing on the pricing page. For the data groundwork, read the AI-ready data foundations guide.
Sources
- NIST, AI Risk Management Framework, accessed 17 November 2026.
- OWASP, Top 10 for Large Language Model Applications, accessed 17 November 2026.
- Microsoft Learn, Azure AI Document Intelligence, accessed 17 November 2026.
- Google Cloud, Document AI documentation, accessed 17 November 2026.
- ISO/IEC 42001, AI management systems, accessed 17 November 2026.
Responses (0)
No comments yet. Add the first one below.
Leave a response
Your email address will not be published. Required fields are marked *.