Adding a new document type to a production extraction system used to mean a labeling project: gather samples, annotate fields, train, measure, ship weeks later. Today you can describe the fields you want in plain language and get a usable extraction from a layout the model has never seen, in an afternoon.
That is a real change, and it explains why nearly every document processing roadmap has been rewritten. It also explains the wall teams hit next, which arrives as a deceptively simple question from operations, risk, or audit: how do you know it got it right?
It is the right question, and it has no model-shaped answer. An extraction that looks correct in a demo and one you can act on unchecked are different things, and that distance is not closed by a better model. It is closed by where the model sits and what runs around it. RAND found that roughly 80% of AI projects fail to deliver their intended business value — and very few of those are model quality failures. They are architectural failures.
What foundation models genuinely unlocked
The wins are real. The cold start problem is largely solved — a model that has never seen your document type still produces a credible first extraction, turning a six-week gate into a day-one starting point. The long tail is reachable: types arriving a few hundred times a year were economically stranded, too few to justify training data and too many to handle manually. And semantic work is in scope — summarizing a correspondence thread, reconciling values across documents, flagging that a form contradicts a policy on record.
None of what follows argues against these models. We build with them: Hyperscience develops its own vision language model, ORCA, and supports third-party frontier models from whichever provider already holds a customer’s precommitted spend. The argument is about where they sit.
Why “point a model at the PDF” stalls
Sending raw pages straight to a multimodal model works in a notebook. Four things break it at scale.
Accuracy degrades on noisy input. When a model has to find the signal inside broken tables, running headers, and irrelevant pages, reasoning quality falls off — and that is where hallucination risk concentrates, in text generated that was never on the page. This is not a problem that a larger model reliably solves. Reading a page accurately and reasoning well about its contents are different skills, and the second does not imply the first.
You cannot threshold what you cannot calibrate. A production system runs on a number: the confidence score that decides whether a field is auto-accepted or sent to a person. Asking a model how sure it is does not produce that number — in a study of nine LLMs, self-reported confidence predicted correctness at close to chance (AUROC 0.51–0.70), and in six of the nine, more than half of the wrong answers still cleared the model’s own optimal confidence threshold. Token probabilities do better, but a score you can safely auto-accept has to come from outside the model: we ground ours in observed accuracy, routing a continuous sample of output to human QA and letting automation proceed only when the measured number clears the threshold the business process requires.
Decisions have to be explainable later. A document processed in March may need explaining a year on: which value came from which position on which page, under what configuration. That is now regulation, the EU AI Act requires event logging for high-risk systems (Article 12), meaningful human oversight (Article 14), and six months minimum log retention (Article 19). Free text with no link back to the page does not satisfy that.
You pay the same rate whatever the task needs. Send every page to a frontier model and a date lookup is billed at the same per-token price as multi-document reasoning. Feed raw pages and you compound the problem by paying to read the layout noise that a cleaner payload would have dropped. While inference gets cheaper, Stanford’s AI Index shows a 280-fold drop at constant quality over two years, that’s spread unevenly, from 9-fold to 900-fold a year depending on the task, and there is no guarantee providers keep a cheap tier for the simple work. A single-model pipeline simply cannot exploit the spread; routing each task to the smallest model that clears the accuracy bar can.
Stage one: establish what the document is
Before a model reasons about a document, the system should know what that document is and where its structured content sits. Classification and field-level extraction on known types are what purpose-built models are good at: fast, cheap, deterministic enough to regression-test, and, critically, able to return a calibrated confidence per field. Hyperscience runs these models alongside LLMs in one platform, and because they are CPU-based, that layer runs at enterprise scale without a GPU bill on every page. It is not a preprocessing chore; it is what makes everything downstream measurable.
Stage two: hand over a clean payload
Instead of passing raw page text to the model, pass the document as structured data — JSON or Markdown holding just the fields the task needs, with the layout noise removed. Running headers, broken table borders, and pages irrelevant to the task are gone before the model sees anything.
This matters because a model reconstructing what a page was trying to say has less attention left for the task you actually asked about. Structured input is the format models handle most reliably. Turning complex back-office documents into LLM- and RAG-ready data is the core of what the Hypercell platform does.
One caveat: condensing can remove context the task needed — a summary may depend on narrative text that field extraction discards. Condense per task, not globally, and keep the path back to the source page.
Stage three: route by complexity and by confidence
Routing by complexity is the obvious half: classification and standard extraction go to small specialized models; deep reasoning and cross-document work go to frontier models, receiving a clean payload rather than raw pages. We ship this as inference layering — orchestration that balances each workload across CPU-based models, GPU-based VLMs like ORCA, and third-party frontier models, choosing the cheapest tier that clears the accuracy bar. One of the largest US mortgage providers uses exactly this mix for income and asset verification, because no single tier was right for every field on the page.
The half that gets skipped is routing by confidence. Extractions below threshold should neither go straight to a human queue nor be silently accepted. They are the best candidates for a second, more capable pass — a frontier model looking at the original page region for exactly the fields in doubt.
Answering the question
All of this is structure. It earns its keep only if it lets you answer the question this post opened with — and “the model is usually right” is not an answer anyone in risk or audit will accept.
Ground every output. An extracted value should carry a pointer to where it came from on the page. That makes review fast, audit possible, and hallucination visible instead of invisible.
Set the accuracy target first, then let the system meet it. This is why the Hyperscience VLM framework is built to be accuracy-harnessed: you define the threshold your process requires, and the platform brings in a person whenever a result falls short. The target is an input, not something you discover afterward from a sample audit.
Treat human review as designed behavior, not failure. Spend reviewer attention only where the system is genuinely uncertain, and capture corrections as a training signal. A pipeline that improves from its own review queue compounds; one that does not, plateaus. Plan for the shift from human-in-the-loop to human-on-the-loop as automation climbs.
Measure automation and error rate together. Evaluate on held-out data at the field level, and never report straight-through processing alone. Either number is gameable by itself: you can hit 99% accuracy by sending everything to a human, or full automation by accepting everything.
Enforce business rules outside the model. Schema validation, cross-field arithmetic, and reference-data checks are cheap, deterministic, and catch a class of errors no prompt reliably prevents. If a total does not equal the sum of its line items, you do not need a model to tell you something is wrong.
The governance layer underneath matters as much as the models. Model versioning, traceability, and audit history are what let you answer the question a year later rather than only on the day of processing. And for regulated environments, controls like FedRAMP High authorization and PII redaction and masking are the difference between an architecture you can describe and one you can deploy.
Decouple to scale
Separating extraction from reasoning is not only about efficiency. It is about optionality. Models improve faster than most procurement cycles, and if your system’s correctness depends on the behavior of one model, every upgrade becomes a migration and a revalidation. If extraction, confidence scoring, routing, and review are separable, adopting a better model is a configuration change.
That is why we built the platform model-agnostic rather than betting on one provider. Whichever frontier models you have already standardized on — and most enterprises have, for reasons of cloud contract, data residency, or procurement rather than benchmarks — they plug into the same orchestration, thresholding, and observability layer as our own ORCA. Build so that the model is the part you can replace, and your answer to how do you know it got it right is the part you keep.