The client
Our client is a European health tech startup offering healthcare as a service: subscription care that brings video consultations, health assessments, wearable data and long-term condition tracking into one platform. A central part of that promise is the patient's lab history. Patients upload the results they get from their laboratory as PDFs, and the platform turns them into structured data their doctor can actually work with: every test identified, with proper units, normal ranges and a trend line across years of results.
The catch is that those PDFs come from whichever laboratory the patient happens to use. Different countries, different layouts, different languages, different names for the same test. The platform had to read all of them.
The problem: lab reports are written for people
A lab report is designed to be read by a human on paper. Every laboratory chain formats it differently. Test names are printed in the local language, in English, or in both at once, sometimes switching alphabet mid-word. One lab prints "СУЕ", another "ERS", a third "Westergreen", and all three mean the same sedimentation rate test. Units differ between labs for the same analyte. Section headings that tell a human "this block is urine" are easy for an OCR step to miss entirely.
The first-generation reader, a single model with a fuzzy text-similarity matcher, recognised about 74% of test rows. Worse than the misses were the hits: hemoglobin was sometimes filed as HbA1c, a diabetes marker; triglycerides landed on thyroglobulin, a thyroid protein, because the names look alike to a string comparison. In a medical product that class of error is disqualifying. A wrong value in a patient's chart does more damage than a missing one.
A multilayered agentic pipeline
We rebuilt extraction as a sequence of narrow agents, each with one job and a verifiable output, instead of one model asked to do everything at once.
The first layer is OCR plus structured extraction: a model reads the PDF and returns the laboratory name, the report date, and one record per test row with the raw printed label, the value, the comparator, the unit string and the reference range. A classification step then tags each row with its specimen: blood, urine, stool, and so on, inferred from the document's own structure.
The second layer is a matching cascade that links each extracted row to the platform's canonical catalogue of test definitions. Stage one scores the label against every definition's names, codes and multilingual keywords. If that produces no confident hit, stage two consults a synonym dictionary: aliases scraped from laboratory catalogues, aliases confirmed by human reviewers, and aliases harvested from medical references. A normalisation step folds Cyrillic and Latin homoglyphs together first, so "АсАТ" and "ASAT" collide the way a human reader would expect.
Every candidate, at every stage, has to pass clinical safety gates before it is accepted. The specimen gate is hard: a blood-only test definition can never be matched to a urine row, whatever the text similarity says. The unit gate is a tiebreaker: when two candidates score nearly the same, the one whose unit dimension agrees with the printed unit wins, which is how the pipeline tells Lymphocyte% from Lymphocyte# reliably.
The result of all this is a confidence ladder rather than a guess. High-confidence rows are stored automatically. Mid-confidence rows go to a human review queue carrying their best candidate. Low-confidence rows are marked unmatched and surfaced. The pipeline never quietly picks a plausible answer, and every row that needed judgement keeps a full trace of the signals that fired, so a reviewer can see exactly why it landed where it did.
Because no layer depends on a specific report layout or language, the pipeline works on reports it has never seen. A new laboratory, from any country, costs at most a synonym set; the machinery stays the same.
Sovereign by design: GDPR-compliant AI
Patient lab reports are sensitive health data, so the AI architecture was drawn around a hard boundary. At runtime, while a patient's document is being processed, the pipeline calls exactly one AI provider: Mistral, a European company, for OCR and structured extraction. No patient document or extracted result is ever sent to a US model provider during processing.
Other models do contribute, but only offline and only on non-patient data. The knowledge-base enrichment described below runs on a development machine against the test catalogue, and its output ships as ordinary database rows. Production reads those rows without making a single outbound AI call beyond the Mistral extraction.
Every AI call is logged with its model, token counts, latency and cost, giving the client a complete audit trail and a live view of spend. The rebuilt pipeline processes a report in about ten seconds for roughly one cent, down from about a minute and seven cents per document in the earlier design.
A human in the loop, with a dashboard built for the job
AI on medical data needs a place where a human can see what it did and overrule it, so we built one: an operations dashboard where a clinical reviewer watches the pipeline work.
The dashboard shows every report broken into buckets: matched automatically, waiting for review, unmatched, and failed outright, with a per-laboratory breakdown that immediately shows which lab's reports are drifting. Drilling into a report shows every row alongside its match trace.
The review queue is where the loop closes. For each borderline row the reviewer sees the printed label, the extracted value and the candidate definition, and confirms or rejects with one click. A confirmation does two things: it fixes that row, and it writes the label into the synonym dictionary, so the next report that prints the same alias matches automatically. A rejection is remembered too, so the same mistake is never proposed twice. The system gets better every week it is used, and the improvement is driven by clinicians rather than by retraining.
Failures are handled with the same care. If a document turns out to be unreadable, or something goes wrong mid-parse, the report is flagged for the operator and the patient's parsing credit is refunded automatically so they can upload again at no cost.
Knowledge bases, wired in and measured
Text similarity alone can never know that three unrelated-looking strings all name the same test. That knowledge lives in reference sources, so we wired them in.
The synonym dictionary draws from three places. Scrapers pull the public test catalogues of six national laboratory chains, around two and a half thousand external test names mapped to internal definitions, refreshed whenever a lab updates its site. MedlinePlus, the US National Library of Medicine's lab-test reference, contributes canonical names and common aliases for the long tail, harvested once offline. And the reviewers' confirmed corrections accumulate on top, capturing the local quirks no public reference records.
None of it is taken on faith. A daily eval harness replays the full pipeline against a fixed corpus of real historical reports and records auto-match rate, average confidence, precision, recall and F1. If any of them drifts past a threshold, the team gets an email before patients notice anything. Every tuning change, every new synonym batch, every model update lands against a measured baseline.
The outcome
The pattern
The interesting lesson sits above the lab domain. A single large model asked to "read the document" will be impressive on the demo and unaccountable in production. What worked here was structure: several narrow agents in sequence, each with an output you can check; domain safety gates with veto power over the AI; a confidence ladder that routes doubt to a human instead of hiding it; a dashboard where those humans teach the system; and an eval harness that measures every change against reality. That shape carries to any workflow where messy human documents have to become data someone will act on, and where being wrong quietly is the one thing you cannot afford.