Can AI make the clinical record more complete?
A study that measures, note by note, whether an AI assistant that reviews the documentation and flags what is missing produces more complete records than a plain automated extraction.
PoktaCare is the clinical record we build for specialists: a record defined by what it manages to capture, not by how good it looks. A specialist has minutes, not hours, to apply each clinical guideline to a real patient, and whatever is not written down in the note stops existing for the next clinician who reads it. This page is the evidence: we measure, note by note, whether our AI assistant closes that gap better than a plain extraction.
More about PoktaCare →The record only works if the note is complete.
A clinical registry for biologic therapies depends on each note carrying the data the disease demands. The study's question is direct: does an AI system that reviews the note and warns about what is missing ("the Grader") leave records more complete than an AI that only extracts what it finds?
We work over 110 real notes, certified complete by the treating physician, across two disease variants: 64 rheumatoid arthritis and 46 spondyloarthritis. Because each note is already complete, an empty field in the extraction is not a gap in the note: it is something the system failed to recover. That is what we measure.
From an idea to an evaluation that runs end to end.
- 01
Corpus locked Done
110 notes certified complete; the structural split verified at 64 RA / 46 SpA.
- 02
Per-disease instruments defined Done
A census of which measurements each variant of the form requires (DAS-28 and related in RA; ASDAS, BASDAI and related in SpA).
- 03
Methods built Done
Plain extraction (the baseline) and the Grader, both measured against the same record.
- 04
Base model locked Done
gpt-oss-120b, an open model with measured performance on clinical conversations, the same for every method.
- 05
Full corpus sweep Done
110 notes × 2 methods = 220 extractions, 0 failures, fixed provenance and under US$0.20 of compute.
- 06
Grader evaluation and iteration Done
Each run is recorded as a study entry. The first showed a completeness gap traced to two fields; the second, with the counting rules ported to the Grader, closed it to a statistical tie. The study exists to track that progression transparently.
- 07
Reference labels and more models In progress
A blind-labeled subset to measure accuracy, not just completeness, plus a comparison across several models.
What is already measured.
Disease classification is scored against each note's true diagnosis, which we know: it is a real measure of accuracy, not just of filling fields in. Completeness ≠ accuracy: a filled field can still be wrong, and that is what the blind labels, now under construction, resolve.
Every run is a study entry, which is how we track progress.
The evaluation did its job: on the first run, the Grader landed about 4.2 points below on completeness. Tracing it, the whole gap came from two fields: the tender and swollen joint counts (the central clinical measure for tracking how active the disease is). The rules for recovering that count, reading a negation as a zero, counting named joints one by one, had been tuned in the reference method but never carried over to the Grader.
The second run, a new study entry rather than a replacement of the first, ran the full corpus with those rules ported to the Grader. The gap closed to −0.2 points, with a confidence interval that crosses zero, in plain terms the difference is so small it cannot be told apart from zero with the data we have: a statistical tie. Both runs stay on the record, because the study exists precisely to track that progression transparently, not to erase it.
Field by field, the parity holds.
Of the data each note demands, here is how each method's capture compares. The assistant is now ahead on the tender joint count (the counting rules ported to the Grader); the swollen joint count remains the outstanding gap.
The tie holds across both variants.
It is not an average hiding one broken variant: rheumatoid arthritis and spondyloarthritis show the same pattern on their own.
This is built with clinical judgment, not just code.
The collaborating rheumatologist
Defines what counts as a "complete note" per disease and tunes the Grader's voice, what is worth flagging and what is not. That judgment is the heart of the system, and it is theirs.
The specialist community
Feedback on the instruments required per diagnosis and on what makes a completeness warning clinically useful, or intrusive.
Rigor and data care, by design.
- Patient data never leaves the machine. No note, no name, no identifiable data is uploaded to the cloud or stored in the project repository.
- Every run is reproducible. It is sealed with its exact code version; it never runs over half-finished work.
- The analysis is pre-registered. The primary outcome and the blind control were fixed before running, not chosen after seeing the results.
- Completeness is not the same as accuracy. A filled field does not prove it is correct; accuracy will be measured against blind labels, now under construction.