What has been measured
Before each release, Priorfolio is run against a 29-page test packet whose right answers are known. The table shows the results. They show that the rules work on a clean example. They say nothing yet about how it does on your faxes, and the last section of this page explains why.
| Check | Planted in the packet | Found |
|---|---|---|
| Pages grouped into the right documents | 25 documents | 25 |
| Clinical documents dated from a labeled date | 19 | 19 |
| Documents sent twice | 3 | 3 |
| Page belonging to another patient | 1 | 1 |
| Scanned pages with no text layer, read by OCR | 4 | 4 |
| Studies mentioned in notes and not sent | 5 | 5 |
| Studies wrongly listed as missing | 0 expected | 0 |
| Psychiatric drugs with a documented stop reason | 7 | 7 |
| Drugs two sources dose differently | 1 | 1 |
| Note mostly copied from an earlier note | 1 | 1 |
How to read this
We wrote this packet ourselves, with invented patients, so that we would know the right answers. It is printed in a clean typewriter face. Real faxes are skewed, faint and stamped over, and they come from hundreds of different record systems. A perfect score on this packet tells us that a change did not break a rule that used to work.
What has not been measured
- Error rates on real faxed packets. We do not hold any, because packets are never sent to us.
- How often a number read from a poor scan is wrong. Low-confidence pages are flagged, but the misread rate is unknown.
- Whether each lens marks what a practicing specialist would mark. The lists have not yet been reviewed line by line by clinicians in each field.
Help us measure it
If your practice can share de-identified packets with the correct answers marked, we will publish the results here, good or bad. Use the Contact page.
The known failure modes are listed on What it gets wrong.