Health records, then AI: ask what actually moved

The test this piece applies, and hands over at the end, is a single question asked five ways: what actually moved? A process measure, a clinical action, or a patient outcome — and who did the work that made it move. The five questions are set out in full, run on a real claim, and available as a prompt you can paste into any assistant. What follows first is the evidence they were built from.

Why this is answerable now

For twenty years, the largest builder of medical-record systems in Africa was not a software company. It was the US AIDS programme. Through PEPFAR, the CDC and USAID paid university partners and contractors to build electronic medical records for HIV care — Malawi’s touchscreen system descended from Baobab Health’s work, KenyaEMR, UgandaEMR, Nigeria’s NMRS, and Zambia’s SmartCare. Most, though not SmartCare, were built on the open-source OpenMRS platform or its data model. In Zambia alone the US put over $300 million into health information systems since 2005. These were built so HIV programmes could count — patients on treatment, retention, viral suppression — because the funder’s reporting demanded it.

That history matters for reading the evidence, because it determined what was measured. Almost everything we know about whether EMRs work in these settings comes from HIV programmes, in a handful of countries, with donor-funded staff doing the data entry. And in January 2025 that arrangement ended abruptly: a US executive order froze foreign assistance, and by August 86% of USAID’s awards had been terminated. Ethiopia lost roughly 10,000 data clerks; South Africa saw more than 15,000 PEPFAR-funded staff furloughed, data capturers among them. The software mostly stayed. The people paid to feed it did not — which, as the evidence below shows, is the part that was doing the work.

The one mortality result, read properly

The strongest study in this literature is a quasi-experimental event study of 106 Malawian HIV clinics, published in the Review of Economics and Statistics in 2025 (open PDF). Adopting the national point-of-care EMR reduced annual deaths by 11% on the conventional estimator and 20% on one robust to staggered rollout, averaged across post-adoption years, reaching 28% by the fifth year — an effect that builds rather than switches on, from about 6% in year one. Effects concentrate in children under ten. The design holds up: no significant pre-trends on mortality, and the result survives four estimators built for exactly this kind of staggered rollout. One caveat belongs here rather than in a footnote, because this piece disqualifies other studies for it: the authors record that in some clinics the EMR arrived alongside a power source and computer-literacy training, so what was measured is that package, not the software by itself.

Read the mechanism, though, because it is the whole story. The authors attribute the gains to efficiency — clinics could see who had lapsed from treatment, trace them, and bring them back. And in the same paper, they report a null: no significant effect on TB testing or treatment, referrals, or CD4 orders, with point estimates often negative, and none on whether an underweight child improved by follow-up. The EMR prompted clinicians and the prompts did not change what they did. What changed was the clinic’s ability to notice an absence and act on it — work performed by people, on foot and by phone.

Two caveats the headline number carries. The 28% is the year-five endpoint, the endpoint of the path rather than its average, identified off the earliest adopters. And the widely quoted $448 per life saved is not a total cost: the paper’s own appendix shows it is a hardware bill of materials — workstations, cabinets, cabling, connectivity — with no line for training, salaries, maintenance or replacement. That is a defensible incremental figure, and the authors argue the point-of-care design deliberately avoided extra staffing. But it is not comparable to the roughly $4,500 per life saved for the most effective charity programmes that the paper sets beside it, which is a full-programme cost. One more thing worth saying plainly: the study is ten months old, with no critique or replication yet — not because it has survived scrutiny — we have seen no independent re-analysis, and the data are proprietary, though the journal requires authors to document an application route.

At national scale, the record often got worse

The case for EMRs usually rests on data quality, and at flagship sites that case is strong: when Uganda’s Infectious Diseases Institute moved from clerk transcription to provider-entered data, recorded discrepancies on four HIV data elements fell from as high as 94% to under 13%. But that is an uncontrolled before-and-after at one elite institute across four years, during which the clinic also added a real-time quality-assurance programme and dropped its standardised forms. It does not describe what happens at scale, and at scale the picture inverts.

In Rwanda, a comparison of 3,467 records across 50 facilities found the EHR less complete than the paper it replaced: viral-load results present in 76.4% of electronic records against 94.7% on paper, viral-load dates matching in under a third, and only two of fifty facilities reaching acceptable concordance. In Kenya, an assessment across 53 KenyaEMR facilities found the system agreeing with its own paper source on 11.9 of 20 data elements, with 31% of records carrying at least one missing value among nine required fields — and, in the 27 facilities reassessed after a dedicated data-quality intervention, 13.6 of 20. That is the ceiling a deliberate fix achieved.

Worse, the outcome that funders care about most is the one the records get wrong. A collaborative analysis of 505,634 patients across 57 African cohorts moved five-year retention from 52.1% to 66.6% once routine records were corrected using tracing data — on the assumption, imported from a separate meta-analysis of tracing studies, that 20.8% of those recorded as lost to follow-up had died and 35.9% had silently transferred. Those weights carry wide intervals (mortality 11.3–35.1%), so the correction follows from the assumption rather than testing it. A South African study of eight primary-care facilities using the national HIV database found 36% of patient outcomes misclassified. The database does not know what happened to the patient; the person who goes and looks does.

And the third pillar of the usual case — that EMRs slash reporting burden — turns out to have no measured evidence behind it at all. The often-cited figure, that Malawi’s quarterly cohort reports once took up to five days of clinic-closing manual counting and became a button-press, comes from a programme report written by the system’s own developers. The five days is an uncited, hedged upper bound that was never timed, with no before-and-after measurement. The same page records that the generated reports came out wrong, from “incorrect software logic,” and took several iterations to fix. We could not find a study that measures reporting time in these settings.

Deployed is not used

Underneath all of this sits a usage problem that most coverage statistics hide. A census attempt across Kenya’s 376 KenyaEMR facilities, which returned usable server logs from 213 of them, found a mean of 18.1% of authorized accounts active per month — the authors attribute much of that to dormant accounts — and that only 9.4% of facilities were fully paperless, with 52.6% hybrid and 38.0% entering everything retrospectively. In Uganda, 94.3% of upper-tier public facilities run paper and electronic systems side by side, with an average of 4.81 systems each and only 1.9% fully digital.

The cost side is thinner than it should be but points one way. Time-motion studies at three rural Kenyan health centres found total patient time in clinic roughly doubled after EMR implementation — 37 to 81 minutes at one site, 56 to 106 at another — with about three-quarters of the increase spent waiting, though the authors caution that concurrent growth in visits and staffing confounds the result. What fell was not clerical burden but patient-facing time: nurses’ and clerks’ patient-care time dropped after implementation, by around 80% at one site.

AI meets the same test — and produces the same shape

Into this arrives clinical AI, and the argument for it is strong on paper: if the value ran through documentation and follow-up rather than through changing clinical decisions, then tools that write, check and chase are aimed at the right target. The evidence base is now good enough to test that. For LLM-based tools it comes overwhelmingly from one deployment, the Penda Health primary-care chain in Kenya. For rule-based digital decision support — the older, unglamorous kind that encodes a guideline as a flowchart — the African evidence base is far larger and almost entirely null, which is the context every LLM claim should be read against.

The first result to circulate widely was the 2025 study of “AI Consult”, an LLM safety net embedded in Penda’s EHR, reporting 16% fewer diagnostic errors and 13% fewer treatment errors — a claim we examined when it first appeared. Three things about it are easy to get wrong. It is often described as observational. It was not: within each of 15 clinics, clinicians were randomly assigned to have the tool or not, 57 with and 49 without — though the paper calls itself a pragmatic cluster-assigned study, and it was never registered. The errors are physician ratings of the written record. The study did collect one patient-reported outcome — whether patients said they were still not feeling better at follow-up — at 3.8% with the AI against 4.3% without, a difference that did not reach significance. And the diagnostic and treatment effects were null until Penda layered on peer champions, one-on-one coaching and leaderboards; only the history and investigation effects survived the induction period. The measured AI benefit depended on exactly the kind of human support layer the 2025 funding collapse removed elsewhere.

Then, eleven months later, the harder test. A registered, Gates-funded pragmatic cluster-randomized trial of the same tool across the same Penda network — 16 clinics, 103 clinical officers, 9,691 patients — was published in Nature Medicine. On the patient outcome it was null: 14-day treatment failure 2.2% with AI against 2.0% without (aOR 0.77, 95% CI 0.55–1.08). On documentation it was positive: appropriate diagnosis and comprehensiveness both improved. That is not evidence that AI does nothing: a confidence interval that wide leaves the question open. It is evidence that the shape of the EMR result repeated itself in a randomized design, under the best conditions anyone has arranged for it, with the vendor’s own product and a well-run clinic chain.

A third paper from the same deployment rarely makes it into the summaries. A retrospective physician-panel review of 1,469 encounters found actively harmful AI recommendations in 7.8% of them, 37 of those major — and that clinicians fully adopted 22% of those harmful recommendations and partially adopted a further 37%, leaving 42% unacted on. The same review found the effect running the other way too: in 8.0% of encounters the assistant caught and mitigated a risk in the clinician’s own notes, and where its advice was followed at all, beneficial recommendations were taken up roughly twice as often as harmful ones (118 against 67). What the authors flag is the asymmetry in the other direction — 362 separate instances where beneficial guidance was offered and ignored. Its own authors put mitigations on the record: no adverse patient outcomes were found on follow-up, ten of the 37 major flags came from a single evaluator, and a post-study audit judged many to be judgement calls. It is also not independent — same lead authors as the trial, Penda co-authors, two holding Penda stock options. Treat it as a signal worth watching, and one that belongs beside the headline result.

A fourth paper from the same clinics answers the question the other three skip: whether clinicians use the thing at all. Across 258,106 clinical episodes, they opened the assistant in 21.7% — adoption climbing from 4% to 47% over the study — and used it least when they felt confident, which the authors say leaves clinician-initiated querying “liable to overconfidence-related underutilisation”. That is the deployed-is-not-used pattern from the EMR era reappearing intact, one software generation later.

The scribe question is at an earlier stage. A preprint from Groote Schuur Hospital’s neurosurgery division in Cape Town scored ambient-AI notes against contemporaneous handwritten ones across 49 encounters and found the AI notes far better — mean SOAP quality 4.9 against 2.9, with moderate-to-severe error rates at least fivefold higher in the handwritten notes and clinically significant impact in 38.8% of handwritten notes against 2.0% of AI ones. Raters agreed strongly with each other. They were also not blinded, the scribe vendor has a co-author, the encounter audio serves as both the AI’s only input and the scoring reference, and it is 49 encounters in one division that has not yet been peer-reviewed. Handwriting is a low bar, and clearing it is worth something in a system drowning in paper. But it says nothing about patients, and neither does anything else: as of August 2026 we could find no AI scribe study anywhere in the world — not in Africa, not in the far larger US and European deployments — tested against a patient health outcome.

One further mortality number belongs on the record precisely because it cuts against the technology. A Nigerian trial of an AI digital stethoscope in obstetric care — SPEC-AI, 1,195 women across six teaching hospitals — reported an unexplained excess of all-cause deaths in the AI arm, 12 against 3, hazard ratio 4.20 (95% CI 1.18–14.87). It is a secondary, unpowered finding with no adjustment for multiple comparisons, cardiovascular mortality was null, and the trial’s own primary endpoint was how much disease was found rather than how much occurred, which is why it is absent from the tables above. It should not be read as evidence of harm — and it should not be quietly dropped either.

How to read the next claim

The pattern across every strand of this evidence is consistent enough to use as a test. Ask what moved. In this literature, process and documentation measures are the easiest things to move — and the easiest to measure; clinical-action outcomes move inconsistently; and hard patient outcomes are not as rare as the field implies — roughly a dozen African trials have measured death, though only two were powered to detect a difference in it. What is rare is a positive one. No trial powered to detect a mortality difference has found one, and six point estimates favour the control arm. Two results point the other way and both deserve their weight: a Zanzibar messaging trial found fewer perinatal deaths as a secondary outcome, and a Kenyan TB adherence platform moved a composite containing death — where the largest effect was in the arm with a trained human support team attached. The one study that found fewer deaths outright, in Malawi, was not randomized. Even the documentation gains are not uniform, as the Rwandan and Kenyan assessments above show: at national scale the electronic record was sometimes worse than the paper it replaced. A systematic review of 25 randomized eHealth trials across ten sub-Saharan African countries found the same split — behaviour changed (OR 1.21, 95% CI 1.05–1.40, pooled over 11 studies) while biological outcomes stayed undetected (OR 1.17, 0.89–1.54, pooled over 6). When a vendor or a funder cites a documentation improvement as though it were a health improvement, that is the gap they are stepping over.

Ask who does the work the tool depends on. Malawi’s mortality result required tracers who went and found people. Kenya’s loss-to-follow-up result required community social workers making calls and home visits — and even then only 23–31% of lapsed patients were relinked. Penda’s AI gains required coaches and champions. Every result in this file that touched a patient had a person attached to it. The tracing and data-entry workforce behind the Malawi and Kenyan results was donor-funded, and that funding is what collapsed in 2025; Penda’s coaching layer was paid for by a private clinic chain, which is its own kind of fragility.

Ask whether the outcome is the thing or the record of the thing. The Kenyan alert trial measured documented loss to follow-up while the alerts explicitly instructed staff to revise that documentation; an unknown share of the improvement is reclassification. When routine records misclassify a third of outcomes, any study whose endpoint is drawn from those records inherits the error.

Ask what else the same paper reports. This is the cheapest check available and it worked every time here: the Malawi paper contains a null on care at the visit, the Kenyan trial’s parent publication contains a marginal clinical result its daughter paper omits, the Malawi programme report contains an admission that the reports came out wrong, and the Penda deployment contains a harms audit. In each case the inconvenient finding sat inside the source already being cited.

Five questions do most of the work. They are what this review applied to every study in the tables above, and they are as useful on a press release as on a paper.

1. What actually moved — a process measure, a clinical action, or a patient outcome? Documentation and process metrics are the easiest things to move and to measure. Name which of the three the headline number belongs to, and check whether a patient outcome was measured at all.
2. Who does the work the tool depends on? Look for the humans in the mechanism — tracers, data clerks, coaches, community workers. If the effect required a paid support layer, the result does not transfer to a setting without one.
3. Is the outcome the thing, or the record of the thing? An intervention that instructs staff to update a record, measured by that record, may be capturing reclassification rather than real change. Ask how the endpoint was ascertained and how often routine records are wrong.
4. What else does the same source report? The cheapest check available. Look inside the cited paper for a null, a harm, a failure or a caveat that the headline skipped — it is usually there.
5. Is there a later or better-designed study of the same product? An early uncontrolled or unregistered result is often superseded within months. Check for a registered trial, an independent evaluation, or a replication before treating the first number as settled.

None of this says the systems were a waste. An 11–20% reduction in annual deaths, reaching 28% by year five, is a large effect, and clinics that can find their lapsed patients save lives. It says something narrower and more useful: these systems earn their keep as instruments of follow-up and accountability, staffed by people. Improving patient outcomes through better bedside decisions is the thing they have not yet been shown to do. The AI wave is currently producing evidence of exactly the same kind, which is a reason to fund the tracing and the coaching, and a reason to be sceptical of anyone who tells you the model itself is the intervention.

The five questions, run on one claim

Take the headline from that first Penda study. It comes from a real study, the number is accurately reported, and the tool is a serious one.

You are evaluating an impact claim about {{SUBJECT}}, using the "Ground Truth" method (groundtruth.health). I will give you a study, evaluation, press release, or vendor claim. {{MODE}}

1. What actually moved — a process measure, a clinical action, or a patient outcome? Documentation and process metrics are the easiest things to move and to measure. Name which of the three the headline number belongs to, and check whether a patient outcome was measured at all.
2. Who does the work the tool depends on? Look for the humans in the mechanism — tracers, data clerks, coaches, community workers. If the effect required a paid support layer, the result does not transfer to a setting without one.
3. Is the outcome the thing, or the record of the thing? An intervention that instructs staff to update a record, measured by that record, may be capturing reclassification rather than real change. Ask how the endpoint was ascertained and how often routine records are wrong.
4. What else does the same source report? The cheapest check available. Look inside the cited paper for a null, a harm, a failure or a caveat that the headline skipped — it is usually there.
5. Is there a later or better-designed study of the same product? An early uncontrolled or unregistered result is often superseded within months. Check for a registered trial, an independent evaluation, or a replication before treating the first number as settled.

Prefer primary sources; if a claim can only be traced to a press release or blog post, treat the number as unverified. Distinguish "no evidence of benefit" (a study that could not detect an effect) from "evidence of no benefit"; a wide confidence interval means the first, and leaves the question open. Do not fill gaps with assumptions — say "not stated" wherever the source is silent.

CLAIM / STUDY TO EVALUATE:
{{CLAIM}}

Sources and method

Every study here was pulled to its primary source and read for its design as well as its headline number, and each effect size was checked against that source independently. Where a study cannot bear the weight commonly placed on it, the table says so in the design column. Facts current as of August 14, 2026. Corrections are published in the open corrections log.