When Dermatology's Real-World Data Agrees With the Gold Standard, and When It Doesn't

A study published online September 11 in the Journal of the American Academy of Dermatology set out to answer a narrow but useful question: when dermatology researchers mine electronic health records for clinical conclusions, how often do those conclusions hold up against higher-level evidence like randomized trials and meta-analyses?

Author Samuel Tringali examined 34 EHR-derived studies, which together generated 144 distinct clinical outcomes, and compared each against the best available comparison evidence in its field. Overall concordance landed at roughly 80%, according to reporting from Healio.

The agreement rate was not uniform across evidence types. Concordance with EHR study conclusions reached 100% for outcomes drawn from randomized clinical trials and prospective cohort studies, according to the same account, while it dropped to 79.2% for meta-analyses and 77.8% for retrospective cohort comparisons. That spread matters more than the topline number, since it suggests EHR data track prospective, tightly monitored comparisons better than they track the pooled, heterogeneous synthesis that meta-analyses represent.

Tringali was explicit that the finding is not a case for treating database research as a substitute for controlled trials. "Dermatology research is increasingly using large clinical databases, so clinicians are going to encounter more studies based on real-world EHR data," he told Healio, while cautioning that the results do not mean EHR-derived evidence is equivalent to randomized-trial or meta-analysis findings.

The value he pointed to instead is complementary: large clinical databases make it possible to study uncommon adverse events and long-term outcomes that are difficult or sometimes impractical to capture in a trial designed to run a few years with a few hundred patients.

That caveat lines up with a broader pattern documented outside dermatology. A 2022 review in Dermatologic Therapy on real-world evidence for biologic agents noted that randomized controlled trials, for all their internal rigor, study small, carefully controlled patient groups that can lack external validity and generalizability, which is precisely the gap real-world data is meant to fill. And a population-based cohort study in the United Kingdom, published via PubMed, found that concordance between two major electronic health record systems, the Clinical Practice Research Datalink and Hospital Episode Statistics, was itself poor when restricted to cancers of the same anatomical site, with even greater discordance among patients with psoriasis. Electronic records disagreeing with each other, not just with trials, is part of the same measurement problem Tringali's paper is describing.

None of this settles the question of which single number should stand in for how trustworthy an EHR-derived dermatology finding is. It does narrow where clinicians should apply skepticism: an EHR study echoing a well-designed prospective cohort deserves more confidence than one whose only point of comparison is a meta-analysis pooling trials that never resembled real-world patients in the first place.