Beyond Accuracy: Why Healthcare AI Demands Better Metrics

One of the first lessons in any machine learning course is that accuracy - the proportion of correctly classified instances - is a natural and intuitive measure of how well a model performs. It is easy to calculate, easy to report, and easy to misunderstand. In healthcare, where models are deployed not to classify emails as spam but to determine whether a critically ill patient needs immediate intervention, accuracy can be not just uninformative but actively dangerous. Reflecting on what was discussed in class, I find that the gap between a model's reported accuracy and its real-world usefulness is widest precisely where the stakes are highest. This essay explores why overall accuracy misleads in healthcare AI, illustrates those failures through clinical use cases, and makes the case for a richer, context-specific approach to model evaluation.

The central problem with accuracy in medicine is that disease is rare relative to health. Most patients who arrive at a hospital emergency department do not have sepsis. Most people who attend a mental health crisis line are not at imminent risk of suicide. Most patients discharged after a hospitalization will not be readmitted within thirty days. When the negative class - "no disease," "no crisis," "no readmission" - dominates the dataset, a model that does nothing more than predict "negative" for every single patient will achieve an accuracy that, on paper, looks impressive.

Consider a concrete classroom example: if only 5% of patients in a dataset have the target condition, a completely passive model that always outputs "not at risk" achieves 95% accuracy. This is the accuracy paradox, and it renders accuracy effectively meaningless as a standalone metric in imbalanced clinical settings (Sreedharan et al., 2024). The model has learned nothing about disease; it has merely learned to exploit base rates. Such a system would never generate a true positive, yet it would pass naive evaluation with flying colours.

This problem is not theoretical. In real-world healthcare datasets, class imbalance ratios of 1:10 to 1:50 are routine. Sepsis affects a minority of ICU patients. Thirty-day readmission rates hover around 15-20% for general medical populations. Acute psychiatric crises represent a small fraction of all mental health contacts. In every one of these cases, a model that ignores the positive class can appear to function while providing zero clinical benefit.

Sepsis is a time-critical condition where treatment delays are directly associated with increased mortality. A sepsis prediction model built on an ICU dataset where sepsis cases represent, say, 10% of all patient hours could achieve 90% accuracy simply by predicting "no sepsis" at every time step. Yet that model would miss every true case - the exact patients for whom early warning systems are designed.

Bomrah et al. (2024), in a scoping review of machine learning for sepsis prediction spanning 1,147,202 patients across 29 studies, found that the performance metrics that actually mattered to clinical utility were AUROC, sensitivity, and specificity - not overall accuracy. Their findings highlighted that models with superior sensitivity and AUROC values, particularly those using deep feature extraction, outperformed simpler models that might have reported higher raw accuracy by being biased toward the negative class. In sepsis detection, the clinical cost of a false negative - a missed sepsis case - is catastrophically high: progressive organ dysfunction, septic shock, and potentially preventable death. This asymmetry in error costs means that sensitivity (recall) must be prioritised, even at the cost of some specificity. Framing it in terms we covered in class: we want the model's positive predictive behaviour to be correct, not just its majority-class behaviour.

Sreedharan et al. (2024), in a systematic review and meta-analysis of AI applications in emergency and critical care, confirmed this by evaluating models across sensitivity, specificity, positive predictive value, and negative predictive value - all of which provide a much fuller picture of clinical utility than accuracy alone.

Mental health triage presents a different but equally sharp illustration of accuracy's limitations. The task here might be to identify individuals who are at risk of self-harm or suicide from a pool of people seeking online or in-person mental health support. The positive class - those at genuine imminent risk - is small. A model optimised for accuracy could easily classify almost everyone as "low risk" and look good numerically while systematically missing the individuals who need urgent escalation.

In mental health, a false negative is not just a statistical artifact - it is a person who needed care and did not receive it. Gutierrez et al. (2024), in a mixed-methods systematic review of AI in online mental healthcare covering 29 studies, documented AI applications in triage, treatment monitoring, and prediction of dropout and response. Their review noted that despite these promising applications, the quality of the available evidence was low to very low - a finding that underscores not just methodological issues but also the fact that many studies failed to report the nuanced metrics needed to assess whether these tools could actually work in the real world. When a triage system only reports accuracy, clinicians cannot know whether it is catching the right patients.

Islam et al. (2025) similarly found, in their systematic review of AI-based risk assessment tools for mental health, that existing models could outperform traditional clinical methods when correctly implemented - but that future research must address biases, transparency, and representation of underserved populations. The failure mode they identified is not that accuracy is mathematically wrong, but that it rewards models that are good at predicting what is already common, not what is clinically urgent.

The right approach for mental health triage is to prioritise sensitivity and positive predictive value (precision), and to explicitly define the acceptable false-negative rate based on the clinical context - whether the system is used as a screener (where sensitivity matters most) or as a referral gatekeeper (where precision matters more).

Hospital readmission prediction models are used to identify patients at risk of returning to hospital within 30 days of discharge, enabling targeted discharge planning and follow-up care. The base rate for readmission in many datasets is 15-20%, creating an 80-85% majority negative class.

Fathy et al. (2025), in a comprehensive review of ICU readmission prediction models spanning statistical, machine learning, and deep learning approaches, observed that while ML and DL models have "moderately improved" prediction, further progress is needed before they can support reliable clinical decision support. The review explicitly discussed the performance metrics used to evaluate models and noted the gap between statistical performance and clinical applicability. Importantly, a model that achieves 85% accuracy by predicting "no readmission" for everyone has learned nothing and would waste clinical resource by providing no usable signal for discharge coordinators.

For readmission prediction, the practical requirement shifts. Because the goal is to allocate limited care-management resources - follow-up calls, social worker visits, pharmacy counselling - precision becomes important alongside recall. Flagging everyone as high risk is operationally useless; clinicians need a shortlist of genuinely high-risk patients. This means the F1 score (the harmonic mean of precision and recall), or ideally AUROC combined with calibration metrics, provides far more actionable information than accuracy.

The Area Under the Receiver Operating Characteristic Curve (AUROC) has become the de facto standard for binary clinical prediction models, and for good reason. While accuracy collapses the full confusion matrix into a single number that is sensitive to class distribution, AUROC measures a model's ability to discriminate between positive and negative cases across all possible classification thresholds. It asks: across the entire range of operating points, how consistently can this model rank a randomly selected positive case higher than a randomly selected negative case? A perfect discriminator scores 1.0; random guessing scores 0.5.

In a class-imbalanced healthcare dataset, accuracy can be inflated to 95% by a model that simply predicts the majority class, while its AUROC will correctly reveal that it performs no better than chance (approximately 0.50). Conversely, a clinically useful model that correctly identifies the rare positives while generating some false positives might score 87% accuracy - lower than the null model - but will have an AUROC of 0.80 or above, faithfully reflecting its discriminatory power (Sreedharan et al., 2024; Bomrah et al., 2024).

AUROC also has a practical clinical interpretation: it expresses the probability that a model will score a patient with the target condition higher than a patient without it. A clinician reading an AUROC of 0.85 for a sepsis model can understand that if they take one septic patient and one non-septic patient at random, the model will correctly rank the septic patient as higher risk 85% of the time. This directly maps onto how triage and risk-stratification systems actually work.

From a clinical decision-making standpoint, AUROC is also threshold-independent. Clinicians can select a decision threshold that matches their local context: in a resource-poor setting with high sepsis prevalence, they might lower the threshold to maximise sensitivity, accepting more false positives. In an elective readmission-prevention programme with limited care-management staff, they might raise the threshold to favour precision. The AUROC captures the model's potential across all these operational scenarios, while accuracy fixes one threshold and reports a single, contextually stripped number.

Recent methodological work has reinforced this position. Research published on medRxiv (2026) found that AUROC and the Area Under the Precision-Recall Curve (AUPRC) remained stable across different class imbalance correction methods, while raw accuracy, precision, and F1 scores fluctuated considerably depending on the method used. This makes AUROC a more reliable and consistent basis for comparing models or tracking performance over time - properties that are essential for governance and clinical oversight of AI systems.

A key insight from reflecting on these three use cases is that there is no single "correct" alternative to accuracy - the right metric depends on the clinical task, the cost of errors, and the intended use of the model's output.

Sepsis detection is a high-sensitivity task. The cost of a false negative is a missed sepsis case and potential death. The cost of a false positive is an unnecessary investigation or a precautionary antibiotic dose - costly and inconvenient, but rarely fatal. Here, sensitivity (recall) should be the primary metric, supported by AUROC for model comparison and calibration measures for threshold selection. A model with 80% sensitivity and 70% specificity is far more useful than one with 60% sensitivity and 99% specificity, even though the latter might look better by raw accuracy.

Mental health triage requires careful balancing of sensitivity and positive predictive value. False negatives (missed crises) carry severe consequences, but in a high-volume triage context, an extremely low threshold will generate so many false positives that the system becomes unusable - clinicians suffer alert fatigue, and the signal is buried in noise. The appropriate metrics are sensitivity, PPV (precision), the F-beta score (where beta > 1 to weight recall more heavily), and AUROC for model selection. Calibration is also critical: a model should give a risk score that accurately reflects true probability, not just a rank.

Readmission prediction typically operates in a resource-allocation context. The question is not "will this patient be readmitted?" (which no model answers reliably) but "which patients should we prioritise for targeted follow-up?" Here, precision matters at the operating threshold - since resources are finite - and AUROC should be used for model selection. The C-statistic (equivalent to AUROC for binary outcomes) has long been the benchmark in clinical prediction literature, and in readmission models it is standard to expect values above 0.70-0.75 to justify deployment (Fathy et al., 2025).

A summary of this context-specific approach might look like this:

While AUROC and sensitivity/precision trade-offs are significant improvements over raw accuracy, it is worth acknowledging that even these metrics do not fully capture clinical utility. A model's AUROC tells us about its discriminatory ability but says nothing about whether its probability estimates are reliable - a model could perfectly rank all patients but systematically overestimate risk by a factor of two, leading clinicians to intervene on patients who would not actually deteriorate.

This is why calibration - the agreement between predicted probabilities and observed outcomes - is increasingly recognised as a necessary companion to AUROC in clinical AI evaluation. A well-calibrated model that says "this patient has a 30% chance of readmission" should, in a large sample, be readmitted 30% of the time. Poor calibration can undermine clinical trust and lead to suboptimal decisions even when discriminatory performance looks strong.

Furthermore, clinical utility should ultimately be assessed by whether model use improves patient outcomes - through randomised or quasi-experimental designs wherever possible. The UC San Diego COMPOSER AI model for sepsis prediction, for example, demonstrated a 17% reduction in mortality in an emergency department setting (UC San Diego Health, 2024), which is the kind of outcome-level evidence that no metric alone can provide but that all metrics are ultimately in service of.

Accuracy is not a useless metric - it is a useful metric in the wrong contexts, and healthcare is almost always the wrong context. The combination of class imbalance, asymmetric error costs, and high-stakes consequences makes accuracy a metric that flatters bad models and obscures the performance of good ones. As we considered in class, a model predicting "no disease" for every patient can look excellent by accuracy while offering nothing clinically. AUROC offers a robust, threshold-independent, and clinically interpretable alternative that captures discriminatory ability across all operating points. Context-specific calibration of which metrics to prioritise - sensitivity for sepsis detection, balanced precision and recall for mental health triage, AUROC and precision for readmission prediction - is not a technical nicety but a clinical necessity. The goal of AI in healthcare is not to build models that score well on benchmarks. It is to build systems that help clinicians make better decisions for patients. That starts with measuring the right things.

Bomrah, S., Uddin, M., Upadhyay, U., Komorowski, M., Priya, J., & Dhar, E. (2024). A scoping review of machine learning for sepsis prediction - feature engineering strategies and model performance: a step towards explainability. Critical Care, 28, Article 204. https://doi.org/10.1186/s13054-024-04948-6

Fathy, W., Emeriaud, G., & Cheriet, F. (2025). A comprehensive review of ICU readmission prediction models: From statistical methods to deep learning approaches. Artificial Intelligence in Medicine, 103126. https://doi.org/10.1016/j.artmed.2025.103126

Gutierrez, G., Stephenson, C., Eadie, J., Asadpour, K., & Alavi, N. (2024). Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review. Frontiers in Psychiatry, 15, Article 1356773. https://doi.org/10.3389/fpsyt.2024.1356773

Islam, S., Shahriyar, R., Agarwala, A., Zaman, M., Ahamed, S., & Rahman, R. (2025). Artificial intelligence-based risk assessment tools for sexual, reproductive and mental health: a systematic review. BMC Medical Informatics and Decision Making, 25, Article 174. https://doi.org/10.1186/s12911-025-02864-5

Sreedharan, J. K., Saleh, F., Alqahtani, A., Albalawi, I. A., Gopalakrishnan, G. K., & Alahmed, H. A. (2024). Applications of artificial intelligence in emergency and critical care diagnostics: a systematic review and meta-analysis. Frontiers in Artificial Intelligence, 7, Article 1422551. https://doi.org/10.3389/frai.2024.1422551

UC San Diego Health. (2024, January 23). Study: AI surveillance tool successfully helps to predict sepsis, saves lives. https://health.ucsd.edu/news/press-releases/2024-01-23-study-ai-surveillance-tool-successfully-helps-to-predict-sepsis-saves-lives

Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019). Calibration: The Achilles heel of predictive analytics. BMC Medicine, 17, Article 230. https://doi.org/10.1186/s12916-019-1466-7