{"id":263811,"date":"2025-11-05T17:49:10","date_gmt":"2025-11-05T17:49:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/263811\/"},"modified":"2025-11-05T17:49:10","modified_gmt":"2025-11-05T17:49:10","slug":"journal-of-medical-internet-research-16","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/263811\/","title":{"rendered":"Journal of Medical Internet Research"},"content":{"rendered":"<p>Introduction<\/p>\n<p class=\"abstract-paragraph\">The increasing adoption of supervised machine learning (ML) in clinical prediction models (diagnostic or prognostic) based on tabular data has sparked considerable debate regarding its comparative performance against traditional statistical logistic regression (LR), particularly for binary outcomes such as mortality or the occurrence of adverse events []. Although supervised ML approaches have demonstrated clear superiority in classifying unstructured clinical data such as medical images and texts [], their added value for the classification of clinical tabular data, where structured data are organized in tables, typically with rows representing individual cases and columns representing individual characteristics, remains uncertain and context-dependent [].<\/p>\n<p>Definition of Statistical LR Versus ML-Based LR<\/p>\n<p class=\"abstract-paragraph\">The distinction between statistical LR and ML-based LR is frequently blurred in both literature and practice []. Many studies loosely refer to any penalized LR model as ML, despite fundamental methodological differences. To enable valid comparisons between these approaches, it is therefore crucial to clearly delineate their boundaries ().<\/p>\n<p>Table 1. Definitions of statistical logistic regression and machine learning\u2013based logistic regression.MethodDefinitionStatistical logistic                 regressionA parametric model operating under conventional statistical assumptions, including linearity and independence, employing fixed hyperparameters without data-driven optimization, using prespecified candidate predictors based on clinical or theoretical justification. This aligns with traditional epidemiological approaches where model specification precedes data analysis.Machine learning\u2013based logistic regressionAn adaptive variant where model specification becomes part of the analytical process itself, hyperparameters like penalty terms are tuned through cross-validation, predictors may be selected algorithmically from a broader set of candidates, and the analytical focus shifts decisively toward predictive performance. While mathematically similar to statistical logistic regression, this approach embodies the machine learning philosophy of learning from data.<\/p>\n<p class=\"abstract-paragraph\">In this paper, we adopt the definition proposed in a previous systematic review [], which characterizes statistical LR as a theory-based model that operates under strict assumptions and does not involve data-driven optimization of predictive performance through hyperparameter tuning but relies on subject knowledge from researchers or experts to specify the model structure. Although penalized LR models may involve hyperparameter tuning and variable selection, they remain theory-involved and do not intrinsically capture nonlinearities or interactions, and the core assumptions of LR still apply. Conversely, ML models are defined as methods that autonomously learn patterns from data (ie, data-driven hyperparameter tuning or predictor selection) [], including ML-based LR and other supervised ML methods (eg, random forest, boosting, neural networks, support vector machine) that intrinsically handle complex interactions without manual specification beforehand.<\/p>\n<p>Comparative Performance: What Was Already Known<\/p>\n<p class=\"abstract-paragraph\">A 2019 meta-regression of 145 low-risk-of-bias comparisons between statistical LR and ML binary clinical prediction models on tabular data showed no performance benefit of ML over statistical LR []. However, this comparison was limited to discrimination measured by the area under the receiver operating characteristic curve (AUROC), as other performance metrics were not frequently reported. Notably, 79% (56\/71) of the studies did not report calibration performance, and only one study reported clinical utility. Clinical utility is commonly assessed through decision curve analysis, which estimates the clinical value of a prediction model at the population level by considering the consequences of decisions made based on its output [], specifically, the benefit of correctly predicting true positives and the harm of incorrectly predicting false positives. A step-by-step guide to this method is available here [].<\/p>\n<p class=\"abstract-paragraph\">Each performance metric captures a distinct aspect of model performance, with its own strengths and limitations. A model may achieve a high AUROC yet still have poor calibration and potentially harmful clinical consequences if the predicted probabilities are systematically overestimated or underestimated; vice versa, a well-calibrated model may still have poor discriminative ability. This highlights the need for comprehensive evaluation across multiple performance domains, including discrimination, calibration, classification metrics, clinical utility, and fairness. Therefore, focusing solely on marginal gains in AUROC between LR and ML can be misleading and inadequate for guiding future research.<\/p>\n<p class=\"abstract-paragraph\">Furthermore, the meta-regression of AUROC did not explore the underlying sources of performance differences (eg, sample size, number of predictors, use of hyperparameter tuning). Therefore, it is still unclear whether the observed variation in performance reflects true algorithmic superiority or is instead driven by dataset characteristics or modelling procedures. For example, the systematic review highlighted that more than half of the included studies did not clearly report their hyperparameter tuning strategies.<\/p>\n<p class=\"abstract-paragraph\">The comparison should not only focus on the difference in performance but also on the stability of performance []. Even when statistical LR outperforms ML in certain metrics (eg, AUROC), this does not necessarily imply that the predictions are stable or reproducible, that is, applying the same model development procedure to different samples of the same size drawn from the same underlying population can result in substantially different predictions for the same individual. This issue is particularly pronounced when using small development datasets, which leads to more different models in the multiverse, often with vastly unstable individual predictions []. Adherence to the minimum sample size recommendations is one way to mitigate this issue [,]. Notably, a 2023 systematic review reported that 73% of the binary clinical prediction models using statistical LR had sample sizes below the recommended minimum threshold []. ML algorithms are generally more data-hungry than LR to achieve stable performance. For example, one study demonstrated that random forest may require more than 20 times the number of events for each candidate predictor compared to statistical LR [].<\/p>\n<p>Research Recommendation for Reporting: Improving the Transparency of Each Step<\/p>\n<p class=\"abstract-paragraph\">In light of the methodological issues identified in the current literature rather than focusing solely on determining the inherent superiority of one modelling approach over another, greater attention should be directed toward ensuring the rigor and transparency of modelling procedures. This includes clear documentation of data preprocessing steps, sample size justifications, modelling decisions, hyperparameter tuning strategies (eg, grid or random search), feature selection techniques (including filter methods like correlation analysis, wrapper methods like recursive feature elimination, or embedded methods such as least absolute shrinkage and selection operator [LASSO]), model performance evaluation methods and metrics, and model explanation methods (eg, Shapley Additive Explanations [SHAP] [], Submodular Pick Local Interpretable Model-agnostic Explanations [SP-LIME] [], and Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence [CERTIFAI] models []).<\/p>\n<p>No Free Lunch Theorem<\/p>\n<p class=\"abstract-paragraph\">There is no universal golden method for clinical prediction models [,] and whether the benefit of each algorithm (either statistical or ML) can be fulfilled is highly subject to the dataset characteristics (eg, sample size, class imbalance, nonlinearity, number of candidate predictors []) and data quality (eg, completeness, accuracy []).<\/p>\n<p class=\"abstract-paragraph\">Each algorithm has its unique strengths and limitations in handling different data characteristics (). For example, Categorical Boosting is particularly effective for datasets with high-cardinality categorical variables, as it includes built-in techniques to encode categories without extensive preprocessing []. eXtreme Gradient Boosting [] and Light Gradient-Boosting Machine [] are known for their computational efficiency, performance in data with complex feature interactions, and native handling of missing data, but are less interpretable than LR. Deep learning, a subfield of ML, uses multilayered neural networks to simulate human decision-making []. While capable of learning highly complex nonlinear relationships from extremely large and high-dimensional datasets, deep learning models are generally more data-hungry, less interpretable, and require significantly more computational resources than traditional ML methods, which may limit their transparency and clinical applicability []. Although efforts in Explainable Artificial Intelligence (XAI) are advancing, current ML models often fall short of the level of clarity and trust required for clinical implementation []. On the other hand, LR is highly interpretable and performs well on small sample sizes when predictors have an approximately linear relationship with the outcome, but it may struggle with complex nonlinearities or large numbers of correlated predictors. The smaller the sample size available, the more we must rely on external information or inputs from experts to determine the features\/predictors.<\/p>\n<p>Table 2. Strengths and weaknesses of statistical logistic regression and machine learning in binary clinical prediction models based on tabular data.AspectStatistical logistic regressionSupervised machine learningLearning processTheory-driven; relies on expert knowledge for model specification and candidate predictor selectionData-driven; directly and automatically learn relationship from dataAssumptions in data structureHigh (eg, interactions, linearity)Low; handle complex, nonlinear relationshipAssumptions in model specificationHigh; use default valueLow; data-driven hyperparameter tuningUser input in creation and selection of candidate predictorsHigh; researchers need to investigate the nonlinearity of continuous variables and interaction effects and systematic review or expert opinion of candidate predictors before developing the modelLow; models automatically capture nonlinearity and interactions, no need for researchers to investigate nonlinearity and interaction effects between variablesFlexibilityLow; constrained by linearity assumptions but can be improved by adding penaltyHighComplexityLow; simple, parametric modelHighPerformance on complex dataLowHighSample size requirement for stable performanceLowHigh; data-hungrinessInterpretability (in-processing decision-making process)High; white-box nature, model coefficients are directly interpretable, can also be presented using graphical score charts or nomogramsLow; black-box nature, decision-making process is not transparentExplainability (postprocessing explanation)HighLow; complex to explain to end users, requires post hoc methods like Shapley Additive Explanations for explanationDeployment easeHighLowComputational costLowHigh<\/p>\n<p class=\"abstract-paragraph\">Therefore, the choice of the algorithm should be tailored to the structure, quality, and characteristics of the dataset. Ultimately, the development of clinical prediction models involves unavoidable trade-offs. There is no single algorithm that excels across all performance metrics (fairness, accuracy, generalizability, stability, parsimony, and interpretability). Researchers must prioritize certain metrics depending on the model\u2019s intended application and target population. For instance, model parsimony, where a model with fewer predictors may sacrifice some accuracy for simplicity, can be crucial in enhancing user acceptance, as overly complex models may reduce usability. Additionally, discussions with stakeholders (eg, health care providers, patients) regarding the most relevant features or desired trade-offs can guide model development.<\/p>\n<p class=\"abstract-paragraph\">Clinical tabular datasets often exhibit characteristics that tend to favor LR over ML models []. These include small to moderate sample sizes, relatively high levels of noise, a limited number of candidate predictors (ie, low dimension), and typically binary outcomes (). Such conditions can constrain the ability of complex ML algorithms to demonstrate superior performance. Moreover, LR\u2019s well-recognized interpretability and trustworthiness [] further reinforce its widespread use in clinical prediction modelling, and it is typically used as a reference model for performance benchmarking in ML studies. However, ML approaches may warrant consideration when they demonstrate clear superiority in performance, supported by model explainability to help build trust among clinicians and end users. To date, no consensus exists on how to evaluate or compare model interpretability and explainability across different methods [].<\/p>\n<p>Table 3. Mismatches between the characteristics of clinical data and supervised machine learning\u2019s strengths.AspectCharacteristics in clinical dataSupervised machine learning\u2019s relative strength compared to statistical logistic regressionCommentsData modalityMostly single modal data (tabular data)Excels with multimodal data (image, scan, text, or signal)Clinical datasets often lack the multimodal richness that enables MLa models to fully demonstrate their advantagesData qualityHigh noise due to errors, missingness, or inconsistent measurement (low signal-to-noise ratio)Performs better for data with high signal-to-noise ratioNoise dilutes true signals, and ML models tend to overfit to noisy artifacts without careful data preprocessingSample sizeOften small to moderateBenefit from large-scale datasets. More \u201cdata-hungry\u201dAlthough sample sizes are improving in some registries, they are often insufficient to train complex ML architectures robustlyPredictorsTypically, a small set of clinically meaningful predictors with high linearity and low order of interaction termsExcels with high-dimensional, nonlinear interactions, temporally rich dataML\\&#8217;s strength in handling high-dimensional, time-series data with higher nonlinearity and interaction termsPredictionPredominantly binary classification (eg, event occurrence: yes\/no).Advantage in multiclass classification and regressionSimple binary classification problems often diminish the additional complexity that ML can handle<\/p>\n<p>aML: machine learning.<\/p>\n<p>Policy and Research Recommendation: Shifting From Model Debate to Data Quality<\/p>\n<p class=\"abstract-paragraph\">Amid increasing interest in complex models, it is crucial to reorient clinical prediction modelling from a model-centric to a data-centric paradigm []. The quality, structure, and representativeness of data are far more critical to model performance than the complexity of models. In clinical settings, prediction models serve best as a second set of eyes, complementing clinical judgment rather than replacing it []. However, without high-quality data, even the most sophisticated models will propagate existing biases and limitations, as the saying goes, \u201cgarbage in, garbage out.\u201d<\/p>\n<p class=\"abstract-paragraph\">As the number of clinical prediction models continues to grow, policymakers and funding bodies should prioritize investment in data quality infrastructure, including standardized phenotyping, consistent variable definitions, and robust data curation practices. Since all models are trained on historical data that inherently reflect systemic limitations, model complexity cannot resolve errors rooted in the data; in fact, they might amplify the bias and make unfair decisions in underrepresented or marginal groups such as defined by sex, ethnicity, or deprivation []. In contrast, thoughtful data preprocessing and transparent reporting of modelling strategies are foundational to developing reliable, generalizable, reproducible, and trustworthy decision support tools []. In addition, more effort is needed to expand the candidate predictors available in health data [], such as integrating lifestyle factors collected through wearables and a range of medical devices [].<\/p>\n<p class=\"abstract-paragraph\">This shift in emphasis from modelling sophistication to data stewardship is essential to ensure that clinical prediction tools genuinely enhance, rather than undermine, the quality and equity of patient-centered care.<\/p>\n<p>None declared.<\/p>\n<p style=\"font-style: italic\">Edited by  A Mavragani; submitted 19.May.2025; peer-reviewed by S Silvey,  J Li; comments to author 24.Jul.2025; revised version received 28.Jul.2025; accepted 14.Aug.2025; published 05.Nov.2025.<\/p>\n<p>\u00a9Yanan Hu, Xin Zhang, Valerie Slavin, Yitayeh Belsti, Sofonyas Abebaw Tiruneh, Emily Callander, Joanne Enticott. Originally published in the Journal of Medical Internet Research (https:\/\/www.jmir.org), 05.Nov.2025.<\/p>\n<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (https:\/\/creativecommons.org\/licenses\/by\/4.0\/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https:\/\/www.jmir.org\/, as well as this copyright and license information must be included.<\/p>\n","protected":false},"excerpt":{"rendered":"Introduction The increasing adoption of supervised machine learning (ML) in clinical prediction models (diagnostic or prognostic) based on&hellip;\n","protected":false},"author":2,"featured_media":263812,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[49,48,121925,61],"class_list":["post-263811","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-ca","tag-canada","tag-machine-learning-data-quality-clinical-prediction-models-logistic-regression-clinical-data-binary-classfication","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/263811","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=263811"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/263811\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/263812"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=263811"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=263811"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=263811"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}