Beyond p value: Why clinical meaning and negative results matter?

Statistical significance has become the dominant language of biomedical research. p value below a predefined threshold, usually 0.05, often determines whether a study is ‘successful’, or ‘practice-changing’. However, an overreliance on statistical significance risks obscuring two essential truths: Not all statistically significant findings may be clinically meaningful, and not all clinically meaningful studies necessarily produce statistically significant results.

Statistical significance suggests whether an observed difference is unlikely to have occurred by chance, but it does not tell us whether that difference would matter to patients or practice. A large, well-powered glaucoma trial may detect a mean intraocular pressure reduction of a few mm Hg between treatment arms and report a highly significant p value. Would that difference influence disease progression, treatment decisions or patient’s quality of life – that is what would matter in practice. When p values are interpreted as providing superiority, the distinction between statistical detection and clinical impact becomes blurred.

This gap is widening in the era of big data, multicentric trials and advanced analytics. As sample sizes become larger, even trivial effects would achieve statistical significance. Without careful contextualisation, such results risk the adoption of newer, costlier or more complex interventions without much or none real-world advantage.

Clinical significance, by contrast, is grounded in patient-centred outcomes, for example meaningful visual improvement, reduced treatment burden, enhanced safety and better quality of life. Metrics such as effect size, confidence intervals, absolute risk reduction, number needed to treat and minimum clinically important difference provide a more balanced assessment of the impact.^[1] Unfortunately, these measures are often underemphasised, and p value makes the headlines.

Understanding these distinctions between statistical and clinical significance is crucial for interpreting study findings accurately:^[2]

1. When a result is both statistically and clinically significant, it means that the effect is unlikely due to chance and is meaningful in a real-world context.

2. A result that is statistically significant but not clinically significant suggests that while the effect is unlikely due to chance, its magnitude is too small to be meaningful. This could be due to a very large sample size, which may have detected even a very small difference which otherwise lacks practical impact.

3. A result that is clinically significant but not statistically significant indicates a meaningful effect that may not have attained statistical significance either due to small sample size or variability.

4. When a result is neither statistically nor clinically significant, it suggests that the observed effect is likely due to chance and also lacks any practical importance.

Closely linked to this concern is the systematic underreporting of negative or neutral studies. When the focus revolves around statistical significance, well-designed studies that demonstrate no meaningful difference between interventions are less likely to be acknowledged; but it needs to be remembered that negative results are not failures; they are critical contributions to the evidence base as well. A rigorous trial showing that a new drug, device or surgical modification offers no advantage over the existing standards can be instrumental in disproving theories; avoiding redundant efforts, deviations from practice and unnecessary costs and guiding the direction of future studies.^[3] Suppression of negative results also perpetuates the misconception that the success of a study lies in always ‘finding something significant’. Knowing what does not work at all or what works no better than prevailing options is equally important.

Research work is gauged more by the importance of the question and robustness of the methodology than by the direction or statistical significance of the results.^[3] Clinicians, as readers, must also ask a fundamental question when interpreting research – ‘Would this difference meaningfully change my practice or benefit my patient?’ Ultimately, the goal of research is not to produce ‘significant’ numbers, but to generate trustworthy evidence that can improve patient care.