Abstract: Why the P-Value Alone Is Not Enough: The Impact of Reporting Confidence Intervals in the Plastic Surgery Literature (Plastic Surgery The Meeting)

Saturday, September 24, 2016

Osama Samargandi, MD, MHSc , Dalhousie University, Halifax, NS, Canada

Mona Al-Taha, BBA, MD Candidate (Class of 2017) , Dalhousie University, Halifax, NS, Canada

Kit Moran, MSc, MD Candidate (Class of 2019) , Dalhousie University, Halifax, NS, Canada

Sarah Al Youha, MD, PhD , Dalhousie University, Halifax, NS, Canada

Michael Bezuhly, MD, MSc, SM, FRCSC, FAAP , Plastic & Reconstructive Surgery, IWK Health Centre, Halifax, NS, Canada

Purpose: The p-value is one of the most utilized descriptors in statistical analysis;however, when reported in isolation, it does not convey the effect size of a treatment ^1,2. The reporting of confidence intervals is an essential adjunct to determine the clinical value of treatment, as it permits an assessment of the effect size^1,3. The objective of this study was to assess the reporting of confidence intervals in clinical trials within the plastic surgery literature.

Methods: The seven highest impact plastic surgery journals of different domains were screened using MEDLINE for clinical trials in the years 2006, 2009, 2012, and 2015. Studies were randomized based on a predetermined sample size. Various characteristics including the Jadad quality score, statistical significance of the study findings, year of publication, journal impact factor, and participation of a methodologist were documented and their influence on the use of confidence interval was examined.

Results: Two independent reviewers analyzed 135 articles. There was substantial inter-rater agreement (kappa=0.78). Although, 86% of the studies reported a p-value, only 27% reported the confidence intervals. The quality of the studies had a median Jadad score of 2 out of 5 (IQR 0-3.75). Bivariate analysis revealed that a higher Jadad score (p=0.023) and inclusion of a research methodologist (p=0.002) were associated with the reporting of confidence intervals. Multivariate analysis revealed similar findings, while journal impact factor, year of publication and statistical significance were not correlated with confidence interval reporting.

Conclusion: Confidence intervals are under-reported in the plastic surgery literature. The impact of this problem is discussed. To improve the reporting quality of clinical trials, results should always include the confidence intervals to avoid the misinterpretation of the effect size of a statistically significant result.

29731 Why the P-Value Alone Is Not Enough: The Impact of Reporting Confidence Intervals in the Plastic Surgery Literature