ORIGINAL ARTICLE

Steps for the decision making based on the statistical analyses

Francisco Winter dos Santos Figueiredo^{I, II}; Fernando Adami^I

^IGrupo de estudos de Enfermagem Baseada em Evidências- EBE. Faculdade de Medicina do ABC, Santo André, Brasil
^IILaboratório de Epidemiologia e Análise de dados da Faculdade de Medicina do ABC, Santo André, Brasil

Correspondence

ABSTRACT

Researchers and students in training are uncertain when they need to conclude their research based on results. However, for many reasons, they address their research questions based mainly on p-values. In this article, we present a framework for decision-making, based on statistical analyses, and describe fundamental aspects of rational decisions. Additionally, we discuss several aspects of the obtainment of scientific evidence, for example, how statistical analyses and research questions are related. Finally, we developed a step-by-step framework for decision-making based on statistical analyses. With this article, we intend to contribute to the development of research, and guide researchers and students regarding this important and neglected subject.

Keywords: biostatistics, scientific research, statistical analyses

In a recent article, Baker¹ makes reference to the statement released by the American Statistical Association (ASA) about the risk of making decisions based primarily on p-values by researchers, which has become a common practice followed by a considerable number of scientists worldwide.

Definitely, this is a highly relevant topic meriting considerable discussion, as the search for statistic results can lead to certain issues, from the scientific viewpoint, and the clinical perspective, as well. As an example, we can mention publication bias, which can occur in systematic reviews-as a consequence of the fact that studies whose results are not statistically significant, have slim chances of publication, compared with those whose results have such significance-and the possible release of drugs without any therapeutic efficacy for general marketing.

There is respectable literature related to the use and misuse of p-values. Additionally, Nuzzo² warns about this rooted culture. As a possible solution, the author suggests that researchers should consider the confidence intervals of estimates and statistical adjustment for confounding variables and replicability of experiments. Thus, such practices are directly related to researchers' lack of knowledge regarding study designs, complete lack of planning for the performance of trials and, particularly, with poor understanding of the meaning of p-values.

Another important fact often overlooked by researchers is that the making of decisions about the acceptance or rejection of a hypothesis involves interval estimates and probability values, as well as the applicability of biological, clinical, and epidemiological results, which are as important as the estimates obtained.

On this matter, we point out the statement made by Leek & Peng³, "p-values are only the tip of the iceberg." The design and conduction of a successful study has many stages, and decisions made earlier during data analysis have greater impacts on results.

Since the first step is defining the research question, such a research question should be defined clearly for accurate statistical analyses. Well-known methodologies are highly useful in the process of defining research questions, such as PICOS⁴ (Population, Intervention, Comparison, Outcome, Study Design) for experimental analytical studies, PEOS (Population, Exposure, Outcome, Study Design) for observational analytical studies, and POS (Population, Outcome and Study Design) for descriptive studies.

Once the research question has been defined clearly, the next step is to establish a plausible model from the theoretical viewpoint, supporting the possible analyses that should be conducted. After knowing the data distribution, the best analyses for addressing the research question that meets the assumptions of the statistical tests and their relationship with the theoretical model that has already been defined⁵ should be selected.

After conducting the most appropriate test, point and interval estimates should be obtained using the statistical models that facilitate adjustment for the variables that might confound the obtained results⁵.

The analysis of p-values alone should not be the last stage in decision-making, as presented in the flowchart.

The p-value is only one of the parameters that should be analyzed, and therefore, we developed the framework described in this paper to guide researchers and students in their research. Therefore, the making of the decision of accepting or rejecting a hypothesis should be guided by scientific literature and the practical knowledge and experience of the researcher.

REFERENCES

1. Baker M. Statisticians issue warning on P values Statement aims to halt missteps in the quest for certainty. Nature. 2016;531(7593):151. DOI: http://dx.doi.org/10.1038/nature.2016.19503        [ Links ]

2. Nuzzo R. Statistical errors: P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume. Nature. 2014;506(7487):150-2. DOI: http://dx.doi.org/10.1038/506150a        [ Links ]

3. Leek JP, Peng PD. P values are just the tip of the iceberg. Nature. 2015;520(7549):612. DOI: http://dx.doi.org/10.1038/520612a        [ Links ]

4. Boudin F, Nie JY, Bartlett JC, Grad R, Pluye R, Dawes M. Combining classifiers for robust PICO element detection. BMC Med Infor Decision Making. 2010;10:29. DOI: https://doi.org/10.1186/1472-6947-10-29        [ Links ]

5. Hauber AB, González JM, Groothuis-Oudshoorn CG, Prior T, Marshall DA, Cunningham C, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR Conjoint Analysis Good Research Practices Task Force. Value Health. 2016;19(4):300-15. DOI: https://doi.org/10.1016/j.jval.2016.04.004        [ Links ]

Correspondence:
winterfigueiredo@gmail.com

Manuscript received: May 2017
Manuscript accepted: October 2017
Version of record online: December 2017