preview

The Extent And Consequences Of P Hacking

Better Essays

Reflection on “The extent and consequences of P hacking in Science” Megan L. Head, Luke Holman, Rob Lanfear, Andrew T. Kahn, Michael D. Jennions The replication crisis is a topic that has gained a fair amount of attention in the past years. It centers on the finding that numerous of the studies that have been conducted in the field of psychology cannot be replicated. Only recently a paper was published in Nature stating that less than half of the studies in psychology can be reproduced. It has been shown that this effect is not only applicable to studies of the psychological science, but also to medicine, technology, mathematical sciences, and many other disciplines in which research is being done. The mechanisms (X) that are responsible …show more content…

All of these methods fall under the category of p hacking, also known as data dredging, data fishing, data snooping, or equating fitting. If p hacking had a slogan, it would say, “If you torture data long enough, they will confess”. It refers to the method of analyzing data in different ways in order to gain the desired p value of 0.05 (or a p value of 0.001, for purposes of simplicity, in the following, I will refer to a p value of 0.05 as significance level). The present paper addresses this topic by giving an overview of different forms of p hacking, explaining how p curves are used to assess p hacking and doing two types of analyses to determine the extent of p hacking. They conclude with a section on the recommendations how to reduce p hacking in the future. Head et al. use two different approaches to determine the extent of p hacking in their analysis. The first method they pursue is the so-called text mining. This method uses automatic text evaluation with predetermined parameters. In this case, Head et al. search for p values in all open access papers that are accessible through the data base pubmed. They include all p values that are reported exactly (i.e. ‘p = ‘). It had been shown in previous statistical analyses through text mining that p hacking is present, but the objection had been expressed, that only the abstracts section was being examined. Here, Head et al. present text-mining data from both the results and the abstracts sections of all

Get Access