False positives and other statistical errors in standard analyses of eye movements in reading
Authors: von der Malsburg, T. and Angele, B.
Journal: Journal of Memory and Language
Volume: 94
Pages: 119-133
ISSN: 0749-596X
DOI: 10.1016/j.jml.2016.10.003
Abstract:In research on eye movements in reading, it is common to analyze a number of canonical dependent measures to study how the effects of a manipulation unfold over time. Although this gives rise to the well-known multiple comparisons problem, i.e. an inflated probability that the null hypothesis is incorrectly rejected (Type I error), it is accepted standard practice not to apply any correction procedures. Instead, there appears to be a widespread belief that corrections are not necessary because the increase in false positives is too small to matter. To our knowledge, no formal argument has ever been presented to justify this assumption. Here, we report a computational investigation of this issue using Monte Carlo simulations. Our results show that, contrary to conventional wisdom, false positives are increased to unacceptable levels when no corrections are applied. Our simulations also show that counter-measures like the Bonferroni correction keep false positives in check while reducing statistical power only moderately. Hence, there is little reason why such corrections should not be made a standard requirement. Further, we discuss three statistical illusions that can arise when statistical power is low, and we show how power can be improved to prevent these illusions. In sum, our work renders a detailed picture of the various types of statistical errors than can occur in studies of reading behavior and we provide concrete guidance about how these errors can be avoided.
https://eprints.bournemouth.ac.uk/27412/
Source: Scopus
False Positives and Other Statistical Errors in Standard Analyses of Eye Movements in Reading.
Authors: von der Malsburg, T. and Angele, B.
Journal: J Mem Lang
Volume: 94
Pages: 119-133
ISSN: 0749-596X
DOI: 10.1016/j.jml.2016.10.003
Abstract:In research on eye movements in reading, it is common to analyze a number of canonical dependent measures to study how the effects of a manipulation unfold over time. Although this gives rise to the well-known multiple comparisons problem, i.e. an inflated probability that the null hypothesis is incorrectly rejected (Type I error), it is accepted standard practice not to apply any correction procedures. Instead, there appears to be a widespread belief that corrections are not necessary because the increase in false positives is too small to matter. To our knowledge, no formal argument has ever been presented to justify this assumption. Here, we report a computational investigation of this issue using Monte Carlo simulations. Our results show that, contrary to conventional wisdom, false positives are increased to unacceptable levels when no corrections are applied. Our simulations also show that counter-measures like the Bonferroni correction keep false positives in check while reducing statistical power only moderately. Hence, there is little reason why such corrections should not be made a standard requirement. Further, we discuss three statistical illusions that can arise when statistical power is low, and we show how power can be improved to prevent these illusions. In sum, our work renders a detailed picture of the various types of statistical errors than can occur in studies of reading behavior and we provide concrete guidance about how these errors can be avoided.
https://eprints.bournemouth.ac.uk/27412/
Source: PubMed
False positives and other statistical errors in standard analyses of eye movements in reading
Authors: von der Malsburg, T. and Angele, B.
Journal: JOURNAL OF MEMORY AND LANGUAGE
Volume: 94
Pages: 119-133
eISSN: 1096-0821
ISSN: 0749-596X
DOI: 10.1016/j.jml.2016.10.003
https://eprints.bournemouth.ac.uk/27412/
Source: Web of Science (Lite)
False positives and other statistical errors in standard analyses of eye movements in reading
Authors: von der Malsburg, T. and Angele, B.
Journal: Journal of Memory and Language
Volume: 94
Pages: 119-133
ISSN: 0749-596X
DOI: 10.1016/j.jml.2016.10.003
Abstract:© 2016 Elsevier Inc. In research on eye movements in reading, it is common to analyze a number of canonical dependent measures to study how the effects of a manipulation unfold over time. Although this gives rise to the well-known multiple comparisons problem, i.e. an inflated probability that the null hypothesis is incorrectly rejected (Type I error), it is accepted standard practice not to apply any correction procedures. Instead, there appears to be a widespread belief that corrections are not necessary because the increase in false positives is too small to matter. To our knowledge, no formal argument has ever been presented to justify this assumption. Here, we report a computational investigation of this issue using Monte Carlo simulations. Our results show that, contrary to conventional wisdom, false positives are increased to unacceptable levels when no corrections are applied. Our simulations also show that counter-measures like the Bonferroni correction keep false positives in check while reducing statistical power only moderately. Hence, there is little reason why such corrections should not be made a standard requirement. Further, we discuss three statistical illusions that can arise when statistical power is low, and we show how power can be improved to prevent these illusions. In sum, our work renders a detailed picture of the various types of statistical errors than can occur in studies of reading behavior and we provide concrete guidance about how these errors can be avoided.
https://eprints.bournemouth.ac.uk/27412/
Source: Manual
Preferred by: Bernhard Angele
False Positives and Other Statistical Errors in Standard Analyses of Eye Movements in Reading.
Authors: von der Malsburg, T. and Angele, B.
Journal: Journal of memory and language
Volume: 94
Pages: 119-133
eISSN: 1096-0821
ISSN: 0749-596X
DOI: 10.1016/j.jml.2016.10.003
Abstract:In research on eye movements in reading, it is common to analyze a number of canonical dependent measures to study how the effects of a manipulation unfold over time. Although this gives rise to the well-known multiple comparisons problem, i.e. an inflated probability that the null hypothesis is incorrectly rejected (Type I error), it is accepted standard practice not to apply any correction procedures. Instead, there appears to be a widespread belief that corrections are not necessary because the increase in false positives is too small to matter. To our knowledge, no formal argument has ever been presented to justify this assumption. Here, we report a computational investigation of this issue using Monte Carlo simulations. Our results show that, contrary to conventional wisdom, false positives are increased to unacceptable levels when no corrections are applied. Our simulations also show that counter-measures like the Bonferroni correction keep false positives in check while reducing statistical power only moderately. Hence, there is little reason why such corrections should not be made a standard requirement. Further, we discuss three statistical illusions that can arise when statistical power is low, and we show how power can be improved to prevent these illusions. In sum, our work renders a detailed picture of the various types of statistical errors than can occur in studies of reading behavior and we provide concrete guidance about how these errors can be avoided.
https://eprints.bournemouth.ac.uk/27412/
Source: Europe PubMed Central
False positives and other statistical errors in standard analyses of eye movements in reading
Authors: von der Malsburg, T. and Angele, B.
Journal: Journal of Memory and Language
Volume: 94
Issue: June
Pages: 119-133
ISSN: 0749-596X
Abstract:In research on eye movements in reading, it is common to analyze a number of canonical dependent measures to study how the effects of a manipulation unfold over time. Although this gives rise to the well-known multiple comparisons problem, i.e. an inflated probability that the null hypothesis is incorrectly rejected (Type I error), it is accepted standard practice not to apply any correction procedures. Instead, there appears to be a widespread belief that corrections are not necessary because the increase in false positives is too small to matter. To our knowledge, no formal argument has ever been presented to justify this assumption. Here, we report a computational investigation of this issue using Monte Carlo simulations. Our results show that, contrary to conventional wisdom, false positives are increased to unacceptable levels when no corrections are applied. Our simulations also show that counter-measures like the Bonferroni correction keep false positives in check while reducing statistical power only moderately. Hence, there is little reason why such corrections should not be made a standard requirement. Further, we discuss three statistical illusions that can arise when statistical power is low, and we show how power can be improved to prevent these illusions. In sum, our work renders a detailed picture of the various types of statistical errors than can occur in studies of reading behavior and we provide concrete guidance about how these errors can be avoided.
https://eprints.bournemouth.ac.uk/27412/
Source: BURO EPrints