Table of Contents

1. Introduction; 2. The basics of hypothesis testing; 3. Robustness of the Two-sample t-test; 4. Adding data increases the Type I error rate: optional stopping; 5. ANOVA can be extremely conservative; 6. ANOVA handles only one type of multiple testing problem; 7. Power analyses should consider all relevant tests; 8. The only p-value you can plan for is zero; 9. Subjects and trials do not trade off evenly; 10. Replication is a poor way to control Type I error; 11. Identifying improper methods through excess success; 12. Preregistration may be useful but is not necessary for good science; 13. Hypothesis testing is a variation of signal detection theory; 14. Using signal detection theory to analyze reported results of hypothesis testing; 15. Conclusions.

Promotional Information

Describes several properties of hypothesis testing, with special emphasis on analyses common to studies of perception.