The Value of Accepting the Null Hypothesis
1 Background
In standard frequentist models, we cannot formally accept the Null Hypothesis \(H_0\), but can only reject, or fail to reject, \(H_0\).
Bayesian models allow one to both accept and reject \(H_0\) (Kruschke and Liddell 2018).
Accepting \(H_0\) may be very scientifically valuable, and may have consequences for affirming similarity, universality, or treatment invariance (Gallistel 2009; Morey, Homer, and Proulx 2018). The ability to accept \(H_0\) may also lead to a lower likelihood of the publication bias that results from frequentist methods predicated upon the rejection of \(H_0\) (Kruschke and Liddell 2018).
This handout is written from a Bayesian perspective. However, even from a traditional frequentist statistical perspective, it may be helpful to think about the value of results that are not statistically significant.
A finding of a null result is dependent on having enough statistical power that one might plausibly detect an effect were an effect to exist.
2 Important Substantive Cases
The Value of Accepting the Null Hypothesis \(H_0\)
| case | description | H_0 | example |
|---|---|---|---|
| Equivalence Testing | Equivalence Of 2 Treatments Or Interventions | $$\beta_1 = \beta_2$$ | The effect of Treatment 1 is indistinguishable from the effect of Treatment 2 (especially important if one treatment is much more expensive, or time consuming than another). |
| Equivalence Testing | Equivalence Of 2 Groups On An Outcome | $$\bar{y_1} = \bar{y_2}$$ or in multilevel modeling $$u_0 = 0$$ | People identifying as men and people identifying as women are more similar than different with regard to psychological processes (Hyde2005). |
| Retiring Interventions | There Is No Evidence That Intervention X Is Effective | $$\beta_{intervention} = 0$$ | Evidence consistently suggests that a particular treatment has near zero effect. |
| Contextual Equivalence | Equivalence of a Predictor Across Contexts (Moderation) | $$\beta_{interaction} = 0$$ or in multilevel modeling $$u_k = 0$$ | Warm and supportive parenting is equally beneficial across different contexts or countries. |
| Family Member Equivalence | Equivalence of a Predictor Across Family Members | $$\beta_{parent1} = \beta_{parent2}$$ | Parenting from one parent is equivalent to parenting from another parent |
| Full Mediation | Association of x and y Is Completely Mediated; No Direct Effect | $$\beta_{xmy} \neq 0$$ $$\beta_{xy} = 0$$ | The relationship of the treatment and the outcome is completely mediated by mechanism m. |
| No Mediation | No Indirect Effect; Association of x and y Is Not Mediated by m | $$\beta_{xmy} = 0$$ $$\beta_{xy} \neq 0$$ | The relationship of the treatment and the outcome is not mediated at all by mechanism m. |
| Theory Simplification | Removing An Association From A Theory | $$\beta_x = 0$$ | There is no evidence that x is associated with y. |
| Theory Rejection | Rejecting A Theory | $$\beta_{theory} = 0$$ | There is strong evidence (contra Theory X) that x is not associated with y. |