Skip to content

bayes

Functions for working with p-values and Bayes factors.

This module provides functions for converting between p-values and Bayes factors, as well as calculating Bayes factors from posterior distributions. These are useful for hypothesis testing and interpreting statistical evidence.

as_bf(pvalue)

Convert a one-sided p-value to a Bayes factor.

This Bayes factor quantifies evidence in favour of a directional effect over the null hypothesis of no such effect.

Parameters:

Name Type Description Default
pvalue NDArray | float

The p-value(s) to be converted to Bayes factor(s). Can be a single value or an array of values.

 required

Returns:

Type Description
NDArray | float

The corresponding Bayes factor(s). The return type matches the input type.
See Also

as_pvalue : Convert a Bayes factor to a p-value.

Notes

The Bayes factor is derived from the one-sided p-value using a Bayesian interpretation, which quantifies evidence in favor of a directional effect over the null hypothesis of no such effect.

See [1]_ for theoretical details and implications.

References

.. [1] Marsman, Maarten, and Eric-Jan Wagenmakers. "Three Insights from a Bayesian Interpretation of the One-Sided P Value." Educational and Psychological Measurement, vol. 77, no. 3, 2017, pp. 529-539. doi:10.1177/0013164416669201.

Examples:

>>> import numpy as np
>>> from samesame.bayes import as_bf
>>> as_bf(0.5)
np.float64(1.0)
>>> as_bf(np.array([0.05, 0.1, 0.5]))
array([19., 9., 1.])
Source code in src/samesame/bayes.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def as_bf(
    pvalue: NDArray | float,
) -> NDArray | float:
    """
    Convert a one-sided p-value to a Bayes factor.

    This Bayes factor quantifies evidence in favour of a directional effect
    over the null hypothesis of no such effect.

    Parameters
    ----------
    pvalue : NDArray | float
        The p-value(s) to be converted to Bayes factor(s). Can be a single
        value or an array of values.

    Returns
    -------
    NDArray | float
        The corresponding Bayes factor(s). The return type matches the
        input type.

    See Also
    --------
    as_pvalue : Convert a Bayes factor to a p-value.

    Notes
    -----
    The Bayes factor is derived from the one-sided p-value using a
    Bayesian interpretation, which quantifies evidence in favor of a
    directional effect over the null hypothesis of no such effect.

    See [1]_ for theoretical details and implications.

    References
    ----------
    .. [1] Marsman, Maarten, and Eric-Jan Wagenmakers. "Three Insights from
       a Bayesian Interpretation of the One-Sided P Value." *Educational and
       Psychological Measurement*, vol. 77, no. 3, 2017, pp. 529-539.
       doi:10.1177/0013164416669201.

    Examples
    --------
    >>> import numpy as np
    >>> from samesame.bayes import as_bf
    >>> as_bf(0.5)
    np.float64(1.0)
    >>> as_bf(np.array([0.05, 0.1, 0.5])) # doctest: +NORMALIZE_WHITESPACE
    array([19., 9., 1.])
    """
    if np.any(np.logical_or(pvalue >= 1, pvalue <= 0)):
        raise ValueError("pvalue must be within the open interval (0, 1).")
    pvalue = np.clip(pvalue, 1e-10, 1 - 1e-10)  # Avoid numerical issues at extremes
    return 1.0 / np.exp(logit(pvalue))  # evidence in favor of an effect

as_pvalue(bayes_factor)

Convert a Bayes factor of a directional effect to a one-sided p-value.

The Bayes factor quantifies evidence in favour of a directional effect over the null hypothesis of no such effect.

Parameters:

Name Type Description Default
bayes_factor float | NDArray

The Bayes factor(s) to be converted to p-value(s). Can be a single value or an array of values.

 required

Returns:

Type Description
float | NDArray

The corresponding p-value(s). The return type matches the input type.
See Also

as_pvalue : Convert a Bayes factor to a p-value.

Notes

The equivalence between the Bayes factor in favor of a directional effect and the one-sided p-value for the null of no such effect is discussed in [1]_.

References

.. [1] Marsman, Maarten, and Eric-Jan Wagenmakers. "Three Insights from a Bayesian Interpretation of the One-Sided P Value." Educational and Psychological Measurement, vol. 77, no. 3, 2017, pp. 529–539, https://doi.org/10.1177/0013164416669201.

Examples:

>>> import numpy as np
>>> from samesame.bayes import as_pvalue
>>> as_pvalue(1)
np.float64(0.5)
>>> as_pvalue(np.array([19.0, 9.0, 1.0]))
array([0.05, 0.1 , 0.5 ])
Source code in src/samesame/bayes.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
def as_pvalue(
    bayes_factor: float | NDArray,
) -> float | NDArray:
    """
    Convert a Bayes factor of a directional effect to a one-sided p-value.

    The Bayes factor quantifies evidence in favour of a directional effect
    over the null hypothesis of no such effect.

    Parameters
    ----------
    bayes_factor : float | NDArray
        The Bayes factor(s) to be converted to p-value(s). Can be a single
        value or an array of values.

    Returns
    -------
    float | NDArray
        The corresponding p-value(s). The return type matches the input type.

    See Also
    --------
    as_pvalue : Convert a Bayes factor to a p-value.

    Notes
    -----
    The equivalence between the Bayes factor in favor of a directional effect
    and the one-sided p-value for the null of no such effect is discussed in
    [1]_.

    References
    ----------
    .. [1] Marsman, Maarten, and Eric-Jan Wagenmakers. "Three Insights from a
       Bayesian Interpretation of the One-Sided P Value." *Educational and
       Psychological Measurement*, vol. 77, no. 3, 2017, pp. 529–539,
       https://doi.org/10.1177/0013164416669201.

    Examples
    --------
    >>> import numpy as np
    >>> from samesame.bayes import as_pvalue
    >>> as_pvalue(1)
    np.float64(0.5)
    >>> as_pvalue(np.array([19.0, 9.0, 1.0])) # doctest: +NORMALIZE_WHITESPACE
    array([0.05, 0.1 , 0.5 ])
    """
    if np.any(bayes_factor <= 0):
        raise ValueError("bayes_factor must be strictly positive.")
    bf_ = np.clip(bayes_factor, 1e-10, 1e10)  # Ensure numerical stability
    pvalue = expit(-np.log(bf_))
    return pvalue

bayes_factor(posterior, threshold=0.0, adjustment=0)

Compute the Bayes factor for a test of direction given a threshold.

The Bayes factor quantifies the evidence in favor of the hypothesis that the posterior distribution exceeds the given threshold compared to the hypothesis that it does not.

Parameters:

Name Type Description Default
posterior NDArray

An array of posterior samples.

 required
threshold float

The threshold value to test against. Default is 0.0.

 0.0
adjustment (0, 1)

Adjustment to apply to the Bayes factor calculation. Default is 0.

 0

Returns:

Type Description
float

The computed Bayes factor.
See Also

as_pvalue : Convert a Bayes factor to a p-value.

as_bf : Convert a p-value to a Bayes factor.

Notes

The Bayes factor is a measure of evidence which compares the likelihood of the data under two competing hypotheses. In this function, the numerator represents the proportion of posterior samples exceeding the threshold, while the denominator represents the proportion of samples not exceeding the threshold. If all samples exceed the threshold, the Bayes factor is set to infinity, indicating overwhelming evidence in favor of the hypothesis.

The adjustment parameter allows for slight modifications to the Bayes factor calculation, which can be useful in specific contexts such as sensitivity analyses.

See [1]_ for further theoretical details and practical implications of this approach.

References

.. [1] Marsman, Maarten, and Eric-Jan Wagenmakers. "Three Insights from a Bayesian Interpretation of the One-Sided P Value." Educational and Psychological Measurement, vol. 77, no. 3, 2017, pp. 529-539. doi:10.1177/0013164416669201.

Examples:

>>> import numpy as np
>>> from samesame.bayes import bayes_factor
>>> posterior_samples = np.array([0.2, 0.5, 0.8, 0.9])
>>> bayes_factor(posterior_samples, threshold=0.5)
np.float64(1.0)
Source code in src/samesame/bayes.py 
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
def bayes_factor(
    posterior: NDArray,
    threshold: float = 0.0,
    adjustment: Literal[0, 1] = 0,
) -> float:
    """
    Compute the Bayes factor for a test of direction given a threshold.

    The Bayes factor quantifies the evidence in favor of the hypothesis that
    the posterior distribution exceeds the given threshold compared to the
    hypothesis that it does not.

    Parameters
    ----------
    posterior : NDArray
        An array of posterior samples.
    threshold : float, optional
        The threshold value to test against. Default is 0.0.
    adjustment : {0, 1}, optional
        Adjustment to apply to the Bayes factor calculation. Default is 0.

    Returns
    -------
    float
        The computed Bayes factor.

    See Also
    --------
    as_pvalue : Convert a Bayes factor to a p-value.

    as_bf : Convert a p-value to a Bayes factor.

    Notes
    -----
    The Bayes factor is a measure of evidence which compares the likelihood of
    the data under two competing hypotheses. In this function, the numerator
    represents the proportion of posterior samples exceeding the threshold,
    while the denominator represents the proportion of samples not exceeding
    the threshold. If all samples exceed the threshold, the Bayes factor is
    set to infinity, indicating overwhelming evidence in favor of the
    hypothesis.

    The adjustment parameter allows for slight modifications to the Bayes
    factor calculation, which can be useful in specific contexts such as
    sensitivity analyses.

    See [1]_ for further theoretical details and practical implications of
    this approach.

    References
    ----------
    .. [1] Marsman, Maarten, and Eric-Jan Wagenmakers. "Three Insights from a
       Bayesian Interpretation of the One-Sided P Value." *Educational and
       Psychological Measurement*, vol. 77, no. 3, 2017, pp. 529-539.
       doi:10.1177/0013164416669201.

    Examples
    --------
    >>> import numpy as np
    >>> from samesame.bayes import bayes_factor
    >>> posterior_samples = np.array([0.2, 0.5, 0.8, 0.9])
    >>> bayes_factor(posterior_samples, threshold=0.5)
    np.float64(1.0)
    """
    return _bayes_factor(posterior, threshold, adjustment)