Weighting API¶

Functions and data classes for reweighting source and target comparisons toward common support.

Public functions¶

`samesame.weights.domain_weights(*, source, target, reweight=ReweightMode.BOTH, shrinkage=0.5)` ¶

Turn domain probabilities into weights that focus on common support.

Give it separate P(target|x) arrays for source and target observations — the probability that each row belongs to target rather than source, from a domain classifier. The prior ratio n_source / n_target is inferred from their lengths, so pass arrays aligned to the scores you intend to test. Values are clipped to [1e-6, 1 - 1e-6] before ratios to avoid infinities; clipping guards the arithmetic but does not rescue a poorly estimated classifier.

Use the domain probability to build weights; use a separate, interpretable score (risk, error, confidence, or outlier score) for the harm test. Membership is not outcome quality.

Parameters:

Name	Type	Description	Default
`source`	`ArrayLike`	Domain probabilities `P(target\|x)` for source observations, each in `[0, 1]`. Estimate out of sample (e.g., `cross_val_predict`) or otherwise honestly.	required
`target`	`ArrayLike`	Domain probabilities `P(target\|x)` for target observations, each in `[0, 1]`. Estimate out of sample.	required
`reweight`	`('source', 'target', 'both')`	Which group(s) to adjust toward common support. Default `'both'` — reweight both toward mutual support. `'source'` reweights source toward target; `'target'` does the reverse. Accepts a plain string or :class:`ReweightMode`.	`'source'`
`shrinkage`	`float`	Shrinkage `λ` in `[0, 1]` (default `0.5`) blending RIW toward uniform weights. `0` is the plain density ratio — strongest correction, highest variance. `1` is uniform — no correction. `0.5` is the recommended default; start there and inspect `ESS/n` before lowering (lower = more aggressive correction).	`0.5`

Returns:

Type	Description
`ImportanceWeights`	Normalized weights per group, each summing to its sample size. Inactive groups keep weight `1` for every observation.

Raises:

Type	Description
`ValueError`	If probabilities are outside `[0, 1]`, empty, or non-finite; if `shrinkage` is outside `[0, 1]` or non-finite; or if `reweight` is invalid.

See Also

ImportanceWeights : Container that normalizes and validates weights. EffectiveSampleSize : Per-group ESS and ESS/n interpretation. ReweightMode : "source", "target", "both" in plain language. samesame.shift.test_shift : Any-shift test that can consume weights. samesame.shift.test_harm : Directional test that can consume weights.

Notes

Start unweighted. Use weights only when poor feature overlap is a real concern — weighting changes the population the test describes and is not a default correction.
Estimate P(target|x) out of sample and keep it separate from the harm score. Domain probability describes membership; it says nothing about whether an outcome is good or bad.
Call .effective_sample_size() on the result and compare each ESS to its n via ESS/n. A low ratio (e.g., substantially below 0.5) warns that a few observations dominate. If ESS/n stays low at shrinkage=0.5, the groups may not have enough common support for a reliable weighted comparison — consider leaving the comparison unweighted. The often-quoted ESS < n/4 is only a rough illustrative heuristic, not a validated cutoff.

References

Kish, L. (1965). Survey Sampling. Wiley. Bickel, S. et al. (2007). Discriminative learning for differing training and test distributions. ICML 24:81-88. Yamada, M. et al. (2013). Relative density-ratio estimation. Neural Comput. 25(5):1324-1370. Elvira, V. et al. (2022). Rethinking the effective sample size. Int. Stat. Rev. 90(3):525-550.

Examples:

>>> import numpy as np
>>> from samesame.weights import domain_weights
>>> source = np.array([0.25, 0.4])
>>> target = np.array([0.6, 0.75])
>>> w = domain_weights(source=source, target=target)
>>> np.round(w.source, 4)
array([0.7692, 1.2308])
>>> w.effective_sample_size().source < 2.0
True

Source code in src/samesame/weights.py

def domain_weights(
    *,
    source: ArrayLike,
    target: ArrayLike,
    reweight: ReweightMode | str = ReweightMode.BOTH,
    shrinkage: float = 0.5,
) -> ImportanceWeights:
    """
    Turn domain probabilities into weights that focus on common support.

    Give it separate ``P(target|x)`` arrays for source and target
    observations — the probability that each row belongs to target rather
    than source, from a domain classifier. The prior ratio
    ``n_source / n_target`` is inferred from their lengths, so pass arrays
    aligned to the scores you intend to test. Values are clipped to
    ``[1e-6, 1 - 1e-6]`` before ratios to avoid infinities; clipping
    guards the arithmetic but does not rescue a poorly estimated classifier.

    Use the domain probability to build *weights*; use a separate,
    interpretable score (risk, error, confidence, or outlier score) for
    the harm test. Membership is not outcome quality.

    Parameters
    ----------
    source : ArrayLike
        Domain probabilities ``P(target|x)`` for source observations, each
        in ``[0, 1]``. Estimate out of sample (e.g., ``cross_val_predict``)
        or otherwise honestly.
    target : ArrayLike
        Domain probabilities ``P(target|x)`` for target observations, each
        in ``[0, 1]``. Estimate out of sample.
    reweight : {'source', 'target', 'both'} or ReweightMode, optional
        Which group(s) to adjust toward common support. Default
        ``'both'`` — reweight both toward mutual support. ``'source'``
        reweights source toward target; ``'target'`` does the reverse.
        Accepts a plain string or :class:`ReweightMode`.
    shrinkage : float, optional
        Shrinkage ``λ`` in ``[0, 1]`` (default ``0.5``) blending RIW
        toward uniform weights. ``0`` is the plain density ratio — strongest
        correction, highest variance. ``1`` is uniform — no correction.
        ``0.5`` is the recommended default; start there and inspect
        ``ESS/n`` before lowering (lower = more aggressive correction).

    Returns
    -------
    ImportanceWeights
        Normalized weights per group, each summing to its sample size.
        Inactive groups keep weight ``1`` for every observation.

    Raises
    ------
    ValueError
        If probabilities are outside ``[0, 1]``, empty, or non-finite; if
        ``shrinkage`` is outside ``[0, 1]`` or non-finite; or if
        ``reweight`` is invalid.

    See Also
    --------
    ImportanceWeights : Container that normalizes and validates weights.
    EffectiveSampleSize : Per-group ESS and ESS/n interpretation.
    ReweightMode : ``"source"``, ``"target"``, ``"both"`` in plain language.
    samesame.shift.test_shift : Any-shift test that can consume weights.
    samesame.shift.test_harm : Directional test that can consume weights.

    Notes
    -----
    * Start unweighted. Use weights only when poor feature overlap is a
      real concern — weighting changes the population the test
      describes and is not a default correction.
    * Estimate ``P(target|x)`` out of sample and keep it separate from
      the harm score. Domain probability describes membership; it says
      nothing about whether an outcome is good or bad.
    * Call ``.effective_sample_size()`` on the result and compare each
      ESS to its ``n`` via ``ESS/n``. A low ratio (e.g., substantially
      below ``0.5``) warns that a few observations dominate. If ``ESS/n``
      stays low at ``shrinkage=0.5``, the groups may not have enough common
      support for a reliable weighted comparison — consider leaving the
      comparison unweighted. The often-quoted ``ESS < n/4`` is only a rough
      illustrative heuristic, not a validated cutoff.

    References
    ----------
    Kish, L. (1965). *Survey Sampling*. Wiley.
    Bickel, S. et al. (2007). Discriminative learning for differing training
        and test distributions. *ICML* 24:81-88.
    Yamada, M. et al. (2013). Relative density-ratio estimation. *Neural
        Comput.* 25(5):1324-1370.
    Elvira, V. et al. (2022). Rethinking the effective sample size.
        *Int. Stat. Rev.* 90(3):525-550.

    Examples
    --------
    >>> import numpy as np
    >>> from samesame.weights import domain_weights
    >>> source = np.array([0.25, 0.4])
    >>> target = np.array([0.6, 0.75])
    >>> w = domain_weights(source=source, target=target)
    >>> np.round(w.source, 4)
    array([0.7692, 1.2308])
    >>> w.effective_sample_size().source < 2.0
    True
    """
    reweight_enum = _coerce_reweight(reweight)
    source_p = _as_prob_vector(source, name="source")
    target_p = _as_prob_vector(target, name="target")

    source_p = np.clip(source_p, _CLIP, 1.0 - _CLIP)
    target_p = np.clip(target_p, _CLIP, 1.0 - _CLIP)

    lam = float(shrinkage)
    if not np.isfinite(lam) or lam < 0.0 or lam > 1.0:
        raise ValueError("shrinkage must be in [0, 1] and finite.")

    n_source, n_target = len(source_p), len(target_p)
    prior_ratio = n_source / n_target

    # density ratio r = p/(1-p) * prior_ratio
    source_r = (source_p / (1.0 - source_p)) * prior_ratio
    target_r = (target_p / (1.0 - target_p)) * prior_ratio

    # RIW formulas (Yamada et al. 2013)
    # source: r / ((1-lam) + lam*r), target: 1 / (lam + (1-lam)*r)
    out_source = np.ones(n_source, dtype=np.float64)
    out_target = np.ones(n_target, dtype=np.float64)

    if reweight_enum in (ReweightMode.SOURCE, ReweightMode.BOTH):
        out_source = source_r / ((1.0 - lam) + lam * source_r)
    if reweight_enum in (ReweightMode.TARGET, ReweightMode.BOTH):
        out_target = 1.0 / (lam + (1.0 - lam) * target_r)

    return ImportanceWeights(source=out_source, target=out_target)

Public data classes¶

`samesame.weights.ImportanceWeights` `dataclass` ¶

Validated, ready-to-use importance weights for source and target.

Bring your own sample weights, or let :func:domain_weights estimate them from domain probabilities P(target|x). Either way, this class validates, normalizes, and carries them to the test.

Reweighting changes which observations count more; it does not change nominal group sizes in the permutation test — each group's weights are normalized to sum to that group's size, so the labels still permute over n_source + n_target slots.

Parameters:

Name	Type	Description	Default
`source`	`ArrayLike`	Raw weights for source observations.	required
`target`	`ArrayLike`	Raw weights for target observations.	required

Attributes:

Name	Type	Description
`source`	`NDArray[float64]`	Weights for source observations, normalized to sum to `len(source)`. Inactive groups (per :class:`ReweightMode`) stay at `1`.
`target`	`NDArray[float64]`	Weights for target observations, normalized to sum to `len(target)`. Inactive groups stay at `1`.

See Also

domain_weights : Estimate weights from P(target|x). EffectiveSampleSize : Diagnose weight concentration via :meth:effective_sample_size. samesame.shift.test_shift : The tests that consume these weights.

Notes

On construction, inputs are coerced to finite one-dimensional float arrays, checked for non-negativity, and normalized per group. An inactive group keeps weight 1 for every observation.

Examples:

>>> import numpy as np
>>> from samesame.weights import ImportanceWeights
>>> w = ImportanceWeights(source=np.array([0.5, 1.5]), target=np.array([1.0, 1.0]))
>>> float(w.source.sum()), float(w.target.sum())
(2.0, 2.0)

Source code in src/samesame/weights.py

@dataclass(frozen=True, repr=False)
class ImportanceWeights:
    """
    Validated, ready-to-use importance weights for source and target.

    Bring your own sample weights, or let :func:`domain_weights` estimate
    them from domain probabilities ``P(target|x)``. Either way, this class
    validates, normalizes, and carries them to the test.

    Reweighting changes which observations count more; it does not change
    nominal group sizes in the permutation test — each group's weights are
    normalized to sum to that group's size, so the labels still permute
    over ``n_source + n_target`` slots.

    Parameters
    ----------
    source : ArrayLike
        Raw weights for source observations.
    target : ArrayLike
        Raw weights for target observations.

    Attributes
    ----------
    source : NDArray[np.float64]
        Weights for source observations, normalized to sum to ``len(source)``.
        Inactive groups (per :class:`ReweightMode`) stay at ``1``.
    target : NDArray[np.float64]
        Weights for target observations, normalized to sum to ``len(target)``.
        Inactive groups stay at ``1``.

    See Also
    --------
    domain_weights : Estimate weights from ``P(target|x)``.
    EffectiveSampleSize : Diagnose weight concentration via
        :meth:`effective_sample_size`.
    samesame.shift.test_shift : The tests that consume these weights.

    Notes
    -----
    On construction, inputs are coerced to finite one-dimensional float
    arrays, checked for non-negativity, and normalized per group. An
    inactive group keeps weight ``1`` for every observation.

    Examples
    --------
    >>> import numpy as np
    >>> from samesame.weights import ImportanceWeights
    >>> w = ImportanceWeights(source=np.array([0.5, 1.5]), target=np.array([1.0, 1.0]))
    >>> float(w.source.sum()), float(w.target.sum())
    (2.0, 2.0)
    """

    source: NDArray[np.float64]
    target: NDArray[np.float64]

    def __post_init__(self) -> None:
        src = _normalize(_as_weight_array(self.source, name="weights.source"))
        tgt = _normalize(_as_weight_array(self.target, name="weights.target"))
        object.__setattr__(self, "source", src)
        object.__setattr__(self, "target", tgt)

    def __repr__(self) -> str:
        def _render(a: NDArray[np.float64]) -> str:
            return np.array2string(a, threshold=8, edgeitems=2)

        return f"{type(self).__name__}(source={_render(self.source)}, target={_render(self.target)})"

    def effective_sample_size(self) -> EffectiveSampleSize:
        """
        How much independent information remains after weighting.

        Returns Kish's ``(sum w)² / sum w²`` (Kish, 1965) per group. Uniform
        weights give ``ESS == n``; concentrated weights where a handful
        dominate push ESS toward ``1``. Interpret ``ESS/n`` continuously; a
        low ratio warns that the weighted result leans on a few observations.

        If ESS stays low even at ``shrinkage=0.5``, the groups may lack
        enough common support for a reliable weighted comparison — consider
        leaving the comparison unweighted.

        Returns
        -------
        EffectiveSampleSize
            ESS per group (``.source``, ``.target``).

        References
        ----------
        Kish, L. (1965). *Survey Sampling*. Wiley, New York.
        Elvira, V. et al. (2022). Rethinking the effective sample size.
            *Int. Stat. Rev.* 90(3):525-550.

        Examples
        --------
        >>> import numpy as np
        >>> from samesame.weights import domain_weights
        >>> w = domain_weights(source=np.array([0.25, 0.4]), target=np.array([0.6, 0.75]))
        >>> ess = w.effective_sample_size()
        >>> round(ess.source, 4)
        1.8989
        """
        src = float(self.source.sum() ** 2 / (self.source**2).sum())
        tgt = float(self.target.sum() ** 2 / (self.target**2).sum())
        return EffectiveSampleSize(source=src, target=tgt)

`effective_sample_size()` ¶

How much independent information remains after weighting.

Returns Kish's (sum w)² / sum w² (Kish, 1965) per group. Uniform weights give ESS == n; concentrated weights where a handful dominate push ESS toward 1. Interpret ESS/n continuously; a low ratio warns that the weighted result leans on a few observations.

If ESS stays low even at shrinkage=0.5, the groups may lack enough common support for a reliable weighted comparison — consider leaving the comparison unweighted.

Returns:

Type	Description
`EffectiveSampleSize`	ESS per group (`.source`, `.target`).

References

Kish, L. (1965). Survey Sampling. Wiley, New York. Elvira, V. et al. (2022). Rethinking the effective sample size. Int. Stat. Rev. 90(3):525-550.

Examples:

>>> import numpy as np
>>> from samesame.weights import domain_weights
>>> w = domain_weights(source=np.array([0.25, 0.4]), target=np.array([0.6, 0.75]))
>>> ess = w.effective_sample_size()
>>> round(ess.source, 4)
1.8989

Source code in src/samesame/weights.py

def effective_sample_size(self) -> EffectiveSampleSize:
    """
    How much independent information remains after weighting.

    Returns Kish's ``(sum w)² / sum w²`` (Kish, 1965) per group. Uniform
    weights give ``ESS == n``; concentrated weights where a handful
    dominate push ESS toward ``1``. Interpret ``ESS/n`` continuously; a
    low ratio warns that the weighted result leans on a few observations.

    If ESS stays low even at ``shrinkage=0.5``, the groups may lack
    enough common support for a reliable weighted comparison — consider
    leaving the comparison unweighted.

    Returns
    -------
    EffectiveSampleSize
        ESS per group (``.source``, ``.target``).

    References
    ----------
    Kish, L. (1965). *Survey Sampling*. Wiley, New York.
    Elvira, V. et al. (2022). Rethinking the effective sample size.
        *Int. Stat. Rev.* 90(3):525-550.

    Examples
    --------
    >>> import numpy as np
    >>> from samesame.weights import domain_weights
    >>> w = domain_weights(source=np.array([0.25, 0.4]), target=np.array([0.6, 0.75]))
    >>> ess = w.effective_sample_size()
    >>> round(ess.source, 4)
    1.8989
    """
    src = float(self.source.sum() ** 2 / (self.source**2).sum())
    tgt = float(self.target.sum() ** 2 / (self.target**2).sum())
    return EffectiveSampleSize(source=src, target=tgt)

`samesame.weights.EffectiveSampleSize` `dataclass` ¶

Kish effective sample size — how much information is left after weighting.

Kish's ESS (sum w)² / sum w² (Kish, 1965): uniform weights keep every voice — ESS == n; when a few observations shout while the rest whisper, ESS slides toward 1. It interpolates between n (uniform) and 1 (one observation dominates).

Compare each ESS to its n via the ratio ESS/n. There is no universal cutoff from Kish; interpret ESS/n as a continuous diagnostic. A low ratio — for example substantially below 0.5 or, as a rough illustrative heuristic, ESS < n/4 — means the weighted result leans on a few observations and should be interpreted cautiously, not as a hard validation rule. The n/4 figure is a package heuristic with no published empirical threshold (see Elvira et al., 2022 for caveats on ESS-based cutoffs).

Parameters:

Name	Type	Description	Default
`source`	`float`	Effective sample size for source weights.	required
`target`	`float`	Effective sample size for target weights.	required

Attributes:

Name	Type	Description
`source`	`float`	ESS for the source weights.
`target`	`float`	ESS for the target weights.

See Also

ImportanceWeights.effective_sample_size : Compute this from weights. samesame.weights.domain_weights : Where shrinkage trades bias for stability.

References

Kish, L. (1965). Survey Sampling. Wiley, New York. Elvira, V., Martino, L., Robert, C. P. (2022). Rethinking the effective sample size. International Statistical Review 90(3):525-550. https://doi.org/10.1111/insr.12500

Source code in src/samesame/weights.py

@dataclass(frozen=True)
class EffectiveSampleSize:
    """
    Kish effective sample size — how much information is left after weighting.

    Kish's ESS ``(sum w)² / sum w²`` (Kish, 1965): uniform weights keep
    every voice — ``ESS == n``; when a few observations shout while the
    rest whisper, ESS slides toward ``1``. It interpolates between ``n``
    (uniform) and ``1`` (one observation dominates).

    Compare each ESS to its ``n`` via the ratio ``ESS/n``. There is no
    universal cutoff from Kish; interpret ``ESS/n`` as a continuous
    diagnostic. A low ratio — for example substantially below ``0.5`` or,
    as a rough illustrative heuristic, ``ESS < n/4`` — means the weighted
    result leans on a few observations and should be interpreted cautiously,
    not as a hard validation rule. The ``n/4`` figure is a package heuristic
    with no published empirical threshold (see Elvira et al., 2022 for
    caveats on ESS-based cutoffs).

    Parameters
    ----------
    source : float
        Effective sample size for source weights.
    target : float
        Effective sample size for target weights.

    Attributes
    ----------
    source : float
        ESS for the source weights.
    target : float
        ESS for the target weights.

    See Also
    --------
    ImportanceWeights.effective_sample_size : Compute this from weights.
    samesame.weights.domain_weights : Where shrinkage trades bias for stability.

    References
    ----------
    Kish, L. (1965). *Survey Sampling*. Wiley, New York.
    Elvira, V., Martino, L., Robert, C. P. (2022). Rethinking the effective
        sample size. *International Statistical Review* 90(3):525-550.
        https://doi.org/10.1111/insr.12500
    """

    source: float
    target: float

Public enum¶

`samesame.weights.ReweightMode` ¶

Bases: StrEnum

Which group(s) to reweight toward common support.

Reweighting does not invent information where the groups do not overlap — it changes which observations count more. Pick the mode that matches where the fringe lives. Pass a member or its plain string value to :func:domain_weights.

Attributes:

Name	Type	Description
`SOURCE`	`ReweightMode`	Reweight source toward target; target unchanged. Use when source has low-overlap observations outside target support.
`TARGET`	`ReweightMode`	Reweight target toward source; source unchanged. Use when target has low-overlap observations outside source support.
`BOTH`	`ReweightMode`	Reweight both groups toward their mutual support (default). Use when both groups have low-overlap regions.

See Also

domain_weights : The function that consumes this choice. samesame.weights.ImportanceWeights : What you get back.

Examples:

>>> from samesame.weights import ReweightMode
>>> ReweightMode("both") == ReweightMode.BOTH
True

Source code in src/samesame/weights.py

class ReweightMode(StrEnum):
    """
    Which group(s) to reweight toward common support.

    Reweighting does not invent information where the groups do not
    overlap — it changes which observations count more. Pick the mode
    that matches where the fringe lives. Pass a member or its plain
    string value to :func:`domain_weights`.

    Attributes
    ----------
    SOURCE : ReweightMode
        Reweight source toward target; target unchanged. Use when source
        has low-overlap observations outside target support.
    TARGET : ReweightMode
        Reweight target toward source; source unchanged. Use when target
        has low-overlap observations outside source support.
    BOTH : ReweightMode
        Reweight both groups toward their mutual support (default).
        Use when both groups have low-overlap regions.

    See Also
    --------
    domain_weights : The function that consumes this choice.
    samesame.weights.ImportanceWeights : What you get back.

    Examples
    --------
    >>> from samesame.weights import ReweightMode
    >>> ReweightMode("both") == ReweightMode.BOTH
    True
    """

    SOURCE = "source"
    TARGET = "target"
    BOTH = "both"

Weighting API¶

Public functions¶

samesame.weights.domain_weights(*, source, target, reweight=ReweightMode.BOTH, shrinkage=0.5) ¶

Public data classes¶

samesame.weights.ImportanceWeights dataclass ¶

effective_sample_size() ¶

samesame.weights.EffectiveSampleSize dataclass ¶

Public enum¶

samesame.weights.ReweightMode ¶

`samesame.weights.domain_weights(*, source, target, reweight=ReweightMode.BOTH, shrinkage=0.5)` ¶

`samesame.weights.ImportanceWeights` `dataclass` ¶

`effective_sample_size()` ¶

`samesame.weights.EffectiveSampleSize` `dataclass` ¶

`samesame.weights.ReweightMode` ¶