Skip to content

optora.divergences.f_divergence

f-divergences (phi-divergences) between discrete probability distributions.

PhiDivergence

Bases: Divergence

General f-divergence generated by a caller-supplied convex function.

For discrete distributions represented as nonnegative tensors that sum to one along their last dimension, computes

\[ D_\phi(p \,\|\, q) = \sum_i q_i \, \phi\!\left(\frac{p_i}{q_i}\right) \]

for a convex generator \(\phi\) with \(\phi(1) = 0\). ChiSquareDivergence and TotalVariationDivergence are PhiDivergence instances with a fixed generator; researchers can subclass or instantiate PhiDivergence directly with any other convex generator to define further phi-divergence-based ambiguity sets in optora.dro.phi_dro.

Attributes:

Name Type Description
phi

Convex generator function with phi(1) = 0, applied elementwise to the density ratio p / q.
eps

Small positive constant used to clamp q away from zero before dividing, avoiding division by zero without branching.

__init__(phi, eps=1e-12)

Initialize the phi-divergence.

Parameters:

Name Type Description Default
phi Callable[[Tensor], Tensor]

Convex generator function with phi(1) = 0, applied elementwise to the density ratio p / q.

 required
eps float

Small positive constant used to clamp q away from zero before dividing.

 1e-12

Raises:

Type Description
ValueError

If eps is not positive.

forward(p, q)

Compute the phi-divergence of p from q.

Parameters:

Name Type Description Default
p Tensor

Candidate distribution, a nonnegative tensor that sums to one along its last dimension.

 required
q Tensor

Reference distribution with the same shape as p.

 required

Returns:

Type Description
Tensor

A scalar tensor holding \(D_\phi(p \,\|\, q)\), clamped to be
Tensor

nonnegative to absorb floating-point error near zero.

ChiSquareDivergence

Bases: PhiDivergence

Chi-square divergence of a candidate distribution from a reference.

Computes

\[ D_{\chi^2}(p \,\|\, q) = \sum_i \frac{(p_i - q_i)^2}{q_i}, \]

the phi-divergence generated by \(\phi(t) = (t-1)^2\). optora.dro.phi_dro uses this divergence to define chi-square ambiguity sets.

__init__(eps=1e-12)

Initialize the chi-square divergence.

Parameters:

Name Type Description Default
eps float

Small positive constant used to clamp q away from zero before dividing.

 1e-12

Raises:

Type Description
ValueError

If eps is not positive.

forward(p, q)

Compute the chi-square divergence of p from q.

Uses the closed form \((p-q)^2 / q\) directly instead of routing through PhiDivergence's generic \(q \, \phi(p/q)\) path: \(q \, (p/q - 1)^2\) and \((p-q)^2 / q\) are algebraically identical, but the generic path divides by q and then multiplies by q again after squaring, a round trip that costs an extra elementwise operation and loses precision by squaring an already-divided ratio before rescaling it back up.

Parameters:

Name Type Description Default
p Tensor

Candidate distribution, a nonnegative tensor that sums to one along its last dimension.

 required
q Tensor

Reference distribution with the same shape as p.

 required

Returns:

Type Description
Tensor

A scalar tensor holding \(D_{\chi^2}(p \,\|\, q)\), clamped to be
Tensor

nonnegative to absorb floating-point error near zero.

TotalVariationDivergence

Bases: PhiDivergence

Total variation divergence of a candidate distribution from a reference.

Computes

\[ D_{\mathrm{TV}}(p \,\|\, q) = \frac{1}{2} \sum_i |p_i - q_i|, \]

the phi-divergence generated by \(\phi(t) = \frac{|t-1|}{2}\). optora.dro.phi_dro uses this divergence to define total-variation ambiguity sets.

__init__(eps=1e-12)

Initialize the total variation divergence.

Parameters:

Name Type Description Default
eps float

Small positive constant used to clamp q away from zero before dividing.

 1e-12

Raises:

Type Description
ValueError

If eps is not positive.

forward(p, q)

Compute the total variation divergence of p from q.

Uses the closed form \(\frac{1}{2}\sum |p-q|\) directly instead of routing through PhiDivergence's generic \(q \, \phi(p/q)\) path: for \(q > 0\), \(q \, |p/q - 1| = |p-q|\), so the q factor and the division it required cancel out algebraically. This removes the division (and its eps clamp) from the computation entirely rather than merely guarding it.

Parameters:

Name Type Description Default
p Tensor

Candidate distribution, a nonnegative tensor that sums to one along its last dimension.

 required
q Tensor

Reference distribution with the same shape as p.

 required

Returns:

Type Description
Tensor

A scalar tensor holding \(D_{\mathrm{TV}}(p \,\|\, q)\), clamped to
Tensor

be nonnegative to absorb floating-point error near zero.