`optora.divergences.kl`¶

Kullback-Leibler divergence between discrete probability distributions.

`KLDivergence` ¶

Kullback-Leibler divergence of a candidate distribution from a reference.

For discrete distributions represented as nonnegative tensors that sum to one along their last dimension, computes

\[ D_{\mathrm{KL}}(p \,\|\, q) = \sum_i p_i \log \frac{p_i}{q_i}. \]

optora.dro.kl_dro uses this divergence to define KL-based ambiguity sets.

Attributes:

Name	Type	Description
`eps`		Small positive constant used to clamp `p` and `q` away from zero before taking the logarithm, avoiding division by zero and `log(0)` without branching on masked entries.

Initialize the KL divergence.

Parameters:

Name	Type	Description	Default
`eps`	`float`	Small positive constant used to clamp `p` and `q` away from zero before taking the logarithm.	`1e-12`

Raises:

Type	Description
`ValueError`	If `eps` is not positive.

Compute the KL divergence of p from q.

Parameters:

Name	Type	Description	Default
`p`	`Tensor`	Candidate distribution, a nonnegative tensor that sums to one along its last dimension.	required
`q`	`Tensor`	Reference distribution with the same shape as `p`.	required

Returns:

Type	Description
`Tensor`	A scalar tensor holding \(D_{\mathrm{KL}}(p \,\\|\, q)\), clamped to be
`Tensor`	nonnegative to absorb floating-point error near zero.