Symbolic

In addition to numeric (floating-point) probabilities, dit supports symbolic distributions whose probabilities are sympy expressions. Information measures then return sympy expressions that can be manipulated, evaluated, and simplified exactly.

Symbolic support requires sympy (an optional dependency):

pip install dit[symbolic]

Constructing a symbolic distribution

The dit.symbolic subpackage provides convenience constructors. Use symbols() to create probability symbols (positive by default) and symbolic_distribution() to build a distribution:

In [1]: from dit.symbolic import symbolic_distribution, symbols, simplify

In [2]: import sympy

In [3]: p = symbols('p')

In [4]: d = symbolic_distribution(['0', '1'], [p, 1 - p])

In [5]: d.is_symbolic()
Out[5]: True

The pmf preserves the symbolic values rather than coercing them to floats:

In [6]: list(d.pmf)
Out[6]: [p, 1 - p]

Computing measures

Shannon and the multivariate/closed-form measures return sympy expressions. For example, the entropy of a symbolic coin is the binary entropy function:

In [7]: from dit.multivariate import entropy, total_correlation

In [8]: entropy(d)
Out[8]: -p*log(p)/log(2) + (p - 1)*log(1 - p)/log(2)

Results are returned unsimplified; call simplify() (a thin wrapper around sympy.simplify()) when a canonical form is desired, or substitute a concrete value with sympy.Expr.subs():

In [9]: entropy(d).subs(p, sympy.Rational(1, 2))
Out[9]: 1

For measures that involve a minimum (e.g. I_min, I_mmi, CAEKL), a plain .subs can occasionally fail with a sympy “not comparable” error when it leaves unsimplified constant arguments inside a Min. Use evaluate() to numerically evaluate such a result at a point robustly:

In [10]: from dit.symbolic import evaluate

In [11]: evaluate(entropy(d), {p: 0.25})
Out[11]: 0.8112781244591328

For a “giant bit” (two perfectly correlated bits), the joint entropy, mutual information, and total correlation all equal \(H(p)\):

In [12]: gb = symbolic_distribution(['00', '11'], [p, 1 - p])

In [13]: simplify(total_correlation(gb))
Out[13]: (-p*log(p) + (p - 1)*log(1 - p))/log(2)

Supported measures

Symbolic computation is supported for the closed-form measures:

• Shannon: entropy, conditional entropy, mutual information.

• Multivariate: co-information, total correlation, dual total correlation, interaction information, O-information, TSE complexity, cohesion.

• Divergences: cross entropy, Kullback-Leibler divergence.

• Common informations: Gács-Körner and the other combinatorial forms.

• PID: the closed-form redundancy measures (e.g. I_min, I_mmi).

Optimization-based common informations

The Wyner and Exact common informations are variational (a minimisation over an auxiliary variable) and have no general closed form, so the numeric backends use iterative solvers. A best-effort backend="symbolic" is nonetheless available for small symbolic distributions:

In [14]: from dit.multivariate import wyner_common_information

In [15]: a = symbols('a')

In [16]: dsbs = symbolic_distribution(['00', '01', '10', '11'],
   ....:                              [(1 - a) / 2, a / 2, a / 2, (1 - a) / 2])
   ....: 

In [17]: wyner_common_information(dsbs, backend="symbolic")  # doctest: +SKIP
Out[17]: ((a - 1)*log(1 - a) - (sqrt(1 - 2*a) - 1)*log(1/2 - sqrt(1 - 2*a)/2) + (sqrt(1 - 2*a) + 1)*log(sqrt(1 - 2*a)/2 + 1/2) + log(2/a**a))/log(2)

The symbolic backend proceeds by (a) analytic short-circuits — the common informations are squeezed between the dual total correlation and the joint entropy, so equal bounds (e.g. a giant bit, the XOR source) or independence give the answer immediately; (b) a generic KKT / reduced-gradient solve at small auxiliary cardinality; and (c) structural (symmetry-injected) ansätze for recognised symmetric sources such as the doubly-symmetric binary source (whose Wyner common information is \(1 + h(a) - 2h(a_0)\) with \(a_0(1 - a_0) = a/2\)). A symbolic distribution is routed to this backend automatically; pass backend="symbolic" explicitly for a numeric distribution.

When no closed form is reachable (e.g. the Exact common information of the doubly-symmetric binary source, which has no simple closed form), the backend raises SymbolicOptimizationError rather than returning an approximation. Other optimization-based measures (BROJA, I_proj/I_IG/I_GH, and the secret-key-agreement rates) remain numeric-only.

Notes and limitations

• Only linear probability space is supported for symbolic distributions.

• Because the sign of a free symbol is not decidable, a probability is treated as a structural zero only when it is literally zero. Normalisation is not checked when free symbols are present.

• Simplification of logarithms (e.g. recognising log(1/p) == -log(p)) may require sympy assumptions; probability symbols created via symbols() are positive=True to help.

API

symbolic_distribution(outcomes, pmf, rv_names=None, validate=True, **kwargs)

Construct a Distribution with symbolic probabilities.

This is a convenience wrapper around Distribution that sympifies the pmf entries so that the resulting distribution stores exact, symbolic probabilities.

Parameters:

• outcomes (sequence) – The outcomes, as accepted by Distribution (e.g. a list of strings such as ['00', '11']).

• pmf (sequence) – The probabilities, as numbers and/or sympy expressions. Each entry is passed through sympy.sympify().

• rv_names (list of str, optional) – Names for each random variable.

• validate (bool) – If True, validate normalisation after construction. For pmfs containing free symbols normalisation is not decidable and is skipped.

• **kwargs – Additional keyword arguments forwarded to Distribution.

Returns:

d – A distribution with d.is_symbolic() True.

Return type:

Distribution

symbols(names, positive=True, **assumptions)

Create sympy symbols suitable for use as probabilities.

A thin wrapper around sympy.symbols() that defaults to positive=True (appropriate for probabilities, and helpful for simplification).

Parameters:

• names (str) – Symbol names, e.g. 'p' or 'a b c' (see sympy.symbols()).

• positive (bool) – Whether the symbols are assumed positive. Defaults to True.

• **assumptions – Additional sympy assumptions forwarded to sympy.symbols().

Returns:

syms

Return type:

Symbol or tuple of Symbol

simplify(expr, **kwargs)

Simplify a symbolic measure result.

A thin wrapper around sympy.simplify() for convenience, so callers need not import sympy directly.

Parameters:

• expr (sympy expression) – The expression to simplify (e.g. the return value of a measure).

• **kwargs – Forwarded to sympy.simplify().

Returns:

simplified

Return type:

sympy expression

evaluate(expr, subs)

Numerically evaluate a symbolic measure result at a point.

This is a robust alternative to expr.subs(subs) for expressions that contain Min/Max (as produced by e.g. I_min/I_mmi or CAEKL). sympy’s Min/Max can raise ValueError (“not comparable”) when a plain .subs leaves unsimplified constant arguments; evaluating through sympy.lambdify() sidesteps that by comparing the arguments numerically.

Parameters:

• expr (sympy expression or number) – The expression to evaluate (e.g. a measure result).

• subs (dict) – Mapping of symbols to numeric values.

Returns:

value – The numeric value of expr at subs.

Return type:

float