Delta^k and Gamma^k

The \(\Delta^k\) and \(\Gamma^k\) measures [Var26] form a parameterized family of higher-order information measures. They unify several existing multivariate measures by recognizing them as special cases tuned by an integer order parameter \(k\).

\(\Delta^k\) is defined in terms of the S-information \(\mathcal{S}\) and the Total Correlation \(\mathcal{T}\):

\[\Delta^k(X_{0:n}) = \mathcal{S}(X_{0:n}) - k\mathcal{T}(X_{0:n})\]

Since the S-information is the sum of the total correlation and the Dual Total Correlation \(\mathcal{D}\), this is equivalent to \(\Delta^k = \mathcal{D} + (1 - k)\mathcal{T}\). It is arranged into a hierarchy of increasingly high-order synergies: if \(\Delta^k(X) < 0\) the system is dominated by interactions of order greater than \(k\), while if \(\Delta^k(X) > 0\) it is dominated by interactions of order lower than \(k\).

\(\Gamma^k\) is the entropic conjugate of \(\Delta^k\), obtained by exchanging the roles of the total correlation and dual total correlation:

\[\Gamma^k(X_{0:n}) = \mathcal{S}(X_{0:n}) - k\mathcal{D}(X_{0:n})\]

equivalently \(\Gamma^k = \mathcal{T} + (1 - k)\mathcal{D}\). It is arranged into a hierarchy of increasingly high-order redundancies.

For particular values of \(k\), both measures recover known quantities:

\(k\)	\(\Delta^k\)	\(\Gamma^k\)
0	S-information	S-information
1	dual total correlation	total correlation
2	negative O-information	O-information

In [1]: from dit.multivariate import delta_k, gamma_k

In [2]: from dit.example_dists import n_mod_m

In [3]: d = n_mod_m(5, 2)

In [4]: delta_k(d, 2)
Out[4]: 3.0

In [5]: gamma_k(d, 2)
Out[5]: -3.0

API

delta_k(dist, k, rvs=None, crvs=None)

Compute the Delta^k measure, a parameterized family of higher-order information measures.

It is defined as \(\Delta^k = \mathcal{S} - k\mathcal{T}\), where \(\mathcal{S}\) is the S-information and \(\mathcal{T}\) is the total correlation. Since the S-information is the sum of the total correlation and the dual total correlation \(\mathcal{D}\), this is equivalent to \(\Delta^k = \mathcal{D} + (1 - k)\mathcal{T}\).

Special cases recover known measures: \(\Delta^0\) is the S-information, \(\Delta^1\) is the dual total correlation, and \(\Delta^2\) is the negative O-information. For larger k, the measure is sensitive to increasingly high-order synergies.

Parameters:

• dist (Distribution) – The distribution from which the Delta^k measure is calculated.

• k (int) – The order parameter.

• rvs (list, None) – A list of lists. Each inner list specifies the indexes of the random variables used to calculate the Delta^k measure. If None, then it is calculated over all random variables, which is equivalent to passing rvs=dist.rvs.

• crvs (list, None) – A single list of indexes specifying the random variables to condition on. If None, then no variables are conditioned on.

Returns:

D_k – The Delta^k measure.

Return type:

float

Examples

>>> d = dit.example_dists.n_mod_m(5, 2)
>>> dit.multivariate.delta_k(d, 2)
3.0

Raises:

ditException – Raised if dist is not a joint distribution or if rvs or crvs contain non-existant random variables.

gamma_k(dist, k, rvs=None, crvs=None)

Compute the Gamma^k measure, the entropic conjugate of the Delta^k measure.

It is defined as \(\Gamma^k = \mathcal{S} - k\mathcal{D}\), where \(\mathcal{S}\) is the S-information and \(\mathcal{D}\) is the dual total correlation. Since the S-information is the sum of the total correlation \(\mathcal{T}\) and the dual total correlation, this is equivalent to \(\Gamma^k = \mathcal{T} + (1 - k)\mathcal{D}\).

Special cases recover known measures: \(\Gamma^0\) is the S-information, \(\Gamma^1\) is the total correlation, and \(\Gamma^2\) is the O-information. For larger k, the measure is sensitive to increasingly high-order redundancies.

Parameters:

• dist (Distribution) – The distribution from which the Gamma^k measure is calculated.

• k (int) – The order parameter.

• rvs (list, None) – A list of lists. Each inner list specifies the indexes of the random variables used to calculate the Gamma^k measure. If None, then it is calculated over all random variables, which is equivalent to passing rvs=dist.rvs.

• crvs (list, None) – A single list of indexes specifying the random variables to condition on. If None, then no variables are conditioned on.

Returns:

G_k – The Gamma^k measure.

Return type:

float

Examples

>>> d = dit.example_dists.n_mod_m(5, 2)
>>> dit.multivariate.gamma_k(d, 2)
-3.0

Raises:

ditException – Raised if dist is not a joint distribution or if rvs or crvs contain non-existant random variables.