Interaction Information

The interaction information is equal in magnitude to the Co-Information, but has the opposite sign when taken over an odd number of variables:

\[\II{X_{0:n}} = (-1)^{n} \cdot \I{X_{0:n}}\]

Interaction information was first studied in the 3-variable case which, for \(X_{0:3} = X_0X_1X_2\), takes the following form:

\[\II{X_0 : X_1 : X_2} = \I{X_0 : X_1 | X_2} - \I{X_0 : X_1}\]

The extension to \(n > 3\) proceeds recursively. For example,

\[\begin{split}\II{X_0 : X_1 : X_2 : X_3} &= \II{X_0 : X_1 : X_2 | X_3} - \II{X_0 : X_1 : X_2} \\ &= \I{X_0 : X_1 | X_2, X_3} - \I{X_0 : X_1 | X_3} \\ &\qquad - \I{X_0 : X_1 | X_2} + \I{X_0 : X_1}\end{split}\]

For three variables the sign flip means synergy is positive and redundancy negative, the opposite of the co-information convention:

In [1]: from dit.multivariate import coinformation, interaction_information

In [2]: xor = dit.example_dists.Xor()

In [3]: coinformation(xor)
Out[3]: -1.0

In [4]: interaction_information(xor)
Out[4]: 1.0

See also

For more information, see Co-Information.

API

interaction_information(dist, rvs=None, crvs=None)[source]

Calculates the interaction information.

Parameters:
  • dist (Distribution) – The distribution from which the interaction information is calculated.

  • rvs (list, None) – The indexes of the random variable used to calculate the interaction information between. If None, then the interaction information is calculated over all random variables.

  • crvs (list, None) – The indexes of the random variables to condition on. If None, then no variables are condition on.

Returns:

II – The interaction information.

Return type:

float

Raises:

ditException – Raised if dist is not a joint distribution or if rvs or crvs contain non-existant random variables.