Disequilibrium and the LMPR Complexity

Lamberti, Martin, Plastino, and Rosso have proposed a complexity measure [LMPR04] disigned around the idea of being a measure of “distance from equilibrium”, or disequilibrium, multiplied by a measure of “randomness”. Here, they measure “randomness” by the (normalized) Entropy:

\[\H{X}/\log_2{|X|}\]

and the disequilibrium as a (normalized) Jensen-Shannon Divergence:

\[\JSD{X || P_e} / Q_0\]

where \(P_e\) is a uniform distribution over the same outcome space as \(X\), and \(Q_0\) is the maximum possible value of the Jensen-Shannon divergence of a distribution with \(P_e\).

The LMPR complexity does not necessarily behave as one might intuitively hope. For example, the LMPR complexity of the xor and “double bit” with independent bit are identical:

In [1]: from dit.other.disequilibrium import *

In [2]: d1 = dit.Distribution(['000', '001', '110', '111'], [1/4]*4)

In [3]: d2 = dit.Distribution(['000', '011', '101', '110'], [1/4]*4)

In [4]: LMPR_complexity(d1)
Out[4]: 0.2894598616025801

In [5]: LMPR_complexity(d2)
Out[5]: 0.2894598616025801

This is because they are both equally “far from equilibrium” with four equiprobable events over the space of three binary variables, and both have the same entropy of two bits.

This implies that the LMPR complexity is perhaps best applied to a ScalarDistribution, and is not suitable for measuring the complexity of dependencies between variables.

API

disequilibrium(dist, rvs=None, rv_mode=None)[source]

Compute the (normalized) disequilibrium as measured the Jensen-Shannon divergence from an equilibrium distribution.

Parameters:
  • dist (Distribution) – Distribution to compute the disequilibrium of.
  • rvs (list, None) – The indexes of the random variable used to calculate the diseqilibrium. If None, then the disequilibrium is calculated over all random variables. This should remain None for ScalarDistributions.
  • rv_mode (str, None) – Specifies how to interpret the elements of rvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted.
Returns:

D – The disequilibrium.

Return type:

float

LMPR_complexity(dist, rvs=None, rv_mode=None)[source]

Compute the LMPR complexity.

Parameters:
  • dist (Distribution) – Distribution to compute the LMPR complexity of.
  • rvs (list, None) – The indexes of the random variable used to calculate the LMPR complexity. If None, then the LMPR complexity is calculated over all random variables. This should remain None for ScalarDistributions.
  • rv_mode (str, None) – Specifies how to interpret the elements of rvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted.
Returns:

C – The LMPR complexity.

Return type:

float