Necessary Conditional Entropy

The necessary conditional entropy [CPC10] quantifies the amount of information that a random variable \(X\) necessarily must carry above and beyond the mutual information \(\I{X : Y}\) to actually contain that mutual information:

\[\H{X \dagger Y} = \H{ X \mss Y | Y}\]

API

necessary_conditional_entropy(*args, **kwargs)[source]

Calculates the necessary conditional entropy \(\H[X \dagger Y]\). This is the entropy of the minimal sufficient statistic of X about Y, given Y.

Parameters:
  • dist (Distribution) – The distribution from which the necessary conditional entropy is calculated.
  • rvs (list, None) – The indexes of the random variable used to calculate the necessary conditional entropy. If None, then the entropy is calculated over all random variables.
  • crvs (list, None) – The indexes of the random variables to condition on. If None, then no variables are conditioned on.
  • rv_mode (str, None) – Specifies how to interpret rvs and crvs. Valid options are: {‘indices’, ‘names’}. If equal to ‘indices’, then the elements of crvs and rvs are interpreted as random variable indices. If equal to ‘names’, the the elements are interpreted as random variable names. If None, then the value of dist._rv_mode is consulted, which defaults to ‘indices’.
Returns:

H – The necessary conditional entropy.

Return type:

float

Raises:

ditException – Raised if rvs or crvs contain non-existant random variables.

Example