Operations

There are several operations possible on joint random variables. Let’s consider the standard xor distribution:

In [1]: d = dit.Distribution(['000', '011', '101', '110'], [1/4]*4)

In [2]: d.set_rv_names('XYZ')

Marginal

marginal() returns a distribution containing only the random variables specified; marginalize() returns one containing all free variables except the ones specified. Both accept positional names or a single list. See Algebra for the chain-rule product and quotient that reconstruct joints from these pieces.

In [3]: print(d.marginal(['X', 'Y']))
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'))
Base:     linear

x            p(X,Y)
('0', '0')   1/4
('0', '1')   1/4
('1', '0')   1/4
('1', '1')   1/4

In [4]: print(d.marginalize(['X', 'Y']))
Class:    Distribution
Alphabet: (('0', '1'),)
Base:     linear

x        p(Z)
('0',)   1/2
('1',)   1/2

Distribution.marginal(*args)[source]

Marginalise to keep only the specified free variables.

Given (conditioned) variables are always kept.

Supports two call signatures:

marginal('X', 'Y') – positional variable names
marginal(['X', 'Y']) – list of names (or integer indices)

Parameters:: *args (str, or a single list/tuple) – The free variable names to keep. Integer indices are auto-resolved to dimension names.
Returns:: result
Return type:: Distribution

Distribution.marginalize(*args)[source]

Marginalise out (remove) the specified free variables.

Supports two call signatures:

marginalize('X') – positional variable names
marginalize(['X']) – list of names (or integer indices)

Parameters:: *args (str, or a single list/tuple) – The free variable names to remove.
Returns:: result
Return type:: Distribution

Conditional

Native condition_on() takes positional variable names and returns a single conditional Distribution:

In [5]: p_z_xy = d.condition_on('X', 'Y')

In [6]: print(p_z_xy)
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'), ('0', '1'))
Base:     linear

x                 p(Z|X,Y)
('0', '0', '0')   1
('0', '1', '1')   1
('1', '0', '1')   1
('1', '1', '0')   1

Passing a list or the crvs / rvs keywords uses the older compatibility form, which returns a pair (marginal, list_of_conditionals) — one slice per outcome of the conditioning variables:

In [7]: marginal, cdists = d.condition_on(['X', 'Y'], rvs=['Z'])

In [8]: print(marginal)
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'))
Base:     linear

x            p(X,Y)
('0', '0')   1/4
('0', '1')   1/4
('1', '0')   1/4
('1', '1')   1/4

In [9]: print(cdists[0])
Class:    Distribution
Alphabet: (('0', '1'),)
Base:     linear

x        p(Z)
('0',)   1

Distribution.condition_on(*cond_vars, rvs=None, crvs=None)[source]

Condition on the specified free variables.

Supports two call signatures:

Native: condition_on('X', 'Y') – positional var names. Returns a single conditional Distribution.
dit-compat: any of these forms triggers the dit-compatible return format (marginal, list_of_conditionals):
- condition_on(crvs=['X'], rvs=['Y'])
- condition_on(['X'], rvs=['Y']) (positional crvs)
- condition_on(crvs=['X'])
The returned list contains one Distribution per outcome of the conditioning variable.

Parameters:

*cond_vars (str, or a single list/tuple) – Variable names to condition on. If a single list/tuple is passed and rvs is provided, it is interpreted as crvs (dit-compat positional form).
rvs (list, optional) – Variables to keep in the conditional (dit-compat API).
crvs (list, optional) – Variables to condition on (dit-compat API).

Returns:

result – A single conditional distribution (native), or a (marginal, list_of_Distributions) tuple (dit-compat).

Return type:

Distribution or tuple

Examples

>>> p_xyz.condition_on('Z')   # native: returns p(X,Y|Z)
>>> p_xyz.condition_on('X', 'Y')  # native: returns p(Z|X,Y)
>>> marg, cdists = p_xyz.condition_on(crvs=['Z'])  # dit-compat
>>> marg, cdists = p_xyz.condition_on(['Z'], rvs=['X'])  # dit-compat

Join

We can construct the join of two random variables:

\[X \join Y = \min \{ V | V \imore X \land V \imore Y \}\]

Where \(\min\) is understood to be minimizing with respect to the entropy.

In [10]: from dit.algorithms.lattice import join

In [11]: print(join(d, ['XY']))
Class:    Distribution
Alphabet: (0, 1, 2, 3)
Base:     linear

x   p(x)
0   1/4
1   1/4
2   1/4
3   1/4

join(dist, rvs, int_outcomes=True)[source]

Returns the distribution of the join of random variables defined by rvs.

Parameters:

dist (Distribution) – The distribution which defines the base sigma-algebra.
rvs (list) – A list of lists. Each list specifies a random variable to be joined with the other lists. Each random variable can defined as a series of unique indexes. Multiple random variables can use the same index. For example, [[0, 1], [1, 2]].
int_outcomes (bool) – If True, then the outcomes of the join are relabeled as integers instead of as the atoms of the induced sigma-algebra.

Returns:

d – The distribution of the join.

Return type:

Distribution

insert_join(dist, idx, rvs, support_only=False)[source]

Returns a new distribution with the join inserted at index idx.

The join of the random variables in rvs is constructed and then inserted into at index idx.

Parameters:

dist (Distribution) – The distribution which defines the base sigma-algebra.
idx (int) – The index at which to insert the join. To append the join, set idx to be equal to -1 or dist.outcome_length().
rvs (list) – A list of lists. Each list specifies a random variable to be met with the other lists. Each random variable can defined as a series of unique indexes. Multiple random variables can use the same index. For example, [[0, 1], [1, 2]].
support_only (bool) – If True, only consider outcomes with non-zero probability.

Returns:

d – The new distribution with the join at index idx.

Return type:

Distribution

Meet

We can construct the meet of two random variables:

\[X \meet Y = \max \{ V | V \iless X \land V \iless Y \}\]

Where \(\max\) is understood to be maximizing with respect to the entropy.

In [12]: from dit.algorithms.lattice import meet

In [13]: outcomes = ['00', '01', '10', '11', '22', '33']

In [14]: d2 = dit.Distribution(outcomes, [1/8]*4 + [1/4]*2)

In [15]: d2.set_rv_names('XY')

In [16]: print(meet(d2, ['X', 'Y']))
Class:    Distribution
Alphabet: (0,)
Base:     linear

x   p(x)
0   1

The meet above is trivial because meet() works on the full sample space, the Cartesian product of the alphabets, where \(X\) and \(Y\) share no nontrivial event. Restricting to the support of the distribution recovers the common random variable that the Gács-Körner Common Information is built from:

In [17]: from dit.algorithms.lattice import dist_from_induced_sigalg, meet_sigalg

In [18]: print(dist_from_induced_sigalg(d2, meet_sigalg(d2, ['X', 'Y'], support_only=True)))
Class:    Distribution
Alphabet: (0, 1, 2)
Base:     linear

x   p(x)
0   1/4
1   1/4
2   1/2

meet(dist, rvs, int_outcomes=True)[source]

Returns the distribution of the meet of random variables defined by rvs.

Parameters:

dist (Distribution) – The distribution which defines the base sigma-algebra.
rvs (list) – A list of lists. Each list specifies a random variable to be met with the other lists. Each random variable can defined as a series of unique indexes. Multiple random variables can use the same index. For example, [[0, 1], [1, 2]].
int_outcomes (bool) – If True, then the outcomes of the meet are relabeled as integers instead of as the atoms of the induced sigma-algebra.

Returns:

d – The distribution of the meet.

Return type:

Distribution

insert_meet(dist, idx, rvs, support_only=False)[source]

Returns a new distribution with the meet inserted at index idx.

The meet of the random variables in rvs is constructed and then inserted into at index idx.

Parameters:

dist (Distribution) – The distribution which defines the base sigma-algebra.
idx (int) – The index at which to insert the meet. To append the meet, set idx to be equal to -1 or dist.outcome_length().
rvs (list) – A list of lists. Each list specifies a random variable to be met with the other lists. Each random variable can defined as a series of unique indexes. Multiple random variables can use the same index. For example, [[0,1],[1,2]].
support_only (bool) – If True, only consider outcomes with non-zero probability.

Returns:

d – The new distribution with the meet at index idx.

Return type:

Distribution

Minimal Sufficient Statistic

This method constructs the minimal sufficient statistic of \(X\) about \(Y\): \(X \mss Y\):

\[X \mss Y = \min \{ V | V \iless X \land \I{X:Y} = \I{V:Y} \}\]

In [19]: from dit.algorithms import insert_mss

In [20]: d2 = dit.Distribution(['00', '01', '10', '11', '22', '33'], [1/8]*4 + [1/4]*2)

In [21]: print(insert_mss(d2, -1, [0], [1]))
Class:    Distribution
Alphabet: (('0', '1', '2', '3'), ('0', '1', '2', '3'), (0, 1, 2))
Base:     linear

x               p(X0,X1,X2)
('0', '0', 2)   1/8
('0', '1', 2)   1/8
('1', '0', 2)   1/8
('1', '1', 2)   1/8
('2', '2', 0)   1/4
('3', '3', 1)   1/4

Again, \(\min\) is understood to be over entropies.

mss(dist, rvs, about=None, int_outcomes=True)[source]

Parameters:

dist (Distribution) – The distribution which defines the base sigma-algebra.
rvs (list) – A list of random variables to be compressed into a minimal sufficient statistic.
about (list) – A list of random variables for which the minimal sufficient static will retain all information about.
int_outcomes (bool) – If True, then the outcomes of the minimal sufficient statistic are relabeled as integers instead of as the atoms of the induced sigma-algebra.

Returns:

d – The distribution of the minimal sufficient statistic.

Return type:

Distribution

Examples

>>> d = Xor()
>>> print(mss(d, [0], [1, 2]))
Class:    Distribution
Alphabet: (0, 1)
Base:     linear
x   p(x)
0   0.5
1   0.5

insert_mss(dist, idx, rvs, about=None)[source]

Inserts the minimal sufficient statistic of rvs about about into dist at index idx.

Parameters:

dist (Distribution) – The distribution which defines the base sigma-algebra.
idx (int) – The location in the distribution to insert the minimal sufficient statistic.
rvs (list) – A list of random variables to be compressed into a minimal sufficient statistic.
about (list) – A list of random variables for which the minimal sufficient static will retain all information about.

Returns:

d – The distribution dist modified to contain the minimal sufficient statistic.

Return type:

Distribution

Examples

>>> d = Xor()
>>> print(insert_mss(d, -1, [0], [1, 2]))
Class:          Distribution
Alphabet:       ('0', '1') for all rvs
Base:           linear
Outcome Class:  str
Outcome Length: 4
RV Names:       None
x      p(x)
0000   0.25
0110   0.25
1011   0.25
1101   0.25