Quickstart

The basic usage of dit corresponds to creating distributions, modifying them if need be, and then computing properties of those distributions. First, we import:

>>> import dit

Suppose we have a really thick coin, one so thick that there is a reasonable chance of it landing on its edge. Here is how we might represent the coin in dit. Each outcome is a tuple with one entry per random variable; unnamed variables are called X0, X1, ….

>>> d = dit.Distribution(['H', 'T', 'E'], [.4, .4, .2])
>>> print(d)
Class:    Distribution
Alphabet: (('E', 'H', 'T'),)
Base:     linear

x        p(X0)
('E',)   0.2
('H',)   0.4
('T',)   0.4

Calculate the probability of H and also of the combination H or T.

>>> d['H']
0.4
>>> d.event_probability(['H', 'T'])
0.8

Calculate the Shannon entropy and extropy of the joint distribution.

>>> float(dit.shannon.entropy(d))
1.5219280948873621
>>> float(dit.other.extropy(d))
1.1419011889093373

Create a distribution where Z = xor(X, Y).

>>> import dit.example_dists
>>> d = dit.example_dists.Xor()
>>> d.set_rv_names(['X', 'Y', 'Z'])
>>> print(d)
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'), ('0', '1'))
Base:     linear

x                 p(X,Y,Z)
('0', '0', '0')   0.25
('0', '1', '1')   0.25
('1', '0', '1')   0.25
('1', '1', '0')   0.25

Calculate the Shannon mutual informations I[X:Z], I[Y:Z], and I[X,Y:Z].

>>> float(dit.shannon.mutual_information(d, ['X'], ['Z']))
0.0
>>> float(dit.shannon.mutual_information(d, ['Y'], ['Z']))
0.0
>>> float(dit.shannon.mutual_information(d, ['X', 'Y'], ['Z']))
1.0

Calculate the marginal distribution P(X,Z) and print its probabilities as fractions.

>>> d2 = d.marginal(['X', 'Z'])
>>> print(d2.to_string(exact=True))
Class:    Distribution
Alphabet: (('0', '1'), ('0', '1'))
Base:     linear

x            p(X,Z)
('0', '0')   1/4
('0', '1')   1/4
('1', '0')   1/4
('1', '1')   1/4

Convert the distribution probabilities to log (base 3.5) probabilities, and access its probability mass function.

>>> d2.set_base(3.5)
>>> d2.pmf
array([-1.10658951, -1.10658951, -1.10658951, -1.10658951])

Draw 5 random samples from this distribution.

>>> dit.math.prng.seed(1)
>>> d2.rand(5)
[('0', '1'), ('1', '0'), ('0', '0'), ('0', '1'), ('0', '0')]

Source and Channel Coding

Beyond measures, dit builds explicit codes and ships a catalog of channels.

The dit.coding module constructs lossless source codes (Shannon, Fano, Shannon-Fano-Elias, Huffman, length-limited Huffman, Golomb/Rice, Tunstall, and the universal integer codes) and reports their code-theoretic properties (rate, redundancy, efficiency, the Kraft sum, and whether the code is prefix-free / uniquely decodable / optimal).

>>> from dit.coding import huffman
>>> d = dit.Distribution(['a', 'b', 'c', 'd', 'e'], [0.4, 0.2, 0.2, 0.1, 0.1])
>>> code = huffman(d)
>>> code.average_length()
2.2
>>> float(code.source_entropy())
2.1219280948873624
>>> code.is_optimal(), code.is_prefix_free()
(True, True)

It also builds binary (GF(2)) channel codes — linear block codes (repetition, parity-check, Hamming, Reed-Muller, Golay) as well as LDPC, polar, and convolutional codes — and evaluates them against a noisy channel supplied as a conditional Distribution p(Y|X).

>>> from dit.coding import hamming
>>> code = hamming(3)
>>> code.length, code.dimension, code.minimum_distance()
(7, 4, 3)
>>> bsc = dit.example_channels.binary_symmetric_channel(0.05)
>>> float(code.probability_of_error(bsc, method='exact'))
0.04438054218749993

The dit.example_channels module is a catalog of canonical discrete memoryless channels (binary symmetric, binary erasure, Z-channel, q-ary symmetric/erasure, noisy typewriter, …). Each constructor returns a conditional Distribution p(Y|X) ready for dit.algorithms.channel_capacity or the coding layer above.

>>> from dit.algorithms import channel_capacity
>>> bec = dit.example_channels.binary_erasure_channel(0.25)
>>> round(float(channel_capacity(bec)[0]), 10)
0.75