Multivariate measures of information generally attempt to capture some global property of a joint distribution. For example, they might attempt to quantify how much information is shared among the random variables, or quantify how “non-indpendent” in the joint distribution is.
These quantities, currently just the Shannon entropy, measure the total amount of information contained in a set of joint variables.
These measures all reduce to the standard Shannon Mutual Information for bivariate distributions.
It is perhaps illustrative to consider how each of these measures behaves on two canonical distributions: the giant bit and parity.
These measures all somehow measure shared information, but do not equal the mutual information in the bivaraite case.