References
S. A. Abdallah and M. D. Plumbley. A measure of statistical complexity based on predictive information with application to finite spin systems. Physics Letters A, 376(4):275–281, 2012.
Hamidreza Abin and Amin Gohari. On the source model key agreement problem. IEEE Transactions on Information Theory, 72(8):5317–5334, 2026. doi:10.1109/tit.2026.3691758.
Rudolf Ahlswede and Imre Csiszár. Common randomness in information theory and cryptography. I. Secret sharing. IEEE Transactions on Information Theory, 39(4):1121–1132, 1993. doi:10.1109/18.243431.
B. Allen, B. C. Stacey, and Y. Bar-Yam. An information-theoretic formalism for multiscale structure in complex systems. arXiv preprint arXiv:1409.4708, 2014.
Shun-ichi Amari. Information geometry on hierarchy of probability distributions. Information Theory, IEEE Transactions on, 47(5):1701–1711, 2001.
Venkat Anantharam, Amin Gohari, Sudeep Kamath, and Chandra Nair. On maximal correlation, hypercontractivity, and the data processing inequality studied by Erkip and Cover. arXiv preprint arXiv:1304.6133, 2013.
T. W. Anderson and Leo A. Goodman. Statistical inference about Markov chains. The Annals of Mathematical Statistics, 28(1):89–110, 1957. doi:10.1214/aoms/1177707039.
Evan Archer, Il Memming Park, and Jonathan W. Pillow. Bayesian entropy estimation for countable discrete distributions. Journal of Machine Learning Research, 15(81):2833–2868, 2014.
Suguru Arimoto. An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Transactions on Information Theory, 18(1):14–20, 1972. doi:10.1109/TIT.1972.1054753.
Erdal Arıkan. Source polarization. In 2010 IEEE International Symposium on Information Theory, 899–903. 2010. arXiv:1001.3087, doi:10.1109/ISIT.2010.5513567.
Shahab Asoodeh and Flavio P Calmon. Bottleneck problems: an information and estimation-theoretic view. Entropy, 22(11):1325, 2020. doi:10.3390/e22111325.
Nihat Ay, Daniel Polani, and Nathaniel Virgo. Information decomposition based on cooperative game theory. Kybernetika, 56(5):979–1014, 2020. arXiv:1910.05979.
Wajid Aziz and Muhammad Arif. Multiscale permutation entropy of physiological time series. In 2005 Pakistan Section Multitopic Conference, 1–6. 2005. doi:10.1109/INMIC.2005.334494.
Lalit R. Bahl, John Cocke, Frederick Jelinek, and Josef Raviv. Optimal decoding of linear codes for minimizing symbol error rate. IEEE Transactions on Information Theory, 20(2):284–287, 1974. doi:10.1109/TIT.1974.1055186.
Christoph Bandt and Bernd Pompe. Permutation entropy: a natural complexity measure for time series. Physical Review Letters, 88(17):174102, 2002. doi:10.1103/PhysRevLett.88.174102.
Pradeep Kr Banerjee and Virgil Griffith. Synergy, redundancy and common information. arXiv preprint arXiv:1509.03706, 2015.
Pradeep Kr Banerjee, Eckehard Olbrich, Jürgen Jost, and Johannes Rauh. Unique informations and deficiencies. In 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 32–38. IEEE, 2018.
Pradeep Kr. Banerjee, Johannes Rauh, and Guido Montúfar. Computing the unique information. arXive preprint arXiv:1709.07487, 2017.
Y. Bar-Yam. Multiscale complexity/entropy. Advances in Complex Systems, 7(01):47–63, 2004.
Jonathan Barrett, Noah Linden, Serge Massar, Stefano Pironio, Sandu Popescu, and David Roberts. Nonlocal correlations as an information-theoretic resource. Physical Review A, 71(2):022101, 2005. doi:10.1103/PhysRevA.71.022101.
Jonathan Barrett and Stefano Pironio. Popescu-Rohrlich correlations as a unit of nonlocality. Physical Review Letters, 95(14):140401, 2005. doi:10.1103/PhysRevLett.95.140401.
Salman Beigi and Amin Gohari. Phi-entropic measures of correlation. arXiv preprint arXiv:1611.01335, 2016.
A. J. Bell. The co-information lattice. In S. Makino S. Amari, A. Cichocki and N. Murata, editors, Proc. Fifth Intl. Workshop on Independent Component Analysis and Blind Signal Separation, volume ICA 2003, 921–926. New York, 2003. Springer.
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, and Jürgen Jost. Shared information—new insights and problems in decomposing information in complex systems. In Proceedings of the European Conference on Complex Systems 2012, 251–269. Springer, 2013.
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay. Quantifying unique information. Entropy, 16(4):2161–2183, 2014.
Chunhua Bian, Chang Qin, Qianli D. Y. Ma, and Qinghong Shen. Modified permutation-entropy analysis of heartbeat dynamics. Physical Review E, 85(2):021906, 2012. doi:10.1103/PhysRevE.85.021906.
Patrick Billingsley. Statistical methods in Markov chains. The Annals of Mathematical Statistics, 32(1):12–40, 1961. doi:10.1214/aoms/1177705136.
Richard E. Blahut. Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory, 18(4):460–473, 1972. doi:10.1109/TIT.1972.1054855.
Juan A. Bonachela, Haye Hinrichsen, and Miguel A. Muñoz. Entropy estimates of small data sets. Journal of Physics A: Mathematical and Theoretical, 41(20):202001, 2008. doi:10.1088/1751-8113/41/20/202001.
Léon Brillouin. The Negentropy Principle of Information. Volume 24. American Institute of Physics, 1953.
Nicolas Brunner and Paul Skrzypczyk. Nonlocality distillation and postquantum theories with trivial communication complexity. Physical Review Letters, 102(16):160403, 2009. doi:10.1103/PhysRevLett.102.160403.
Marat V. Burnashev. Data transmission over a discrete channel with feedback. random transmission time. Problems of Information Transmission, 12(4):250–265, 1976.
Flavio P Calmon, Ali Makhdoumi, and Muriel Médard. Fundamental limits of perfect privacy. In 2015 IEEE International Symposium on Information Theory (ISIT), 1796–1800. 2015. doi:10.1109/ISIT.2015.7282765.
Cristian Calude. Information and Randomness: An Algorithmic Perspective. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2nd edition, 2002. ISBN 3540434666.
Deeparnab Chakrabarty, Prateek Jain, and Pravesh Kothari. Provable submodular minimization using wolfe's algorithm. In Advances in Neural Information Processing Systems, volume 27. 2014.
Chung Chan, Ali Al-Bashabsheh, Javad B Ebrahimi, Tarik Kaced, and Tie Liu. Multivariate mutual information inspired by secret-key agreement. Proceedings of the IEEE, 103(10):1883–1913, 2015.
Chung Chan and Tie Liu. Agglomerative info-clustering. arXiv preprint arXiv:1701.04926, 2017.
Anne Chao and Tsung-Jen Shen. Nonparametric estimation of Shannon's index of diversity when there are unseen species in sample. Environmental and Ecological Statistics, 10(4):429–443, 2003. doi:10.1023/A:1026096204727.
Anne Chao, Y. T. Wang, and Lou Jost. Entropy and the species accumulation curve: a novel entropy estimator via discovery rates of new species. Methods in Ecology and Evolution, 4(11):1091–1100, 2013. doi:10.1111/2041-210X.12108.
Daniel Chicharro and Stefano Panzeri. Redundancy and synergy in dual decompositions of mutual information gain and information loss. arXiv preprint arXiv:1612.09522, 2016.
Eric Chitambar, Ben Fortescue, and Min-Hsiu Hsieh. Distributions attaining secret key at a rate of the conditional mutual information. In Advances in Cryptology – CRYPTO 2015, volume 9216 of Lecture Notes in Computer Science, 443–462. 2015. doi:10.1007/978-3-662-48000-7_22.
Eric Chitambar, Min-Hsiu Hsieh, and Andreas Winter. The private and public correlation cost of three random variables with collaboration. IEEE Transactions on Information Theory, 62(4):2034–2043, 2016. doi:10.1109/TIT.2016.2530086.
Matthias Christandl, Renato Renner, and Stefan Wolf. A property of the intrinsic mutual information. In IEEE international symposium on information theory, 258–258. 2003.
Ferdinando Cicalese, Luisa Gargano, and Ugo Vaccaro. Minimum-entropy couplings and their applications. IEEE Transactions on Information Theory, 65(6):3436–3451, 2019. doi:10.1109/TIT.2019.2894519.
John F. Clauser, Michael A. Horne, Abner Shimony, and Richard A. Holt. Proposed experiment to test local hidden-variable theories. Physical Review Letters, 23(15):880–884, 1969. doi:10.1103/PhysRevLett.23.880.
Madalena Costa, Ary L. Goldberger, and C.-K. Peng. Multiscale entropy analysis of complex physiologic time series. Physical Review Letters, 89(6):068102, 2002. doi:10.1103/PhysRevLett.89.068102.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Interscience, New York, second edition, 2006. ISBN 0471241954.
F. H. C. Crick. The origin of the genetic code. Journal of Molecular Biology, 38(3):367–379, 1968. doi:10.1016/0022-2836(68)90392-6.
László Csirmaz. A short proof of the Gács–Körner theorem. 2023. arXiv:2306.14718.
Imre Csiszár. Generalized cutoff rates and Rényi's information measures. IEEE Transactions on Information Theory, 41(1):26–34, 1995. doi:10.1109/18.370121.
Imre Csiszár and Prakash Narayan. Secrecy capacities for multiple terminals. IEEE Transactions on Information Theory, 50(12):3047–3061, 2004. doi:10.1109/TIT.2004.838380.
Paul Cuff. Distributed channel synthesis. IEEE Transactions on Information Theory, 59(11):7071–7096, 2013. doi:10.1109/TIT.2013.2279330.
Paul Warner Cuff, Haim H Permuter, and Thomas M Cover. Coordination capacity. IEEE Transactions on Information Theory, 56(9):4181–4206, 2010.
Kyle J. A. Down and Pedro A. M. Mediano. A logarithmic decomposition and a signed measure space for entropy. arXiv preprint arXiv:2409.03732, 2024. arXiv:2409.03732.
Bradley Efron and Charles Stein. The jackknife estimate of variance. The Annals of Statistics, 9(3):586–596, 1981. doi:10.1214/aos/1176345462.
David A. Ehrlich, Andreas C. Schneider, Viola Priesemann, Michael Wibral, and Abdullah Makkeh. A measure of the complexity of neural representations based on partial information decomposition. Transactions on Machine Learning Research, 2023. URL: https://openreview.net/forum?id=R8TU3pfzFr.
Abbas El Gamal and Young-Han Kim. Network Information Theory. Cambridge University Press, 2011. doi:10.1017/CBO9781139030687.
Bilal Fadlallah, Badong Chen, Andreas Keil, and José Príncipe. Weighted-permutation entropy: a complexity measure for time series incorporating amplitude information. Physical Review E, 87(2):022911, 2013. doi:10.1103/PhysRevE.87.022911.
Shaun Fallat, Steffen Lauritzen, Kayvan Sadeghi, Caroline Uhler, Nanny Wermuth, and Piotr Zwiernik. Total positivity in Markov structures. The Annals of Statistics, 45(3):1152–1184, 2017. doi:10.1214/16-AOS1478.
Leopold Fejér. Über trigonometrische Polynome. Journal für die reine und angewandte Mathematik, 146:53–82, 1916. doi:10.1515/crll.1916.146.53.
C. Finn and J. T. Lizier. Pointwise partial information decomposition using the specificity and ambiguity lattices. Entropy, 20(4):297, 2018.
Conor Finn and Joseph T. Lizier. Generalised measures of multivariate information content. Entropy, 22(2):216, 2020. doi:10.3390/e22020216.
Seth Frey, Paul L Williams, and Dominic K Albino. Information encryption in the expert management of strategic uncertainty. arXiv preprint arXiv:1605.04233, 2016.
William A. Gale and Geoffrey Sampson. Good-Turing frequency estimation without tears. Journal of Quantitative Linguistics, 2(3):217–237, 1995. doi:10.1080/09296179508590051.
Weihao Gao, Sreeram Kannan, Sewoong Oh, and Pramod Viswanath. Estimating mutual information for discrete-continuous mixtures. In Advances in Neural Information Processing Systems 30, 5986–5997. 2017.
Joshua Garland, Ryan G. James, and Elizabeth Bradley. Model-free quantification of time-series predictability. Physical Review E, 90(5):052910, 2014. doi:10.1103/PhysRevE.90.052910.
Michael Gastpar and Erixhen Sula. Relaxed Wyner's common information. In 2019 IEEE Information Theory Workshop (ITW), 1–5. 2019. doi:10.1109/ITW44776.2019.8989086.
Nicolas Gisin and Stefan Wolf. Linking classical and quantum key agreement: is there “bound information”? In Advances in Cryptology – CRYPTO 2000, volume 1880 of Lecture Notes in Computer Science, 482–500. 2000. arXiv:quant-ph/0005042.
Naveen Goela. Polarized random variables: maximal correlations and common information. In 2014 IEEE International Symposium on Information Theory, 1643–1647. 2014. doi:10.1109/ISIT.2014.6875112.
Amin Gohari and Venkat Anantharam. Comments on “information-theoretic key agreement of multiple terminals—part i”. IEEE Transactions on Information Theory, 63(8):5440–5442, 2017.
Amin Gohari, Onur Günlü, and Gerhard Kramer. On achieving a positive rate in the source model key agreement problem. arXiv preprint arXiv:1709.05174, 2017.
Amin Aminzadeh Gohari and Venkat Anantharam. Information-theoretic key agreement of multiple terminals—part I. IEEE Transactions on Information Theory, 56(8):3973–3996, 2010.
Chetan Gohil, Oliver M. Cliff, James M. Shine, Ben D. Fulcher, and Joseph T. Lizier. Cross mutual information. In 2025 IEEE Information Theory Workshop (ITW). 2025. arXiv:2507.15372.
André F. C. Gomes and Mário A. T. Figueiredo. Orders between channels and implications for partial information decomposition. Entropy, 25(7):975, 2023.
André F. C. Gomes and Mário A. T. Figueiredo. A measure of synergy based on union information. Entropy, 26(3):271, 2024. doi:10.3390/e26030271.
Allison E Goodwell and Praveen Kumar. Temporal information partitioning: characterizing synergy, uniqueness, and redundancy in interacting environmental variables. Water Resources Research, 53(7):5920–5942, 2017.
Peter Grassberger. Finite sample corrections to entropy and dimension estimates. Physics Letters A, 128(6–7):369–373, 1988. doi:10.1016/0375-9601(88)90193-4.
Peter Grassberger. Entropy estimates from insufficient samplings. 2003. arXiv:physics/0307138.
R. M. Gray and A. D. Wyner. Source coding for a simple network. The Bell System Technical Journal, 53(9):1681–1721, 1974.
Virgil Griffith, Edwin KP Chong, Ryan G James, Christopher J Ellison, and James P Crutchfield. Intersection information based on common randomness. Entropy, 16(4):1985–2000, 2014.
Virgil Griffith and Christof Koch. Quantifying synergistic mutual information. In Guided Self-Organization: Inception, pages 159–190. Springer, 2014.
Aaron J. Gutknecht, Michael Wibral, and Abdullah Makkeh. Bits and pieces: understanding information decomposition from part-whole relationships and formal logic. Proceedings of the Royal Society A, 477(2251):20210110, 2021. doi:10.1098/rspa.2021.0110.
Peter Gács and János Körner. Common information is far less than mutual information. Problems of Control and Information Theory, 2(2):149–162, 1973.
T. S. Han. Linear dependence structure of the entropy space. Information and Control, 29:337–368, 1975.
Frank Harary. On the notion of balance of a signed graph. Michigan Mathematical Journal, 2(2):143–146, 1953. doi:10.1307/mmj/1028989917.
Malte Harder, Christoph Salge, and Daniel Polani. Bivariate measure of redundant information. Physical Review E, 87(1):012130, 2013.
Jean Hausser and Korbinian Strimmer. Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks. Journal of Machine Learning Research, 10:1469–1484, 2009.
Michael Horstein. Sequential transmission using noiseless feedback. IEEE Transactions on Information Theory, 9(3):136–143, 1963. doi:10.1109/TIT.1963.1057832.
Robin A. A. Ince. Measuring multivariate redundant information with pointwise common change in surprisal. Entropy, 19(7):318, 2017.
Robin A. A. Ince. The Partial Entropy Decompostion: decomposing multivariate entropy and mutual information via pointwise common surprisal. arXive preprint arXiv:1702.01591, 2017.
Robin A. A. Ince, Bruno L. Giordano, Christoph Kayser, Guillaume A. Rousselet, Joachim Gross, and Philippe G. Schyns. A statistical framework for neuroimaging data analysis based on mutual information estimated via a Gaussian copula. Human Brain Mapping, 38(3):1541–1573, 2017. doi:10.1002/hbm.23471.
Ryan G. James and James P. Crutchfield. Multivariate dependence beyond shannon information. Entropy, 19(10):531, 2017. doi:10.3390/e19100531.
Ryan G. James, Jeffrey Emenheiser, and James P. Crutchfield. Unique information via dependency constraints. arXiv preprint arXiv:1709.06653, 2017.
Ryan G. James, Jeffrey Emenheiser, and James P. Crutchfield. Unique information and secret key agreement. Entropy, 21(1):12, 2019. doi:10.3390/e21010012.
Sudeep Kamath and Venkat Anantharam. A new dual to the Gács-Körner common information defined via the Gray-Wyner system. In 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 1340–1346. IEEE, 2010. doi:10.1109/allerton.2010.5707069.
Dan Kandel, Yossi Matias, Ron Unger, and Peter Winkler. Shuffling biological sequences. Discrete Applied Mathematics, 71(1–3):171–185, 1996. doi:10.1016/S0166-218X(97)81456-4.
F. Kaspar and H. G. Schuster. Easily calculable measure for the complexity of spatiotemporal patterns. Physical Review A, 36(2):842–848, 1987. doi:10.1103/PhysRevA.36.842.
Richard W. Katz. On some criteria for estimating the order of a Markov chain. Technometrics, 23(3):243–249, 1981. doi:10.2307/1267787.
Benjamin J. Killian, Joslyn Yundenfreund Kravitz, and Michael K. Gilson. Extraction of configurational entropy from molecular simulations via an expansion approximation. The Journal of Chemical Physics, 127(2):024107, 2007. doi:10.1063/1.2746329.
Michael Kleinman, Alessandro Achille, Stefano Soatto, and Jonathan Kao. Gacs-korner common information variational autoencoder. arXiv preprint arXiv:2205.12239, 2022.
Murat Kocaoglu, Alexandros G. Dimakis, Sriram Vishwanath, and Babak Hassibi. Entropic causal inference. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31. 2017. doi:10.1609/aaai.v31i1.10674.
Artemy Kolchinsky. A novel approach to the partial information decomposition. Entropy, 24(3):403, 2022.
Artemy Kolchinsky and Bernat Corominas-Murtra. Decomposing information into copying versus transformation. arXiv preprint arXiv:1903.10693, 2019.
Artemy Kolchinsky, Brendan D Tracey, and Steven Van Kuyk. Caveats for information bottleneck in deterministic scenarios. arXiv preprint arXiv:1808.07593, 2018.
Ioannis Kontoyiannis, Paul H. Algoet, Yuri M. Suhov, and Abraham J. Wyner. Nonparametric entropy estimation for stationary processes and random fields, with applications to English text. IEEE Transactions on Information Theory, 44(3):1319–1327, 1998. doi:10.1109/18.669425.
Raphail E. Krichevsky and Victor K. Trofimov. The performance of universal encoding. IEEE Transactions on Information Theory, 27(2):199–207, 1981. doi:10.1109/TIT.1981.1056331.
Klaus Krippendorff. Ross ashby's information theory: a bit of history, some solutions to problems, and what we face today. International Journal of General Systems, 38(2):189–212, 2009.
Mariusz Kubkowski and Jan Mielniczuk. Asymptotic distributions of empirical interaction information. Methodology and Computing in Applied Probability, 23(1):291–315, 2020. doi:10.1007/s11009-020-09783-0.
Dimitris Kugiumtzis. Transfer entropy on rank vectors. Journal of Nonlinear Systems and Applications 3(2):73–81, 2012. arXiv:1007.0357.
G. R. Kumar, C. T. Li, and A. El Gamal. Exact common information. In Information Theory (ISIT), 2014 IEEE International Symposium on, 161–165. IEEE, 2014.
János Körner. Coding of an information source having ambiguous alphabet and the entropy of graphs. In Transactions of the 6th Prague Conference on Information Theory, Statistical Decision Functions, Random Processes, 411–425. Prague, 1973. Academia.
Frank Lad, Giuseppe Sanfilippo, and Gianna Agrò. Extropy: a complementary dual of entropy. arXiv preprint arXiv:1109.6440, 2011.
P. W. Lamberti, M. T. Martin, A. Plastino, and O. A. Rosso. Intensive entropic non-triviality measure. Physica A: Statistical Mechanics and its Applications, 334(1):119–131, 2004.
H. O. Lancaster. The Chi-Squared Distribution. Wiley, London, 1969.
Steffen Lauritzen, Caroline Uhler, and Piotr Zwiernik. Total positivity in exponential families with application to binary variables. The Annals of Statistics, 49(3):1436–1459, 2021. doi:10.1214/20-AOS2007.
Abraham Lempel and Jacob Ziv. On the complexity of finite sequences. IEEE Transactions on Information Theory, 22(1):75–81, 1976. doi:10.1109/TIT.1976.1055501.
Nikolai Leonenko, Luc Pronzato, and Vippal Savani. A class of Rényi information estimators for multidimensional densities. The Annals of Statistics, 36(5):2153–2182, 2008. doi:10.1214/07-AOS539.
Alexander Lex, Nils Gehlenborg, Hendrik Strobelt, Romain Vuillemot, and Hanspeter Pfister. Upset: visualization of intersecting sets. IEEE transactions on visualization and computer graphics, 20(12):1983–1992, 2014.
Cheuk Ting Li and Abbas El Gamal. Extended Gray–Wyner system with complementary causal side information. IEEE Transactions on Information Theory, 64(8):5862–5878, 2018. arXiv:1701.03207, doi:10.1109/TIT.2017.2777487.
Cheuk Ting Li and Abbas El Gamal. Strong functional representation lemma and applications to coding theorems. IEEE Transactions on Information Theory, 64(11):6967–6978, 2018. arXiv:1701.02827, doi:10.1109/TIT.2018.2865570.
Jianhua Lin. Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1):145–151, 1991. doi:10.1109/18.61115.
Wei Liu, Ge Xu, and Biao Chen. The common information of n dependent random variables. In Communication, Control, and Computing (Allerton), 2010 48th Annual Allerton Conference on, 836–843. IEEE, 2010.
Joseph T Lizier, Benjamin Flecker, and Paul L Williams. Towards a synergy-based approach to measuring information modification. In Artificial Life (ALIFE), 2013 IEEE Symposium on, 43–51. IEEE, 2013.
Antoni Lozano, Bernardino Casas, Chris Bentz, and Ramon Ferrer-i-Cancho. Fast calculation of entropy with Zhang's estimator. 2017. arXiv:1707.08290.
Aobo Lyu, Andrew Clark, and Netanel Raviv. Explicit formula for partial information decomposition. In 2024 IEEE International Symposium on Information Theory (ISIT), 2329–2334. 2024. arXiv:2402.03554, doi:10.1109/ISIT57864.2024.10619361.
David JC MacKay. Information theory, inference and learning algorithms. Cambridge university press, 2003.
Tobias Mages and Christian Rohner. Decomposing and tracing mutual information by quantifying reachable decision regions. Entropy, 25(7):1014, 2023.
Konstantin Makarychev, Yury Makarychev, Andrei Romashchenko, and Nikolai Vereshchagin. A new class of non-Shannon-type inequalities for entropies. Communications in Information and Systems, 2(2):147–166, 2002. doi:10.4310/CIS.2002.v2.n2.a3.
Ali Makhdoumi, Salman Salamatian, Nadia Fawaz, and Muriel Médard. From the information bottleneck to the privacy funnel. In 2014 IEEE Information Theory Workshop (ITW), 501–505. 2014. doi:10.1109/ITW.2014.6970882.
Abdullah Makkeh, Daniel Chicharro, Dirk Oliver Theis, and Raul Vicente. Introducing a differentiable measure of pointwise shared information. Physical Review E, 103(3):032149, 2021.
Abdullah Makkeh, Dirk Oliver Theis, and Raul Vicente. BROJA-2PID: a robust estimator for bivariate partial information decomposition. Entropy, 20(4):271, 2018. doi:10.3390/e20040271.
Rostislav Matveev and Andrei Romashchenko. Beyond mutual information: extension profiles and shape functions of random variable pairs. 2026. arXiv:2606.23849.
Rostislav Matveev and Andrei Romashchenko. Spectral conditions for the Ingleton inequality. 2026. arXiv:2602.16536.
Gerald Matz and Pierre Duhamel. Information geometric formulation and interpretation of accelerated Blahut-Arimoto-type algorithms. In Proceedings of the IEEE Information Theory Workshop (ITW), 66–70. San Antonio, TX, USA, 2004. doi:10.1109/ITW.2004.1405276.
Ueli Maurer and Stefan Wolf. The intrinsic conditional mutual information and perfect secrecy. In IEEE international symposium on information theory, 88–88. Citeseer, 1997.
Ueli M. Maurer. Secret key agreement by public discussion from common information. IEEE Transactions on Information Theory, 39(3):733–742, 1993. doi:10.1109/18.256484.
W. J. McGill. Multivariate information transmission. Psychometrika, 19(2):97–116, 1954.
Pedro A. M. Mediano, Fernando Rosas, Robin L. Carhart-Harris, Anil K. Seth, and Adam B. Barrett. Beyond integrated information: a taxonomy of information dynamics phenomena. 2019. arXiv:1909.02297.
Pedro A. M. Mediano, Fernando E. Rosas, Andrea I. Luppi, Robin L. Carhart-Harris, Daniel Bor, Anil K. Seth, and Adam B. Barrett. Toward a unified taxonomy of information dynamics via integrated information decomposition. Proceedings of the National Academy of Sciences, 122(39):e2423297122, 2025. doi:10.1073/pnas.2423297122.
George A. Miller. Note on the bias of information estimates. In Henry Quastler, editor, Information Theory in Psychology: Problems and Methods, pages 95–100. Free Press, Glencoe, IL, 1955.
Ilya Nemenman, William Bialek, and Rob de Ruyter van Steveninck. Entropy and information in neural spike trains: progress on the sampling problem. Physical Review E, 69(5):056111, 2004. doi:10.1103/PhysRevE.69.056111.
Ilya Nemenman, Fariel Shafee, and William Bialek. Entropy and inference, revisited. In Advances in Neural Information Processing Systems 14, 471–478. MIT Press, 2002. arXiv:physics/0108025.
Xueyan Niu and Christopher J Quinn. A measure of synergy, redundancy, and unique information using information geometry. In 2019 IEEE International Symposium on Information Theory (ISIT), 3127–3131. IEEE, 2019.
Eckehard Olbrich, Nils Bertschinger, and Johannes Rauh. Information decomposition and synergy. Entropy, 17(5):3501–3517, 2015.
Daniel P Palomar and Sergio Verdú. Lautum information. IEEE transactions on information theory, 54(3):964–975, 2008.
Liam Paninski. Estimation of entropy and mutual information. Neural Computation, 15(6):1191–1253, 2003. doi:10.1162/089976603321780272.
Stefano Panzeri and Alessandro Treves. Analytical estimates of limited sampling biases in different information measures. Network: Computation in Neural Systems, 7(1):87–107, 1996. doi:10.1080/0954898X.1996.11978656.
Jef Pauwels, Nicolas Gisin, and Renato Renner. Bipartite bound information exists. arXiv preprint arXiv:2607.25838, 2026.
Haim Permuter, Paul Cuff, Benjamin Van Roy, and Tsachy Weissman. Capacity of the trapdoor channel with feedback. IEEE Transactions on Information Theory, 54(7):3150–3165, 2008. doi:10.1109/TIT.2008.924681.
Shawn D. Pethel and Daniel W. Hahs. Exact significance test for Markov order. Physica D: Nonlinear Phenomena, 269:42–47, 2014. arXiv:1302.1500, doi:10.1016/j.physd.2013.11.014.
Giuseppe Pica, Eugenio Piasini, Daniel Chicharro, and Stefano Panzeri. Invariant components of synergy, redundancy, and unique information among three variables. arXiv preprint arXiv:1706.08921, 2017.
Dimitris N. Politis and Joseph P. Romano. The stationary bootstrap. Journal of the American Statistical Association, 89(428):1303–1313, 1994. doi:10.1080/01621459.1994.10476870.
Sandu Popescu and Daniel Rohrlich. Quantum nonlocality as an axiom. Foundations of Physics, 24(3):379–385, 1994. doi:10.1007/BF02058098.
Vinod M. Prabhakaran and Manoj M. Prabhakaran. Assisted common information with an application to secure two-party sampling. IEEE Transactions on Information Theory, 60(6):3413–3434, 2014. arXiv:1206.1282, doi:10.1109/TIT.2014.2316011.
Rick Quax, Omri Har-Shemesh, and Peter M. A. Sloot. Quantifying synergistic information using intermediate stochastic variables. Entropy, 19(2):85, 2017. doi:10.3390/e19020085.
Murali Rao, Yunmei Chen, Baba C Vemuri, and Fei Wang. Cumulative residual entropy: a new measure of information. Information Theory, IEEE Transactions on, 50(6):1220–1228, 2004.
Borzoo Rassouli and Deniz Gündüz. On perfect privacy. IEEE Journal on Selected Areas in Information Theory, 2(1):177–191, 2021. doi:10.1109/JSAIT.2021.3053432.
Johannes Rauh. Secret sharing and shared information. arXiv preprint arXiv:1706.06998, 2017.
Johannes Rauh, Pradeep Kr Banerjee, Eckehard Olbrich, Jürgen Jost, and Nils Bertschinger. On extractable shared information. arXiv preprint arXiv:1701.07805, 2017.
Johannes Rauh, Nils Bertschinger, Eckehard Olbrich, and Jurgen Jost. Reconsidering unique information: towards a multivariate information decomposition. In Information Theory (ISIT), 2014 IEEE International Symposium on, 2232–2236. IEEE, 2014.
Johannes Rauh, Maik Schünemann, and Jürgen Jost. Properties of unique information. arXiv preprint arXiv:1912.12505, 2019.
Renato Renner, Juraj Skripsky, and Stefan Wolf. A new measure for conditional mutual information and its properties. In IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY, 259–259. 2003.
Renato Renner and Stefan Wolf. New bounds in secret-key agreement: the gap between formation and secrecy extraction. In Advances in Cryptology — EUROCRYPT 2003, volume 2656 of Lecture Notes in Computer Science, 562–577. Springer, 2003. doi:10.1007/3-540-39200-9_35.
Fernando Rosas, Vasilis Ntranos, Christopher J Ellison, Sofie Pollin, and Marian Verhelst. Understanding interdependency through complex information sharing. Entropy, 18(2):38, 2016.
Fernando E Rosas, Pedro AM Mediano, Michael Gastpar, and Henrik J Jensen. Quantifying high-order interdependencies via multivariate extensions of the mutual information. Physical Review E, 100(3):032305, 2019.
Fernando E. Rosas, Pedro A. M. Mediano, Borzoo Rassouli, and Adam B. Barrett. An operational information decomposition via synergistic disclosure. Journal of Physics A: Mathematical and Theoretical, 53(48):485001, 2020. doi:10.1088/1751-8121/abb723.
Mostafa Rostaghi and Hamed Azami. Dispersion entropy: a measure for time-series analysis. IEEE Signal Processing Letters, 23(5):610–614, 2016. doi:10.1109/LSP.2016.2542881.
Salman Salamatian, Asaf Cohen, and Muriel Médard. Maximum entropy functions: approximate Gács-Körner for distributed compression. In Information Theory and Applications Workshop (ITA). 2016. arXiv:1604.03877.
Jossy Sayir. Iterating the Arimoto-Blahut algorithm for faster convergence. In Proceedings of the IEEE International Symposium on Information Theory (ISIT), 235. Sorrento, Italy, 2000. doi:10.1109/ISIT.2000.866533.
J. Pieter M. Schalkwijk and Thomas Kailath. A coding scheme for additive noise channels with feedback–I: no bandwidth constraint. IEEE Transactions on Information Theory, 12(2):172–182, 1966. doi:10.1109/TIT.1966.1053879.
E. Schneidman, S. Still, M. J. Berry, W. Bialek, and others. Network information and connected correlations. Phys. Rev. Lett., 91(23):238701, 2003.
Dino Sejdinovic, Arthur Gretton, and Wicher Bergsma. A kernel test for three-variable interactions. In Advances in Neural Information Processing Systems, volume 26. 2013. arXiv:1306.2281.
Ofer Shayevitz and Meir Feder. Optimal feedback communication via posterior matching. IEEE Transactions on Information Theory, 57(3):1186–1222, 2011. doi:10.1109/TIT.2011.2104992.
David Sigtermans. A path-based partial information decomposition. Entropy, 22(9):952, 2020.
Noam Slonim, Gurinder Singh Atwal, Gašper Tkačik, and William Bialek. Information based clustering. arXiv preprint q-bio/0511043, 2005.
Bernd Streitberg. Lancaster interactions revisited. The Annals of Statistics, 18(4):1878–1885, 1990. doi:10.1214/aos/1176347885.
Christopher C. Strelioff, James P. Crutchfield, and Alfred W. Hübler. Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling. Physical Review E, 76(1):011106, 2007. doi:10.1103/PhysRevE.76.011106.
Steven P. Strong, Roland Koberle, Rob R. de Ruyter van Steveninck, and William Bialek. Entropy and information in neural spike trains. Physical Review Letters, 80(1):197–200, 1998. doi:10.1103/PhysRevLett.80.197.
DJ Strouse and David J Schwab. The deterministic information bottleneck. arXiv preprint arXiv:1604.00268, 2016.
H. Te Sun. Multiple mutual informations and multiple interactions in frequency data. Information and Control, 46:26–45, 1980.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
Howell Tong. Determination of the order of a Markov chain by Akaike's information criterion. Journal of Applied Probability, 12(3):488–497, 1975. doi:10.2307/3212863.
Giulio Tononi, Olaf Sporns, and Gerald M Edelman. A measure for brain complexity: relating functional segregation and integration in the nervous system. Proceedings of the National Academy of Sciences, 91(11):5033–5037, 1994.
H. Tyagi, P. Narayan, and P. Gupta. When is a function securely computable? Information Theory, IEEE Transactions on, 57(10):6337–6350, 2011.
Gregory Valiant and Paul Valiant. Estimating the unseen: improved estimators for entropy and other properties. Journal of the ACM, 64(6):1–41, 2017. doi:10.1145/3125643.
Francisco José Valverde-Albacete and Carmen Peláez-Moreno. The multivariate entropy triangle and applications. In Hybrid Artificial Intelligent Systems, pages 647–658. Springer, 2016.
Marten van Dijk. On a special class of broadcast channels with confidential messages. IEEE Transactions on Information Theory, 43(2):712–714, 1997. doi:10.1109/18.556128.
Cristiano Varin, Nancy Reid, and David Firth. An overview of composite likelihood methods. Statistica Sinica, 21(1):5–42, 2011.
Thomas F. Varley. Decomposing past and future: integrated information decomposition based on shared probability mass exclusions. PLOS ONE, 18(3):e0282950, 2023. doi:10.1371/journal.pone.0282950.
Thomas F. Varley. A scalable synergy-first backbone decomposition of higher-order structures in complex systems. npj Complexity, 1(1):9, 2024. doi:10.1038/s44260-024-00011-1.
Thomas F. Varley. Generalized decomposition of multivariate information. PLOS ONE, 19(2):e0297128, 2024. doi:10.1371/journal.pone.0297128.
Thomas F. Varley. The many faces of multivariate information. arXiv preprint arXiv:2601.08030, 2026.
Thomas F. Varley, Maria Pope, Maria Grazia Puxeddu, Joshua Faskowitz, and Olaf Sporns. Partial entropy decomposition reveals higher-order information structures in human brain activity. Proceedings of the National Academy of Sciences, 120(30):e2300888120, 2023. doi:10.1073/pnas.2300888120.
Praveen Venkatesh, Keerthana Gurushankar, and Gabriel Schamberg. Capturing and interpreting unique information. arXiv preprint arXiv:2302.11873, 2023.
Sergio Verdu and Tsachy Weissman. The information lost in erasures. Information Theory, IEEE Transactions on, 54(11):5030–5058, 2008.
S. Verdú and T. Weissman. Erasure entropy. In Information Theory, 2006 IEEE International Symposium on, 98–102. IEEE, 2006.
K. B. Viswanatha, E. Akyol, and K. Rose. The lossy common information of correlated sources. IEEE Transactions on Information Theory, 60(6):3238–3253, 2014. doi:10.1109/TIT.2014.2315805.
Xiang Wan, Can Yang, Qiang Yang, Hong Xue, Xiaodan Fan, Nelson L. S. Tang, and Weichuan Yu. BOOST: a fast approach to detecting gene-gene interactions in genome-wide case-control studies. The American Journal of Human Genetics, 87(3):325–340, 2010. doi:10.1016/j.ajhg.2010.07.021.
Qing Wang, Sanjeev R. Kulkarni, and Sergio Verdú. Divergence estimation for multidimensional densities via $k$-nearest-neighbor distances. IEEE Transactions on Information Theory, 55(5):2392–2405, 2009. doi:10.1109/TIT.2009.2016060.
S. Watanabe. Information theoretical analysis of multivariate correlation. IBM Journal of research and development, 4(1):66–82, 1960.
P. Whittle. Some distribution and moment formulae for the Markov chain. Journal of the Royal Statistical Society: Series B (Methodological), 17(2):235–242, 1955. doi:10.1111/j.2517-6161.1955.tb00197.x.
Paul L Williams and Randall D Beer. Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515, 2010.
Paul L Williams and Randall D Beer. Generalized measures of information transfer. arXiv preprint arXiv:1102.1507, 2011.
Andreas Winter. Secret, public and quantum correlation cost of triples of random variables. In IEEE International Symposium on Information Theory, 2270–2274. 2005. doi:10.1109/ISIT.2005.1523752.
Yihong Wu and Pengkun Yang. Minimax rates of entropy estimation on large alphabets via best polynomial approximation. IEEE Transactions on Information Theory, 62(6):3702–3720, 2016. doi:10.1109/TIT.2016.2548468.
A. D. Wyner. The common information of two dependent random variables. Information Theory, IEEE Transactions on, 21(2):163–179, 1975.
Aaron D. Wyner and Jacob Ziv. A theorem on the entropy of certain binary sequences and applications: Part I. IEEE Transactions on Information Theory, 19(6):769–772, 1973. doi:10.1109/TIT.1973.1055107.
Raymond W Yeung. Information theory and network coding. Springer, 2008.
Hubert P. Yockey. Information Theory, Evolution, and the Origin of Life. Cambridge University Press, Cambridge, 2005.
Lei Yu, Houqiang Li, and Chang Wen Chen. Generalized common informations: measuring commonness by the conditional maximal correlation. arXiv preprint arXiv:1610.09289, 2017. arXiv:1610.09289.
Zhen Zhang. On a new non-Shannon type information inequality. Communications in Information and Systems, 3(1):47–60, 2003. doi:10.4310/CIS.2003.v3.n1.a4.
Zhiyi Zhang. Entropy estimation in Turing's perspective. Neural Computation, 24(5):1368–1389, 2012. doi:10.1162/NECO_a_00266.
M. Zwick. An overview of reconstructability analysis. Kybernetes, 33(5/6):877–905, 2004.