MI and CMI are measures of correlation that are frequently used in classical and quantum SIT (Shannon Information Theory). MI stands for Mutual Information. CMI stands for Conditional Mutual Information (you can pronounce CMI as “see me”). MI and CMI obey many identities and inequalities that are very useful. For example, MI and CMI are always non-negative. I’ve waxed poetic about CMI many times before in this blog. For example, in the following post:
The thing that I want to discuss now that I haven’t discussed before is a pictorial way of representing CMI identities and inequalities.
In quantum SIT, CMI is used to define squashed entanglement. Squashed entanglement seems to occupy a privileged position in the pantheon of entanglement measures because it’s the only known definition of entanglement that obeys the Monogamy Inequality for mixed states, something considered to be a very desirable trait for an entanglement measure. I’ll talk more about squashed entanglement and prove that it satisfies monogamy in a future post, but in this post I’ll restrict myself to speaking mostly about classical SIT. I’ll do this only for simplicity. Let me emphasize that everything that I say in this post generalizes easily to quantum SIT.
Let me review some definitions. I’ll represent random variables by underlined letters. In classical SIT, one defines
- the entropy (i.e., the variance or spread) of by
- the conditional spread (of given ) by
- the mutual information (MI) (i.e., the correlation) between and by
- the CMI by
Classical Bayesian networks (CB nets) are (directed acyclic) networks of random variables. In the slide below, I portray a typical CB net with nodes labeled by the random variables through . Then I show how to represent an MI and 2 CMIs for that CB net.
One can represent a CMI right on the CB net, but for simplicity I’ve represented the CMI in a diagram that doesn’t contain some of the nodes and that contains none of the lines with single arrowheads that belong to the CB net.
The convention is clear. Let us say CMI has 3 slots .
- lines with single arrowheads: belong to the CB net
- lines with no arrowheads: one of them connects all the random variables in the first slot of CMI. Another connects all the random variables in the second slot of CMI
- lines with arrowheads at both ends: connect all variables in slot 1 with all variables in slot 2.
- double circled nodes: correspond to the random variables in slot 3.
In the slide below, I show how to represent two examples of the chain rule for CMI.
Quantum Bayesian networks (QB nets) are a generalization of CB nets to quantum mechanics. The above two slides are for CB nets. However, very similar results hold for QB nets. That is one of the cool things about QB nets. That one can easily translate a CB net to its counterpart QB net or vice versa. This facilitates comparisons between the classical and quantum cases.