# Quantum Bayesian Networks

## September 7, 2012

### Of Balloons and Steel Balls

Filed under: Uncategorized — rrtucci @ 7:26 pm

Yesterday I listened on the radio to many great speeches being delivered at the DNC (Democratic National Convention). Furthermore, as you may have gathered from my previous blog posts, recently I’ve been obsessively interested in SIT (Shannon Information Theory)—to the point that I’m beginning to see SIT in everything, even in the DNC. In both SIT and the DNC, there are sender nodes and receiver nodes. The receiver nodes in turn become sender nodes themselves.

I like to think of SIT as the study of a special type of bayesian network. The bayesian networks of SIT have nodes of two kinds: one kind of node stores “messages”; the other kind stores “codewords”. Codeword nodes are like helium filled balloons; they don’t store information in a very high density format. On the other hand, message nodes are like steel balls; they are small, compact, dense storers of information.

The main two problems of SIT were first proposed and completely solved by Shannon in his 1948 paper. They are Source Coding and Channel Coding. Both problems can be represented by Bayesian networks (as can all other problems in SIT, and in the universe, as far as I’m concerned). Here is how I portray these two problems in my paper “Shannon Information Theory without Shedding Tears over Delta & Epsilon Proofs or Typical Sequences

In this figure, random variables are denoted by underlining. “Codewords” are denoted by a letter raised to the n’th power, like $\underline{x}^n$. Estimates of a quantity are denoted by that quantity with a hat over it; for example, $\widehat{\underline{m}}$ is an estimate of $\underline{m}$.

Notice that in source coding, the information density goes from low to high to low again. The high density is in the middle. For channel coding, it’s the opposite. Information density goes from high to low to high again. The low density is in the middle.

P.S. I find that writing this blog is very useful to me. For one thing, it helps me to organize my ideas. Blogging reminds me of a quote I read recently:

“The art of writing is the art of discovering what you believe.” Gustave Flaubert