Frequently, people who don’t know about Bayesian networks ask me, why don’t I use (artificial) neural networks instead, for mechanical decision making. They seem to think that Bayesian nets, whatever those are, can’t possibly be as good as neural nets. After all, neural nets mimic the human brain, a device which mother Nature has been improving for millions of years. I’d like to explain here to those people that neural nets are to Bayesian nets what a Chevy is to a car. That is to say, neural nets are a specific model of Bayesian nets. Neural nets are Bayesian nets for which the nodes in the network (a directed acyclic graph) have assigned to them a very special type of probability matrix. For general Bayesian nets, the node probability matrices are completely general. If a node has input arrows labeled by the variables and output arrows labeled by the variable , then a neural net is constrained to have , where , , is Dirac’s delta function, and is some convenient function. For instance, for some and weight vector of positive real numbers.
Using B nets with these special “deterministic” node probability matrices has some advantages and disadvantages. (e.g. advantages: less calculational cost. disadvantages: deterministic systems like rule based systems are “too literal”, they are easier to confuse, less robust, less forgiving of contradictions and missing data). Read some criticisms of neural nets here.
Often, Biology comes up with brilliant solutions to problems. But Biology has a constrained palette of materials and environment to work with. So man can sometimes best Biology: for example, man can build flying machines called airplanes and rockets and helicopters, which are superior to birds in many ways.
One should also keep in mind that artificial neural nets are only a model of Nature, not Nature itself. How good a model is it? Perhaps real neurons are not completely deterministic. I don’t know if this hypothesis has been tested for, and ruled in or out. Let me speculate. Maybe idiot savants have B networks with node probability matrices that are much sharper than normal, and for this reason, they can calculate in an instant what day of the week was Aug 26, 1915, but they get very uncomfortable and confused every time you move the furniture around. Maybe some types of insanity arise in individuals whose B net has node probability matrices that are broader, noisier than normal. Maybe when we sleep, we are creating new nodes, new arrows, and re-adjusting the probability matrices of our existing nodes.
By the way, Fuzzy Logic is often mentioned by the popular press as a reasonable alternative to Bayesian networks. The only good thing about Fuzzy Logic is it’s name. Fuzzy logic is a hodgepodge, poorly motivated technique that isn’t based on probability theory, and thus is difficult (read impossible) to scale consistently to larger problems. Read some criticisms of Fuzzy Logic here.
To be fair, Bayesian nets are often criticized too, because doing B net calculations on a conventional classical computer often requires doing a huge number of elementary operations (additions). I believe that by using quantum computation, we can reduce the time complexity of many B nets calculations to sqrt of their classical complexity. This has already been proven to be the case for simulated annealing.