Quantum Bayesian Networks

February 22, 2017

Quantum Fog’s weight in bnlearn units

Filed under: Uncategorized — rrtucci @ 2:42 am

In a recent blog post entitled “R are Us. We are all R now”, I expressed my great admiration for the R statistical computer language, and I announced the addition to the Quantum Fog (QFog) GitHub repository of a Jupyter notebook called “Rmagic for dummies” which explains how something called Rmagic allows one to run both Python and R in the same Jupyter notebook.

In 2 other earlier blog posts, I also expressed great admiration for something else, bnlearn, an open source computer program written in R by Marco Scutari for learning classical Bayesian networks (cbnets) from data. I consider bnlearn the gold standard of bnet learning software.

The main purpose of this blog post is to announce that the QFog GitHub repo now has a folder of Jupyter notebooks comparing QFog to bnlearn. This is a perfect application of Rmagic to comparing two applications that can do some of the same things but one app is written in R while the other is written in Python. Pitting QFog against bnlearn is highly beneficial to us developers of QFog because it shows us what needs to be improved and suggests new features that would be worthwhile to add.

QFog can do certain things that bnlearn can’t (most notably, QFog can do both classical and quantum bnets, whereas bnlearn can only do classical bnets), and vice versa (for instance, bnlearn can do bnets with continuous (Gaussian) node probability distributions, whereas QFog can only handle discrete PDs), but there is much overlap between the 2 software packages in the area of structure and parameter learning of classical bnets from data.

A cool feature of the folder of Jupyter notebooks comparing bnlearn and QFog is that most notebooks in that folder can be spawned and run from a single “master” notebook. This amazing ability of the “master” notebook to create and direct a zombie horde of other notebooks is achieved thanks to an open source Python module called “nbrun” (notebook run).

qfog-bnlearn-scales

Advertisements

April 29, 2017

Miss Quantum Computing, may I introduce to you Miss Bayesian Hierarchical Models and Miss MCMC?

Filed under: Uncategorized — rrtucci @ 5:49 pm

Warning: Intense talk about computer software ahead. If you are a theoretical computer scientist, you better stop reading this now. Your weak constitution probably can’t take it.

When you enter the nerd paradise and secret garden that is Bayesforge.com (a free service on the Amazon cloud), you will see one folder named “Classical” and another named “Quantum”. Here is a screenshot of this taken from Henning Dekant’s excellent post in Linkedin

The “Quantum” folder contains some major open source quantum computing programs: Quantum Fog, Qubiter, IBM-QisKit (aka kiss-kit), QuTip, DWave, ProjectQ, Rigetti

The “Classical” folder contains some major Bayesian analysis open source programs: Marco Scutari’s bnlearn (R), Kevin Murphy’s BNT (Octave/matlab), OpenPNL (C++/matlab), PyMC, PyStan.

The idea is to promote cross fertilization between “Quantum” and “Classical” Bayesian statisticians.

Today I want to talk mostly about PyMC and PyStan. PyMC and PyStan deal with “Hierarchical Models” (Hmods). The other programs in the “Classical” folder deal with “Bayesian Networks”(Bnets).

Bnets and Hmods are almost the same thing. The community of people working on Bnets has Judea Pearl as one of its distinguished leaders. The community of people working on Hmods has Andrew Gelman as one of its distinguished leaders. You might know Gelman (Prof. at Columbia U.) from his great blog “Statistical Modeling, Causal Inference, and Social Science” or from one of his many books

Both PyStan and PyMC do MCMC (Markov Chain Monte Carlo) for Hmods. They are sort of competitors but also complementary.

PyStan (its GitHub repo here) is a Python wrapper of a computer program written in C++ called Stan. According to Wikipedia, “Stan is named in honour of Stanislaw Ulam, pioneer of the Monte Carlo method.” Prof. Gelman is one of the fathers of Stan (I mean the program, of course).

PyMC comes in 2 incompatible versions: 2.X and 3.X. Version 3 is more advanced and intends to replace Ver 2. PyMC2’s best sampler is a Metropolis-Hastings (MH) sampler. PyMC3 contains an MH sampler, but it also contains the “No U turns” or “NUTS” sampler that is supposed to be much faster than MH for large networks. Currently, Bayesforge contains only PyMC2, but the next version will contain both PyMC2 and PyMC3. As an added bonus, PyMC3 comes with Theano, one of the leading deep neural networks frameworks.

Check out this really cool course:

Sta-663 “Statistical Programming” , available at GitHub, taught at Duke U. by Prof. Chi Wei Cliburn Chan.

This wonderful course has some nice jupyter notebooks illustrating the use of PyMC2, PyMC3 and PyStan. Plus it contains discussions of many other statistical programmimg topics. I love it. It has a similar philosophy to BayesForge, namely to do statistical programming with jupyter notebooks because they are great for communicating your ideas to others and allow you to combine seamlessly various languages like Python, R, Octave, etc

March 20, 2017

BNT and PNL, two masterpieces of Bayesian Networks retro-art

Filed under: Uncategorized — rrtucci @ 8:58 pm

An update on the latest adventures of our company artiste-qb.net.

In previous blog posts, I waxed poetic about Marco Scutari’s open source software called bnlearn for learning the structure of bnets (Bayesian networks) from data. bnlearn is written in the language R whereas Quantum Fog is written in Python. But by using Jupyter notebooks with Rmagic installed, we have been able to write some notebooks running both QFog and bnlearn side by side in the same notebook for the same bnets, and to compare outputs of both programs. That is a good bench-marking exercise for the bnet learning side of QFog, but what about it’s bnet inference side?

Two open source programs that are very good at doing bnet inference (and many other things too) are BNT (Bayes Net Toolbox, by Kevin Murphy et al) and OpenPNL (PNL = Probabilistic Networks Library, written by Intel. I like to call it PaNeL to remember the letters quickly).

So our next adventure is to learn how to use BNT and PNL and to compare them to QFog.

BNT is written in Matlab. PNL is written in C++ but it includes Matlab wrappers for most of its functions. Both BNT and PNL are very mature and comprehensive programs. Since its core is written in C++ rather than Matlab, we expect PNL to be much faster than BNT for large networks.

As you already know if you’ve ever checked Matlab’s pricing, the software is very costly for everyone except students. However, this is one case when the open source gods have listened to our prayers. Octave is a free, open source program that can run 99% of a Matlab program and the few differences between Matlab and Octave (mostly in the plotting packages) are well documented. Furthermore, one can run Octave in a Jupyter notebook, either on an Octave kernel or on a Python kernel with octavemagic (oct2py) installed.

So in order to compare QFog to bnlearn, we’ve had to start using Jupyter notebooks on R kernel  or on Python kernel with Rmagic. And in order to compare QFog with BNT&PNL, we’ve had to start using Jupyter notebooks on Octave kernel or on Python kernel with octavemagic. We have seen the light and we are now believers in a holy trinity of computer languages (diversity and open source is our strength, Mr Trump):

Python, R, Octave
(our polyglot notebooks)

Curiously, Duke Univ. offers a course called “Computational Statistics in Python” that also advocates the use of Jupyter notebooks, and the languages Python, R and Matlab intermixed to do statistics. So when two cultures independently come up with the same idea, it’s probably a good one, don’t you think?

Since BNT is written in Matlab, running it does not require any compilation. PNL, on the other hand, is written in C++, so it does. Compiling PNL has proven a difficult task because the software is ten years old, but, after a lot of sweat and tears, our wiz Henning Dekant has managed to compile it (a few issues remain).

BNT was last changed in a bigly way circa 2007 (or even earlier) and PBL on 2006. (bnlearn, by comparison, is still very active). BNT and PNL belong to what I like to call the first bnet revolution (inference by junction tree) whereas bnlearn belongs to the second revolution (structure learning from Markov blankets). Even though PNL belongs to the first, not second, revolution, it is a major mystery to me why Intel abandoned it. PNL is a very impressive, large and mature piece of software. A lot of work, love and passion seems to have gone into it. Then sometime in mid 2006, it seems to have been abandoned in a hurry, and now it looks like a ghost town, or the deserted island in the video game Myst. I already know how the game Myst ends. If anyone knows why Intel stopped PNL development circa 2006, I would appreciate it if you would tell me, either in public in this blog or in private by email. Luckily, Intel had the wisdom to make PNL open source. PNL will go on, because 💍OPEN SOURCE IS FOREVER💍.

Sorry for the length of this blog post (almost as long as a Scott Aaronson blog post or a never ending New Yorker article).

myst.jpg
mystmeme.jpg

August 14, 2016

Quantum Fog on the verge of becoming Sentient: it can now distinguish between (the words) “Good” and “Evil”

Filed under: Uncategorized — rrtucci @ 7:29 pm

midnigh-garden_good_evilYou have to start somewhere. First those 2 words, then … the Oxford Dictionary?

I am pleased to announce that I and http://www.artiste-qb.net have added a new, major addition to Quantum Fog. QFog can now learn classical (and quantum) Bayesian Networks from data fairly well by today’s standards.

As far as I am concerned, the gold standard for software that learns bnets from data is bnlearn, by Marco Scutari. To show my readers how the current Quantum Fog and the current bnlearn compare, I took a snapshot of a portion of the home page of http://www.bnlearn.com, the portion that enumerates the various algorithms that bnlearn can do, and I put a red check-mark next to those that QFog can now do too. As you can see, QFog is still behind bnlearn, but not by too much.

qfog-bnlearn

So why am I trying to replicate bnlearn, isn’t that silly? Because bnlearn is in R, whereas I want to write something in Python, using Pandas. Furthermore, I want to write a software library that allows you to analyze BOTH, classical and quantum bnets alongside each other.

Pandas is a Python library that replicates many of the statistical capabilities of R. R is super popular among statisticians, but Pandas, less than a decade old, has also received many plaudits from that community. The original author of Pandas, Wes McKinney, has written a wonderful book about Pandas, numpy and, more generally, about doing data science with Python.

There are very close ties between the R and Python communities, and it’s fairly easy to call R subroutines from Python and vice versa. Pandas was Wes McKinney’s love poem to R. In the future, I and http://www.artiste-qb.net are planning to use bnlearn subroutines often. At first, I’m sure that most bnlearn subroutines will perform better than those of Quantum Fog and that we can improve QFog a lot by comparing its performance, architecture, and output with that of bnlearn.

There are certain aspects of bnlearn that we haven’t replicated yet. For example, bnlearn does continuous (just Gaussian) bnets whereas we don’t yet. In the quantum case, Gaussian continuous distributions would entail coherent and squeezed coherent states. Let the LIGO people worry about that.

On the other had, at this point, QFog’s inference capabilities are better than those of bnlearn. QFog can do the message passing join tree algorithm and bnlearn can’t. (At present, bnlearn can do inference only using Monte Carlo)

And then there is the Judea Pearl do-calculus, both for classical and quantum bnets. Neither bnlearn nor QFog can do that yet, but some day soon… BayesiaLab is way ahead of everyone else in that regard. They already have a beautiful graphical implementation of the Judea Pearl do-calculus stuff for classical bnets.

Added later: Judea Pearl do-calculus has also been implemented in the following R package. Thanks to M.S. for telling me about this:
https://cran.r-project.org/web/packages/pcalg/index.html

July 14, 2016

Quantum Fog, a quantum computer simulator based on quantum Bayesian networks, can now Think (at least better than a rock)

Filed under: Uncategorized — rrtucci @ 6:51 pm

Today, I added a folder called “learning” to Quantum Fog. QFog is a quantum computer simulator based on Bayesian Networks (bnets). Classical Bayesian networks are what earned Judea Pearl a Turing Prize. Quantum Fog implements seamlessly both classical Bayesian networks and their quantum generalization, quantum Bayesian networks.

The way I see it, the field of classical Bayesian networks has had 2 Springs.

The first Spring was about 20 years ago and it was motivated by the discovery of the join tree message passing algo which decreased significantly the complexity of doing inference with bnets. That complexity is exponential regardless, but the join tree algo makes it exponential in the size of the largest clique of the graph.

The second Spring is occurring right now, and it is motivated by the discovery of various algorithms for learning bnets by computer from the data. Immediately after the first Spring, bnet inference could be done fairly quickly, but the bnet had to be divined manually by the user, a formidable task for bnets with more than a handful of nodes. But nowadays, that situation has improved considerably, as you can see by looking at my 2 favorite open source libraries for learning bnets from data:

  1. bnlearn. Very polished, R language. Written by Marco Scutari
  2. neuroBN, a less polished but very pedagogically helpful to me. Python language. Written by Nicholas Cullen

Its new ”learning” folder gives Quantum Fog a rudimentary capability for learning both classical bnets and quantum bnets from data.(so far QFog can only do Naive Bayes and Chow Liu Tree algos. Soon will add Hill Climbing, Tabu, GrowShrink, IAMB and PC algos) Previous workers like Scutari and Cullen only consider cbnets. Quantum Fog aims to cover both cbnets and qbnets seamlessly. We hope QFog can eventually generate ”programs” (instruction sequences) that can be run on real quantum computer hardware.

Quantum Fog goes to pet school

Canine-Cognition-Center3

Create a free website or blog at WordPress.com.

%d bloggers like this: