Yann LeCun, the godfather of superstitious AI

Yann LeCun believes that autoregressive LLMs like ChatGPT are a fad, an “off-ramp” for AI. He believes that they will be replaced by “Objective Driven AI” (ODAI). According to him, ODAI will (1) be able to reason like a human, (2) not suffer from hallucinations.

Yann has been working on ODAI for nine years (according to him), but still has no working prototype or even evidence that his unrealized pipe dream will accomplish 1 and 2. Nine years is a long time in AI! The “Attention is all you need” paper that invented transformers —a key ingredient of LLMs, was written in 2017, 6 years ago. Not surprisingly once you get to know Yann through his Tweets, his ODAI plans or opinions haven’t changed one bit in light of that paper or the LLM events that followed.

Real artists ship! I’ve been working on Mappa Mundi for less than a year and I already have a working prototype and 3 open source apps:

1. https://github.com/rrtucci/mappa_mundi
2. https://github.com/rrtucci/scumpy
3. https://github.com/rrtucci/SentenceAx
4. CausalFitBit (soon)

When your design is good, it comes to fruition quickly. When it’s bad, you can be brilliant and spend decades banging your head against the wall, and still not get it to work.

I don’t believe LLMs will fade soon, or that they will be replaced by ODAI. Just like bicycles, LLMs will be hard to replace, because they already perform their job (i.e., curve fitting the space of human generated text) quite efficiently. I believe discarding LLMs is silly, like throwing away the baby with the bath water. Instead, I believe human-like reasoning can be achieved by adding stuff to LLMs. Unlike Yann and his vaporware, I already have open source software that gives strong evidence that my approach works.

As Steve Jobs would say: “One Last Thing”.

My software Mappa Mundi (MM) is fairly immune to hallucinations (because the LLM is used only to summarize some input text, and that summary is only used at an intermediate stage—it’s not the end product). Furthermore, unlike ODAI, my architecture can distinguish correlation from causation. This is a crucial difference, as I will explain next.

For MM, world maps are a directory (atlas) of causal DAGs.

I believe the world maps that will be produced by Yann’s ODAI are highly deficient and nix the whole ODAI program from the start. This is how Yann defines his world maps.

An objective function C(f(x,a),y) that must be minimized with respect to a.
– x: current observation from sensors (or a prompt)
– a: imagined action sequence
– f(x,a): a model of the world that predicts the outcome of taking actions a
– C(f(x,a),y): a cost function measuring the…

— Yann LeCun (@ylecun) October 28, 2023

Yann’s world maps only capture correlations. They don’t distinguish between correlations and causations (DCC). To do DCC, according to Judea Pearl, who has worked on DCC for 40 years, ODAI would have to use causal DAGs, but it doesn’t. Also, Yann’s world maps are not interpretable (I), nor easily transportable(T). A DAG atlas does DCC and is I & T

A DAG atlas, because it’s a collection of easily interpretable graphs, can be easily corrected (“aligned”). Just omit some DAGs. Yann’s world maps will not be easy to correct and will be full of SUPERSTITIONS, because they equate correlations to causations.

The ultimate irony. Yann LeCun, the proud atheist, wants to be remembered as the godfather of superstitious (religious) AIs. Yann is trading hallucinations for superstitions. In my opinion, his cure is far worse than the disease. Whether you are an atheist or a believer in a religion, you will probably agree that religion/superstition has been used as an excuse for many a war throughout human history.

A DAG atlas will also be very transportable, not just between usecases, but between software libraries.

In the future, all LLMs will come with a DAG atlas