The Master Algorithm

A Poem by Robert Ronnow

                              Some say the scientific method
                              Is the ultimate algorithm and others
                              Prefer prayer.

For symbolists, all intelligence can be reduced to manipulating symbols, in the same way that a mathematician solves equations by replacing expressions by other expressions. Symbolists understand that you can't learn from scratch: you need some initial knowledge to go with the data. They've figured out how to incorporate pre-existing knowledge into learning, and how to combine different pieces of knowledge on the fly in order to solve new problems. Their master algorithm is inverse deduction, which figures out what knowledge is missing in order to make a deduction go through, and then makes it as general as possible.

                              Tea
                              In its simplicity
                              Can sustain concentration

For connectionists, learning is what the brain does, and so what we need to do is reverse engineer it. The brain learns by adjusting the strengths of connections between neurons, and the crucial problem is figuring out which connections are to blame for which errors and changing them accordingly. The connectionists' master algorithm is back propagation, which compares a system's outputs with the desired one and then successively changes the connections in layer after layer of neurons so as to bring the output closer to what it should be.

                              Hungry and cold
                              A holy condition
                              A warrior's position in the world

Evolutionaries believe that the mother of all learning is natural selection. If it made us, it can make anything, and all we need to do is simulate it on the computer. The key problem that evolutionaries solve is learning structure: not just adjusting parameters, like back propagation does, but creating the brain that these adjustments can then fine-tune. The evolutionaries' master algorithm is genetic programming, which mates and evolves computer programs in the same way that nature mates and evolves organisms.

                              Arithmetic
                              A good s**t's the metric
                              Of a dying man

Bayesians are concerned above all with uncertainty. All learned knowledge is uncertain, and learning itself is a form of uncertain inference. The problem then becomes how to deal with noisy, incomplete, and even contradictory information without falling apart. The solution is probabilistic inference, and the master algorithm is Bayes' theorem and its derivatives. Bayes' theorem tell us how to incorporate new evidence into our beliefs, and probabilistic inference algorithms do that as efficiently as possible.

                              I can't believe
                              I won't live forever, therefore,
                              I invented an afterlife to supplement reincarnation

For analogizers, the key to learning is recognizing similarities between situations and thereby inferring other similarities. If two patients have similar symptoms, perhaps they have the same disease. The key problem is judging how similar two things are. The analogizers' master algorithm is the support vector machine, which figures out which experiences to remember and how to combine them to make new predictions.

                              Prepare for a powerful anesthesia
                              Chemical processes irresistible
                              A good and perfect rest