Discovery of knowledge from data- that’s the goal of what Pedro Domingos calls “The Master Algorithm.” This master algorithm is a single algorithm capable of discovering all knowledge from data. A single algorithm to automate discovery.
Machine Learning algorithms today are aware of our favorite movies and TV shows, can filter out spam emails, and can assign a name to a face in a photograph (a.k.a tagging on Social Media). They can analyse bodily fluids, recognize cancerous tissue, and recommend patients treatment options.
While reading the book, Machine Learning for Dummies, I came across the “Five Tribes of Machine Learning” which are essentially five algorithmic techniques and strategies to solve a problem in machine learning. The authors describe these five techniques as follows:
Symbolists
This tribe resonates with logic and philosophy. Its origins stem from “Symbolic Artificial Intelligence” or better known as “Good Old Fashioned AI” or “Rule-based AI systems.”
In this category of AI algorithms, Logic (Logical connectives: AND, OR, and NOT; Boolean output: True or False) is the main problem solving technique.
The conventional Machine Learning algorithm requirements like large amounts of data and training are not needed for this tribe of algorithms to do its work. The problem solving technique employed here is representing problems as symbols and then applying logic to reach to a solution to the given problem. Now symbols can be anything, from numbers and letters, words and a book to you and me. To represent these symbols we put them within parenthesis like:
(Aarushi) or (chair) or (reader)
To describe objects, we use “relations” which is an adjective placed in front of a symbol, like:
smart (person)
In the example “smart” is the adjective or “relation” and (person) is the symbol.
Another use of relations is to describe the interaction between two or more symbols, like:
write(Aarushi, articles).
So that would translate to “Aarushi writes articles.”
A collection of all such symbols is known as a “knowledge base.”
Another term that I came across while reading about symbolists was the inverse deduction technique to solve problems. According to Jesus Rodriguez, instead of the classical model of starting with a premise and looking for the conclusions, inverse deduction starts with a set of premises and conclusions and works backward to fill in the gaps.
Connectionists
This is probably the most well-known tribe among the five. The origin of this tribe is in neuroscience and re-engineering the brain. Yep, that’s right- Neural Networks.
Neural networks built to resemble the human neurons. Essentially, each “neuron” is an algorithm that models the real-world counterpart and it solves a small piece of the problem, and the use of many neurons in parallel solves the problem as a whole.
What exactly does a neuron do? A neuron receives inputs, processes those inputs, and generates an output.
Neural networks are adaptive systems which basically means that they have the ability to learn.
This group relies on back propagation to solve problems. Each connection between a pair of neurons has a “weight” (how much a particular input weighs into the result) and by adjusting these weights and “biases” (which can be thought of as an intercept in a linear equation. It is an additional parameter in the Neural Network which is used to adjust the output along with the weighted sum of the inputs to the neuron. Moreover, bias value allows you to shift the activation function to either right or left), we can tweak our algorithm to get the output we desire.
The goal is to continue changing the weights and biases until such time as the actual output matches the target output. Each neuron passes information to the next neuron in line until the group of neurons creates a final output.
Evolutionaries
The origin of this tribe is in evolutionary computing. This group relies on genetic programming to solve problems.
Ever heard of the phrase “Survival of the Fittest?” According to Wikipedia, “Survival of the fittest” is a phrase that originated from Darwinian evolutionary theory as a way of describing the mechanism of natural selection. Genetic Algorithms are based on this concept.
This strategy can be used when you require a perfect optimized solution in a sea of possibilities and when it’s impossible to check every single possibility (brute force search) for a potential solution because the search space is extremely extremely vast. In Professor Daniel Shiffman’s words, genetic algorithms are used to design a system that uses evolutionary algorithms to evolve a behavior, design or optimize a search problem.
We start with a “population” of potential solutions. While applying the genetic algorithm strategy, we eliminate the weaker solutions and the stronger solutions pass on their “characteristics” to the next stronger generation. Each new generation is optimized. Each new generation is closer to the solution. (Check out https://rednuht.org/genetic_cars_2/ to see a genetic algorithm in action!)
There are three key principles that need to be followed for a genetic algorithm to function optimally. 1) Variation (a diversity of traits in a given population) ; 2) Heredity (children receive characteristics or qualities of parent) ; 3) Selection (Survival of the fittest).
Bayesians
This origin of this tribe is in statistics. That spam filter that gmail uses to save you from the headache of doing it yourself? That’s an example of the Bayesian tribe. This group relies on probabilistic inference to solve problems.
Analogizers
The origin of this tribe is in psychology. This group relies on kernel machines to solve problems. Here the terms SVM or Support Vector Machine is thrown around quite often. So what is a Support Vector Machine?
Support Vector Machines are a useful technique for data classification and regression analysis. SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik and Chervonenkis. Kernel machines recognize patterns in data. By recognizing the pattern of one set of inputs and comparing it to the pattern of a known output, you can create a problem solution. The goal is to use similarity to determine the best solution to a problem.
On combining these five techniques, we can ultimately reach “The Master Algorithm.”
*******************
References:
- bbva.com- “The five tribes of ‘machine learning’”
- salesforce.com- “Machine Learning: 6 Real World Examples”
- Machine Learning for Dummies
- bdtechtalks.com- “what is symbolic artificial intelligence”
- Crash Course– “Symbolic AI”
- medium.com– “Jesus Rodriguez- The Five Tribes of Machine Learning”
- natureofcode.com- “The nature of code”
- intellipaat.com
- Genetic Algorithms @ The Coding Train by Daniel Shiffman
Man that was one long article! It took me two days to briefly touch upon all these concepts and write about them. I’ll be focusing more on the Bayesian in the future articles, hence I haven’t really elaborated much on this topic.
Want to join in on my learning journey? Feel free to subscribe!
