Over the last several months, I have had multiple folks calling me about my experiences with Deep Learning. They ask questions about Machine Learning, Data Analysis, Neural Nets, etc. I actually started with what most folks think of as Artificial Intelligence(AI) over twenty years ago. I read dozens of papers going back to the 70s and 80s. I learned and applied multiple packages to solve a number of problems. I was a neophyte. For me, AI was about studying how human beings solved problems. From what was learned, software and hardware were developed and the two were applied to solve problems.
At the time, I looked at the following approaches:
Rule Based — Expert Systems
From Wikipedia:
“In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert.[1] Expert systems are designed to solve complex problems by reasoning about knowledge, represented mainly as if–then rules rather than through conventional procedural code.[2] The first expert systems were created in the 1970s and then proliferated in the 1980s.[3] Expert systems were among the first truly successful forms of artificial intelligence (AI) software.[4][5][6][7][8].”
The thought behind the expert system was to interview one or more experts in a particular field. gather the expert’s knowledge in a specific field and map the expertise to a series of If – then rules. I was able to map rule based systems to problems that spanned the military to the financial industry. It was relatively trivial to develop systems with GUI based systems that were relatively easy to trouble shoot and maintain. The bad thing was that rule based systems equate to representing real life as 1s and 0s, and real life is not that way.
Fuzzy Logic:
From Wikipedia:
“Classical logic only permits conclusions which are either true or false. However, there are also propositions with variable answers, such as one might find when asking a group of people to identify a color. In such instances, the truth appears as the result of reasoning from inexact or partial knowledge in which the sampled answers are mapped on a spectrum.”
The idea is to instead of trying to force life into yes or no, to build relationships upon words that humans more closely relate. If I asked how to build an application that controlled a train that was stopping at a platform, the engineer talked about slowly stopping — not 3 MPH. It becomes evident that there are places for binary expert systems and fuzzy logic based controllers.
Genetic Algorithms
I think one of the very rewarding things about trying to understand the human mind and body is that you never know what you will discover — something that you would never have directly thought.
Below is an image of an antenna designed by NASA — really the result of designing, implementing and running an algorithm that emulates how humans adapt to different situations.
Pretty cool.
From Wikipedia:
“In computer science and operations research, a genetic algorithm (GA) is a metaheuristic inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms (EA). Genetic algorithms are commonly used to generate high-quality solutions to optimization and search problems by relying on bio-inspired operators such as mutation, crossover and selection.[1]“
This is really about optimization — searching for not necessarily the best solution, but a solution that gets you from point A to point B. Think about police officers that need to navigate a dark room. And the algorithms are based upon how our minds and bodies have adapted to survive.
Neural Nets – The Beginning.
And the final leg of my early venture into AI was the neural net based upon a feed forward perceptron.
Again, from Wikipedia:
“In machine learning, the perceptron is an algorithm for supervised learning of binary classifiers (functions that can decide whether an input, represented by a vector of numbers, belongs to some specific class or not).[1] It is a type of linear classifier, i.e. a classification algorithm that makes its predictions based on a linear predictor function combining a set of weights with the feature vector. The algorithm allows for online learning, in that it processes elements in the training set one at a time.”
After having learned the strengths and weaknesses of the four approaches to solving problems, Expert Systems, Fuzzy Logic Controllers, Genetic Algorithms, and Neural Nets, I decided to pick what worked best to solving certain types of problems. Eventually, I was able to develop an effective threat detection application. As I wrote earlier, you can never tell where the journey will take you.
Time to fast forward to today. I think of all the things that I have learned along the way by poking and probing. I think of powerful packages like Python, sci-learn, Keras, Tensorflow, Theano, etc. I can see it all coming together.
I have done a fair amount of work with computer vision and OpenCV. I had a problem that I needed to solve. First, I had to identify whether someone was carrying a rifle, shotgun, handgun, suicide vest, etc.
Enter Convolutional Neural Nets(CNN):
If you map back to the perceptron, emulating the human brain — you can start to understand how a CNN learns from experiences much like the simple perceptron and applies what is called pooling layers which are simply filters breaking a problem into small parts to be learned.
The second part of a problem that I needed to solve was intent. Because someone has a weapon, it does not mean that the person is a threat.
If you think about threat identification, the first step is to determine whether the object is hostile or friendly. If you look at someone and they have a big gun in their hand, your interest increases. But you still don’t know what the object intends on doing. So time and what that object does over the next second or two is going to be an indicator of intent.
Enter Recurrent Neural Nets(RNN):
One thing about an RNN is that they understand the concept of time. They can store what has happened T-1 time frames and add that information plus current information to provide an answer.
So the solution becomes rack and stack hundreds of CNNs feeding hundreds of RNNs across hundreds of servers to monitor thousands of objects in real time.
Some Things To Consider:
1. Data is the driving force. Understanding your data helps choose algorithms and provides solutions.
2. Not all data is equal — examples to learn need to be carefully selected.
3. Data needs to be cleaned and mapped to distributions.
4. GPUs and the Cloud make almost anything possible.
5. Vision, audio, physiological data and time all help to refine context.
6. Understanding and adapting sensors so that they work in the real world are as important as back-end servers.
This list can go on and on. If you have questions:
troy@biosension.com
6035056500.