What Are Artificial Neural Networks, Really?

Machine Learning has been a topic receiving a lot of attention in the last few years, partly because of the improvements reached in computer hardware and software that enable experimentation with such concepts to be as cheap as we’d like it to be.

Whenever we talk about Machine Learning, the term Deep Learning comes across in a way or another. This is because the latter revolutionized the way we think and apply the broader aspects of Machine Learning.

Biological Neural Networks

It makes all the difference to think about the one thing that defines our effectiveness as human beings is the thing that we understand the least — the human brain. A set of billions of small, heavily interconnected components (called Neurons) that exchange electrical signals whose values determine what the surrounding neurons should exchange with their neighbors. This master synchronicity depicts what thought and memory in the brain are.

Artificial Neural Networks

Neuroscientists and computer scientists who understood the power of the human brain realized that introducing the world to an artificial form of this biological masterpiece could be anything but short of benefit. This artificial form — called an Artificial Neural Network (ANN) — looks something like the following:

If we look thoroughly, the artificial depiction still comprises of the little components that we talked about (neurons), and each one of them is connected to a few neighboring ones. This ANN is built through code, to the point that the whole thing becomes one call away from effect (a function call to a model).

It is worth mentioning that a lot of people argue that the resemblance between biological and artificial neural networks stops at this point, emphasizing on the fact that an ANN only looks like any very small part of the brain, but doesn’t act like it, attributing that to the fact that we still don’t fully understand how the brain works — as a whole.

But, why would we go through the hassle?

Yes, this simple manifestation of the brain might have led to a lot of debates and misinformation regarding how the brain really works, but it has revolutionized the world of computing. From image classification to speech and facial recognition, most tech giants have adopted the concept of ANNs into their applications. In Machine Learning, the objective is always to build something that generalizes well. This means that we are constantly trying to build a program and train it very well (by tweaking mathematical functions and terms) so that it can make a hypothesis about new inputs that it has never seen before. However, one shortcoming that we often face is the fact that a lot of the applications cannot be limited to one linear function. That is, not all applications have inputs and outputs that are able to fall on the same line. Moreover, it is not always possible for a mathematical function to be modeled in a way that is able to predict really complex problems.

That is where ANNs come into play. If we think of a program (or a model) that predicts house prices but doesn’t generalize well, we can introduce some non-linear complexity to it and try perfecting — to an extent — the newly formed combination. That is exactly what an ANN is. We take our house prices predictor and duplicate it in a formation (ANN architecture) and link those duplicates with variables that show how important one is to the other (weights). Artificial neurons in an ANN play the role of the duplicates in this context.

Types of ANNs

ANNs have a few types and formations that depend on two things; network architecture and activations. Network architecture (i.e. topology) is how the neural network is designed. This includes the number of neurons/nodes, the number of layers and how those neurons are interconnected. Activations include the functions that neurons use to determine what output to feed into their neighboring neurons. Different ANNs are similar yet very different as well. They are conceptually similar, as they follow the same theoretical background and require mostly the same knowledge. However, they are different in the way they are applied. For example, most image classification applications include a Convolutional Neural Network (CNN) due to convolution’s effectiveness in Image Processing as a field. On the other hand, Speech Recognition applications rely heavily on Recurrent Neural Networks (RNNs) as no other architecture represents sequential data in such a fruitful manner.

Artificial Neural Networks have introduced Machine Learning to the efficiency of modeling classifiers in a non-linear manner. This changed the way we think about complex Machine Learning and Pattern Recognition problems.