What are Neural Networks?

Neural networks are used everywhere these days. They change user inputs through an algorithm consisting of some fancy math, to output something useful. Chatbots like ChatGPT are based on neural networks. Moreover, Netflix uses a neural network to recommend new movies and series for you to watch. The input into the network is the movies/series you have already watched. As output, unwatched movies/series pop up in your recommendations. Detecting skin cancer is another example of how neural networks can be used. Some phone applications allow you to take a picture of a skin mole. A neural network will then tell you the risk that this mole is cancerous.

A Flowery Example

Imagine you see a flower of the iris plant, which has many subtypes. If you want to know which subtype it is, you could take a picture with Google Lens to get the answer. This app uses a kind of neural network. But how does such a network decide on the correct subtype of iris in your picture? There are three very similar-looking subtypes: setosa, versicolor, and virginica. They have sepals and petals, as you can see in Figure 1A. We can measure the length and the width of these sepals and petals (in centimeters) and collect this information in a dataset. Figure 1B shows measurements from six flowers chosen randomly, two for each subtype, from the entire dataset of 150 observations. As you can see, the lengths and widths are not the same for each subtype. When working with neural networks, we call the characteristics of the flower (such as sepal length) features while the subtypes are called categories. The idea is that we will use these features to figure out the category of iris flower. We call this “predicting”.

Try to discover some patterns in the flower dataset in Figure 1B. Focus on the petal and sepal widths. How are these features different for each subtype? (now try Exercise 1 in Box 1)

Creating a Neural Network

Neurons

To help you understand neural networks, we first need to talk about what a neural network consists of [1, 2]. One of the most important components are called neurons (hence the name “neural network”) because their function was inspired by nerve cells called neurons in the brain [3]. Figure 1C shows an overview of a neural network that could be used to analyze the flower data. The neurons are represented by circles. Every neural network always has an input layer and output layer. In the input layer (green), the number of neurons is always equal to the number of features, so in our case there are four neurons (N1, N2, N3, and N4). For the output layer (orange), the number of neurons always equals the number of categories, three in our case (N9, N10, and N11), corresponding to the three flower subtypes (now try Exercise 2 in Box 2).

Weights and Biases

Neural networks also contain other/middle layers (purple) called hidden layers, with a certain number of neurons. These layers process the input to generate the output. There could be as many hidden layers as you want in the network. Each hidden layer could also contain as many neurons as you like. For our data, one hidden layer with a relatively small number of neurons (four) will work well. Each neuron in a layer has connections to each neuron in the next layer (shown by the arrows). For example, neuron N5 in the hidden layer has connections to all four neurons in the input layer.

In practice, these connections are actual numbers, and we call them weights (w1, w2 etc.). So, the number of weights corresponds to the number of connections. The purpose of these weights is to transform the input values (sepal length, width, etc.) into different numbers. This helps the network to decide which category the flower belongs to. You can also think of the weights as the strength of the connection between neurons. So, a large value means that the neuron has a lot of influence on the connected neuron. Each neuron, except the input neurons, also has another number called a bias (b1, b5, etc.) that further helps to determine the category. This number basically tweaks the results from the weights to make the network more flexible. For example, a large bias further strengthens the role of that neuron (now try Exercise 3 in Box 3).

Training a Network

How does the neural network determine the category from the features of the flower? For example, if we plugged in the width and length values from the first row in Figure 1B, how would the network know this is a setosa flower? To help you understand this, we need to talk about training and testing a neural network.

To start, we randomly separate the dataset into a training set and a test set. In general, the training set should be larger than the test set. More training data tend to lead to more accurate predictions. But there needs to be enough data to test the network on, so it is a bit of a trade-off. In our case, we can use 2/3 of the flowers (100) for the training set and the rest (50) for the test set (If you would like to read more about training a network, see this article).

When we first give the network the training set, it does not know the correct category for each flower, and it basically guesses. At this point we say that the network is untrained. The weights are random numbers, while the biases are zero (Figure 2A). We feed the network all the features of the 100 flowers from the training set and ask it to predict which type of flower the data are from. In the beginning, the output will often be incorrect, and we tell the network when it is wrong. We can express how wrong the network is, which is called the error. At the start of training the error will be large, but after each run the network adjusts its weights and biases to make the error smaller. Step by step, the network learns patterns in the data and starts to predict the correct categories for each presented flower. For our network, the training stage took about 5 s—but for large datasets it can take hours or longer (now try Exercise 4 in Box 4).

Testing a Network and Accuracy

After enough training, the error will become really small, and the network will settle on certain weights and biases (Figure 2B).

However, those numbers are pretty meaningless at first glance. That is why networks are considered black boxes. Still, it will likely make the right predictions. There is one catch though: the network has settled down on weights and biases that are optimal for the training set. What would happen if we showed the network new examples of iris flowers that it has never seen before? This is the ultimate test, and so we call this “testing the network.” We do this using the test set of flower data that we created earlier. For this new data, we often calculate the accuracy of the network to determine its performance. The accuracy is the percentage of flowers that are predicted correctly. Often, the accuracy will not be 100%. Values between 90 and 100% are considered pretty good (now try Exercise 5 in Box 5).

It is finally time to use our network (from Figure 1C) to predict the categories for our test set. (You can play around with this yourself on this website.) Figure 3 shows a snippet of code to create and run our network (you do not need to understand the code snippet). The bottom of the figure shows the results, in the form of a confusion matrix and the accuracy of the network. Do not get confused by the confusion matrix! It merely shows the correct and incorrect predictions. The rows refer to the true flower categories, while the columns represent the category predictions made by the network. So, all the correct predictions end up along the diagonal, while incorrect predictions appear in the other cells. Our network mistakenly classified two virginica flowers as versicolor, so we ended up with an accuracy of 96%. Remember, our network has never seen these 50 flowers before and still managed to determine the right category for 48 of them. Pretty impressive, right?

Limitations of Neural Networks

As cool as neural networks may seem, they do have certain shortcomings. For one, we often say that these networks are black boxes. That means we do not know how the features are used to predict the categories of the flowers. For example, it could be that a single feature, such as petal width, is very important for predicting the flower category. The network would not be able to tell us this. There are other algorithms that can explain their decision during prediction. An example is multiple linear regression, which you can read about in this article.

Another issue is that the network’s performance heavily depends on the training data. The final weights and biases are based on the training data. These weights and biases are then used to predict categories for the new test data. Imagine if the training data are not like the actual iris flowers and thus are very different from the test data. For example, maybe for some reason the training data only came from flowers of each category with very small sepal and petal widths and lengths. If we train our network with this unusual set of training data, then the weights and biases will be rooted in this data. So, when we feed the trained network random new flowers (the test set), its performance will likely be poor.

What to Remember

Neural networks use input information about an object (petal and sepal width a flower, for example) to predict something about that object (such as flower subtype). The network has weights and biases that are updated during the training stage. During the test stage the network can often predict new observations that it has never seen before with good accuracy. So, getting back to our initial example, ChatGPT has learned the response to lots of different input prompts and can now take your message and predict a useful response!

Glossary

Neural Networks: ↑ A complex function that creates output based on the input.

Algorithm: ↑ A process that solves a specific problem based on a set of mathematical rules.

Dataset: ↑ A structured collection of data with the variables in the columns and the observations in the rows.

Bias: ↑ In the context of neural networks, refers to a number connected to each neuron that helps the network to determine the correct category.

Error: ↑ A number to represent how wrong the network is when it predicts the categories.

Confusion Matrix: ↑ A table that shows the number of correct predictions on the diagonal while the incorrect predictions are in the other cells.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank several of our colleagues at Leiden University and at Karolinska Institutet as well as an anonymous child for their helpful comments on previous versions of this manuscript.

AI Tool Statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Additional Materials

https://python-kr7xiht74wpxndrfffbxzz.streamlit.app