What does it mean to train a Neural Network?

Imagine you have just adopted a puppy. This puppy knows nothing: it doesn't know how to sit, it doesn't know where to do its business, it doesn't even know it should come when you call it. But with patience and training, you can teach it. A neural network is like that newly adopted puppy: it knows nothing at first, but it can learn over time and it takes a lot of practice.

What is a neural network?

A neural network is nothing more than a huge collection of decimal numbers (like 0.234 or 7.891). These numbers don't do anything useful at first, they are like a caveman trying to use a smartphone.

But, if you adjust them correctly, they can become incredibly useful. These numbers, called "weights", are what the network uses to make decisions or solve problems. Training is the process that gives meaning to those numbers.

How does a neural network learn?

Let's think of a simple example:

We want our network to learn to add two numbers, like 1 + 2.

1. Trial and error:

The network starts completely lost. You ask:

“What is 1 + 2?”
The network answers: “999”.
ERROR. It's not correct.

2. Adjust the numbers:

Every time the network fails, it adjusts a little bit those decimal numbers (the "weights") to try to improve the next time. And when we say a little bit, it's thousandths or millionths.

3. Repeat thousands of times:

You ask again: “What is 1 + 2?”
It answers: “5”.
ERROR. But this time it's a little closer.
Adjust the numbers again and try again.

After thousands or even millions of attempts, the network finally learns that 1 + 2 = 3.

Why does it seem complicated?

In this example, we only want the network to learn to add two numbers. That's relatively easy because the problem is small.

But imagine training a neural network for much more complex things, like:

Recognizing faces in a photo.
Talking to you like ChatGPT does.
Predicting the weather.

To solve problems like those, the network needs millions or billions of decimal numbers that need to be adjusted. That takes a lot of time, a lot of computing power, and a lot of patience.

What happens after training?

Once the network is trained, those decimal numbers have the correct "weight" to answer correctly most of the time. In other words, the network now knows how to add, recognize faces, or talk to you.

But, and this is important, a trained network is not always perfect. It can make mistakes, especially if you ask it questions it never saw during training.

Why is it like training a puppy?

Without training: The network (or the puppy) knows nothing and will make many mistakes.
With practice: After many repetitions, it learns what to do and does it well.
There are always limits: Just as a trained puppy can't learn algebra, a neural network can only do what it was trained to do.

Training a neural network may seem like magic, but at the end of the day, it's just a process of trial, error, and constant adjustments. And that's what makes it possible for machines to be smarter.