First Principle of Machine Learning

Machine learning is very popular today. Despite the fancy terms and complex jargon often associated with it, machine learning is simply a tool — much like a sword and axe. It is a tool that help human to recognize patterns in the world.

The Goal of Machine Learning

The goal of machine learning is to discover pattern in data.

Specifically, the goal is to create a mathematical function (or pattern) that can accurately map one set of data to another.

For example, imagine we have a list of house sizes and their corresponding prices. We have the data, but we don't know the exact function that links the size of a house to its price. This is where machine learning can help.

First attempt of training. we initialize the parameters with random values, which results in a model that is initially wrong (as shown by the orange line). The model’s predictions differ from the actual values, known as the loss. Our goal is to minimize this loss.

Gradient Descent

By nature, the lower a loss, the better a model. We need to make a model with lowest loss. So, our goal of machine learning can be restated as this: to find a model’s parameters that minimize the loss.

To minimize the loss, imagine you’re adjusting two knobs on a radio to find your favorite station. Initially, you might turn both knobs randomly. You notice that one knob is quicker at finding the station, while the other is slower. As you experiment, you start fine-tuning each dial based on how effective it is.

Just like tuning a radio to find a favorite station, adjusting the model’s parameters involves checking how changes affect the loss. This adjustment process is guided by something called the gradient, which indicates how quickly the loss changes as we adjust the parameters.

Optimizer

Intuitively, if the gradient is higher, we need to make bigger adjustments to the parameters. If the gradient is lower, we should make smaller, more gradual changes. There are various optimizers to help with this tuning, but the basic idea remains the same.

Repeat

Repeat this process: define new parameters-> produce a wrong model -> calculate the loss -> determine the gradient -> adjust the parameters accordingly -> define new parameters-> … Keep repeating these steps until finding the optimal parameters that minimize the loss.

After several times of adjusting the parameters, the model’s predictions finally align with the actual data (notice how the orange line fits the blue dots).

Conclusion

Congratulations! You’ve grasped the concept of machine learning without needing to dive into math and code. In summary, machine learning aims to create the best function to map an input dataset to an output dataset. The process is straightforward: we start with a model that’s initially wrong, learn from its mistakes, and use that knowledge to improve the model over time. To me, this approach offers a valuable philosophical lesson about learning and growth.