Understanding Reinforcement Learning with Neural Networks Part 5: Connecting Reward, Derivative, and Step Size

In the

So, the step size is 0.5.

Next, we update the bias by subtracting the step size from the old bias value 0.0:

This means the probability of going to Place A is:

Choosing Again

We now pick a random number between 0 and 1, and get 0.9.

Now we compute the difference between the ideal value 1.0 and the actual value 0.4.

Using this, we calculate the derivative with respect to the bias, which gives:

Checking the Reward

Now we check whether this was actually a good decision.

Place B gives a large portion of fries, but our hunger input is 0.0, meaning we are not very hungry.

So this was not a good choice.

Therefore, the reward is:

Reward = -1

Updating with Reward

We multiply the derivative by the reward:

-0.6 x -1 = 0.6

So the updated derivative becomes 0.6.

Second Step Update

Now we calculate the step size again:

Now the probability of going to Place B has decreased.

This means that when hunger is low, the model is more likely to choose Place A, which is the correct behavior.

This shows that the reinforcement learning algorithm, specifically policy gradients, is working as expected.

In the next article, we will explore how to further train the model using different input values.

Looking for an easier way to install tools, libraries, or entire repositories?

Try Installerpedia: a community-driven, structured installation platform that lets you install almost anything with minimal hassle and clear, reliable guidance.

Just run:

ipm install repo-name

… and you’re done! 🚀