Skip to content

Commit b158880

Browse files
authored
clarify backward in time = earlier
1 parent b0ad3e4 commit b158880

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

ch8/rl/README.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,11 @@ Now the situation is reversed: the `Rew` layer shows that the reward has been pr
5858

5959
* Do `Step Trial` to process the rest of the trial, and switch to viewing `Train Trial Plot`.
6060

61-
The plot shows that the "dopamine spike" of TD delta has moved forward one step in time. This is the critical feature of the TD algorithm: by learning to anticipate rewards one time step later, it ends up moving the dopamine spike earlier in time.
61+
The plot shows that the "dopamine spike" of TD delta has moved backward (earlier) one step in time. This is the critical feature of the TD algorithm: by learning to anticipate rewards one time step later, it ends up moving the dopamine spike earlier in time.
6262

6363
* Keep doing more `Step Trial` (or just `Train`).
6464

65-
You should see that the spike moves "forward" in time with each `Step Trial`, but can't move any further than the onset of the CS at time step 10.
65+
You should see that the spike moves "backward" in time with each `Step Trial`, but can't move any further than the onset of the CS at time step 10.
6666

6767
We can also examine the weights to see what the network has learned.
6868

0 commit comments

Comments
 (0)