Reinforcement Learning - Return Function

tsaitea · Dec 11, 2019

$Equation: R_t = r_{t+1} + r_{t+2} + r_{t+3} + r_{t+4} + ... = sum from k=0 to infinity of r_{t+k+1}.$

Where did the k come from? I was expecting the index to be t.

Ibix · Dec 12, 2019

##t## tells you the index of the first term you are summing over, so you can't sum over that. You need to sum over a new index, called a dummy index - they've chosen ##k##. Note that if you expand the sum, you will replace ##k## with 0 in the zeroth term, replace it with 1 in the first term, and so on - ##k## disappears when you expand the sum. If you'd summed over ##t##, ##t## would likewise be replaced and disappear when you expanded sum, which is not what you want.

Reinforcement Learning - Return Function

Discussion

Similar threads