- #1
tsaitea
- 19
- 0
- TL;DR
- In this post on the Return function is indexed by k?
Where did the k come from? I was expecting the index to be t.
Reinforcement Learning - Return Function
- Thread starter tsaitea
- Start date
-
- Tags
- Function
Click For Summary
SUMMARY
The discussion clarifies the use of the dummy index ##k## in reinforcement learning return functions, emphasizing that it is necessary to avoid summing over the primary index ##t##. The choice of ##k## allows for proper expansion of the sum without losing the original index context. When expanding the sum, ##k## is replaced sequentially by integers starting from 0, ensuring clarity in the summation process. This distinction is critical for accurate calculations in reinforcement learning algorithms.
PREREQUISITES- Understanding of reinforcement learning concepts
- Familiarity with summation notation in mathematics
- Knowledge of index manipulation in algorithms
- Basic grasp of return functions in machine learning
- Study the role of dummy variables in mathematical summations
- Explore reinforcement learning algorithms, focusing on return functions
- Learn about the implications of index choices in algorithm design
- Investigate the mathematical foundations of summation in machine learning
Researchers, students, and practitioners in machine learning, particularly those focusing on reinforcement learning and algorithm design.
Where did the k come from? I was expecting the index to be t.
- #2
- 13,661
- 16,361
##t## tells you the index of the first term you are summing over, so you can't sum over that. You need to sum over a new index, called a dummy index - they've chosen ##k##. Note that if you expand the sum, you will replace ##k## with 0 in the zeroth term, replace it with 1 in the first term, and so on - ##k## disappears when you expand the sum. If you'd summed over ##t##, ##t## would likewise be replaced and disappear when you expanded sum, which is not what you want.