Markov, find a selection strategy which maximizes probability

[skaterboy1]

Homework Statement

There are three patchs in which an animal can forage.
Patch 1: Risk of predation is 0. Probability of finding food is 0. Energy value is 0.
Patch 2: Risk of predation is 0.004. Probability of finding food is 0.4 and energy gain is 3.
patch 3: risk of predation is 0.02. Probability of finding food is 0.6 and energy gain is 5.

Foraging in any patch uses one unit of energy reserves. Energy reserves below 4 indicate death. maximum energy capacity for animal is ten units.

Solve this problem for 20 foraging periods to find a patch selection strategy which maximizes the animals probability of survival over this period.

Homework Equations

?

The Attempt at a Solution

Three patchs; N=3
States: risk, food findings and energy gain.
action: Choosing patchs 1, 2 or 3.

How do I solve this problem?

 

[Ray Vickson]

You could try to set it up as a standard Markov Decision Problem, where you want to maximize the expected value of a reward function. In this case, the reward is r = 1 if energy at t = 20 is >= 5 and is 0 otherwise. You need to identify states, decisions and decision-dependent transition probabilities.

RGV

 

[skaterboy1]

You could try to set it up as a standard Markov Decision Problem, where you want to maximize the expected value of a reward function. In this case, the reward is r = 1 if energy at t = 20 is >= 5 and is 0 otherwise. You need to identify states, decisions and decision-dependent transition probabilities.

RGV

Why is the reward r=1 if energy is >= 5 (not 4?) and why 0 otherwice?

 

[skaterboy1]

Do you have any sample problems to show me?

 

[Ray Vickson]

Why is the reward r=1 if energy is >= 5 (not 4?) and why 0 otherwice?

OK, it should be r = 1 if energy >= 4 and 0 otherwise.

RGV

 

[Ray Vickson]

Do you have any sample problems to show me?

Do you not have a textbook? Are there no lecture notes? Anyway, Google is your friend: try "Markov decision problem".

RGV

 

[skaterboy1]

http://wdl.persiangig.com/pages/download/?dl=http://rnezhadhasan.persiangig.com/narmafzar/1210-5.pdf