Find a function from a Recursive Formula

[loveinla]

Hi,

I'd like to solve a function $$W_{t}^{x}$$ from a recursive formula below.

$$W_{t}^{x}=P_{t}^{x}*W_{t+1}^{x+1}+(1 - P_{t}^{x})*W_{t+1}^{x}$$,

where $$P_{t}^{x} = \frac{E[\theta^{x+1}*(1 - \theta)^{t-x}]}{E[\theta^{x}*(1-\theta)^{t-x}]}$$, $${\underset{t\to\infty, x\to 0}{lim}}W_{t}^{x} =0$$. Here $$E$$ stands for expectation.

Any suggestions? Thanks.

 

[Stephen Tashi]

$$E$$ stands for expectation.

Expectation with respect to which variable? Over what range of integration?

 

[dodo]

Expectation with respect to which variable? Over what range of integration?

I'd imagine on the only free variable in the expression for P, namely $\theta$, and over $(-\infty,\infty)$. But you do well in asking instead of guessing like yours truly. :)

What worries me most is that, in the recurrence rule for W, the "x" does not get any smaller. So what are supposed to be the initial values?

 

[loveinla]

I'd imagine on the only free variable in the expression for P, namely $\theta$, and over $(-\infty,\infty)$. But you do well in asking instead of guessing like yours truly. :)

What worries me most is that, in the recurrence rule for W, the "x" does not get any smaller. So what are supposed to be the initial values?

Thanks, you are right, the expectation is with respect to $\theta$, which is on [0,1].

The initial value is indeed what I am trying to find.

In fact, it is a one-armed bandit problem with Bernoulli probability $\theta$, but $\theta$ is a random variable we are trying to figure out through the realization.

Any idea on how to determine the initial value $W^{0}_{0}$ or $W^{x}_{t}$?

 

[CellCoree]

I would first suggest checking the validity of the recursive formula by plugging in different values for t and x and seeing if the formula holds true. If it does, then we can proceed with finding a function for W_{t}^{x}.

One possible approach could be to use the method of generating functions, where we define a generating function G(z) = \sum_{t=0}^{\infty}\sum_{x=0}^{\infty}W_{t}^{x}z^{t}y^{x} and try to find a closed form expression for it. This would involve manipulating the recursive formula and using techniques such as partial fraction decomposition and binomial expansion.

Another approach could be to use the concept of Markov chains, where we can view W_{t}^{x} as the probability of being in state (t,x) and use the transition probabilities given by P_{t}^{x} to construct a transition matrix. From there, we can find the stationary distribution of the Markov chain, which would give us a function for W_{t}^{x}.

Both of these approaches may require some mathematical background and may not necessarily yield a simple and straightforward solution. Another option could be to use numerical methods and solve for W_{t}^{x} for specific values of t and x. This could provide a more practical solution for real-world applications.

In any case, it would be important to thoroughly understand the assumptions and limitations of the recursive formula and the resulting function, and to validate the results through further analysis or experimentation.