squenshl said:
The only formula given is ##B^{-1}AB##
I'm completely lost on this.
Given where? There's no B in the problem statement. How do you define your B?
In post #1, you're letting us know that you know that if two matrices represent the same linear operator in different bases, then they are similar (i.e. if one of them is A, then there's a matrix B such that the other one is ##B^{-1}AB##). But if the book hasn't proved a theorem that gives you a formula for the matrix B, then you will have to work directly with the definitions in post #5, and the definition of matrix multiplication: ##(AB)_{ij} =\sum_k A_{ik}B_{kj}##.
Edit: Also, since the problem involves
two changes of ordered bases, you should expect
two matrices to show up in the answer.
This isn't an easy problem, because it requires you to understand the definitions well enough to use them correctly, and because it involves
two changes of ordered basis instead of just one. But it's fairly straightforward in the sense that if you choose a good notation and apply the definitions correctly in every step, you will get the result you want.
The notations I used in post #5 are good enough, but we also need something to distinguish between the ith component of a vector x with respect to U, and the ith component of the same vector with respect to V. You could e.g. denote the former by ##x_i## and the latter by ##x_i'##. In your case, that would be ##x_i## for the ith component with respect to the standard ordered basis, and ##x_i'## for the ith component with respect to the other one.
In general, when you ask for assistance here, you should post your attempt up to the point where you're stuck. If you're stuck right at the start, then you should at least explain what definitions and/or theorems you think you should be using, and what it is about them that confuses you or makes you think they won't solve the problem.
Edit: These are some suggestions to get you started. I will denote the standard ordered bases for ##\mathbb R^n## and ##\mathbb R^m## by E. (If that feels weird, you can use notations like ##E_n## and ##E_m##). I will denote the other two ordered bases by B and C. In my notation, what's given in the problem is that ##[f]_{E,E}=A##. The problem is asking you to use that to find ##[f]_{C,B}##. Post #5 gives you a formula for its ij component. It's ##([f]_{C,B})_{ij}=(fb_j)_i'##. How do you proceed from this? A good start would be to multiply by ##c_i## and sum over i. Then you can use the definitions (and what I'm saying in the last paragraph below) to simplify the right-hand side. When you've taken that as far as you can, you should look for another way to rewrite the left-hand side.
Some other useful notations: Let M be the the linear operator such that ##b_i=Me_i## for all i. Let N be the linear operator such that ##c_i=Ne_i## for all i. Note that ##b_i=Me_i=\sum_j (Me_i)_j e_j =\sum_j ([M]_{E,E})_{ji} e_j##. It's convenient and relatively harmless to simplify the notation from ##[M]_{E,E}## to just M. So we can write ##b_i=\sum_j M_{ji} e_j##.