# Approximation theory

by a.mlw.walker
Tags: golden section, levenberg marquardt, parameter estimation
 P: 153 Yeah I got better too, but mine curves the other way...(attached). That is using D H's idea that c0 is not a parameter in itself. The website that you said you found the whole document at says that T0 is the time of the initial revolution, on the page called manual at the bottom somewhere.... What have you plotted on the y axis? I plotted the cumulative time? Attached Thumbnails
HW Helper
P: 928
 Quote by a.mlw.walker Yeah I got better too, but mine curves the other way...(attached). That is using D H's idea that c0 is not a parameter in itself. The website that you said you found the whole document at says that T0 is the time of the initial revolution, on the page called manual at the bottom somewhere.... What have you plotted on the y axis? I plotted the cumulative time?
I made goof and just now edited my last post with the correct version. Almost a perfect fit (looks identical to yours), though I still used 3 parameters. Looks like you are on the right track.
Mentor
P: 13,602
 Quote by hotvette Hmmm, if that's the case, the author's description of equation 40 is misleading. He indicates the function to minimize is a function of a,b,c0 implying that all three need to be estimated. He should have said the function to minimize is a function of a,b where c0=xxxxx. Not well written.
I'm not gonna argue with you on that one! Look at my post #7: I initially thought that equation #40 was a minimization with respect to those three parameters myself. It took a couple of readings to see that this is not the case.
HW Helper
P: 928
 Quote by D H I'm not gonna argue with you on that one! Look at my post #7: I initially thought that equation #40 was a minimization with respect to those three parameters myself. It took a couple of readings to see that this is not the case.
Looks like our collective understanding is converging. I tried a two parameter fit and the result is nearly identical to a three parameter fit. Almost looks too good.

What still bothers me is the expression T0 = t1 - t0 because it seems to me t0 has to equal zero (based on equation #35).
 Mentor P: 13,602 a.mlw.walker: Unless you object, I'd like to move this back to mathematics, where you originally asked about this topic. You didn't get any bites the first time around because you put the question in "General Math" and you gave the thread a bad title ("please can people advise me on this!"). Threads in general math whose titles are of the form "please help me", are entirely in lower case, and ends with "!" are typically from students asking us to do their homework for them while they go toss down a beer. So, other than perhaps the mentor responsible for that section, nobody even looked at your post. This time around you got bites despite having put the thread in the wrong place (this is not a MechE question) because you gave the thread a good title and kept the original post short and to the point.
 Mentor P: 13,602 Now back to the question at hand: I have noticed on occasion that a non-linear multivariate fit doesn't seem to fit as well as I'd like and it seems to leave some signal in the residuals. Polishing that initial fit is with a second fit on a restricted set of variables oftentimes does the trick (e.g. as was done in this paper) . That polishing is admittedly a bit ad hoc. If that adhocery doesn't work I either resort to something even more ad hoc or I back up and try again with a drastically different technique.
 P: 153 Cool, we are getting to an aggreement. Hotvette got the two and three parameter fits to agree. Hotvette, on the website you found the actual document on, there is a link to a page called manual. Search for this line t0: time of initial revolution in the chapter called 3. Set Ball. That is why i think T0 is just the time for the initial timing. D H If you would like to move it, do so by all means. Apart from it being specific to parameter estimation, I think the computing side of this problem could allow it to be considered as engineering, however i am not bothered. I dont even know if you have already moved it.... Your point on second fits. How do you know when a fit is good enough to not need a second fit? He does one yes, however the curve looks so good that is there some way to determine whether a second fit is necessary? Once we do/dont do a second fit, the next problem is equation 42. Can we talk about what he does here. He uses a and b and some approximations for theta_f to optimize for the three parameters. However he says he can solve it for the one nonlinear parameter first, and the linear parameters linearly. I have read about this, but Im not sure how the method changes. What is the method of trisection here? Google cant find much on it, but i suspect its similar to the golden section search as in it finds a minimum as you vary the non linear parameter of 2pi there will be a minimum error to try and find... Out of interest if you have linear and non linear parameters, can you solve for all of them non linearly and still get the correct linear parameters or does it have to be done the way he mentions - thats only a side note, just wondering... then the linear parameters can be found by any old meansin the linear equation 43.
 P: 153 Guys, I have tried to use the golden section search for equation 41. It runs successfully but doesnt seem to produce a 'better' fit. What do you reckon. I used beta as ab^2 like he said, and c is a constant from the solution of the first part (attached) Attached Thumbnails
 P: 153 hey guys, you gone on holiday?
HW Helper
P: 928
 Quote by a.mlw.walker hey guys, you gone on holiday?
In my case it was because I wasn't sure there was anything more I could add that would be useful. I reviewed the paper one more time and am convinced of the following (others may disagree).

1. Step 1: equation #40 is intended to obtain initial estimates of all three parameters (not two) based on equation #35. Subsequent discussion refers to further refinement steps and has nothing to do with equations #35 and #40.

2. Step 2: parameter refinement for a & b using equation #41 (and definition for c0) isn't intended to get a better fit for equation #35. It is meant to get better estimates of a & b for use in step 3.

3. Step 3: using the refined esimtates for a & b from step 2, obtain estimates for the remaining parameters using equation #43

I believe the intention is to discard the previous equations at each new step. Re why the author chose the particular optimization method for each step, it is difficult to say. Perhaps the author tried several methods each time and found one that seemed to work better in each case.

I seem to recall you asking a question about a curve fitting situation where some parameters are linear and others are non-linear. As far as I know, even if only 1 parameter is non-linear they all need to be treated as non-linear. Even if a least squares problem is linear in all unknowns, it can still be approached as a non-linear problem (and solved in a single iteration).

Hope this helps. I really don't think there is anything more of use I can add.
 P: 153 Great thanks Hotvette, however after equation 42, before 43, the author writes that using the method of trisection the value for phi can be found and then the equation becomes a linear equation (43). How would you find the value of phi without also finding the other parameters - or is this what you are saying. solve it completely non linearly and then using the value of phi improve the other approxiomations linearly? Did you see my graph above, trying to improve the parameter a, gave a worse fit?
HW Helper
P: 928
 Quote by a.mlw.walker Great thanks Hotvette, however after equation 42, before 43, the author writes that using the method of trisection the value for phi can be found and then the equation becomes a linear equation (43). How would you find the value of phi without also finding the other parameters - or is this what you are saying. solve it completely non linearly and then using the value of phi improve the other approxiomations linearly? Did you see my graph above, trying to improve the parameter a, gave a worse fit?
It's difficult to unambiguously interpret the paper . I think it's clear that equation #40 is a 3-parameter problem. It clearly says S(a,b,c0) = xxx and all three are mentioned in the next sentence.

Beyond that it's a bit fuzzy. Equation #41 looks like a single parameter problem (i.e. S(a) = xxx) to refine the value of 'a', thus the use of golden section or trisection. Beta is fixed using the values of a & b that were determined from equation #40. What isn't clear is whether c0 is considered fixed using the same values of a & b that were used for Beta or whether 'a' is still considered a variable. Once you solve equation #41, forget about #40 and #35, they no longer apply (that was my point in the previous post).

What's the ultimate goal of this? Predict where the ball will land?
 P: 153 I suppose thats the ultimate goal. I am applying for a job in finance and have been advised that I have a very good grasp of estimation theory. After hinting the interenet for more complex examples and I came across this years ago so thought that I would try and solve them. My background is mechanical engineering, but usually when i need to fit a curve its to a polynomial not to an equation like in this paper. I have read a little more and emailed the author and have found out that eqn 41 is used to quantify the goodness of the fit. I.e if a changes much then the fit is not very good. D H mentioned that this technique is an 'ad hoc' techinque. As the original fit from equation 40 is better, I will use other methods to describe the goodness of the fit. Just looking at equation 40, the mimization method usually is the sum of the difference between real data and theoretical data (squared). however in equation 40 i cant tell which part is the real data part, can you see what i mean? EDIT: Oh right, eqn 41 is not the sum squared, it is the modulus of the sum, why has the author written this differently?
Mentor
P: 13,602
 Quote by hotvette 1. Step 1: equation #40 is intended to obtain initial estimates of all three parameters (not two) based on equation #35. Subsequent discussion refers to further refinement steps and has nothing to do with equations #35 and #40.
One problem with this: The c0 in equation #35 is a function of b and c1, the latter of which is given by equation #25 or #26.

That said, it does appear after multiple readings of the text around equation #40 that equation #40 is a fit for three parameters. The factor c0 found with this fit is apparently tossed out.

 2. Step 2: parameter refinement for a & b using equation #41 (and definition for c0) isn't intended to get a better fit for equation #35. It is meant to get better estimates of a & b for use in step 3.
Equation #41 is a fit for one parameter, a, not two. Whether the value for b is recomputed from the "frozen" value for beta isn't at all clear. That he said "frozen" does suggest that b does need to be recomputed here.

 3. Step 3: using the refined esimtates for a & b from step 2, obtain estimates for the remaining parameters using equation #43
I agree with that interpretation.

 Re why the author chose the particular optimization method for each step, it is difficult to say. Perhaps the author tried several methods each time and found one that seemed to work better in each case.
Or perhaps he found one that worked better in the one case he had on hand. One thing is certain: This approach wreaks of ad-hocery. Why not fit for all parameters at once? And trisection? Seriously? That is one of the worst optimization techniques around.

This is perhaps a bit disparaging, but it appears that the author knows a limited number of optimization techniques. To overcome the limitations of those techniques he used a lot of ad hoc, ummm, stuff. There's no mention in the paper of the ruggedness of the optimization landscape or of any correlations between his chosen tuning parameters. I suspect there's a lot of nastiness going on such as correlated coefficients and a rugged landscape with a long curving valley. Perhaps another technique would fare better. Simulated annealing or a biologically-motivated technique such as ant search might well attack all of the parameters at once.

One more point: The author obviously had a lot more than five data points (noisy data points at that) at hand. Noisy data does tend to make for a rugged landscape.
 P: 153 Hi, Can I just ask you what you think about the setup of equation 42. Usually with least squares methods you have the actual data - the computed answer, square it, sum it and find the minimum. What part of equation 42 is the actual data part. (apart from theta_f). I am a little conusfed as to how to set the matlab up for this. At the moment I am taking: f = sum(abs(c1.*exp(-2*a*theta_f)+nu*((1+0.5*(4*a^2+1))*cos(theta_f+phi)-2*a*sin(theta_f+phi))+b^2-wf2)) That is the sum of the modulus of everything in the equation. I think this is what i am trying to minimize? I took your advice and decided i would try and solve all the parameters in 42 in one go. I am again using the golden section search. However one question is from the fact that my wf^2 comes out the same as my 'nu' - that is if the upper and lower boundaries start as the same. I just wondered how sensitive this method was to the start values, because changing the start values has a significant effect on the values it calculates
 HW Helper P: 928 I don't know where equation #42 came from but it isn't a least squares problem. It's just some function to be minimized with respect to several variables, meaning the partial derivatives of the function with respect to each variable need to be zero.
 P: 153 Ok so for this one what method would you suggest. I know he does a trisection search (i have never used trisection, D H said it wasnt good, so I attempted using golden section search). However I thjink I am running into what he is talking about. He said he solves for phi between 0 and 2pi using trisection then solved equation 43 using other methods ( i think by hand?). My values for nu and omega^2 always come out the same using this method, so I think this is why he did it his way, However if i want to get all parameters at once what method would you recommend. Can i use the downhill simplex for this, and just set it up as the modulus and the sum rather than squaring it? D H mentioned an ant search which i have been reading about, however I dont think I understand what the algorithm is doing. A nice robust method like that though did sound appealing. His data he uses for equation 42 is in the attached graph. its pretty dirty... x axis is T0 (initial time) y axis is theta_f (fall angle) Attached Thumbnails

 Related Discussions Calculus & Beyond Homework 16 Quantum Physics 14 Beyond the Standard Model 7 Beyond the Standard Model 8