Approximation Theory Help: Parameter Estimation & Fitting Curve

[a.mlw.walker]

I suppose that's the ultimate goal. I am applying for a job in finance and have been advised that I have a very good grasp of estimation theory. After hinting the interenet for more complex examples and I came across this years ago so thought that I would try and solve them. My background is mechanical engineering, but usually when i need to fit a curve its to a polynomial not to an equation like in this paper.

I have read a little more and emailed the author and have found out that eqn 41 is used to quantify the goodness of the fit. I.e if a changes much then the fit is not very good. D H mentioned that this technique is an 'ad hoc' techinque. As the original fit from equation 40 is better, I will use other methods to describe the goodness of the fit.

Just looking at equation 40, the mimization method usually is the sum of the difference between real data and theoretical data (squared). however in equation 40 i can't tell which part is the real data part, can you see what i mean?

EDIT: Oh right, eqn 41 is not the sum squared, it is the modulus of the sum, why has the author written this differently?

 

[D H]

1. Step 1: equation #40 is intended to obtain initial estimates of all three parameters (not two) based on equation #35. Subsequent discussion refers to further refinement steps and has nothing to do with equations #35 and #40.

One problem with this: The c₀ in equation #35 is a function of b and c₁, the latter of which is given by equation #25 or #26.

That said, it does appear after multiple readings of the text around equation #40 that equation #40 is a fit for three parameters. The factor c₀ found with this fit is apparently tossed out.

2. Step 2: parameter refinement for a & b using equation #41 (and definition for c₀) isn't intended to get a better fit for equation #35. It is meant to get better estimates of a & b for use in step 3.

Equation #41 is a fit for one parameter, a, not two. Whether the value for b is recomputed from the "frozen" value for beta isn't at all clear. That he said "frozen" does suggest that b does need to be recomputed here.

3. Step 3: using the refined esimtates for a & b from step 2, obtain estimates for the remaining parameters using equation #43

I agree with that interpretation.

Re why the author chose the particular optimization method for each step, it is difficult to say. Perhaps the author tried several methods each time and found one that seemed to work better in each case.

Or perhaps he found one that worked better in the one case he had on hand. One thing is certain: This approach wreaks of ad-hocery. Why not fit for all parameters at once? And trisection? Seriously? That is one of the worst optimization techniques around.

This is perhaps a bit disparaging, but it appears that the author knows a limited number of optimization techniques. To overcome the limitations of those techniques he used a lot of ad hoc, ummm, stuff. There's no mention in the paper of the ruggedness of the optimization landscape or of any correlations between his chosen tuning parameters. I suspect there's a lot of nastiness going on such as correlated coefficients and a rugged landscape with a long curving valley. Perhaps another technique would fare better. Simulated annealing or a biologically-motivated technique such as ant search might well attack all of the parameters at once.

One more point: The author obviously had a lot more than five data points (noisy data points at that) at hand. Noisy data does tend to make for a rugged landscape.

 

[a.mlw.walker]

Hi, Can I just ask you what you think about the setup of equation 42. Usually with least squares methods you have the actual data - the computed answer, square it, sum it and find the minimum. What part of equation 42 is the actual data part. (apart from theta_f). I am a little conusfed as to how to set the MATLAB up for this.
At the moment I am taking:
f = sum(abs(c1.*exp(-2*a*theta_f)+nu*((1+0.5*(4*a^2+1))*cos(theta_f+phi)-2*a*sin(theta_f+phi))+b^2-wf2))

That is the sum of the modulus of everything in the equation. I think this is what i am trying to minimize?
I took your advice and decided i would try and solve all the parameters in 42 in one go. I am again using the golden section search. However one question is from the fact that my wf^2 comes out the same as my 'nu' - that is if the upper and lower boundaries start as the same. I just wondered how sensitive this method was to the start values, because changing the start values has a significant effect on the values it calculates

 

[hotvette]

I don't know where equation #42 came from but it isn't a least squares problem. It's just some function to be minimized with respect to several variables, meaning the partial derivatives of the function with respect to each variable need to be zero.

 

[a.mlw.walker]

Ok so for this one what method would you suggest. I know he does a trisection search (i have never used trisection, D H said it wasnt good, so I attempted using golden section search).
However I thjink I am running into what he is talking about. He said he solves for phi between 0 and 2pi using trisection then solved equation 43 using other methods ( i think by hand?). My values for nu and omega^2 always come out the same using this method, so I think this is why he did it his way, However if i want to get all parameters at once what method would you recommend. Can i use the downhill simplex for this, and just set it up as the modulus and the sum rather than squaring it?
D H mentioned an ant search which i have been reading about, however I don't think I understand what the algorithm is doing. A nice robust method like that though did sound appealing.

His data he uses for equation 42 is in the attached graph. its pretty dirty...
x axis is T0 (initial time) y-axis is theta_f (fall angle)