Recent content by hotvette

The appendix has a section called Background Material that defines ##\nabla^2 f(x)## as the Hessian. It doesn't contain the word Laplacian. I had a chat with a friend (statistics professor) and he told me he encounters this sort of thing (different uses for the same symbol) all the time.

I found multiple references that use the Laplacian to define the Hessian matrix like what I found in the Optimization book: https://www.mit.edu/~gfarina/2024/67220s24_L12_newton/L12.pdf https://www.geeksforgeeks.org/multivariate-optimization-gradient-and-hessian/#...

I've been studying the book "Numerical Optimization" by Jorge Nocedal and Stephan J. Wright published by Springer, 1999. I'm puzzled by the use of the Laplacian operator ##\nabla^2## in chapter 10 on nonlinear least squares and in the appendix to define the Hessian matrix. The following is from...

Thanks, I'll chew on it.

I think I found a solution to the dilemma. A caution was posted earlier not to confuse variables and values, but the point didn't really hit me until now. I was using ##\beta_k## as a variable but ##x_i## as a specific value of ##x##. I think what resolves the inconsistent use of terminology...

Thanks for the discussion. It was very helpful.

Does the following work? \begin{equation*} L = \sum_{i=1}^m \left[ f_i - y_i + \Delta x_i \cdot \frac{\partial f}{\partial x} (x_i; \vec{\beta}) + \sum_{k=1}^n \Delta \beta_k \cdot \frac{\partial f}{\partial \beta_k} (x_i; \vec{\beta}) \right]^2 \end{equation*} with the clarification that...

I decided to simplify and go back to fundamentals as a way to try to sort this out. Below is the result. Start with the following definition of truncated Taylor Series: $$\begin{equation*} f(x) \approx f(x_0) + \frac{d f(x_0)}{dx} (x-x_0) \end{equation*}$$ then the following should be valid...

I don't think so. The dilemma really shows up when L is expanded, where we let ##r_i = (f_i - y_i)##: $$\begin{align*} L &= \left[ r_1 + \frac{\partial f_1}{\partial x} \Delta x_1 + \sum_{k=1}^n \frac{\partial f_1}{\partial \beta_k} \Delta \beta_k \right]^2 + \left[ r_2 + \frac{\partial...

Thanks for you comment (and catching the error on the sum index). If I wanted to make the expression for L more compact, it would be: $$\begin{equation*} L = \sum_{i=1}^m \left[ f_i - y_i + \frac{\partial f_i}{\partial x_i} \Delta x_i + \sum_{k=1}^n \frac{\partial f_i}{\partial \beta_k} \Delta...

I have another dilemma with terminology that is puzzling and would appreciate some advice. Consider the following truncated Taylor Series: $$\begin{equation*} f(\vec{z}_{k+1}) \approx f(\vec{z}_k) + \frac{\partial f(\vec{z}_k)}{\partial x} \Delta x + \frac{\partial f(\vec{z}_k)}{\partial...

Thanks for the suggestion!

Wow, does this bring back a memory. My high school had no calculus (in 1972), so physics was non-calculus based. When I got to college I put off taking calculus until second semester, but took a calculus based physics class (by mistake) my first semester. I was totally lost. I went back to my...

I write math related tutorials and would appreciate comments/advice about terminology that may be potentially ambiguous or confusing. As an example, I use ##f(x;\beta)## to define a generic continuous function with independent variable ##x## and function parameters ##\beta = (\beta_1, \beta_2...

Thanks!