Couple of days ago I downloaded a book on numerical optimization, hoping to get clearer picture on some techniques.(adsbygoogle = window.adsbygoogle || []).push({});

But, I'm surprised that some of the concepts were not precisely separated from one another.

Namely, in the part of "coordinate descent methods" (cdm), I found that, in the case the contribution of each point

to the function in question is known, one applied the Gauss-Southwell approach to select the point to be updated.

In case the update is done is a sequence from 1 to n, the the approach is referred to as the "cyclic descent method" (cydm).

However, on some other places I've found that Gauss-Seidl method's principle is close to cydm's.

Is Gauss-Seidl actually another name for cydm? I hope someone will clarify this for me.

Another question is regarding the Jacobi method, which is a "hold on" version of Gauss-Seidl, in my understanding

(regarding the point update at the end of iteration). Given certain function f(x) which is quadratic in x,

in order to obtain its minimal, I set the derivative to 0, and obtain the following linear system

Ax=b

which I decide to solve by Jacobi iteration. Given that A is strictly diagonally dominant, is it true that the value of

f(x1), where x1 is obtained by a single Jacobi iteration on some arbitrary x0 (but not the stationary point) satisfies

f(x1)<f(x0)?

**Physics Forums - The Fusion of Science and Community**

# Gauss-Seidl, Gauss-Southwell, Jacobi...

Know someone interested in this topic? Share a link to this question via email,
Google+,
Twitter, or
Facebook

Have something to add?

- Similar discussions for: Gauss-Seidl, Gauss-Southwell, Jacobi...

Loading...

**Physics Forums - The Fusion of Science and Community**