Question about using Newton's Method to solve a system of equations

  • Context: Undergrad 
  • Thread starter Thread starter Ax_xiom
  • Start date Start date
Click For Summary

SUMMARY

The discussion clarifies the computational approach used in Newton's Method for solving systems of nonlinear equations, specifically the step involving the Jacobian matrix \( J(X_k) \). Instead of explicitly computing the inverse \( J^{-1}(X_k) \), the method solves the linear system \( J(X_k) \Delta X_k = -F(X_k) \) using numerical linear algebra techniques such as Gaussian elimination or LU decomposition. This avoids the inefficiency of matrix inversion. The process involves expressing the problem as a system of linear equations and solving it directly, which is computationally more efficient and numerically stable.

PREREQUISITES

  • Newton-Raphson method for nonlinear systems
  • Jacobian matrix and its role in nonlinear equation solving
  • Gaussian elimination and LU decomposition for solving linear systems
  • Matrix inversion concepts and computational complexity

NEXT STEPS

  • Study LU decomposition algorithms for efficient linear system solving
  • Explore numerical stability considerations in solving \( J(X_k) \Delta X_k = -F(X_k) \)
  • Learn implementation of Newton's Method in numerical computing libraries (e.g., SciPy, MATLAB)
  • Investigate sparse matrix techniques for large Jacobian matrices

USEFUL FOR

Numerical analysts, computational scientists, applied mathematicians, and software engineers implementing or optimizing Newton's Method for solving nonlinear systems will benefit from understanding the computational strategies discussed here.

Ax_xiom
Messages
65
Reaction score
4
So the formula used to solve non-linear equations using the Newton-Raphson method is this $$ X_{k+1} = X_k - J^{-1}(X_k)F(X_k) $$ and instead of finding ##J^{-1}(X_k)##, we solve for the change that will be applied to ##X_k## (##\Delta X_k##) using this relation $$J(X_k) \Delta X_k = -F(X_k)$$ and the next iteration will be this $$ X_{k+1} = X_k + \Delta X_k $$ My question is that wouldn't you need to solve for ##J^{-1}(X_k)## anyways when doing that? If so, why do we do that step in the first place?
 
Physics news on Phys.org
We don't solve linear systems numerically by finding an inverse matrix; that is incredibly inefficient. Instead we use other methods, like Gaussian elimination or LU decomposition.
 
pasmith said:
We don't solve linear systems numerically by finding an inverse matrix; that is incredibly inefficient. Instead we use other methods, like Gaussian elimination or LU decomposition.
So what happens comptationally during the ##J(X_k) \Delta X_k = -F(X_k)## step? Do you express the problem as a series of linear equations and solve them?

Edit: I think that might be the case. I was under the impression that using Gaussian manipulation to solve a system of equations would give you the inverse matrix for free, but I don't think that is the case
 
Last edited:
If ##A## is invertible, then solving ##Ax = b## by Gaussian elimination where ##b## consists of zeros except for a 1 in the ##i##th row will give you the ##i##th column of ##A^{-1}##.
 
pasmith said:
If ##A## is invertible, then solving ##Ax = b## by Gaussian elimination where ##b## consists of zeros except for a 1 in the ##i##th row will give you the ##i##th column of ##A^{-1}##.
I'm assuming that this is computationally inefficient and it's much faster to just solve directly.

This is good to know but I attempted to implement this in Excel, and I was only doing it with a 3x3 matrix so it was much easier to just use Excel's MINVERSE() and MMULT() functions
 

Similar threads

  • · Replies 21 ·
Replies
21
Views
4K
  • · Replies 7 ·
Replies
7
Views
3K
Replies
4
Views
3K
  • · Replies 12 ·
Replies
12
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 10 ·
Replies
10
Views
2K