The algorithms involved in finding a solution with numerical methods

In summary, there are several algorithms involved in finding a solution with numerical methods, including numerical differentiation, numerical integration, solution of algebraic and transcendental equations, numerical solution of a system of linear equations, and numerical solution of ordinary differential equations. Other techniques such as curve fitting, interpolation, and solving problems associated with partial differential equations also use numerical methods. These methods often start with an initial guess to generate successive approximations to the solution. While numerical solutions may not capture the full complexity of the real solution, they can still provide a qualitative view and be improved through techniques like interpolation.
  • #1
awholenumber
200
10
what are all the algorithms involved in finding a solution with numerical methods ?

an example

say we were to solve the initial value problem:

Y′ = 2x

y(0) = 0

it's so simple, you could find a formulaic solution in your head, namely y = x2. On the other hand, say we were to use a numerical technique. (yes, i know we don't know how to do this yet, but go with me on this for a second!) the resulting numerical solution would simply be a table of values. To get a better feel for the nature of these two types of solution, let's compare them side by side, along with the graphs we would get based on what we know about each one:

Notice that the graph derived from the formulaic solution is smoothly continuous, consisting of an infinite number of points on the interval shown. On the other hand, the graph based on the numerical solution consists of just a bare eight points, since the numerical method used apparently only found the value of the solution for x-increments of size 0.2.Using numerical solutions

so what good is the numerical solution if it leaves out so much of the real answer? Well, we can respond to that question in several ways:

The numerical solution still looks like it is capturing the general trend of the "real" solution, as we can see when we look at the side-by-side graphs. This means that if we are seeking a qualitative view of the solution, we can still get it from the numerical solution, to some extent.

The numerical solution could even be "improved" by playing "join-the-dots" with the set of points it produces. In fact this is exactly what some solver packages, such as mathematica, do do with these solutions. (mathematica produces a join-the-dots function that it calls interpolatingfunction.)

when actually using the solutions to differential equations, we often aren't so much concerned about the nature of the solution at all possible points. Think about it! Even when we are able to get formulaic solutions, a typical use we make of the formula is to substitute values of the independent variable into the formula in order to find the values of the solution at specific points. Did you hear that? Let me say it again: To find the values of the solution at specific points. This is exactly what we can still do with a numerical solution

numerical methods and errors
interpolation
numerical differentiation
numerical integration
solution of algebraic and transcendental equations
numerical solution of a system of linear equations
numerical solution of ordinary differential equations
curve fitting
numerical solution of problems associated with partial differential equations

fixed point iteration method
bisection and regula false methods
Newton raphson method etc.
Finite differences operators
numerical interpolation
Newton’s and lagrangian formulae
part i
Newton’s and lagrangian formulae
part ii
interpolation by iteration
numerical differentiaton
numerical integration
solution of system of linear
equations
solution by iterations
eigen values
taylor series method
picard’s iteration method
euler methods
runge – kutta methods
predictor and corrector methods
 
Last edited by a moderator:
Physics news on Phys.org
  • #2
rosekidcute said:
what are all the algorithms involved in finding a solution with numerical methods ?
??
Well, you've listed quite a bunch of them already. Do you expect someone in PF to complete your list? Or is it in fact domething else you want to know ?
 
  • #3
BvU ,

thanks for the reply ...

i am so confused right now...

A numerical iteration method or simply iteration method is a mathematical
procedure that generates a sequence of improving approximate solutions for a class of
problems. A specific way of implementation of an iteration method, including the
termination criteria, is called an algorithm of the iteration method. In the problems of
finding the solution of an equation an iteration method uses an initial guess to generate
successive approximations to the solution.

what is this initial guess ??
the initial guess of what ??
 
  • #4
A numerical integration has to start with some value (the initial value, e.g. y(0) = 0) and can then proceed. Good for initial value problems.

When you solve a boundary problem (e.g. y(10) = 100 as a given) using numerical integration, you still have to start somewhere, so you 'guess' the initial value and see where you end up with the integration. (Look up 'shooting method')
 
  • #5
these numerical type questions are a bit hard usually ...
i really thought first we had to try to solve the equation in an ,analytic , symbolic or formulaic way ...before trying the numerical way ...

so it means, when i encounter a question like this .. i have to take an initial value ...


an example

say we were to solve the initial value problem:

Y′ = 2x

y(0) = 0

it's so simple, you could find a formulaic solution in your head, namely y = x2. On the other hand, say we were to use a numerical technique. (yes, i know we don't know how to do this yet, but go with me on this for a second!) the resulting numerical solution would simply be a table of values. To get a better feel for the nature of these two types of solution, let's compare them side by side, along with the graphs we would get based on what we know about each one:

Notice that the graph derived from the formulaic solution is smoothly continuous, consisting of an infinite number of points on the interval shown. On the other hand, the graph based on the numerical solution consists of just a bare eight points, since the numerical method used apparently only found the value of the solution for x-increments of size 0.2.Using numerical solutions

so what good is the numerical solution if it leaves out so much of the real answer? Well, we can respond to that question in several ways:

The numerical solution still looks like it is capturing the general trend of the "real" solution, as we can see when we look at the side-by-side graphs. This means that if we are seeking a qualitative view of the solution, we can still get it from the numerical solution, to some extent.

The numerical solution could even be "improved" by playing "join-the-dots" with the set of points it produces. In fact this is exactly what some solver packages, such as mathematica, do do with these solutions. (mathematica produces a join-the-dots function that it calls interpolatingfunction.)

when actually using the solutions to differential equations, we often aren't so much concerned about the nature of the solution at all possible points. Think about it! Even when we are able to get formulaic solutions, a typical use we make of the formula is to substitute values of the independent variable into the formula in order to find the values of the solution at specific points. Did you hear that? Let me say it again: To find the values of the solution at specific points. This is exactly what we can still do with a numerical solution





then i have to use the algorithm , depending on the type of the question ...this is surely going to take some time to get used to ... because i am not that familiar with many of the terms ...
 
Last edited by a moderator:
  • #6
I see two two-foot long posts and I don't see a difference. Nor a question I can answer ...
 
  • #7
i am sorry for the long posts ... as i learn more , the size of the post seems to be getting smaller ...

i really need to understand how these works ...
lagrange's interpolation formula
Newton raphson method
bisection method
eulers method
Newtons forward
runge kutta method
trapezoidal ruleand i don't have one text that explains it all ...
 
  • #8
Here ?
And I see a lot of your goodies http://ins.sjtu.edu.cn/people/mtang/textbook.pdf as well ! Should keep you off the streets for a while ... :smile:
 
  • #9
BvU ,

thanks a lot for that book .. there is a lot to learn from that book ...
The book contains sufficient material for at least a full year of study

i really need to follow at least this much amount of books to really understand all these ...

Peter Selby, Steve Slavin Practical Algebra_ A Self-Teaching Guide
Tom M. Apostol Calculus, Volume I_ One-Variable Calculus, with an Introduction to Linear Algebra
Tom M. Apostol Calculus, Volume II_ Multi-Variable Calculus and Linear Algebra, with Applications to Differential Equations and Probability
Numerical Analysis
by Richard L. Burden (Author), J. Douglas Faires (Author), Annette M. Burden
Computer oriented numerical methods - N Datta
Computer Fundamentals and Programming in C - J.B. Dixit

but before i delve a bit deeper into the books alone ...

i have two more doubts ...

one is ...

how to find a proper initial guess ...?

Nonlinear optimization routines enable you to find the values of variables that optimize an objective function of those variables. When you use a numerical optimization routine, you need to provide an initial guess, often called a "starting point" for the algorithm. Optimization routines iteratively improve the initial guess in an attempt to converge to an optimal solution. Consequently, the choice of a starting point determines how quickly the algorithm converges to a solution and—for functions with multiple local extrema—to which optimum the algorithm converges.

But how can you produce a "good" starting point for multivariate functions? Some analysts use a constant vector, such as a vector of all 0s or a vector of all 1s. Although this practice is widespread, this default initial guess might not converge to a global extremum. You can usually do better by choosing a starting point that is based on a few easy evaluations of the objective function.

second is ...

i understand the steps involved here ...

ccc_numerical_methods_png_2.png


is this the same thing we did here ??

ccc_numerical_methods_png_3.png

ccc_numerical_methods_png_1.png


ccc_numerical_methods_1.png

ccc_numerical_methods_2.png

first we are given a gradient of the curve or an instantaneous rate of change
then we are trying to find a function that is giving this gradient of the curve or this instantaneous rate of change

then we change the dy/dx , the gradient of the curve or an instantaneous rate of change to ... delta y /delta x
which is an even smaller gradient of the curve or a smaller instantaneous rate of change ...

we start with x = 0 , we get y =0when we increment x by 0.1 , we get the value of y around -0.299

which is another gradient of the curve for delta y / delta x , the smaller instantaneous rate of change


why are we adding up these gradients of the curve or the smaller instantaneous rate of change for these small small change in delta y / delta x?

this means that these smaller gradients of the curve or these smaller instantaneous rate of change has certain property at each value of x ..., the gradient of the curve with respect to x which is indeed "the rate" at which y is changing as x is changing ...

so the function has the property of this "rate" ?

usually , when we try to solve ...

we get a function that has this instantaneous rate of change ...

but when we try to solve it with numerical methods , we only get a " rate " which is the property of that function ?
 
  • #10
Again, several feet of post.
rosekidcute said:
how to find a proper initial guess ...?
Good thing you already prepared an answer. General answer: use all your knowledge, intuition and luck. As if you were a TV detective. Stay critical.

rosekidcute said:
second is ...
Numerically, we use the derivative (which changes from point to point) to approximate the function as linear. Which is good up to order (step size)2. After a step, we re-evaluate the derivative. We make errors of a certain order, but still get an estimate of the function values.

Start reading the book :smile:
 
  • #11
BvU said:
I see two two-foot long posts and I don't see a difference. Nor a question I can answer ...
I deleted the duplicated images
 
  • #12
BvU said:
Start reading the book :smile:
This is advice that has been given in previous threads to you (rosiekidcute). Have you obtained a book, yet?
 
Last edited:
  • #13
BvU ,

thanks ...

Mark44 ,

yes , i have this book ... numerical analysis and i am reading it ...
 
  • #14
rosekidcute said:
yes , i have this book ... numerical analysis and i am reading it ...
It would make more sense to start with a textbook that deals with ordinary differential equations.

You have recently started threads with these titles:
  • Few basic questions about differential equations...
  • A few differential equation questions
  • differential equations and numerical methods questions
  • first order differential equations ??
  • few more numerical methods question...?
Any good differential equations textbook would provide answers for all of the questions you've asked here.
 
Last edited:
  • #15
Mark44,

thanks ...

can you recommend some books on ordinary differential equations ??

i have few more doubts from the previous posts ...
i don't know if i am understanding this properly ...
this thread , post # 9 ...
picture two ..

that question is about finding the "function" that has that instantaneous rate of change ... right ??

we found the function ... that has that rate of change ...

same thread
picture one ,

we are given a function and we are trying to solve it numerically ...

if you take dy/dx of that instantaneous rate of change
you get certain properties of that function at certain points or places ...
then if you increment x ... you get properties of that function again at certain points or places

if you add all the function values ... you get the overall properties of a function ...
and that is the function with all those properties that has given that instantaneous rate of change ...??
 

1. What are numerical methods?

Numerical methods are mathematical techniques used to solve problems that cannot be solved analytically. They involve using algorithms to approximate solutions by breaking down complex problems into smaller, more manageable ones.

2. What is the purpose of using numerical methods?

Numerical methods are used when it is not possible to find an exact solution to a problem. They are especially useful in solving problems that involve large amounts of data or complex equations.

3. How do numerical methods work?

Numerical methods work by breaking down a problem into smaller pieces and then using algorithms to approximate a solution. These algorithms use iterative processes to refine the solution until it reaches a desired level of accuracy.

4. What are some common numerical methods?

Some common numerical methods include the bisection method, Newton's method, and the Gauss-Seidel method. Other popular methods include the Euler method, Runge-Kutta method, and the Monte Carlo method.

5. What are the benefits of using numerical methods?

Numerical methods allow for the solution of complex problems that would be impossible to solve analytically. They also provide a more efficient way to solve problems that involve large amounts of data, and can be used to simulate real-world scenarios and predict outcomes.

Similar threads

Replies
7
Views
2K
  • Calculus and Beyond Homework Help
Replies
4
Views
649
  • Differential Equations
Replies
7
Views
2K
Replies
2
Views
4K
Replies
16
Views
1K
  • Differential Equations
Replies
9
Views
2K
  • Special and General Relativity
Replies
5
Views
868
  • Differential Equations
Replies
2
Views
2K
Replies
2
Views
269
  • Differential Equations
Replies
1
Views
2K
Back
Top