# C, C++, or Python for solving Diff. Eq./other math stuff

1. Jul 22, 2015

### PhotonSSBM

I have a friend who wrote a script for doing Runge Kutta and I didn't even know you could do that in Python. He's kind of poor at explaining stuff so I'm here to ask how one learns to do that. I'm a Physics/Comp. Sci. student who's still self teaching myself a lot of stuff. I've only taken an introductory course in Computer Science with Java and dabbled a little bit in Python. I'm taking my second level of Java using Gaddis' book in a few weeks.

In your opinions am I even at a level where I can do stuff like that? Even if not I'd still like to bust my tail trying to get there so I'm not so far behind, so any advice on where to start would be appreciated. Is C, C++, Python better for it?

2. Jul 22, 2015

### Dr. Courtney

You can go far on Numerical Recipes in C.

3. Jul 22, 2015

### rcgldr

in the case of Runge Kutta, the language isn't going to make much difference. C or C++ would essentially be the same (if you wanted to save values, C++ would give you the option of using vectors instead of arrays). I don't know much about Python. Here is a link to wiki article for Runge Kutta (RK4 is the most common version used). Typically the 'h' used in the wiki article is a time step, Δt.

http://en.wikipedia.org/wiki/Runge–Kutta_methods

4. Jul 22, 2015

### JorisL

The hard part is understanding the algorithm. Luckily for runge kutta this is quite simple.
I don't know any standard text (we had a Dutch book) that handles numerical methods.
But for the example of Runge-Kutta methods you can get there in 2 weeks if you care about mathematical technicalities. (if your maths foundation is sufficiently deep)

5. Jul 22, 2015

### PhotonSSBM

Haha, I see you on PF quite a bit regarding this sort of thing. What resources do some of your students use to learn this stuff on their own?

Edit: Realized JorisL answered my question in previous post.

6. Jul 22, 2015

### FactChecker

An old reference of algorithms in FORTRAN that is extremely well documented is the IBM Scientific Subroutine Package (see http://media.ibm1130.org/1130-106-ocr.pdf )
But it assumes you have a lot of knowledge of the algorithms. Implementing an algorithm is any language can be a problem since all the details have to be right. FORTRAN has a different order of matrix indices and a different smallest index than C, C++, or Python , so that is a potential source of errors. Many languages have scientific packages with the algorithms already implemented. Python is very popular now for scientific calculations as long as speed is not important. C and C++ programs execute much faster.

7. Jul 23, 2015

### fluidistic

8. Jul 23, 2015

### rcgldr

In case you need to solve a second order differential equation, such as acceleration as a function of velocity and/or position, like y" = -y (acceleration as a function of position, such as sin() or cos()).

Code (Text):

equation for acceleration a = f(v, p)

t[i] = current time
p[i] = current position
v[i] = current velocity
a[i] = current acceleration = f(v[i], p[i])

p1 = p[i]
v1 = v[i]
a1 = f(v1, p1)

p2 = p[i] + 1/2 Δt v1
v2 = v[i] + 1/2 Δt a1
a2 = f(v2, p2)

p3 = p[i] + 1/2 Δt v2
v3 = v[i] + 1/2 Δt a2
a3 = f(v3, p3)

p4 = p[i] + Δt v3
v4 = v[i] + Δt a3
a4 = f(v4, p4)

t[i+1] = t[i] + Δt
p[i+1] = p[i] + 1/6 Δt (v1 + 2 v2 + 2 v3 + v4)
v[i+1] = v[i] + 1/6 Δt (a1 + 2 a2 + 2 a3 + a4)
a[i+1] = f(v[i+1], p[i+1])
i      = i + 1

If acceleration is also affected by time, a = f(v,p,t)
Code (Text):

a1 = f(v1, p1, t[i])
a2 = f(v2, p2, t[i] + 1/2 Δt)
a3 = f(v3, p3, t[i] + 1/2 Δt)
a4 = f(v4, p4, t[i] +     Δt)

Last edited: Jul 23, 2015
9. Jul 23, 2015

### PhotonSSBM

You guys are attaching too much value to the specific example of Runge Kutta. I'm trying to gain broad skills in programming Numerical solutions myself, not looking for code written by others. So if you guys have any resources you'd suggest, i.e. books, open courses, etc. Then let me know.

10. Jul 23, 2015

### FactChecker

A classic series of books is Knuth "The Art of Computer Programming". Even if you do not study them, you should be aware of what they are like. The volumes are: "Fundamental Algorithms", "Seminumerical Algorithms", "Sorting and Searching", and "Combinatorial Algorithms". They are very well known texts.

PS. I just checked Amazon. A boxed set of 4 classic technical books for under \$200???!!! Unbelievable! I might buy a second set just to get the 4'th volume and a new edition!

Last edited: Jul 23, 2015
11. Jul 23, 2015

### PhotonSSBM

What would you consider the background necessary to digest these books?

12. Jul 23, 2015

### FactChecker

I don't think that the prerequisites are too severe because he starts with basics. But Knuth is certainly college level and would take a lot of serious study. That is why I recommend that you look at them and see (before you buy). If your interest is in general numerical methods, there are better books. Conte and De Boor, "Elementary Numerical Analysis: An Algorithmic Approach" is a good college level book with FORTRAN code for all the algorithms.

13. Jul 23, 2015

### PhotonSSBM

Will the FORTAN algorithms translate well into more efficient languages (pardon the words "more efficient" if that's ignorant of something. Still new-ish to programming.)

14. Jul 23, 2015

### DrZoidberg

There are also lots of lecture videos you could watch.
e.g.
UC Berkeley - CS61A:
Teaches Python and basic programming skills, algorithms, etc.
This course alone is already sufficient to let you solve many numerical problems in python.

CS61B:
Java and more algorithms

CS61C:
C and Assembler. This course teaches you low level programming.
How programs are executed by the processor, how numbers are represented in binary form, etc.
Useful if you want to optimize your programs to run as fast as possible.

MIT 6.006:
Just algorithms. This course expects that you already know how to program.

15. Jul 24, 2015

### FactChecker

FORTRAN, C, and C++ are all about the same in execution speed. But FORTRAN is not as popular now as C and C++, so I end up translating it just so I can use C compilers, which are everywhere. One tricky problem is that array indices are different. I don't know if there are word-for-word C translations of the book programs, but it would be nice if they were already available.

PS. FORTRAN has some nice features for engineering programming that are not in other languages (like namelist read/write) But I don't see it used much any more.

16. Jul 24, 2015

### rcgldr

Most of those numerical solution books or classes are going to include the algorithms, which may include code examples or descriptions of algorithms with so much detail that you end up just translating the algorithm into code. There are other mathematical fields like error correcting codes (most based on a form of finite field math), that can take up an entire text book on their own.

For a PC, the C / C++ compilers do a good job of optimizing. I don't know about how good the Fortran compilers are for PC, but for mainframes and super computers, certain implemenations of Fortran are highly optimized to take advantage of those machines. Then again, there are extensions to C / C++ compilers like CUDA, OpenCL, ... , that implement parallel operations using graphics processing units in video cards, and the SSE instructions in CPU's, which some compilers take advantage of in their library routines.

Last edited: Jul 24, 2015
17. Jul 25, 2015

### JorisL

I'm quite certain the same holds for C compilers. Either the administrators do it or the company installing the basis system (they always tend to expand in my limited experience) have some optimized compiler.

I know for a fact that for some parallel computing paradigms the processor can and is optimized for the machines it sits on.

18. Jul 26, 2015

### FactChecker

There have been benchmark tests of several languages on several types of calculations. C, C++, and FORTRAN are all in the same category of speed. They are all compiled and significantly faster than interpreted or script languages like Java, MATLAB, Python, Perl, R, etc. I was surprised that there were any noticeable differences between FORTRAN, C, and C++, but there were. FORTRAN would be faster on some and C would be faster on others. If I remember correctly, C++ tended to be a little slower. But FORTRAN, C, and C++ were all roughly the same.

19. Jul 26, 2015

### maka89

You can use Java to do it to start out, and many algorithms you can find on Wikipedia or in Numerical Recipes(They also have a Java edition). Would start doing this as this will allow you to focus more on the algorithms than on the programming language you don't know yet ^^ If you want to solve diff. equations, the Euler method, backward Euler method and Runge-Kutta methods are probably a good place to start(Please correct me if im wrong). Just use google and wikipedia and get cracking :)

Python(SciPy stack) is very good for scientific computing(In fact, I use this for the most part), but in Python it would probably not be wise to make these math functions you are talking about yourself. Rather one would use built-in functions, that are faster.

I would recommend learning C,C++ or FORTRAN(They're all basically the same) in the long run.

Last edited: Jul 26, 2015
20. Jul 26, 2015

### JorisL

That's to be expected, interpreting is just another layer of the program (that you don't see).

It is to be expected that C++ is slower, with OOP capabilities also comes some extra overhead. If done right however, it gets a lot simpler.

This is my view at this stuff. It might be wrong as I'm just a simple physicist.