C, C++, or Python for solving Diff. Eq./other math stuff

[PhotonSSBM]

I have a friend who wrote a script for doing Runge Kutta and I didn't even know you could do that in Python. He's kind of poor at explaining stuff so I'm here to ask how one learns to do that. I'm a Physics/Comp. Sci. student who's still self teaching myself a lot of stuff. I've only taken an introductory course in Computer Science with Java and dabbled a little bit in Python. I'm taking my second level of Java using Gaddis' book in a few weeks.

In your opinions am I even at a level where I can do stuff like that? Even if not I'd still like to bust my tail trying to get there so I'm not so far behind, so any advice on where to start would be appreciated. Is C, C++, Python better for it?

 

[Dr. Courtney]

You can go far on Numerical Recipes in C.

 

[rcgldr]

in the case of Runge Kutta, the language isn't going to make much difference. C or C++ would essentially be the same (if you wanted to save values, C++ would give you the option of using vectors instead of arrays). I don't know much about Python. Here is a link to wiki article for Runge Kutta (RK4 is the most common version used). Typically the 'h' used in the wiki article is a time step, Δt.

http://en.wikipedia.org/wiki/Runge–Kutta_methods

 

[JorisL]

I have a friend who wrote a script for doing Runge Kutta and I didn't even know you could do that in Python. He's kind of poor at explaining stuff so I'm here to ask how one learns to do that. I'm a Physics/Comp. Sci. student who's still self teaching myself a lot of stuff. I've only taken an introductory course in Computer Science with Java and dabbled a little bit in Python. I'm taking my second level of Java using Gaddis' book in a few weeks.

In your opinions am I even at a level where I can do stuff like that? Even if not I'd still like to bust my tail trying to get there so I'm not so far behind, so any advice on where to start would be appreciated. Is C, C++, Python better for it?

The hard part is understanding the algorithm. Luckily for runge kutta this is quite simple.
I don't know any standard text (we had a Dutch book) that handles numerical methods.
But for the example of Runge-Kutta methods you can get there in 2 weeks if you care about mathematical technicalities. (if your maths foundation is sufficiently deep)

 

[PhotonSSBM]

You can go far on Numerical Recipes in C.

Haha, I see you on PF quite a bit regarding this sort of thing. What resources do some of your students use to learn this stuff on their own?

Edit: Realized JorisL answered my question in previous post.

 

[FactChecker]

An old reference of algorithms in FORTRAN that is extremely well documented is the IBM Scientific Subroutine Package (see http://media.ibm1130.org/1130-106-ocr.pdf )
But it assumes you have a lot of knowledge of the algorithms. Implementing an algorithm is any language can be a problem since all the details have to be right. FORTRAN has a different order of matrix indices and a different smallest index than C, C++, or Python , so that is a potential source of errors. Many languages have scientific packages with the algorithms already implemented. Python is very popular now for scientific calculations as long as speed is not important. C and C++ programs execute much faster.

 

[fluidistic]

Here's the implemented Runge-Kutta algorithm in many different languages. See by yourself which one you prefer or understand the most: rosettacode.org/wiki/Runge-Kutta_method.

 

[rcgldr]

In case you need to solve a second order differential equation, such as acceleration as a function of velocity and/or position, like y" = -y (acceleration as a function of position, such as sin() or cos()).

equation for acceleration a = f(v, p)

t[i] = current time
p[i] = current position
v[i] = current velocity
a[i] = current acceleration = f(v[i], p[i])

p1 = p[i]
v1 = v[i]
a1 = f(v1, p1)

p2 = p[i] + 1/2 Δt v1
v2 = v[i] + 1/2 Δt a1
a2 = f(v2, p2)

p3 = p[i] + 1/2 Δt v2
v3 = v[i] + 1/2 Δt a2
a3 = f(v3, p3)

p4 = p[i] + Δt v3
v4 = v[i] + Δt a3
a4 = f(v4, p4)

t[i+1] = t[i] + Δt
p[i+1] = p[i] + 1/6 Δt (v1 + 2 v2 + 2 v3 + v4)
v[i+1] = v[i] + 1/6 Δt (a1 + 2 a2 + 2 a3 + a4)
a[i+1] = f(v[i+1], p[i+1])
i      = i + 1

If acceleration is also affected by time, a = f(v,p,t)

a1 = f(v1, p1, t[i])
a2 = f(v2, p2, t[i] + 1/2 Δt)
a3 = f(v3, p3, t[i] + 1/2 Δt)
a4 = f(v4, p4, t[i] +     Δt)

 

[PhotonSSBM]

You guys are attaching too much value to the specific example of Runge Kutta. I'm trying to gain broad skills in programming Numerical solutions myself, not looking for code written by others. So if you guys have any resources you'd suggest, i.e. books, open courses, etc. Then let me know.

 

[FactChecker]

A classic series of books is Knuth "The Art of Computer Programming". Even if you do not study them, you should be aware of what they are like. The volumes are: "Fundamental Algorithms", "Seminumerical Algorithms", "Sorting and Searching", and "Combinatorial Algorithms". They are very well known texts.

PS. I just checked Amazon. A boxed set of 4 classic technical books for under $200?! Unbelievable! I might buy a second set just to get the 4'th volume and a new edition!

 

[PhotonSSBM]

A classic series of books is Knuth "The Art of Computer Programming". Even if you do not study them, you should be aware of what they are like. The volumes are: "Fundamental Algorithms", "Seminumerical Algorithms", "Sorting and Searching", and "Combinatorial Algorithms". They are very well known texts.

PS. I just checked Amazon. A boxed set of 4 classic technical books for under $200?! Unbelievable! I might buy a second set just to get the 4'th volume and a new edition!

What would you consider the background necessary to digest these books?

 

[FactChecker]

What would you consider the background necessary to digest these books?

I don't think that the prerequisites are too severe because he starts with basics. But Knuth is certainly college level and would take a lot of serious study. That is why I recommend that you look at them and see (before you buy). If your interest is in general numerical methods, there are better books. Conte and De Boor, "Elementary Numerical Analysis: An Algorithmic Approach" is a good college level book with FORTRAN code for all the algorithms.

 

[PhotonSSBM]

I don't think that the prerequisites are too severe because he starts with basics. But Knuth is certainly college level and would take a lot of serious study. That is why I recommend that you look at them and see (before you buy). If your interest is in general numerical methods, there are better books. Conte and De Boor, "Elementary Numerical Analysis: An Algorithmic Approach" is a good college level book with FORTRAN code for all the algorithms.

Will the FORTAN algorithms translate well into more efficient languages (pardon the words "more efficient" if that's ignorant of something. Still new-ish to programming.)

 

[DrZoidberg]

There are also lots of lecture videos you could watch.
e.g.
UC Berkeley - CS61A:
https://www.youtube.com/playlist?list=PLjjZcDhFjV4vDRancNJlRrM6Tqe6efF5k
Teaches Python and basic programming skills, algorithms, etc.
This course alone is already sufficient to let you solve many numerical problems in python.

CS61B:
https://www.youtube.com/playlist?list=PL-XXv-cvA_iAlnI-BQr9hjqADPBtujFJd
Java and more algorithms

CS61C:
https://www.youtube.com/playlist?list=PL-XXv-cvA_iDHtKXLFJbDG-i6L9oDr5X9
C and Assembler. This course teaches you low level programming.
How programs are executed by the processor, how numbers are represented in binary form, etc.
Useful if you want to optimize your programs to run as fast as possible.

MIT 6.006:
https://www.youtube.com/playlist?list=PLUl4u3cNGP61Oq3tWYp6V_F-5jb5L2iHb
Just algorithms. This course expects that you already know how to program.

 

[FactChecker]

Will the FORTAN algorithms translate well into more efficient languages (pardon the words "more efficient" if that's ignorant of something. Still new-ish to programming.)

FORTRAN, C, and C++ are all about the same in execution speed. But FORTRAN is not as popular now as C and C++, so I end up translating it just so I can use C compilers, which are everywhere. One tricky problem is that array indices are different. I don't know if there are word-for-word C translations of the book programs, but it would be nice if they were already available.

PS. FORTRAN has some nice features for engineering programming that are not in other languages (like namelist read/write) But I don't see it used much any more.

 

[rcgldr]

You guys are attaching too much value to the specific example of Runge Kutta. I'm trying to gain broad skills in programming Numerical solutions myself, not looking for code written by others. So if you guys have any resources you'd suggest, i.e. books, open courses, etc. Then let me know.

Most of those numerical solution books or classes are going to include the algorithms, which may include code examples or descriptions of algorithms with so much detail that you end up just translating the algorithm into code. There are other mathematical fields like error correcting codes (most based on a form of finite field math), that can take up an entire textbook on their own.

Will the FORTAN algorithms translate well into more efficient languages?

For a PC, the C / C++ compilers do a good job of optimizing. I don't know about how good the Fortran compilers are for PC, but for mainframes and super computers, certain implemenations of Fortran are highly optimized to take advantage of those machines. Then again, there are extensions to C / C++ compilers like CUDA, OpenCL, ... , that implement parallel operations using graphics processing units in video cards, and the SSE instructions in CPU's, which some compilers take advantage of in their library routines.

 

[JorisL]

For a PC, the C / C++ compilers do a good job of optimizing. I don't know about how good the Fortran compilers are for PC, but for mainframes and super computers, certain implemenations of Fortran are highly optimized to take advantage of those machines.

I'm quite certain the same holds for C compilers. Either the administrators do it or the company installing the basis system (they always tend to expand in my limited experience) have some optimized compiler.

I know for a fact that for some parallel computing paradigms the processor can and is optimized for the machines it sits on.

 

[FactChecker]

I'm quite certain the same holds for C compilers. Either the administrators do it or the company installing the basis system (they always tend to expand in my limited experience) have some optimized compiler.

I know for a fact that for some parallel computing paradigms the processor can and is optimized for the machines it sits on.

What about this part?

There have been benchmark tests of several languages on several types of calculations. C, C++, and FORTRAN are all in the same category of speed. They are all compiled and significantly faster than interpreted or script languages like Java, MATLAB, Python, Perl, R, etc. I was surprised that there were any noticeable differences between FORTRAN, C, and C++, but there were. FORTRAN would be faster on some and C would be faster on others. If I remember correctly, C++ tended to be a little slower. But FORTRAN, C, and C++ were all roughly the same.

 

[maka89]

You can use Java to do it to start out, and many algorithms you can find on Wikipedia or in Numerical Recipes(They also have a Java edition). Would start doing this as this will allow you to focus more on the algorithms than on the programming language you don't know yet ^^ If you want to solve diff. equations, the Euler method, backward Euler method and Runge-Kutta methods are probably a good place to start(Please correct me if I am wrong). Just use google and wikipedia and get cracking :)

Python(SciPy stack) is very good for scientific computing(In fact, I use this for the most part), but in Python it would probably not be wise to make these math functions you are talking about yourself. Rather one would use built-in functions, that are faster.

I would recommend learning C,C++ or FORTRAN(They're all basically the same) in the long run.

 

[JorisL]

There have been benchmark tests of several languages on several types of calculations. C, C++, and FORTRAN are all in the same category of speed. They are all compiled and significantly faster than interpreted or script languages like Java, MATLAB, Python, Perl, R, etc.

That's to be expected, interpreting is just another layer of the program (that you don't see).

I was surprised that there were any noticeable differences between FORTRAN, C, and C++, but there were. FORTRAN would be faster on some and C would be faster on others. If I remember correctly, C++ tended to be a little slower. But FORTRAN, C, and C++ were all roughly the same.

It is to be expected that C++ is slower, with OOP capabilities also comes some extra overhead. If done right however, it gets a lot simpler.

This is my view at this stuff. It might be wrong as I'm just a simple physicist.

 

[rcgldr]

I was surprised that there were any noticeable differences between FORTRAN, C, and C++.

Depends on the machine and the effort put into a compiler. I recall that older super computers like the Cray series focused on FORTRAN because that was the most common language for the programs used on that machine. In addition, machine specific extensions were sometimes added to FORTRAN for various super computers and mainframes. On the PC side, C / C++ is more popular and now there are extensions like CUDA, and OpenCL, for GPU based processing, and MMX / SSE intrinsics for X86 / X64 type processors.