Interested in optimization for scientific computing, where to start?

[rem1618]

I've just started programming with Python this summer, and I'm taking a course in computational physics this semester. I've been really enjoying it and programming in general, but I don't have much knowledge in computer science save for intro stuff (string/list methods, functions/classes, precision digits, a bit of dynamic programming), and I'm interested in learning more. Basically, I don't even know what I don't know. For example, what should I look into in order to understand something like this?

(top answer)
http://stackoverflow.com/questions/...-slower-than-builtin-in-arithmetic-operations

 

[zivo]

You should know that there are different programming languages (bummer!).

A computer doesn't directly understand those strings of commands you're giving him, it's not smart enough, and for that you need another program(s) that translates those strings into something the computer understands, what we call machine code. For some languages you only need one program or two to get from human code to machine code, we call these languages "low-level languages", because they're closer to the string of 1's and 0's. For others you need several jumps, and we call them "high-level languages". As a result of those jumps, high-level languages tend to produce code that's easier to understand/write while being slower overall, while low-level languages tend to be the opposite.

Where does Python come? Python is a high-level language, and most of the under-the-hood stuff is hidden from the coder unless you use specific libraries. Those stuff are taken care-of for you by a translating program that's called an "interpreter", and you won't need to understand all those "blocks" and "bytes" stuff to code in Python at all!

However, while many Python coders don't even try to understand those, I think everyone should, as sometimes the interpreter (which gives you the errors) gives you weird low-level unintelligible errors, and you need to understand them to use the Python scientific libraries (numpy,scipy...etc) to their fullest.Which is why I recommend you learn a low-level language, and I recommend you learn C specifically. (fun fact: Python is made using C! And that stackoverflow question is about CPython, a Python library for calling pseudo-C code into Python) You shouldn't really "learn" it, you're not supposed to write everything you do in Python in C, just learn how things work. Learning C is by far the best way to learn how low-level stuff works, and it will enable you to write faster code in Numpy once you understand how computer memory works.

Also you might want to properly learn a low-level language for scientific computing, since you've probably noticed that Python is slow, while C is great, it's not at all user-friendly and it lacks scientific libraries as it's intended for use for CS specialists. And here I can recommend C++ (C with object oriented stuff), Fortran (I hate it, but it's incredible as it's intended for scientists) or Julia ( a "high-level" language with low-level performance, a bit biased here since I develop libraries in it!).