Bad programming skills = biggest hurdle to astronomy research

  • Thread starter Simfish
  • Start date

chiro

Science Advisor
4,783
127
I'm going to side with two-fish here, especially with regard to working on the bigger projects with dozens and dozens of people.

Working on the bigger projects is where you get a lot of experience in a lot of things. Everything from large scale project design to optimization to effective integration of multiple code bases (think libraries or amalgamation of smaller repositories) is where people need to see the forest from the trees and have a depth that is a synonym for experience.

Also a lot of programming that is taught can be way too theoretical. If you're designing a GUI widget, you have to get your hands dirty and not be stuck in some analysis paralysis where you are overanalyzing the design, structure and so on.

Like twofish said with the writing, to get good at writing you have to write: you can only theorize so much before you have to physically do something to learn.
 

Chronos

Science Advisor
Gold Member
11,398
738
Waiting for data to process is not labor intensive. Waiting for physicists to gather good data is the labor intensive part. Data gathering algorithms are extremely important in this process. Researchers are not blind to this issue and grad students with excellent programming skills are not rare. This may have been issue 30 years ago, but, not now.
 

D H

Staff Emeritus
Science Advisor
Insights Author
15,329
681
And personally I think that's a good thing, since CS classes are often terrible for teaching application programming.
You essentially are talking about the difference between software engineering and computer science. And yes, computer science classes past the introductory CS classes are for the most part terrible about teaching software engineering concepts. The introductory CS classes teach some very basic concepts common to computer science, software engineering, scientific programing, and even IT. Some schools require there students to take at least an introductory CS class, some don't (my undergrad school still does).

What's not surprising is the number of astronomy Ph.D.'s that are terrible programmers, what is more surprising is the number of CS Ph.D.'s that are terrible programmers.
Not all that surprising. I've learned the hard way that resumes from CS and IT grads with no education in engineering or the sciences are best filed circularly.


The big problem with those courses is that they generally don't give you experience in working on hundred-person project teams with millions of source lines of code.
That problem does not pertain just to computer science classes. Failing to teach how to work collaboratively is in my opinion a shortcoming of a lot of science programs. Engineering curricula on the other hand offer lots of opportunities for undergrads to work in team projects, with a lot of the project lead activities performed by a grad student. Participation in such projects, and having some kind of lead role in particular, is something I look for in evaluating prospects.
 
6,814
10
What about applied math courses?
The problem is not with course content, but course format. The way that courses are structured just doesn't lend itself to teach "real world" programming. Among the differences

1) real world problems are invariably team graded. Your "grade" depends a lot on how competent the person next to you is. So is it unfair that you get a bad "grade" because the person next to you is incompetent. It may be unfair, but it's real, and one skill is to figure out how to deal with that.

2) real world problems tend to be vague and ill-defined. Classes you get a well defined assignment. Much of the work in real work programming involves figuring out what you need to do. When you do get a set of marching orders, more often than not, those orders are either contradictory or flat out impossible, and dealing with that is part of real world programming.

3) class assignments are short and throw-away. in the real world, you have to work with a pre-existing system, and you never are in a situation in which you have to start from scratch. Also sometimes you have to deal with something that is badly written.

This means that you tend to have emotional reactions to good/bad code. In a class they teach you rules, but there is no emotional connection to through rules. In real world software development, you see bad code, and you react with horror since you know that you'll be spending the next three weeks going through ten thousand lines of code and fixing things.

You can get a lot more experience if you work on an open source project. Also I wasn't kidding when I said that it helps you if you take a course on writing poetry. Poetry classes are usually set up so that you write something and then you go to a room where everyone else in the class tells you how you can improve it. That's usually the dynamics of code reviews.
 

D H

Staff Emeritus
Science Advisor
Insights Author
15,329
681
The problem is not with course content, but course format. The way that courses are structured just doesn't lend itself to teach "real world" programming.
You're looking at the wrong courses. The sciences really should take a look at engineering education. Cube sats, autonomous vehicles, robots, ... The students work as a part of a team on what is often a multi-year project. That might mean dropping a course on some advanced concept, but that's what grad school is for. Besides, the stereotype of a scientist being someone who works on his own with only a blackboard for company is for the most part fifty years out of date. Most scientists, like most engineers, work in large teams nowadays.

in the real world, you have to work with a pre-existing system, and you never are in a situation in which you have to start from scratch.
Have to start from scratch? Get to start from scratch is more like it. There is nothing like being able to start on something from scratch. No CMMI 3 stuff, very few constraints other than getting the job started. Some other saps have to deal with making that initial design real. Getting that kind of opportunity doesn't happen very often.
Also sometimes you have to deal with something that is badly written.
Or a poor design by the people who don't want to deal with all that CMMI 3 nonsense.
 
104
0
This seems like a good thread to ask this question but what languages would you guys say are the 'best' for writing astrophysics tyoe simulations (i.e. 3+ body problem simulations).

I began with Maple before I had any intentions to program these types of problems and have moved onto Matlab as maple isn't the best for high end computing. I have a feeling C++ would perhaps be a good shout but since I have a course on Matlab this year I'll be mainly working with that. I also find that the more I work with languages like Matlab the more I seem to understand C++ code despite never having worked with it.

I read somewhere that you should look at programming less in terms of learning a language and more in terms of learning the basics and fundementals of programming such as OOP.
 
2
0
This is a website I found a while back, its a great comparison of programming languages and their speed.

http://shootout.alioth.debian.org/fastest-programming-language.php [Broken]

I think you're right in choosing C++ as a language, as you can see, C is minimally faster, but slightly more annoying to code at times so C++ is a happy medium. Speaking of happy medium, I tend to fall into that category when it comes to this topic, my schooling (while not finished) was for Astro-Engineering, but I now hold a job as Linux Sysadmin and have done a rather extensive amount of programming. I think what someone mentioned before its like we have to learn all over again, we've gotten so used to fast computers and not caring about efficiency, but when you are dealing with this large of numbers, it is like going back 30 years with computing power. Unfortunately, I agree that it's a vicious cycle of astronomers not being programmers and vice versa, it makes it hard to program efficiently when you don't know what you're programming, and it takes years of practice to know how to program efficiently.

while(astronomer == programmer)
{
printf(&efficient_program);
}

EDIT: Coming from a background in a variety of languages, I agree with what you said about learning the basics of programming over the language, you'll learn quickly most languages are very similar and you can adapt to them quickly.
 
Last edited by a moderator:

Simfish

Gold Member
814
2
Wow, very interesting link! Why is FOTRAN Intel so slow? My professors always told me to use either C/C++ or FORTRAN since they were the fastest.

FORTRAN is nice for MPI/OpenMP integration too (if you want to parallelize things). I'm sure C also has MPI/OpenMP integration, but I'm not sure if it's as fluid.
 

D H

Staff Emeritus
Science Advisor
Insights Author
15,329
681
This is a website I found a while back, its a great comparison of programming languages and their speed.
Eh. The question raised by Simfish immediately arises on seeing stuff like that:
Wow, very interesting link! Why is FOTRAN Intel so slow? My professors always told me to use either C/C++ or FORTRAN since they were the fastest.
The answer is (at least) twofold:
  1. They are combining metrics from multiple programs, some of which are not Fortran's forte. Who cares how well Fortran performs in handling strings?
  2. The programs appear to be not so well-written. Some of the so-called benchmarks are so poorly written that they don't even compile. This is a failure of the benchmarking, not of the language.
The tests in which Fortran fares very poorly are in part a reflection of the limitations of Fortran (yes, Fortran isn't so good at handling strings), but also appear to be in part due to giving the programming assignment to someone not well-versed in Fortran.

Note well: I am not a Fortran advocate. Far from it; I gladly abandoned the language a couple of decades ago. One doesn't have to be a Fortran advocate to say that those benchmarks are more than a bit suspect. That said, Fortran does not always look so bad at debian.org. In the n-body problem Fortran is the winner: http://shootout.alioth.debian.org/u32q/performance.php?test=nbody [Broken].
 
Last edited by a moderator:

Related Threads for: Bad programming skills = biggest hurdle to astronomy research

Replies
16
Views
2K
  • Posted
Replies
4
Views
3K
Replies
1
Views
4K
Replies
5
Views
4K
Replies
3
Views
7K
Replies
24
Views
5K
Replies
6
Views
6K
Replies
1
Views
352

Physics Forums Values

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving
Top