Computational Physics can be frustrating sometimes

[franznietzsche]

Computational Physics can be frustrating sometimes...

Just realized that my past eleven weeks of work has been entirely invalidated due to single error in my simulation inputs. Arrrg.

On the upside, I think I've got the process down now, so its just a matter of rerunning everything. Its going to be a long week.

Moral of the story: Even if you have twenty different input files, all generated automatically by another program that you wrote on the fly to make life a little easier, check them all by hand. I forgot to add 1 to a single number in a single file. Bah!

Ok, I feel a little better now.

 

[Math Is Hard]

Math is hard, franz.

 

[arildno]

Just realized that my past eleven weeks of work has been entirely invalidated due to single error in my simulation inputs. Arrrg.

On the upside, I think I've got the process down now, so its just a matter of rerunning everything. Its going to be a long week.

Moral of the story: Even if you have twenty different input files, all generated automatically by another program that you wrote on the fly to make life a little easier, check them all by hand. I forgot to add 1 to a single number in a single file. Bah!

Ok, I feel a little better now.

Hooray for the joys of professional calculations!
If there is any comfort in it, reflect on how many billions of dollars other people than you have wasted on the same type of mistakes that you made.

 

[PerennialII]

... yeah, and then when get the ".":s about right can usually start working with "eternal" convergence problems .

 

[Kurdt]

I have a similar problem but I'm tempted to just publish the groundwork and let others worry about it and I can move the program to be a side project. Of course doesn't help now I have no access to the nag library and have to copy numerical recipes etc.

I sympathise man.

 

[franznietzsche]

... yeah, and then when get the ".":s about right can usually start working with "eternal" convergence problems .

Nothing like quite like watching your hydro code go to infinite density and negative infinite internal energy in 0.01 seconds.

Actually this problem was minor enough that we didn't catch it for eleven weeks. I'd run well over 200 cases without noticing. It only affects the results by a few percent, but unfortunately, that's the level we're looking at, so its a problem. Gah.

On the upside, we have two papers going for this work, and only one of them is affected. Which is good, because the other has a deadline for sunday. So much stuff to do.

 

[Clausius2]

Nothing like quite like watching your hydro code go to infinite density and negative infinite internal energy in 0.01 seconds.
.

What are you computing?. I'm just curious.

 

[franznietzsche]

What are you computing?. I'm just curious.

Actually that hydro code is a different project, not the one I'm working on now. Right now I'm working on asteroseismology, trying to model stellar oscillations. Well, more specifically, trying to figure out what problems exist in our current methods of modelling stellar oscillations.

 

[Kalimaa23]

Poor guy. Try theoretical physics. The same problems, one small error being able to invalidate months of work, only you do not know wether or not your correct result has anything to do with the real world or not...

 

[Gokul43201]

Just realized that my past eleven weeks of work has been entirely invalidated due to single error in my simulation inputs. Arrrg.

On the upside, I think I've got the process down now, so its just a matter of rerunning everything. Its going to be a long week.

Moral of the story: Even if you have twenty different input files, all generated automatically by another program that you wrote on the fly to make life a little easier, check them all by hand. I forgot to add 1 to a single number in a single file. Bah!

So, you had to go back and "add one". Fret not, boy. It's happened to the best of them.

You've probably heard the Eddington story involving the fine structure constant. Goes something like this:

'Twas the time when early measurements of alpha had it at about 1/136 and people were pondering its philosophical value to the far ends of their navels. Arthur Eddington, true to his style, came up with a system of16 equations in a whole lot of "universal parameters" that supposedly determined how the universe worked. The result of this was that the number of independent "universal constants" turned out to be n(n+1)/2, where n was the number of equations; for 16 equations this number is 136. So Eddington concluded that alpha revealed the secret about the number of universal constants which, in the long run, determine EVERYTHING.

Subsequent measurements showed that alpha was actually about 1/137. Unfazed, Eddington coolly revised his result saying he'd forgot to include alpha itself as one of the universal constants, so the total should really be 136+1.

Soon after, some clever newswriter poked fun at Eddie with an article titled: Sir Arthur Adding One!

PS: There's other versions of this story (or maybe corrollaries to it) where Eddington claimed something about the universe having 10^136 electrons and protons or somesuch.

 

[franznietzsche]

So, you had to go back and "add one". Fret not, boy. It's happened to the best of them.

You've probably heard the Eddington story involving the fine structure constant. Goes something like this:

'Twas the time when early measurements of alpha had it at about 1/136 and people were pondering its philosophical value to the far ends of their navels. Arthur Eddington, true to his style, came up with a system of16 equations in a whole lot of "universal parameters" that supposedly determined how the universe worked. The result of this was that the number of independent "universal constants" turned out to be n(n+1)/2, where n was the number of equations; for 16 equations this number is 136. So Eddington concluded that alpha revealed the secret about the number of universal constants which, in the long run, determine EVERYTHING.

Subsequent measurements showed that alpha was actually about 1/137. Unfazed, Eddington coolly revised his result saying he'd forgot to include alpha itself as one of the universal constants, so the total should really be 136+1.

Soon after, some clever newswriter poked fun at Eddie with an article titled: Sir Arthur Adding One!

PS: There's other versions of this story (or maybe corrollaries to it) where Eddington claimed something about the universe having 10^136 electrons and protons or somesuch.

The number in question was actually the number of lines in a certain input file that the section of code needed to read. I didn't count the line the number was on.

 

[Astronuc]

I was working on a calculation last week, where I had to do a quick calculation of energy per unit mass. I ran some numbers and my calculation was off by 13.4%. I thought that almost one year's work was invalid. A final report had been distributed, and we were about to initiate new work based on the past year's work.

I got a really sick feeling in my stomach thinking that I was going to have to send out a 'nonconformance'. It was one of those big things that can torpedo a career and endanger a company's future.

I had to walk away from the problem for a while.

I came back and looked at it - then I saw it - I had forgotten to convert the mass basis from UO2 to U - M(UO2) ~ 270, and M(U)~238 - the ratio 1.1344 . . .

The revised calculations showed that the previos work was OK.

Oh - and I've done the same kind of programming error when counting lines. Those are nasty buggers to catch sometimes.

I always like having someone else look at my work or I have to walk away and not think about. I also test programs with hand calcs to see if the results make sense. Developing a gut feeling for what looks right takes experience. Franznietsche - you're lucky that you're learning this very early.

 

[franznietzsche]

I was working on a calculation last week, where I had to do a quick calculation of energy per unit mass. I ran some numbers and my calculation was off by 13.4%. I thought that almost one year's work was invalid. A final report had been distributed, and we were about to initiate new work based on the past year's work.

I got a really sick feeling in my stomach thinking that I was going to have to send out a 'nonconformance'. It was one of those big things that can torpedo a career and endanger a company's future.

I had to walk away from the problem for a while.

I came back and looked at it - then I saw it - I had forgotten to convert the mass basis from UO2 to U - M(UO2) ~ 270, and M(U)~238 - the ratio 1.1344 . . .

The revised calculations showed that the previos work was OK.

Oh - and I've done the same kind of programming error when counting lines. Those are nasty buggers to catch sometimes.

Neither me nor my mentor caught it in the frequency predictions we were getting. They all looked reasonable, and indeed the difference even in the worst cases is only about 3-5% (for some predicted frequencies there was no change), which is about the difference between observational data and the calculations for this particular star. So it wasn't enough to set off any red flags just based on the outputs.

We noticed it when we saw that the core of the star in the pulsation models was not even remotely similar structurally to the core from the evolution models, which are the inputs to the pulsation model codes. That was a big red flag.

I always like having someone else look at my work or I have to walk away and not think about. I also test programs with hand calcs to see if the results make sense. Developing a gut feeling for what looks right takes experience. Franznietsche - you're lucky that you're learning this very early.

Hand calcs for these would be...tough. Doable, but definitely not pleasant.

Of course, now I'm having zoning problems with one of the pulsation codes and I'm getting frequency outputs that are a little wacky. Bah, this is going to take me all week to puzzle out, at least. On the upside, the paper due sunday is pretty much done, one more round of editing before we send it off for review .

 

[Cyrus]

On the upside, the paper due sunday is pretty much done, one more round of editing before we send it off for review

Nice.

-text unlimited-

 

[Astronuc]

Several years ago, I was doing some calculations with a code that had been developed by a then prominent company under a federally mandated QA/QC program, and it had been licensed by federal regulators.

Well I found an error. It was a pretty significant error (code-wise), that dozens of reviewers had not seen. Most calculations using the code probably had insignficant errors, but I was doing a rather uncommon problem, when I noticed some temperatures just didn't make sense. There was a cold region surrounded by a hot region, and that was physically impossible.

So I dug out the source code, and I readily found the error. It was an iteration loop. The loop was supposed to start at the bottom of the column and go upward. Instead, the logic, which was based on a downward march (from a previous by similar code), just stayed at the bottom. So the temperatures, which were supposed to calculated at the top, were being taken from the bottom. I many cases, the temperature differences were slight, but in some rather unusual and uncommon cases, there were fairly big differences.

Fortunately, the code was being phased out after 15+ years. A lot of people were upset and somewhat embarrassed. If the code had stayed in use - all h*** would have broken loose.

 

[neurocomp2003]

franz: do you use other libraries in your code(if so which)...or is everything yours?

 

[franznietzsche]

Several years ago, I was doing some calculations with a code that had been developed by a then prominent company under a federally mandated QA/QC program, and it had been licensed by federal regulators.

Well I found an error. It was a pretty significant error (code-wise), that dozens of reviewers had not seen. Most calculations using the code probably had insignficant errors, but I was doing a rather uncommon problem, when I noticed some temperatures just didn't make sense. There was a cold region surrounded by a hot region, and that was physically impossible.

So I dug out the source code, and I readily found the error. It was an iteration loop. The loop was supposed to start at the bottom of the column and go upward. Instead, the logic, which was based on a downward march (from a previous by similar code), just stayed at the bottom. So the temperatures, which were supposed to calculated at the top, were being taken from the bottom. I many cases, the temperature differences were slight, but in some rather unusual and uncommon cases, there were fairly big differences.

Fortunately, the code was being phased out after 15+ years. A lot of people were upset and somewhat embarrassed. If the code had stayed in use - all h*** would have broken loose.

V&V (Verification and validation) is a huge thing here at the lab. Its how we justify our astrophysics projects to the DOE basically.

The codes I'm using are descendendants of codes originally written in the early seventies, that have been continuously updated with new physics modules being added and replaced to reflect new understandings of important processes. Unfortunately, the style and programming practices used in the are anything but coherent or even logical at times. GOTO statements occur every fifteen lines on average (out of 20,000 lines of code). I've tried to puzzle out some of the routines, but most of it I can't follow. Some of the more recently written stuff I can (like the OPAL stuff, or the newer reaction rate segments) but the main driver is beyond me. There are no loops--its all GOTO statements. If there's a major bug in there, there's no way I could find it. On the upside, every other group that has worked in this subfield with their independent codes gets the same results as our codes (when including the same input physics).

They're used for work in asteroseismology (what I'm doing) but also solar neutrino work. For the sun, they used to be accurate to less than 0.5% until the observational abundances were changed. Now its about 1.5%. But matching observations of larger stars has never been better than a few(5%+) percent. The bit I'm working on is essentially trying to figure out what physics we're missing. It boils down to figuring out how to fake 3-D hydro processes in a 1-D stellar evolution code where the timesteps are on the order of 1e8 years, but the dynamical timescales are much shorter--thousands of years for large scale convection, and from minutes to years for local effects which are the ones I'm interested in for the moment.

 

[franznietzsche]

franz: do you use other libraries in your code(if so which)...or is everything yours?

The only library used by these codes is the snbir library, used by the nonradial pulsation codes to solve the pulsation equation (its essentially the schrodinger equation in 3-D).

edit: Correction, that's the slatec library, the snbir subroutine.

Well, and the OPAL subroutines, I suppose, but those aren't really a library.