Regret your PhD? Would do it all over again?

[twofu]

I'm just an undergrad who has to make a big decision about grad school pretty soon. While I've been sure about this for a long time, I just want to collect as much information as possible :)

Title asks the relevant questions :)

Thanks

 

[ZapperZ]

I'm just an undergrad who has to make a big decision about grad school pretty soon. While I've been sure about this for a long time, I just want to collect as much information as possible :)

Title asks the relevant questions :)

Thanks

Title doesn't give all the relevant scenario. For example, aren't you at least interested to see how this breaks down into different field? You can't compare someone who did a PhD in, say, English, with someone who did a PhD in Electrical Engineering. Totally different experience, and totally different job prospects.

I've already posted the latest statistics regarding physics PhDs in the last post on "So You Want To Be A Physicist" thread. That statistics include such a question for Physics PhDs.

Zz.

 

[twofu]

Title doesn't give all the relevant scenario. For example, aren't you at least interested to see how this breaks down into different field? You can't compare someone who did a PhD in, say, English, with someone who did a PhD in Electrical Engineering. Totally different experience, and totally different job prospects.

I've already posted the latest statistics regarding physics PhDs in the last post on "So You Want To Be A Physicist" thread. That statistics include such a question for Physics PhDs.

Zz.

I wanted anyone who's done a PhD to give their experiences and thoughts on it. Personal experience, not just statistics. And It didn't really come up in my mind that an English PhD would be on a science forum. I assumed the general population that post would be engineering/science fields, which are all related.

 

[G01]

I wanted anyone who's done a PhD to give their experiences and thoughts on it. Personal experience, not just statistics. And It didn't really come up in my mind that an English PhD would be on a science forum. I assumed the general population that post would be engineering/science fields, which are all related.

Zz's point still stands. The can be large differences in experiences and job prospects even among different subfields of physics.

Working in a spectroscopy lab with ultrafast lasers is a different experience than doing high energy theory. Job prospects and opportunities after graduation are very different as well.

If you want an answer that actually helps you, you are going to need to be more specific about what it is you are thinking of studying.

 

[ZapperZ]

I wanted anyone who's done a PhD to give their experiences and thoughts on it. Personal experience, not just statistics. And It didn't really come up in my mind that an English PhD would be on a science forum. I assumed the general population that post would be engineering/science fields, which are all related.

If that's all you care about, then that's fine. But if it gives a MISLEADING impression one way or the other, then it is a disservice! Anecdotes, if they are not taken as that, will tend to skewer one's conclusion of something.

And your dismissal of the statistics is highly disturbing. One would think that you'd want to be aware of that as a baseline. I'm sure there's a valid reason for you asking such a question. I would hate to think that you are drawing up some conclusion on something based simply on anecdotes without knowing what a more general statistics on the identical topic would be.

Zz.

 

[twofu]

Thanks Zz. I've looked at all the statistics, from APS to forums. I realize a lot of these responses will be anecdotes and will not give accurate information. I'm asking this question just to hear different perspectives about people who spent so much time in school. Statistics doesn't tell you how it feels to spend 10 years in school and can't find a job. Also, in general I just want to know what I should expect. All undergrads I know at my university say, "I'm going to go to grad school, do a few postdocs, and be a professor." Some say "I'm going to go to Wall street."

These are inexperienced students, so they can't tell me real experiences. The reason I'm not stating what field I want to hear from is because I, myself, don't know what I'm going to go into. I just know for sure its going to be physics related. And no, I'm not going to chose a field based on these responses.

 

[Jack21222]

I'm just an undergrad who has to make a big decision about grad school pretty soon. While I've been sure about this for a long time, I just want to collect as much information as possible :)

Title asks the relevant questions :)

Thanks

You're going into your junior year, right? Just taking your first "intermediate" level physics classes? What big decision do you have to make, and how soon?

 

[Pyrrhus]

Perhaps, you should ask yourself, Why do you want to do a PhD? Higher Salary? Prestige?.

 

[QuantumCandy]

Try not to overthink it - just make sure that you take all the steps necessary to allow yourself the option to apply for a PhD. The only 'decision' you have to make right now is whether to apply (or prepare to apply) or not; presumably by continuing with your studies now and taking GRE or appropriate tests.

Then, if/when you're accepted to programs, you will make another decision then. I don't think asking others if they regret getting a PhD is a good measure for anything- maybe we can offer you advice in terms of what we might have done differently or something.

For now just focus on doing everything that will allow you to have options later :) Good luck!

 

[twofu]

You're going into your junior year, right? Just taking your first "intermediate" level physics classes? What big decision do you have to make, and how soon?

Since my parents found out I want to get a PhD in something Physics related, they freaked and told me I would be miserable and struggle getting a job. So 2 years worth of undergrad work now I am reconsidering it...I never thought about jobs that that much, I just knew I like physics; I get good grades and I enjoy studying it.

So now I want to hear about perspectives. This is why statistics won't work for me. So far everyone has been interrogating me on this thread and it was probably a bad idea to post it.

Maybe I should have stated that it isn't one big decision but rather should I start looking for research as an undergrad, or apply to something more secure...etc.

 

[twofu]

Perhaps, you should ask yourself, Why do you want to do a PhD? Higher Salary? Prestige?.

I like physics.

 

[Pyrrhus]

I don't think asking others if they regret getting a PhD is a good measure for anything- maybe we can offer you advice in terms of what we might have done differently or something.

I agree. In my case, I would have gone to Applied Math instead of Engineering for my undergraduate studies.

 

[twofu]

Perhaps I really worded the question wrong. Any advice about a PhD and post-PhD along with experience should have been the title.

I just made the title the way it is to advocate people to really tell me their stories.

 

[Jack21222]

Since my parents found out I want to get a PhD in something Physics related, they freaked and told me I would be miserable and struggle getting a job.

You have the strangest parents I've ever heard of, going just by this bit of information. Employment statistics for physics Ph.D holders are out there, that'd be more use than anecdotes.

 

[deRham]

You have the strangest parents I've ever heard of, going just by this bit of information.

Huh? I think even if it is a misconception that physics PhDs have a hard time with jobs, it's not hard to believe why someone would have that idea (i.e. that you spend 5+ years doing overspecialized work that academia cares about and that may not be ultra in demand outside of it at the moment).

If you want an answer that actually helps you, you are going to need to be more specific about what it is you are thinking of studying.

I don't think the microscopic itsy bitsy details are relevant so much as the nature of the PhD broadly. Was it applied? Was it in a theoretical field that was in demand?

It's more of a question about getting a PhD versus not; that is very different from figuring out academic interests. I doubt this poster is far enough along the way to even know.

Also, asking for opinions doesn't imply that one will make a conclusion then and there. I like hearing people's perspectives, so I can think about how they add to my own, as opposed to make the decision for me.

I agree with Zz that dismissing statistics on a matter like this is incredibly unwise. They are not even that trivial to find (especially if you're looking for meaningful ones), and greatly enhance the perspective of individual posters.

 

[twofu]

Huh? I think even if it is a misconception that physics PhDs have a hard time with jobs, it's not hard to believe why someone would have that idea (i.e. that you spend 5+ years doing overspecialized work that academia cares about and that may not be ultra in demand outside of it at the moment).



I don't think the microscopic itsy bitsy details are relevant so much as the nature of the PhD broadly. Was it applied? Was it in a theoretical field that was in demand?

It's more of a question about getting a PhD versus not; that is very different from figuring out academic interests. I doubt this poster is far enough along the way to even know.

Also, asking for opinions doesn't imply that one will make a conclusion then and there. I like hearing people's perspectives, so I can think about how they add to my own, as opposed to make the decision for me.




I agree with Zz that dismissing statistics on a matter like this is incredibly unwise. They are not even that trivial to find (especially if you're looking for meaningful ones), and greatly enhance the perspective of individual posters.

This post is gold and exactly what my intentions were. Thank you.
I also agreed with Zz, and stated already that I have looked at statistics, I just wanted some perspectives.

 

[twofish-quant]

If that's all you care about, then that's fine. But if it gives a MISLEADING impression one way or the other, then it is a disservice! Anecdotes, if they are not taken as that, will tend to skewer one's conclusion of something.

On other other hand you can get yourself in equally as much trouble if you look at statistics uncritically. One problem with statistics is that you may not care about the "average" person making the decision. For example, in a lot of situation, you really don't care about the "average" person, but you care about the worst case situation.

Also, you have problems with self-selection bias, time series issues, and the way that the question is asked. To get a statistically valid survey, you need to put in a lot of work, and uncritically accepting something that is not statistically valid can be worse than nothing. There's also the problem with making statistics "actionable". Suppose I find out that people with blue hats tend to be two centimeters taller than people with people with red hats. Now what?

My wife has a Ph.D. in education and they have to deal with this stuff in doing studies in their field, and you quickly find out that in some situations, uncritical use of statistics is a very, very, very bad thing. One way of making sure that your statistics make sense is to do another study, when you take some people that you asked statistical questions from and then do deep interviewing. My experiences in finance have left me extremely skeptical of statistical data. It *can* be useful, but you have to look at statistics with every bit of skepticism than you look at non-statistical data. One particularly problem with statistical data that you have to take into account is that things change. For example a survey of Ph.D.'s and careers taken with people that graduated or will graduate in 1975, 1985, 1995, 2005, and 2015 is going to be extremely different, and if you lump everything up into one big pile, you get numbers that don't mean anything.

One particular problem is with physicists is small numbers. There are huge difference between different fields, and also differences between different fields of physics, so by the time you take into account all of the differences you are dealing with extremely small sample sizes. For example, last year you are looking at about 20 or so people that got HEP theory Ph.D.'s, and you really can't do random statistical sampling because the numbers are too small.

 

[twofish-quant]

In my situation, getting my Ph.D. has been the best good thing that has happened to me other than getting married and having kids. It's a core part of my life and who I am.

The weird thing is that my situation seems to be the reverse of the OP. I come from a family in which pretty much everyone goes to graduate school, and getting a Ph.D. is the "normal thing to do" and graduation ceremonies are a lot like weddings and funerals.

One thing that was weird was reading family letters and finding out that I was pretty much expected to get a Ph.D., even before I was born, and then talking to my uncle and finding out that I was expected to get a Ph.D. even before my father was born.

One thing that I find strange is the "standard decision making process" that people seem to go through in deciding whether or not to get a Ph.D. In my family, looking at career statistics in order to decide whether or not to get a Ph.D. is a lot like deciding who to get married with, how many kids to have, and what religion to choose based on how much money you can make.

 

[twofish-quant]

I agree with Zz that dismissing statistics on a matter like this is incredibly unwise. They are not even that trivial to find (especially if you're looking for meaningful ones), and greatly enhance the perspective of individual posters.

On the other hand, one reason I'm extremely skeptical of statistics was that when I was twofu's age I got bombarded with a ton of statistics that purported to show that the United States was vastly underproducing Ph.D.'s and that there would be a massive shortage of Ph.D.'s and faculty jobs when I got my degree.

Those numbers turned out to be totally garbage, and it would have been more obvious that those numbers were garbage if I had been able to ask the questions that twofu is asking. Unfortunately for me, the internet wasn't quite as developed then.

 

[ZapperZ]

On the other hand, one reason I'm extremely skeptical of statistics was that when I was twofu's age I got bombarded with a ton of statistics that purported to show that the United States was vastly underproducing Ph.D.'s and that there would be a massive shortage of Ph.D.'s and faculty jobs when I got my degree.

Those numbers turned out to be totally garbage, and it would have been more obvious that those numbers were garbage if I had been able to ask the questions that twofu is asking. Unfortunately for me, the internet wasn't quite as developed then.

You are confusing the statistics itself versus the INTERPRETATION of the statistics. The latter requires making a number of assumptions that are required to put the statistics into some context!

The statistics asking if a PhD recipient would do a PhD again is "naked data". Trying to decipher what the data means (for example, why is the number lower for international students in all categories) is no longer a statistics, but an interpretation of it! The same with "underproducing" PhDs. It requires that one make an assumption on what actually is a required number!

Do not confuse the two and undermining the statistics themselves.

On other other hand you can get yourself in equally as much trouble if you look at statistics uncritically. One problem with statistics is that you may not care about the "average" person making the decision. For example, in a lot of situation, you really don't care about the "average" person, but you care about the worst case situation.

But then, I can turn this around and tell you that without knowing the statistics, you might only hear about the tail ends of the Normal curve and miss the majority that is a more accurate representation of the situation! For each extreme case, there are way more "average" case. So what is the more accurate reflection of the situation? The extreme case? I don't get it. Norway had one major shooting incident in how many years that made the headlines around the world. So using that alone (the worst case situation), you are ready to make a conclusion on the crime rate in Norway? Is this rational?

Zz.

 

[Jack21222]

Huh? I think even if it is a misconception that physics PhDs have a hard time with jobs, it's not hard to believe why someone would have that idea (i.e. that you spend 5+ years doing overspecialized work that academia cares about and that may not be ultra in demand outside of it at the moment).

I've never "struggled" to get a job, and I only have a high school diploma. The more education you have, the more jobs you're qualified for, and the less you'd struggle to get a job.

People in the work force spend 5+ years doing overspecialized work that nobody outside of their company would care about all the time. Other skills are picked up and can be transferred. I started doing pest control sales soon after high school, and did that for six years. When I switched jobs to sell mattresses, my specialized knowledge of the two species of eastern subterranean termite did not stop this mattress company from hiring me, because the sales skills transfer over.

I feel this would be analogous to what's learned in grad school. Sure, employers might not care about what happens to the thermal conductivity of graphene when placed on a new type of substrate, but they will care about the skills you learned along the way. The ability to solve differential equations, computer modeling, practical lab skills... these are all skills that do transfer.

So, like I said, the OP's parents are out of line to discourage their child from continuing their education.

 

[twofish-quant]

The statistics asking if a PhD recipient would do a PhD again is "naked data".

There's no such thing as naked data.

Do not confuse the two and undermining the statistics themselves.

Personally, I think that all data has to be treated critically, and you end up in a lot of trouble if you take any sort of data without thinking about what it means.

The reason I mention this is that I think that the OP ought to be encouraged in asking the question that he did. If it turns out that the statistics and the anecdotal data match up then you have more confidence in trying to figure out what is going on. If they don't then you dig some more and figure out what is going on.

But then, I can turn this around and tell you that without knowing the statistics, you might only hear about the tail ends of the Normal curve and miss the majority that is a more accurate representation of the situation!

Sure, you might get in trouble if you take anecdotal information and incorrectly assume that the data is representative, but you'll run into equally as much trouble if you assume the same thing about statistics. I'm not saying that you should ignore statistics, but it's *vital* that you cross check the stats with other data. Anecdotal information can get you into selection bias, but you have to be very careful with statistics, because they can be biased in the same way. If you are careful with your interviewing technique, you can often pick up selection bias with interviews more readily than you can with a survey.

The curve is almost never normal. What is often the situation is that you are trying to figure out the shape of the curve. One thing that you find in finance is that it is a bad, bad, bad, bad, bad thing to assume normality since extreme events happen far more frequently than the normal distribution.

Also, it really depends on what you care about. For example, if you are in risk management, you don't care about the 99.999% of days when the market doesn't crash. Most days you go to work, and the market doesn't crash, and you go home happy. What you do care about is that one day in 30 years in which the market does crash. The fact that Lehman Brothers only failed once is pretty irrelevant.

Now which one is more "accurate". It depends what you question are asking.

The extreme case? I don't get it. Norway had one major shooting incident in how many years that made the headlines around the world. So using that alone (the worst case situation), you are ready to make a conclusion on the crime rate in Norway? Is this rational?

Again it depends on what you are trying to figure out. You'll get into just as bad a situation if you assume because most days in Norway there are no major shootings that you shouldn't spend money on SWAT teams and police. Most days there are no hurricanes in New Orleans, but that doesn't mean that you shouldn't care about building dams and levees, or that you should ignore that possibility because it is a rare event. On the average, ships won't collide with icebergs, but you still want to have lifeboats ready.

If you work in risk management, you know financial crashes are rare but *YOU DON'T CARE*. You want to find out as much as you can about historical crashes and bulletproof your bank. To use your Norway example. If you are the head of the special anti-terrorist police, you don't care what the general crime rate is. You care that in the one situation when all hell breaks loose, you can survive.

A lot of this is influenced by my daily work. After the Lehman crash, you had a huge number of people getting hired in risk management and model control, and a lot of this work involves having a government official stare at you and ask, if this extreme, improbable event occurs, can you survive, and what will you do? They want to know that if you hit an iceberg, you have enough life rafts and have done the drills so that you don't take down the world economy (again). Telling them that in 99.99% of the days the world won't crash is irrelevant and will get you fired for incompetence. What you really what to do is to look for people that survived iceberg hits or didn't survive iceberg hits, and find out what they did.

Also relying on statistics can be dangerous. In physics and biology you can assume that the laws of physics and the human body don't radically change, but you can't assume this for economics. One thing that got people in trouble was that everyone looked at the numbers for mortgages in 2005 and they looked good. No one was defaulting. Everyone was happy. The loan statistics looked reasonable. What a lot of people didn't do was to just call people up and see what was going on. Forget about the numbers, who are you loaning to and what are the conditions of the loans. People that did that quickly figured out that massive loans were made out to people that couldn't possibly pay them back, and all of the statistics were being fudged, and we were heading to a blowup. The people that I work with managed to survive that part of the crisis because they didn't just trust the numbers and spent a lot of time questioning the statistics, which is why I'm glad that the OP is doing the same thing.

 

[twofish-quant]

So, like I said, the OP's parents are out of line to discourage their child from continuing their education.

I think that the OP's parents have legitimate worries. If we are in the situation where the OP wants to do something and the problem is that his parents have doubts, he/she can contact me in private, and I'd be glad to chat with them about what I think the situation is.

 

[Astronuc]

I somewhat regret not completing it.

Knowing what I know now, I'd do it all over again, with some changes. Now I know what I needed to know then.

I've done plenty of independent research and contributed to the field of practice, which is afterall the point of having a PhD. One doesn't need a PhD to do that, but having a PhD makes it easier for doors of opportunity to open.

 

[ZapperZ]

There's no such thing as naked data.

There is too.

Personally, I think that all data has to be treated critically, and you end up in a lot of trouble if you take any sort of data without thinking about what it means.

The reason I mention this is that I think that the OP ought to be encouraged in asking the question that he did. If it turns out that the statistics and the anecdotal data match up then you have more confidence in trying to figure out what is going on. If they don't then you dig some more and figure out what is going on.

No one said anything about not looking at data critically. But you seems to have confused interpretation, extrapolation, and the likes with actual data. Stating how many PhDs were awarded in a particular year is different than saying that number is low or under some value, thus resulting in a need for more PhDs. The former is data. The latter is interpretation and extrapolation.

Sure, you might get in trouble if you take anecdotal information and incorrectly assume that the data is representative, but you'll run into equally as much trouble if you assume the same thing about statistics. I'm not saying that you should ignore statistics, but it's *vital* that you cross check the stats with other data. Anecdotal information can get you into selection bias, but you have to be very careful with statistics, because they can be biased in the same way. If you are careful with your interviewing technique, you can often pick up selection bias with interviews more readily than you can with a survey.

Then point out the selection bias of this AIP data.

The curve is almost never normal. What is often the situation is that you are trying to figure out the shape of the curve. One thing that you find in finance is that it is a bad, bad, bad, bad, bad thing to assume normality since extreme events happen far more frequently than the normal distribution.

Also, it really depends on what you care about. For example, if you are in risk management, you don't care about the 99.999% of days when the market doesn't crash. Most days you go to work, and the market doesn't crash, and you go home happy. What you do care about is that one day in 30 years in which the market does crash. The fact that Lehman Brothers only failed once is pretty irrelevant.

Now which one is more "accurate". It depends what you question are asking.

Sorry, but I don't buy that, especially when you are the one pushing for the "extreme" case, which is even WORSE at representing the data.

Again it depends on what you are trying to figure out. You'll get into just as bad a situation if you assume because most days in Norway there are no major shootings that you shouldn't spend money on SWAT teams and police. Most days there are no hurricanes in New Orleans, but that doesn't mean that you shouldn't care about building dams and levees, or that you should ignore that possibility because it is a rare event. On the average, ships won't collide with icebergs, but you still want to have lifeboats ready.

If you work in risk management, you know financial crashes are rare but *YOU DON'T CARE*. You want to find out as much as you can about historical crashes and bulletproof your bank. To use your Norway example. If you are the head of the special anti-terrorist police, you don't care what the general crime rate is. You care that in the one situation when all hell breaks loose, you can survive.

A lot of this is influenced by my daily work. After the Lehman crash, you had a huge number of people getting hired in risk management and model control, and a lot of this work involves having a government official stare at you and ask, if this extreme, improbable event occurs, can you survive, and what will you do? They want to know that if you hit an iceberg, you have enough life rafts and have done the drills so that you don't take down the world economy (again). Telling them that in 99.99% of the days the world won't crash is irrelevant and will get you fired for incompetence. What you really what to do is to look for people that survived iceberg hits or didn't survive iceberg hits, and find out what they did.

Also relying on statistics can be dangerous. In physics and biology you can assume that the laws of physics and the human body don't radically change, but you can't assume this for economics. One thing that got people in trouble was that everyone looked at the numbers for mortgages in 2005 and they looked good. No one was defaulting. Everyone was happy. The loan statistics looked reasonable. What a lot of people didn't do was to just call people up and see what was going on. Forget about the numbers, who are you loaning to and what are the conditions of the loans. People that did that quickly figured out that massive loans were made out to people that couldn't possibly pay them back, and all of the statistics were being fudged, and we were heading to a blowup. The people that I work with managed to survive that part of the crisis because they didn't just trust the numbers and spent a lot of time questioning the statistics, which is why I'm glad that the OP is doing the same thing.

Again, you chose the exception rather than the norm. I don't know how we are getting into a discussion of the validity of statistics here. My emphasis has always been the validity of THIS statistics, i.e. the AIP statistics, which I again insist that it shouldn't be ignored. If you have a problem with this particular statistics, then I would like to hear it. Otherwise, you are making unsupported assumptions and claims. You are trying to undermine this set of data by making an obtuse reference to statistics in general.

Zz.

 

[Astronuc]

"Naked data" = "raw data"?

 

[twofish-quant]

Just to reinforce my point about "there is no such thing as a naked number"

There are several specific problems with the AIP number. The big one is that it's not obvious to me how they got the statistics on satisfaction. For the other numbers, you can do a statistical survey of AIP members so there is not likely to be reporting bias, but the only way I can think of in which AIP could have gotten the satisfaction numbers is by mail survey which gets you a number of areas of bias. People that are still in academia are easier to track, and I think it's likely that if you hated your Ph.D., you won't be in a mood to return a survey from AIP. I'd feel a lot better if they posted the response rates. I can sort of guess from the appendix, but I don't want to guess.

The other problem is that these numbers are right after graduation. It would be interesting to see what happens after two, five, ten, and twenty years. Also the way they ask the question will tend to increase the "I love my Ph.D. rate." Something that statisticians no and are very sensitive about is how you ask the question can give you very different answers.

I'm also particularly sensitive about this question. I have enough respect to AIP to believe that they aren't outright crooked, but I have seen surveys by various business and finance schools that manage to massage their statistics to be almost crooked (i.e. all our students are happy and employed), and often the only way to find out that the numbers are crooked is to ask around to people that actually went to the school. I don't think that AIP is intentionally crooked (which is more than I can see for some salary surveys), but I wouldn't rule out the possibility of unintentional bias.

Also being hypercritical about numbers is sort of something that happens if you go into astrophysics. If someone hands me a bunch of data about galaxy counts, I'm going to ask the same sort of questions.

 

[ZapperZ]

Just to reinforce my point about "there is no such thing as a naked number"

There are several specific problems with the AIP number. The big one is that it's not obvious to me how they got the statistics on satisfaction. For the other numbers, you can do a statistical survey of AIP members so there is not likely to be reporting bias, but the only way I can think of in which AIP could have gotten the satisfaction numbers is by mail survey which gets you a number of areas of bias. People that are still in academia are easier to track, and I think it's likely that if you hated your Ph.D., you won't be in a mood to return a survey from AIP. I'd feel a lot better if they posted the response rates. I can sort of guess from the appendix, but I don't want to guess.

The other problem is that these numbers are right after graduation. It would be interesting to see what happens after two, five, ten, and twenty years. Also the way they ask the question will tend to increase the "I love my Ph.D. rate." Something that statisticians no and are very sensitive about is how you ask the question can give you very different answers.

I'm also particularly sensitive about this question. I have enough respect to AIP to believe that they aren't outright crooked, but I have seen surveys by various business and finance schools that manage to massage their statistics to be almost crooked (i.e. all our students are happy and employed), and often the only way to find out that the numbers are crooked is to ask around to people that actually went to the school. I don't think that AIP is intentionally crooked (which is more than I can see for some salary surveys), but I wouldn't rule out the possibility of unintentional bias.

Also being hypercritical about numbers is sort of something that happens if you go into astrophysics. If someone hands me a bunch of data about galaxy counts, I'm going to ask the same sort of questions.

But you are making guesses here. That is what I have been critical of. Lacking an actual DATA to back your claim, you then not only try to throw holes into it, but you are also making counter claim that seriously lack any kind of data (even weak ones) to back your claim. This practice is no better than Intelligent Design supporters poking holes at evolution, while they themselves have nothing.

I can see if you want to say that MY EXPERIENCE is not the same as so-and-so. You can even use me as a counter example, and I can use my experience to counter yours. But to put your experience on par, or even trump over the statistics, when you're lacking of any to back your claim, is a highly dubious practice. For someone who is in your profession, you should know better.

Zz.

 

[twofish-quant]

Again, you chose the exception rather than the norm. I don't know how we are getting into a discussion of the validity of statistics here. My emphasis has always been the validity of THIS statistics, i.e. the AIP statistics, which I again insist that it shouldn't be ignored.

I'm not saying that we should ignore it, but we should look at it critically, and there are a number of obvious biases.

The big one is "how did they get this data?" For the statistics on graduation rates and "what they are doing now" you can get this data by polling AIP members. For this question, I don't see how they can possibly get this statistic other than mailing out surveys and getting responses. This immediately gets you into to very well known biases for this data. People that are employed and employed in a university are just easier to track down. Also, people that are satisified are much more likely to respond to a survey.

You can figure out the level of this bias if you mention how many surveys were sent out and how many people responded. It's not obvious from the paper what those numbers are. Also one thing that sends a lot of alarm bells is the there is no column for "declined to respond."

The other thing that makes me nervous is the way that the ask the question. One thing that you don't want to do in a statistical question is to ask multiple questions in one question, because this tends to confuse people and biases the response. I'll try to go through my wife's books to find material on how to ask a survey question, but the way that they asked the question tends to increase the "yes" response because if you ask two questions at one, you get unreliable answers.

If you have a problem with this particular statistics, then I would like to hear it.

Just did. It's possible that the did things right, but they don't provide nearly enough information to establish this.

Otherwise, you are making unsupported assumptions and claims.

I'm kicking the tires for the used car. Also one thing about getting anecdotal data is that it can tell you something is odd even if you don't know what it is. Just because I can't immediately find out why the data is weak, doesn't mean that it isn't wrong. What I'm saying is that it's a good idea to cross check data.

You are trying to undermine this set of data by making an obtuse reference to statistics in general.

I'm being skeptical about data. I'm also *very* skeptical about data involving degree satistifaction, because I've seen how other people have twisted that sort of data, and the AIP survey doesn't give me enough information to exclude the possibility that they are biasing the data intentionally.

The things that I would like to see are:

1) I think the question is very badly worded. I'll try to find some references on this.
2) I would like to see the number of surveys sent out, the numbers received, and the number of non-responses.
3) I would like to see some evidence that they cross checked the data to check for reporting bias. In particular, if they send out 600 surveys and get back 300, and it turns out that 85% of the people have post-docs, then you have obvious self-reporting bias.
4) Also dividing US Citizens and non-citizens is odd. I wonder why they did that, and what happens if you aggregate the numbers.

Again this is all *basic* stuff. If someone was doing a survey for a high school science project, I wouldn't be holding them to these standards, but we are supposed to be scientists.

 

[nlsherrill]

I somewhat regret not completing it.

Knowing what I know now, I'd do it all over again, with some changes. Now I know what I needed to know then.

I've done plenty of independent research and contributed to the field of practice, which is afterall the point of having a PhD. One doesn't need a PhD to do that, but having a PhD makes it easier for doors of opportunity to open.

what changes would you make? I remember you saying you started in physics and transferred to NE.

btw, did you read my pm I sent awhile ago?

 

[twofish-quant]

But you are making guesses here. That is what I have been critical of. Lacking an actual DATA to back your claim, you then not only try to throw holes into it, but you are also making counter claim that seriously lack any kind of data (even weak ones) to back your claim.

If you don't know, then you don't know. If you don't know then you figure out what it is that you need to find out.

This practice is no better than Intelligent Design supporters poking holes at evolution, while they themselves have nothing.

I have nothing wrong with ID people poking holes at evolution. Poking holes is good. Of course just because evolution has holes make have nothing to do with evolution being right, and just because you have a hole doesn't mean that you are wrong.

Also most of my physics experience in statistics, outside of my wife's experience, comes from being near observational cosmologists. Poking wholes at cosmological statistics is a good thing because it tells you where to spend point the telescope next. The typical problem involves looking at galaxies correlation counts and evolutionary data and trying to figure out what the data says and what it doesn't.

I can see if you want to say that MY EXPERIENCE is not the same as so-and-so. You can even use me as a counter example, and I can use my experience to counter yours. But to put your experience on par, or even trump over the statistics, when you're lacking of any to back your claim, is a highly dubious practice.

I don't think it is. If I suddenly get observational statistics that say that the CMB is highly anisotropic, my first reaction is go back over the data and assume that I made some sort of mistake. Usually you can find an obvious mistake in the data processing. If I get statistical data that says the sky is pink, my first reaction is to assume that there is a mistake.

Now what makes me different from a creationist is that I'm not totally dogmatic about it. If I go through my data, and I still can't find an obvious mistake, then I think some more about experiments to perform, and if after going through more experiments, I *still* find a signal, then I'll change my mind.

However, I know from observational cosmology and my wife's work what a statistically strong study looks like and the AIP study ain't it. If the AIP published a statistically strong study and it conflicted with anecdotal data, then I'd likely conclude some reporting bias in the anecdotal data, but because they leave out critical pieces of information and also because I think the question is very badly worded, right now, it's statistically weak.

One thing that people in the social sciences and medicine are quite aware of is that people *do* unconsciously bias statistics because of experience, and there are some pretty cool statistical tests for teasing out this. It's not that people are evil or intentionally trying to mess with stats. It's just that you have to look very carefully at statistical processing to see that.

For someone who is in your profession, you should know better.

I'm not sure how I'm supposed to respond to this.

 

[twofish-quant]

Also, one thing to make clear is that I don't know or not if the AIP statistics are inaccurate or not. It's data. It's interesting data, and if anecdotal reports are inconsistent then we have an interesting question of what is going on.

What I object to is bashing the OP for asking the questions which I think he should be asking. If he asks a bunch of people and they all give responses which are consistent with the AIP survey, then that means that the AIP survey is more reliable. If not, then we have inconsistent data that we need to try to explain.

 

[HossamCFD]

Also, one thing to make clear is that I don't know or not if the AIP statistics are inaccurate or not. It's data. It's interesting data, and if anecdotal reports are inconsistent then we have an interesting question of what is going on.

What I object to is bashing the OP for asking the questions which I think he should be asking. If he asks a bunch of people and they all give responses which are consistent with the AIP survey, then that means that the AIP survey is more reliable. If not, then we have inconsistent data that we need to try to explain.

This post says it all. I think after 33 posts we may actually want to respond to the OP, like what Astronuc did, rather than debating whether the question is useful or not.

I myself don't have stories to tell. I will begin my Ph.D next October. And although it is extremely unlikely that I will be making any decisions based on what I read in this post, I find the question asked here very interesting and I am curious to read people's perspectives.

 

[twofish-quant]

One other thing since you mentioned ID, let me counter by bringing up Halton Arp. Arp has been arguing since 1965 that statistics prove the quasars are close by, and from time to time a paper gets published that argues that with statistics you can show that the big bang didn't happen or that quasars are close by, and you can look in the astrophysics forum for these arguments.

What you end up finding out is that people that argue this are sloppy with their statistics, and if you drill down into them, the correlations they argue about disappear. Now if you run into a situation where someone publishes a quasar count paper without enough information to drill down to see what is going on, it's probably going to be dismissed out of hand, and the attitude is "we can't figure out what you did wrong since you didn't give us enough data, but you likely messed up somewhere so we aren't going to publish." Arp takes this as evidence that the scientific community is against him and the truth.

So coming from this environment I'm going to be *very* skeptical of statistics data and more trusting in deep anecdotal data. For example, you take twenty quasars at look at them in detail and conclude that "yup they are really far away" and that beats Arp's statistical arguments that they are close by. It's not that I will dismiss statistical data out of hand, but I will insist on some basic cross-checking of data.

Now if you think that statistics always beats anecdotal data, the next time someone in the astrophysics forum brings up Arp and friends and their statistical proof that most quasars are nearby, I'll let you argue with them.

Deal? :-) :-) :-)

 

[Astronuc]

what changes would you make? I remember you saying you started in physics and transferred to NE.

I'd ask more questions, and do a little more investigation! I thought I could figure it out by myself, but I didn't know the what I needed to know at the time. Looking back, the path is more clear.

I'd actually redo my undergrad program to double major in physics and nuclear engineering.

btw, did you read my pm I sent awhile ago?

Yes I did. I will answer it later today.