Endurance->permanent academia position?

[ParticleGrl]

ParticleGrl, you had mentioned (both in this thread and in another thread where I had asked what happened to the physics PhD students you knew personally) that the plurality of your PhD cohort were working on statistical or big data type work. I am curious as to whether their working in this field depended to any great degree on their area of specialty.

It doesn't appear to be the case- I know a biophysicist, a particle theorist (me), an applied math guy who studied stochastic diff-eq who all retrained, as well as some HEP experiment guys. However, by the time we talk about physics phds who are doing big data, people I know get small quickly.

Another reason I'm asking is that in discussions that I've followed with other statisticians, there is much lamenting the perceived lack of understanding or appreciation of statistics on the part of physicists. See, for example, the following amusing comment on the blog from physicist-turned-statistician Cosma Shalizi, currently a professor at CMU.

I would say that there is very little formal statistical training for physicists- everyone I've talked to, even HEP experiment have had to learn a great deal of the statistics as they go.

 

[Locrian]

Quantum mechanics and statistical mechanics made so much more sense to me after the preliminary actuarial exams.

 

[Vanadium 50]

It is true that formal statistics is not part of the standard physics curriculum, and it's also true that HEP has reinvented the wheel more than once (and often not a particularly round wheel at that). I had three courses in probability and statistics in college, which was enough to make me wince at the way statistics is handled.

That said, one of the secrets of industrial statistics is that the textbook is not of much help. If it's in the textbook, chances are it's already in SPSS or SAS and they don't need your help. I made a few extra grand moonlighting as a statistical consultant when I was a postdoc, and my bread and butter was figuring out and explaining what to do when the book doesn't say what to do.

Which is a lot like they way statistics is done in HEP.

 

[ParticleGrl]

That said, one of the secrets of industrial statistics is that the textbook is not of much help. If it's in the textbook, chances are it's already in SPSS or SAS and they don't need your help. I made a few extra grand moonlighting as a statistical consultant when I was a postdoc, and my bread and butter was figuring out and explaining what to do when the book doesn't say what to do.

You would be surprised- in order to know what to do in SPSS or SAS, you have to know what a method is called and how it works, etc. Often having this sort of 'mental decoder ring' is quite valuable for smaller companies who don't have a lot of statisticians.

That said, because my statistics is so spotty (my self-study was and is heavy on machine learning and light on many other things), I've found myself reinventing the wheel on quite a number of occasions (I have a few functions I built in R that later turned out to be less-efficient clones of existing functions).

 

[George Jones]

As a physics student, I took loads of (mainly pure) math courses that were not required, but I only took the one required statistics course.

Cosmologist/astronomer Peter Coles wrote some interesting things about statistics in his blog post
http://telescoper.wordpress.com/2012/12/06/society-counts-and-so-do-astronomers/

 

[StatGuy2000]

It is true that formal statistics is not part of the standard physics curriculum, and it's also true that HEP has reinvented the wheel more than once (and often not a particularly round wheel at that). I had three courses in probability and statistics in college, which was enough to make me wince at the way statistics is handled.

That said, one of the secrets of industrial statistics is that the textbook is not of much help. If it's in the textbook, chances are it's already in SPSS or SAS and they don't need your help. I made a few extra grand moonlighting as a statistical consultant when I was a postdoc, and my bread and butter was figuring out and explaining what to do when the book doesn't say what to do.

Which is a lot like they way statistics is done in HEP.

First of all, I would disagree with your characterization above about statistics. In order for someone to apply a method built into SPSS or SAS, you need to know about the method, how to use, and most important of all, under what circumstances does it apply or not.
Part of the reason why one hears about "lies, damned lies, and statistics" is due to non-statisticians applying the wrong method blindly without understanding what circumstances
such methods would apply to e.g. blindly fitting linear regression without validating the model. These are things that a good statistics course -- in particular a good course on linear models/regression analysis or applied statistics -- should teach (and a good statistics textbook should cover).

Furthermore, the very fact that in HEP (as well as in other fields of physics) that there is a need to reinvent the wheel at least in terms of data analysis suggests to me that either more formal statistical training needs to be offered, or that there needs to be greater interdisciplinary participation between statisticians and research physicists (perhaps in the form of consulting, similar to what statisticians often provide to other faculty members in fields as diverse as medicine, biology, psychology, engineering, etc.)

 

[Vanadium 50]

StatGuy, I am not arguing that that's not the way things should be. (ParticleGrl makes a similar point) I am instead discussing how thing actually are - or at least were. Certainly experimental HEP could do a better job than it does.

My point was that there is - or at least was - an opportunity to help businesses (and making money doing it) using the same kind of statistics as used in experimental HEP.

 

[StatGuy2000]

StatGuy, I am not arguing that that's not the way things should be. (ParticleGrl makes a similar point) I am instead discussing how thing actually are - or at least were. Certainly experimental HEP could do a better job than it does.

My point was that there is - or at least was - an opportunity to help businesses (and making money doing it) using the same kind of statistics as used in experimental HEP.

Fair enough. And I agree with you that the kind of statistics that were used in experimental HEP are the same as that can be used in businesses for data mining/analytics -- one of the reasons why I asked ParticleGrl whether those physics PhD graduates who she knew of that transitioned to statistics/data analytics type work often come from a background in HEP or astronomy.

 

[ParticleGrl]

I agree with you that the kind of statistics that were used in experimental HEP are the same as that can be used in businesses for data mining/analytics

Just to be clear what I tend to think of as the skill Vanadium is talking about isn't actually KNOWING statistics- its having the ability (and perhaps the arrogance, to use the word from Shalizi's blog) to reinvent the wheel to solve the problem.

 

[DarrenM]

This has been a very interesting discussion.

If I may venture a very small detour, I'd like to ask how much of this information also applies to Mathematics/Applied Mathematics PhDs? I know this forum has significantly fewer "math folk" than physics--still, any information would be helpful.

 

[ParticleGrl]

If I may venture a very small detour, I'd like to ask how much of this information also applies to Mathematics/Applied Mathematics PhDs?

This is just an educated guess, but most of the math phds I know were seriously considering the faculty job market after only 1 postdoc, and all of the physics phds I know did two postdocs before approaching the faculty market. This suggests to me that while its still very difficult, its a tad easier.