Endurance->permanent academia position?

[StatGuy2000]

It is true that formal statistics is not part of the standard physics curriculum, and it's also true that HEP has reinvented the wheel more than once (and often not a particularly round wheel at that). I had three courses in probability and statistics in college, which was enough to make me wince at the way statistics is handled.

That said, one of the secrets of industrial statistics is that the textbook is not of much help. If it's in the textbook, chances are it's already in SPSS or SAS and they don't need your help. I made a few extra grand moonlighting as a statistical consultant when I was a postdoc, and my bread and butter was figuring out and explaining what to do when the book doesn't say what to do.

Which is a lot like they way statistics is done in HEP.

First of all, I would disagree with your characterization above about statistics. In order for someone to apply a method built into SPSS or SAS, you need to know about the method, how to use, and most important of all, under what circumstances does it apply or not.
Part of the reason why one hears about "lies, damned lies, and statistics" is due to non-statisticians applying the wrong method blindly without understanding what circumstances
such methods would apply to e.g. blindly fitting linear regression without validating the model. These are things that a good statistics course -- in particular a good course on linear models/regression analysis or applied statistics -- should teach (and a good statistics textbook should cover).

Furthermore, the very fact that in HEP (as well as in other fields of physics) that there is a need to reinvent the wheel at least in terms of data analysis suggests to me that either more formal statistical training needs to be offered, or that there needs to be greater interdisciplinary participation between statisticians and research physicists (perhaps in the form of consulting, similar to what statisticians often provide to other faculty members in fields as diverse as medicine, biology, psychology, engineering, etc.)

 

[Vanadium 50]

StatGuy, I am not arguing that that's not the way things should be. (ParticleGrl makes a similar point) I am instead discussing how thing actually are - or at least were. Certainly experimental HEP could do a better job than it does.

My point was that there is - or at least was - an opportunity to help businesses (and making money doing it) using the same kind of statistics as used in experimental HEP.

 

[StatGuy2000]

StatGuy, I am not arguing that that's not the way things should be. (ParticleGrl makes a similar point) I am instead discussing how thing actually are - or at least were. Certainly experimental HEP could do a better job than it does.

My point was that there is - or at least was - an opportunity to help businesses (and making money doing it) using the same kind of statistics as used in experimental HEP.

Fair enough. And I agree with you that the kind of statistics that were used in experimental HEP are the same as that can be used in businesses for data mining/analytics -- one of the reasons why I asked ParticleGrl whether those physics PhD graduates who she knew of that transitioned to statistics/data analytics type work often come from a background in HEP or astronomy.

 

[ParticleGrl]

I agree with you that the kind of statistics that were used in experimental HEP are the same as that can be used in businesses for data mining/analytics

Just to be clear what I tend to think of as the skill Vanadium is talking about isn't actually KNOWING statistics- its having the ability (and perhaps the arrogance, to use the word from Shalizi's blog) to reinvent the wheel to solve the problem.

 

[DarrenM]

This has been a very interesting discussion.

If I may venture a very small detour, I'd like to ask how much of this information also applies to Mathematics/Applied Mathematics PhDs? I know this forum has significantly fewer "math folk" than physics--still, any information would be helpful.

 

[ParticleGrl]

If I may venture a very small detour, I'd like to ask how much of this information also applies to Mathematics/Applied Mathematics PhDs?

This is just an educated guess, but most of the math phds I know were seriously considering the faculty job market after only 1 postdoc, and all of the physics phds I know did two postdocs before approaching the faculty market. This suggests to me that while its still very difficult, its a tad easier.