Last year of my Ph.D, and having a crisis about my work (data science)

[joshthekid]

Hi all,

I just want to gain some perspective as I am sure there are people in the same boat or have been where I am.

I am in the home stretch of my Ph.D in Biomedical Engineering, I have manuscript published and a couple more in the pipeline. I work in big data and how to use it to come up with more personalized treatment. My path to a Ph.D was not necessarily linear. I started in Physics, moved too engineering, and then back to physics finally getting my B.S. in applied Physics while also dealing with significant health issues. After taking a year looking for a job while driving a bus at a ski resort I decided that if I wanted to be more competitive for engineering jobs I needed an engineering degree. So I entered a non-thesis master of engineering program in biomedical engineering. As an undergrad I had worked in a computational biology lab and really loved it. Anyways, to earn some extra-money on the side I worked for professor who needed someone that was computer savvy to work with some genomics modeling. At the time I had known nothing about data science, so I took a course in machine learning through the C.S. to learn more about the available algorithms. Anyways, I am about done with my Masters when the prof I work for asks me if I want to do my Ph.D working on this stuff. While as a traditional student I struggled a bit because the current paradigm of lecture, homework, test does not mesh well with how I learn but I still love to learn and have always thought about getting a Ph.D. So this was my shot and I took it. I realized that data science was a hot topic on the job market and these skills would set me up well for a career.

Here is where I am having trouble. It seems with data science's popularity days it is popping up everywhere. There are a plethora of online courses, youtube videos, blogs you name it. The thing that is discouraging to me is that most of the stuff I use on a day to day basis requires absolutely no understanding of the theory underlying these algorithms. You do not need to be a Ph.D. to do this stuff, I could teach it to middle schoolers. Thus, I have taken the approach that in order to use any machine learning algorithm I need to know the theory behind it first. So I have spent a good amount of time reading textbooks about probability theory, measure theory, information science, topology, statistical mechanics and this is the part of my Ph.D. which I really love, this is why I keep doing it and all of it is not necessary to do my job. It keeps me up at night.

So I'm just a little shaken because I have spent a significant amount of time learning about the theory and am not as much interested in the actual application. The theory is the truly intellectually rewarding part. I like using math to solve biological problems. I also believe that data science is necessary to be a good computational biologist based on the job descriptions I see. Is there any tips on how I should proceed in my so the intellectual challenge becomes a bigger aspect? Professorships are hard to come and staying funded is nightmare. Besides, my P.I. mostly writes grants and looks at lab finances, does not really do much science anymore.

Hopefully there are some others out there who have been where I am. Thanks

 

[jedishrfu]

Yes, Data Science is a dark art that anyone can pick up and use the tools to create an ML application. However, it will take someone with yours skills to understand what is happening, how effective it is and how to improve its performance.

With your computational biomedical background and published papers, there should be many companies in the biology or pharmaceutical industries interested in hiring you.

In a competitive field, the higher degree will win out over someone with some experience learned from online courses and you have both.

On another note, what languages and tools did you use python, Matlab, r or Julia? Jupyter notebooks, pycharm and Anaconda?

 

[Locrian]

So I'm just a little shaken because I have spent a significant amount of time learning about the theory and am not as much interested in the actual application. The theory is the truly intellectually rewarding part.

There are positions at private companies that are largely research and focus much less on the actual application. They tend to be rare, and the ones I know of don't offer a ton of job security, since it's a real challenge to demonstrate your value in such a position (and you often rely on people who can deliver the application to provide said justification).

Honestly, having seen all three areas (theory, application, integration), the art of enacting the kind of organizational change needed to really leverage advanced analytics (integration) seems more interesting than either of the others, and jobs that touch on all three are really my sweet spot. My guess is that you haven't spent much time with the integration portion. You might not like it, but don't rule it out until you've really engaged with it.

 

[joshthekid]

Yes, Data Science is a dark art that anyone can pick up and use the tools to create an ML application. However, it will take someone with yours skills to understand what is happening, how effective it is and how to improve its performance.

With your computational biomedical background and published papers, there should be many companies in the biology or pharmaceutical industries interested in hiring you.

In a competitive field, the higher degree will win out over someone with some experience learned from online courses and you have both.

On another note, what languages and tools did you use python, Matlab, r or Julia? Jupyter notebooks, pycharm and Anaconda?

Thank you for your comment. For most of my models I use sklearn in python or keras for ANN , occasionally I will use R. Matlab is probably my favorite language when solving most engineering problems in course work, Image processing, and the small of amount of dynamic models I work on. I have yet to really yet to learning for any ML or DS applications.

 

[StatGuy2000]

There are positions at private companies that are largely research and focus much less on the actual application. They tend to be rare, and the ones I know of don't offer a ton of job security, since it's a real challenge to demonstrate your value in such a position (and you often rely on people who can deliver the application to provide said justification).

Honestly, having seen all three areas (theory, application, integration), the art of enacting the kind of organizational change needed to really leverage advanced analytics (integration) seems more interesting than either of the others, and jobs that touch on all three are really my sweet spot. My guess is that you haven't spent much time with the integration portion. You might not like it, but don't rule it out until you've really engaged with it.

Not to mean to hijack this thread, but I'm curious as to what is involved in the integration aspect of "data science". You speak of the organizational change needed to really leverage advanced analytics, but my speculation is that organizations (be they private or public sector) have a tendency to be resistant to change -- certainly radical change, as I imagine might be required to leverage advanced analytics.

I am legitimately asking, as my job has not involved much in terms of integration -- my jobs have all been very applied.