Data Science and the need of programming?

[Chubigans]

Hello. I have an undergraduate degree in CS and I'm beginning grad school to study AI and Machine Learning.

I've spent a lot of time doing "software engineering" (dealing with design patterns, web development, QA, Agile processes, SQL, GUI development) and I'm really sick of it. I don't use much of my computer science knowledge as a programmer. I studied CS because I find beauty in algorithms, but there's not much theory in writing web apps or .NET apps.

I love writing proofs, analyzing algorithms and data structures, writing python and LISP code, etc. I am really into *proofs of concept* as opposed to *ultra robust software*... I like to deal in ideas, not code. Love to read and write papers, love to share ideas and teach... so I'm looking for a career that involves that.

As a data scientist / machine learning engineer with a Ph.D. in CS, would I be expected to write a lot of "application" code, or would I spend the majority of my workday in the realm of theory, creating new ideas and hacking together prototypes in scripting languages?

Is it essentially just an advanced software development job?

Attached is an example job posting from CL:
http://sfbay.craigslist.org/sfc/sci/3722725371.html

 

[Devils]

I'm a software engineer with a computer science degree & over 20 years experience. Like you I'm a bit of a guru on theoretical computer science.

Calling yourself a "data scientist" is the way of the future. It gives a catchy marketing title for those of us with hard science degrees.

You are right, the majority of the code in this world is boring. Most of the world is business producing mundane things, who need mundane software. The "interesting" jobs I've had have been rare.
There is some non-boring work like research or startups, but they are not too many jobs.

Some ideas for less boring jobs:
- specialist govt departments with technical needs eg defence, defence contractors, environment, energy
- anything to do with statistics eg govt depts
- engineering-related jobs
- university - non-academic programming jobs

For fun read http://www.kalzumeus.com/2011/10/28/dont-call-yourself-a-programmer/

 

[ParticleGrl]

I have a phd in physics and have worked two jobs in the "data science" type description. In the first job, there was very little need for coding, but there was also very little in the way of algorithm design. Mostly, it was tiny tweeks of off-the-shelf statistical algorithms, that sort of thing. To get an idea of how far from a robust-software environment, one of the "production models" that predated me was a series of excel notebooks, with data transferred via copy-paste.

In my newer job, I still don't do much in the way of algorithm design, but I do a fair amount of coding. For most business problems, existing methods work very well, so off the shelf statistical models (in R or any of the other statistical packages) are the name of the game. Its less like research into machine learning and more like applied machine-learning. But we do provide finished models in various forms for the client ,which is closer to robust-software, so much more of my time is spent on IT solution/routine coding type stuff. I spent most of yesterday coding up various unit-tests.

 

[Floid]

Almost everyone who works in software would prefer to do "proofs of concepts" and do the fun part of prototyping a new idea and then jump to a new project. The problem is there is no money in that for a company. Proof of concepts don't make money, deployed products do...

On real world software projects the devil is in the details. Coming up with a proof of concept is the easy part. Implementing that proof of concept into a marketable, usable, and stable product is the hard part. It also is the more boring part.