Transitioning from Academia to Data Science

[Written in 2016 in relation to my first tech job — still accurate in 2020.]

I realize there are numerous blog posts on transitions from academia to industry, but each individual’s experience adds value to the narrative.

Since making my transition to industry, I’ve received numerous questions from current PhD students and postdocs asking about my experience. Although it is relatively common for engineering students to transition to industry after their PhDs, there is a growing trend of those with a pure science background. The insights below are related to my current company, a relatively established company with significant resources.

My background: From 2011–2015 I completed a PhD in Population Genetics and a MS in Biomedical Informatics at Stanford University. I loved my four years of graduate school. I can think of few better ways to spend those four years than getting paid to (extremely slowly and slightly) expand the bounds of human knowledge. I thoroughly enjoyed waking up each morning prepared to learn something new and push the bounds of the unknown. However, as my graduate career progressed, I started to feel too far back from the front lines. I wanted to be closer to the action, creating tools and knowledge that could make an impact more quickly. I also became interested in expanding to skill base to allow me to solve more general problems within and outside of bioinformatics.

I decided to explore this interest and take a job in software engineering / data science after graduation. It is definitely a different experience with new challenges, but many of the skills I gained in graduate school have been quite transferable.

Similarities

Bright, creative colleagues and workplace

• There is a surprising quantity of enriching intellectual activities in industry
• Some of the coolest talks I’ve been to at work include a Berkeley astronomer, the CEO of Patagonia and some political scientists.
• We also have journal clubs to discuss recent papers
• As well as activities like hikes and hack days to meet new people and try out new projects

Creativity and control over project

• I have some control over my project direction. In industry, I expected to be given a specific project to complete based on a very structured list. However, there is actually much more creativity and problem-finding than I expected (see Differences for caveat).
• In both cases, you need significant motivation and initiative to keep things afloat and network to acquire needed resources.
• This was important to me because I love problem solving and can have a unique impact on the product.

Domain-specific knowledge can be a barrier everywhere

• Lingo can seem over-whelming at times, but oftentimes the meanings behind complicated terms are much less terrifying than they sound.
• Also, internally each company has their own set of complicated lingo.

Documentation is a problem everywhere

• For some reason I assumed companies would have extremely thorough documentation, but this is definitely not the case. Though industry is a little bit better than academia, it can still be difficult to impossible to tracking down the meaning of a number in a data set or an explanation about how it was created.

Schedule is very flexible

• This one is probably the most specific to my current affiliation. Similar to grad school, it is useful to be around at least between 10–4ish, but working from home or appointments during the day are totally fine. Work progress is more important than the exact hours you are sitting at your desk.

Differences

Rapid prototyping in industry

• This of course is the major difference between academia and industry. In graduate school, I would spend months to years attempting to verify a specific result. This would be considered wasted time here.
• People are immediately helpful at companies. I am accustomed to messaging people and maybe getting help with a problem after a week. But here, people often drop whatever they are doing to help out immediately. I guess it hurts everyone if one person is stuck — I quite like this difference!

More immediate impact vs. (potentially) long term impact

• In industry I can create a new feature and in a mere few weeks see how it is improving the experience of real-life people. My work in graduate school may (or may not) help to guide future researches in that area of work, though it may take years to decades.

Currency difference

• Both fields have a currency in which “successful” individuals must optimize for. In industry that currency is profit margins, while in academia it is publications.

Organizational change

• Organizational changes are common in industry. In graduate school, I was a PhD student in the same lab for 3–4 years and my status underwent very few alterations during that time. In industry, there is rapid turnover of staff, organizational change and goal restructuring. It is common (and a bit unsettling) to wake up one morning and find out your are doing something entirely different with an entirely new group of people.

Like any career transition, there has been an adjustment period, but I am enjoying the new challenge.