When I was doing a cyber training exercise in the military, we had an Air Force general tell us that the best technical officers they had started out with non-technical degrees. They said that officers with undergrad degrees in things like IT, Software Engineering, and Computer Science, weren't very good at thinking outside of the box because everything they did was fairly low level and had a provably correct solution. The officers with degrees like psychology and political science were better at taking in large amounts of information, and deducing a "best" answer to a problem that didn't have a "correct" answer. I noticed the same thing when I started teaching after I got out of the military; people with technical backgrounds wanted a checklist to follow, and didn't really care about the end result as long as the steps were followed. Non-technical students would look at what the result should be, and do whatever was necessary, creating their own checklist as they went along.
As a technical person, I find it interesting that so many technical people are that way.
I start to get into a motivational rut if all I'm doing is working on a list of technical things to complete. Engineering disciplines are some of the few degrees that provide you the ability to both directly solve other people's problems and have close customer contact - that was a major reason why software engineering attracted me as a career rather than just a hobby.
I don't think it's so much that they are that way innately, as we select for it then drill it in further.
For junior and even mid-level engineers, your performance is generally evaluated on checklists of mundane shit. Code is passably neat and organized? Check. Code has and passes tests? Check. Feature checks requirements boxes? Check, check, check.
I think I'm similar (wanting a mix of technical and business problem solving) but I feel like this is the minority of jobs - though I don't have a good way to identify this.
Especially at larger organizations, I've found that the dev role is very much of an "implementer", that your job is to take the design, spec, architecture and put it into place as close to the above as possible.
You're much less likely to be actually sorting out the why/how of the system.
I think it's a learned behavior, they were mainly talking about people fresh out of college who were evaluated based on "use the fastest algorithm for a mostly sorted set of data" type assignments. I worked with a lot of people with technical degrees who learned to think outside the box, but it seemed easier to teach technical knowledge to non-technical people than it was to teach technical people that they needed to change the rigid way of thinking that had been drilled into their head for 4 years.
>Engineering disciplines are some of the few degrees that provide you the ability to both directly solve other people's problems and have close customer contact
Douglas Adams made his career reiterating the cliche engineers do not communicate with the customer.
David Epstein's Range talks a lot about this issue. He mentions the idea of 'kind' and 'unkind' learning environments. He uses golf and Tiger Woods as a 'kind' learning environment. To be a great golf player, you pretty much just have to grind out the hours and work hard at it. The better players have simply just spent more time grinding away. Epstein then uses tennis and Federer as an 'unkind' learning environment. Federer did a lot of other sports and has many other interests than tennis, and other successful players are similar. Using tricks from a variety of sports and arenas is better. Epstein spends the rest of the book talking about other areas like music, testing, and business and compares generalists to specializers, demonstrating that generalists end up better off in unkind learning environments. The now infamous USAF academy testing screw-up is of particular interest here. As most of life is an unkind learning environment, generalists may be better off than specialists over the long haul. Hence, things like a liberal arts education being so powerful and the paradox of the 'well-rounded vs. spiky' person in long term success.
Lately I have been realizing that this is true for me. I can get stuff done on time when there are existing guidelines/checklist. If not, I tend to struggle and take much more time to complete tasks.
I do have a technical degree. But I think my underlying problem is because of how I was brought up. I had strict parents and family members and was always told what to do. Questioning things was not something I was taught and I grew up to always follow the "rules".
And I think that has (kind of) led to my current state where without rules I struggle to make decisions.
>Questioning things was not something I was taught and I grew up to always follow the "rules".
>And I think that has (kind of) led to my current state where without rules I struggle to make decisions.
I worry that the newer generations brought up in zero tolerance environments will come late to questioning authority, if ever. Life is far more nuanced and negotiable than school and zero tolerance leads people to believe and folks like yourself who aren't equipped to think creatively in these situations have been robbed of a valuable skill.
I think this is almost universally true. When the next step isn’t obvious because there are too many different strands of the project to integrate, progress grinds to a halt. Same underlying phenomenon of what novelists call “writer’s block.”
I worked in a defence company that had a strong revolving-door relationship with the military: several of our staff were recruited from the military, often because they had good domain knowledge and if nothing else could help translate customer requirements into a language our engineers could understand. Their technical knowledge varied but I never noticed any great correlation between their military trade (let alone their original degree) and their success in the company. However, something that I observed many times was that the effective ones rapidly rose up through the management and sales hierarchies, often I suspected because their extensive leadership and people skills gained in the military, as well as decisiveness, let them run rings around management who came from a purely scientific or technical background. This seemed to be truer of the officers than other ranks, especially those who had left because their military promotions and careers were limited by availability of roles rather than (lack of) competence.
[Edit] - people skills. One ex-military guy who was recruited into my team was asked by corporate HR how he would deal with 'difficult conversations'. His answer had to do with telling people / families that their best friends / relatives (soldiers under his command) had been killed in action, or persuading one of his squaddies not to marry the local prostitute. When I decided to step back from team leadership he was my first choice as my replacement, and later became one of the best bosses I've ever had.
That's rather interesting. I noticed a lot of friends who are much better at communicating than those who started off on a technical career with a technical degree. Not that being highly technical is a bad thing but in business, you go further as an employee and/or manager if you can grok what's better for a business. Usually that's where communication comes in.
Very cool example you have there. I'm keeping this filed on my Roam Research documents!
It depends on the mindset of the developer or engineer. Some very smart people just want to code the thing according to the spec and not deviate. They don’t really care about the result because it’s not their problem the requirements are changing. They coded the thing based on spec that was given.
I’ll push back a lot and I’ve seen other much more senior technical people that will do the same. A lot of people just want to come to work, heads down coding, and then leave.
Getting stuck on trying to achieve a certain result and changing the rules to get there is usually a sign of a bad program that is not flexible and brittle.
Sounds like there is more of a problem with how the army is asking verse what they expect. If they were more direct in what they wanted vs measuring some scenerio they created.
And if there is a place like no other that needs out of the box thinking, it must be the military. This sounds like a shtick. The kind of mentality that made them put STOVL in the requirements for the F-35.
Look, there is an essential difference between what is an engineer and what is a technical person. The later want direction, projects broken into epics and then stories, and wants steps. Engineers can make that structure and work better in ill defined problem areas. That's not a flaw. What is a flaw is when tech never progresses to engineer. That's bad for the individual, team, company, and client. Leadership then (in the drucker or tqm sense) is important. Look at IBM pre/post gernster and Microsoft pre/post balmer.
In short rigidity on the part of individuals combined with countervailing organizational goals to combatting same is the issue.
VP at a top startup mean very different things to different people. To somebody who values sr titles it’s Substantial. To somebody who views titles as a measure of how much and what you contribute will only view this as substantial in the context of where you operate - VP of the lemonade stand vs VP at a large company (by revenue,employees etc). Most of the experienced crowd on this site will generally take the latter view. Most of the young/fresh grad kind of people on this site will take the former view. As for yourself I would suggest you measure yourself by impact and not title.
Awesome post! Thank you for taking the time to document down your learnings along your journey of pursuing data science. What I really like about your journey is how one thing manages to lead to another in the most amazing way. Some people attribute it to luck, but I guess in your case you really have made your own luck by taking actions to move towards your goal! Congratulations on your accomplishments thus far! I just have a few questions which would love if you can elaborate further on.
Firstly, it seems to me that the online course enabled you to reach top 3% of a Kaggle project which led to a series of career opportunities. However, I believe that being able to reach the top in Kaggle competition is not an easy feat. This kind of process is not really replicable especially for the people who are just starting out in Data Science. In that case, what would you recommend them to do?
And secondly, what is the end goal of a data scientist or at least for you as a data scientist? Would it be going into research? Continue to climb the ladder till you reach the top of the company? Data science and engineering in general are great as a form of intellectual challenge. However, personally sometimes I feel that there is a lack of meaning in the thing that I am doing as engineer/data scientist (especially if the culture of the company is very bottom-line driven). Just want to hear your thoughts! :)
On the first point: I assume you're asking for avenues to practice and demo your work. If so, personal projects are a great way.
The most impressive candidates can demo something they've deployed. This demonstrates the ability to apply what you've learned AND learn the rest of the stuff as needed (e.g., spin up EC2, build basic front end, maintenance, etc.). The latter is more important and gives confidence that you take ownership and likely can do the same (i.e., end-to-end with results) in the company.
On the second point: having meaning in work differs from person to person. Some people enjoy pushing the envelope (research), others enjoy staying hands-on (individual contributor, IC), while some big picture thinkers enjoy coordinating (PMs). It's helpful to think back on what gave you the greatest satisfaction and work with your leaders to be a role that lets you perform your best.
I find it most fulfilling to build data products that are useful, relative to generating tons of revenue. I would be equally happy as an IC doing the hands-on or leading a small team and mentoring them to become more effective.
I have interviewed many applicants to the ML engineer position. What you need to do is to have decent knowledge about ML theory and good dev skills. You don't need to be able to answer the most obtuse theoretical questions, but it is absolutely necessary to have a couple of years of previous experience with neural nets, to have trained many models with your own hands, seen the results, made decisions, rinse and repeat.
There is a high chance of getting hired if you have practical skills. Most candidates are just trying their luck, coming to the interview with abysmal knowledge and experience, even when they have PhD's.
I had almost the exact same experience going from a psychology degree, to coding in SF, to heading up analytics for the Trail Blazers. Luck, self-study, and Kaggle! Really cool to hear that someone else also took the psychology to data analysis route.
One of the interesting aspects of this account is the role of online classes (Coursera and edX).
I always thought that those kinds of things are useful and should be effective but (online) anecdotes suggest they are not. Seems like there is a missing step and the author was able to find that step. (Branded company work?)
Author here: I believe the missing step is being able to APPLY the knowledge and skills (learned from online courses) to create value.
It's not difficult, you just need to know that creating an artifact is not the end of your job--it needs to be used to benefit customers and generate value. Remember to measure the impact after your feature/system is launched.
I've seen many interview candidates struggle with this (missing step) too. Those that get past it were hungry enough to want it, as well as followed up with proper validation and measurement.
I moved into management 5 years in at my startup. At the time I had ~50% of all code commits. I wanted to run a team because there was so much I wanted to create that I couldn't without the help of others. After a few years of VP of Engineering and then CTO I'm back to writing small apps in Rust for free. So while it was enjoyable it didn't scratch the same itch.
So they can make technical decisions. This is one of the reasons I hate programming at companies. I didn't serve in Iraq so I could work my butt off to get a degree and then spend years working professionally while continuing to learn and do side projects to have someone who doesn't know anything tell me what to do.
Ok, this is shameless self promotion masquerading as a casual, "Hey I've got an interesting story". I notice that he conveniently left out the part where Alibaba promoted him to VP, Data Science at the same time he started an MSc at Georga Tech which he completed in two years which is an impressive thing to do while you're working so I suspect there is more to the story. If working in NYC has taught me anything, always be promoting yourself while simultaneously putting everyone else down. "I guess I just worked harder and smarter than all those PhD's around me".
A learning curve is a plot of work required for achieving some amount of learning. Those guys just didn't get that that was how it's being used.
"How hard is it to learn Haskell?" Not "How much time does it take to learn Haskell?"
For instance, people will say some things are easy to learn, hard to master (playing Go). Other things are hard to learn but once you have, you're close to mastering them (kinds of problems that have a trick - like integrating e^ax cos bx).
I didn't recall any books helping much on communication.
With regard to presentation and speaking, I had great leaders around me who provided candid feedback the presentation was off-point, too long, or had to be re-ordered (e.g., when presenting to non-tech folks, results before methodology). It was also useful to "rehearse" any conference presentations at meet-ups and get feedback.
Feedback to improve written communication is harder to come by. Few people feedback on your design documents or reports. For this, I find blogging and seeking feedback from writers whose writing I enjoy to help the most.
Well.... I imagine it comes with the author's field? I've always been vocal about how much I suck at communication in both my personal life and work.
I think I would have landed a super nice position if I wasn't a socially inept person. Bitterness and "reality" or "truths" won't help us, so these days I'm just leaning on being reasonable and as human as possible because nobody likes a person who's constantly pointing out flaws and telling people they are horrible. (thankfully I've never in my life told anyone they are bad at something, at the end of the day this field you can keep improving yourself. I imagine in a field of medicine if you mess up too much that's it for you)
Written communication: blog, short stories, editing others' work
Oral: practice public speaking, either by volunteering in your own context whenever possible (most shy away - you stand out for volunteering to speak) or build confidence and skills outside, eg in Toastmasters.
- progress plateau is scary. whenever i experience this, I try very hard to get out of it.
> I needed practice too. The data team wanted to launch an internal newsletter to spread awareness—I volunteered. Someone had to visit overseas markets to do a data science roadshow—I volunteered. We wanted to present our work at conferences—I volunteered.
> In year two I was internally transferred to the workforce analytics team, working to forecast job demand and build a job recommendation engine to move people within the organization.
I'm hearing this a lot from friends.
> On the side, I continued self-learning. I picked up Python (love it) and took classes in machine learning. Spark, a shiny (pun intended) big data framework was emerging and provided free courses on EdX—I devoured these too.
I'm currently noticing this myself. I'm devouring content on Hack The Box. I'm currently making 16 hour days and am at 200+ hours within 2 weeks, and devoured half of their active boxes.
You don't need a background once you have 16 hour per day (every day) determination. There are people who need more than sheer determination, but enough don't. They simply need Elon Musk level determination.
> They were struggling with accurate product categorization and had heard about my sharing on the Kaggle competition.
Use the right status symbols. It used to be university, now it is being at the top of a competition.
> My family and closest friends thought it was risky. However, deep down, I knew I would regret not accepting the offer.
A potential pitfall that he avoided. I've been heavily hit by this and kind of stalled my life for a year. So yea, he could've listened to them. I know I shouldn't have, but I did. He didn't and pushed forward.
> The failure and embarrassment was very public. But so was the recovery and success.
One that I know from my personal circles: go to a public embarrassment that isn't yours and no one dares to touch and make it into a success. It's not easy to do but if you can do it, then do it, in the right company this will transform your life. Or at least, it transformed the lives that I know of whom did it. They didn't do it consciously though, it's upon reflection that they realize.
> But once in the field, there were PhDs with more experience around me—why did I get promoted above them?
When I read this I'm simply thinking that he didn't get promoted above them. Being a VP is something different, I imagine, than being a data scientist, so different skills are at play. Yes, you need technical competence but you also need VP skills. Let's see what he says.
> The measurable value I created was 3x that of an average data scientist.
Well, he could also communicate that. Could the avg data scientist do that? I don't know, I bet half of them couldn't or wouldn't care.
> I was promoted to be a role model and to mentor the team to deliver and communicate better.
Like I said, VP skills.
-----
To close:
I think he's right in his assessment, simply by looking at my successful friends, family and acquaintances and by constantly asking what they do.
I'd summarize his success as:
A) Get in (I'm struggling super hard at this part, "you can problem solve but have too little experience" is what I get, it's depressive, it's a key stage)
B) Create trust with everyone
C) Be the best pick for the new position out of your team
From those key stages one can distill that technical skill is initially important and communication later.
-----
If anyone wants to hire me as a mix between a hacker and programmer anywhere in the world let me know! I have 1 year of work experience in software engineering (excluding the bootcamp I taught for a year). My Hack The Box profile is looking to be more impressive every day.
Getting to VP level at a "top startup" has little to do with data science, and everything to do with your ability to play the networking game. Depending on the size of the company, VP level jobs are more about management skills than technical proficiency.
Why does this community (HN) have such a proclivity for these types of bitter responses?
This comment makes wild assumptions, doesn't assume best intentions, and really just makes `brenden2` sound like a nasty person. Yet here it is, top comment on the front page.
Nice work OP! You are flexing a very unique set of muscles (technical + leadership + communication + ambition) that very few people have. You will go far in life.
Why do you perceive it as bitter rather than simply factual observations about employment? It is a whole discipline of social science research, eg “Moral Mazes.”
My personal goal is for my beliefs and my participation in a community to above all be accurate. If the fact-based conclusions aren’t very pleasant, that is a function of the state of the world, not the state of participation.
You seem to be reading a bitterness into the comment that is not there. Apart from that your comment is also very true. The op will go very far! Big picture, networking, and management are far more valued than someone with raw technical ability who misses the above.
Indeed. If your goal is to become a VP at a successful startup, you're better off working on your networking/blogging/social/managerial skills than studying anything technical.
Tech startup as a phenomenon of the last 25-ish years is maturing and has become sufficiently mainstream that it has attracted a very wide spectrum of people, so much so that the majority of them are - as you would expect - quite average. It turns out that being average is not enough to beat the odds, and those are quite stacked against early-stage startups. So you end up with fairly low chances of success and the resulting bitterness. As this industry continues to mature, the next step will be calls for regulation, unions, etc.
None of this is new - it's happened to all industries after they became sufficiently mainstream. Take a look at aviation - at first it was the wild west at the beginning of the century, then WW1 and WW2 brought some rapid advances, and in the last 70 years things have been relatively stale. Anyone dreaming of designing the next airliner today is a very different person from those who designed them at the beginning of that cycle.
I am not even saying there's anything wrong with all of this. As an industry matures and becomes more mainstream (aka, affects more people), we have to put some safety mechanisms in place. That means discussions become more about safety and less about achievement, and this attracts a fundamentally different group of people - more average, more bitter.
Kudos to the OP: just like syndacks, I am rooting for him and hope he'll use his enthusiasm to continue to shoot for the stars!
> Tech proficiency is largely irrelevant once you go into management.
The lack of tech proficiency is even an advantage: in the companies where I worked, it is people who sucked badly at the technical job they had, who were moved to management, hoping that they might be less useless there. They are happy, they are paid twice as much as others who could fulfil their tasks, so it must mean they're good at something (something difficult to assess otherwise so money must be the right measurement). And then, since it is almost impossible to differentiate between a good and a bad manager, they could thrive a few decades going up in those roles, jumping from a company to another, boasting about the number of projects they 'made' (which are naturally more numerous than for the people who actually worked deeply on them).
Strange comment. How could you be a good data science leader if you are not a good data scientist? Could you equally be a good CFO if you didn't have technical skills in finance?
My best manager was my least technical manager. He was really good at getting the right work to the right people, controlling expectations, stopping the clients from burying us under requirements, stopping us from hiding behind poorly phrased requirements, communicating setbacks early to turn them into course-corrections rather than trainwrecks, and juggling lots of constraints and dependencies while planning and rescheduling. He wasn't even an interpersonal wizard, but he was really good at keeping his part of the organization running smoothly. Nothing fell through the cracks while he was in charge, and we were all better for it.
This lies in stark contrast to the three other managers I've had for any length of time, who have all been considerably more technical and considerably less good at those other things. They've been able to step in and help fight fires in a way that the above manager couldn't have, but they haven't been as good at keeping their part of the organization running smoothly. Thing is, the latter is a manager's job, and the former isn't.
I mostly see management and implementation as orthogonal skills, with the caveat that experience is industry-specific in both cases.
Im afraid that while this seems logical, it's not how the real world actually is. I've known of many heads of departments with rather grand titles and only a very small percentage of them even know the slightest of what goes on below them or why.
Heads of IT, software departments and even CIOs that cant even login.
In fact, they often move to similar roles in other companies because the new companies believe they must be good to be in that seniority of the current company. When in fact the current company just can't wait for them to go.
After a couple of these moves, they have a whole career of senior positions in immpressive companies.
I can think of about 10 examples within ftse10 and fortune25 companies right now that I've had the "pleasure" to work with.
>How could you be a good data science leader if you are not a good data scientist?
The job of a Data Science leader at a growing company is 50% recruiting and 50% sitting in planning meetings. Even if you started with good IC knowledge after a few years your skills will be rusty as hell. So assuming you know nothing and trusting your team and delegating is going to work infinitely better than trusting your own out of date skills.
edit: And yes, this is from personal knowledge, I've held such titles before and I've had a lot of offers for other such titles.
edit2: As a corollary, promoting someone to management just because they're a good IC is a really bad idea. You want your top ICs to stay ICs if possible. You want your managers to be people who actually can manage and want to manage.
This exactly. I can't begin to list off all the rich dumbasses who had all sorts of titles in startups because they went to the same school, knew the founders, etc. I've had such people have no idea what model they even use for large chunks of their pipeline. Obviously they hire the actual talent and these guys get the credit.
How is this different from any other company though? It happens a lot where I live. You either know the higher ups through some sort of connection(school, event, family, etc) became friends and that's it you landed on a nice position.
I'd love to say it's only prone in startups but it's been plaguing companies forever, especially small-medium sized ones where it's easier because the owner is probably hoping to find able hands, perhaps the huge ones have slightly better processes to counter this type of scenario.
It's not different, all companies are like this. That doesn't make it any better though...
Success is mostly based upon who you know and being at the right place at the right time. That's why it's disingenuous to suggest you can "teach" how to be successful with blog posts like this -- you can't teach someone how to be born into the right family so they can attend the right school and meet the right people.
