Being a data science manager isn't what you think it is
You don't need to become a people manager to be successful
First-time data science and machine learning1 managers are often selected to become a manager because they were once stellar individual contributors (ICs). Many think that becoming a manager is a step up and should be straightforward. After all, managing is just making schedules, meetings, and performance review2, and your team should do as you say, right? It doesn’t seem so hard.
Wrong.
In fact, management is not a step up but an entirely different job. You won’t be able to just bolt on some management work on top of your technical responsibilities. You now have people whose happiness at work rests largely on your shoulders and are responsible for the results but can’t do all the work yourself. In many ways you’re probably the least powerful member of your team.
As a manager, before you know it, you’ll need to give your team the autonomy and control over the work they do and trust them to make decisions and take ownership of their work3. But as you empower your team, you will soon realize how much you’re giving up your own autonomy and control, technical problem-solving, and ability to take care of yourself.
Being a manager in data science is much more difficult than you can imagine.
Losing control
Once you become a manager, you’ll be invited to a lot of meetings4 that seem like a waste of time. Your calendar will be filled with them back to back, leaving little time for other work. You could experience a severe case of FOMO when you miss a meeting, worried that you might have missed out on valuable information and opportunities for collaboration. This is one of the most common new manager traps—the endless cycle of email and meetings.
Unless you start practicing ruthless prioritization and delegating outcomes to your reports, you will lose control of your own time and deliverables. Some meetings will be important to your reports, but you don’t have to be there if it’s not a significant value add5. They can update you on the details later. You should trust your reports to make good decisions on the big and the little things related to their work, assuming you've already achieved alignment with them on a specific end outcome.
Staying technical
The amount of technical work you do depends on the needs of your team at the moment and how much of your own time you choose to invest in it. If your team is relatively small and mostly junior, you might still be able to do some technical work despite being a manager, but you will rarely find quality time to zone in and code in peace.
Over time, you will feel like your technical problem-solving skills are getting rusty, even after doing code and design reviews as part of your futile attempt to be hands-on. While you could offer advice to your team, since you aren’t doing any deep technical work, it’s not uncommon to miss the context and for your advice to end up being too superficial to be useful. This could potentially backfire in the form of skepticism and lost trust from your team.
On the other hand, you might try avoiding management work by coding because you know you’re good at it and can make an immediate impact. You’ll feel tempted to enter the fray whenever the stake is high and the workloads become increasingly onerous. You may think that you’re the only one who can truly champion the team. After all, a mistake from your team could reflect poorly on you, and you aren’t going to let that happen.
The problem with this is you’re stagnating your team’s long term growth by stealing opportunity for your team to step up and face something that scares them. You’re bounding their ability to think critically, experiment and learn through iteration. You’ll need to show your team you trust their judgement.
Ultimately, your job is to create a good circumstance for your team to do good work and then get out of the way. You need to cultivate resilience, rewarding innovation in the face of repeated setbacks and failures. Re-learn the definition of success—you’re measured by your team’s successes, not your own.
Whether you’d like to stay technically relevant or not, being a manager means you will do less of it.
Building trust
Building trust externally with your stakeholders (e.g. senior leadership, product managers, marketers, fellow data science managers) as well as internally with your team is a crucial responsibility as a manager.
When it comes to stakeholder management in a relatively immature field of data science and machine learning, it’s unfortunately common for stakeholders to wonder what your teams are up to and why it’s taking so long to make an impact. Growing out of unrealistic expectations and unclear impact is something you need to be comfortable with doing as a data science manager wherever you go.
Even when you’ve tried so hard to speak their language and understand their incentives, you might still feel like they aren’t listening to you. It’s not that you’re lacking ideas or fluency. It’s that we live in an anti-intellectual culture where we value experts only when we need them to produce something. The onus is always on the technical people and their managers to develop soft skills, while it’s seemingly okay for others to not become more data literate.
Mentoring data scientists can be rewarding, but it’s different from building trust with them. Building trust internally takes immense time and effort, but losing trust is a slippery slope. The cost of making a mistake as a manager is often greater than that of an IC. You try to help your team a little more, they call you a micromanager behind your back. You let them have all the freedom, they call you neglectful. You can’t expect to have your cake and eat it too.
Burnout is real
Being a manager is physically and mentally exhausting6, leaving you with little energy for anything else. There will be days when you feel miserable, and you will feel like there no end to the tunnel. After all, navigating through the intricacies of human emotions and relationships is much more demanding than writing lines of code because there’s no one-size-fits-all approach that works all the time. The expectation is high. You're managing a team, and everyone will look to you for guidance when things don't go right. But know that you don't need to have all the answers. Take care of yourself before you take care of your team. You are part of the team too.
Motivating
Motivating your team to do good work, even in times of uncertainty, is challenging. When there is a layoff at your company and someone on your team is let go, or when the company cuts your compensation or your team’s budget in an attempt to cut costs, for example, trying to sustain your team’s drive through a task, a project, or even a career can feel like pulling yourself out of a swamp by your own hair, especially if your motivation is tanked as well.
Some managers use the idea of “purpose” to motivate individuals, but this is a double-edged sword that could undermine the trust and respect between you and your team. When there's a lack of trust between the manager and the team, purpose is often interpreted as a thinly-veiled attempt to convince people to work more for less pay and lower title under the guise of pursuing higher mission or goal7.
Motivation is inherently intrinsic, and for it to be authentic, managers need to express candor, be patient and stay optimistic.
Ending note
Despite my warnings, if you want to become a data science manager, by all means go ahead. Being a data science manager is fulfilling in many ways—you’ll learn how managers think and the different thought processes they go through, gain experience in performance management and advocating for your team, and most importantly, learn about the importance of taking care of yourself. Know that it’s perfectly normal to not like being a manager and take up a technical role again. Career growth is never linear, and you don’t have to be a people manager to become successful.
In future posts, I plan to cover how I approach managing all these as a data science manager. If you haven’t done so yet, sign up below for free to receive my newsletter straight to your inbox 👇
👋 While I won’t make promises, I’d love to hear your suggestions on future topics. What is one thing you’re interested in reading and learning about?
Acknowledgement
A huge thanks to for proofreading this post and her helpful suggestions! Sign up for her newsletter for free doodles on data science and machine learning.
About this newsletter
I’m , and I’m a data scientist, machine learning engineer, and software engineer. is a newsletter about data science, machine learning, tech startups, management and career, with occasional excursions into other fun topics that come to my mind. All opinions in this newsletter are my own.
I loosely define data science and machine learning (DSML) as an overarching field of leveraging data and algorithms to solve a complex problem. You may be managing data analysts in guise of data scientists, serving analytics (and experimentation) needs across the company, or managing ML engineers, building products like engineering. In still many companies where DS and ML are an embedded function within the company, they are not given more priority or attention they deserve than software engineering because products are often thought to work just fine without ML and deep analytics.
The way the industry works is that management skills tend to be undervalued compared to technical skills and experience at the company. Most tech companies have a bias towards in-house conversion of current ICs into managers. This is good in a way that they have a lot of context around the product, tech and workflow, and bad that the majority of ICs, even the most talented, have no idea how to manage. Most managers don’t receive a formal training either. In early startups already in “management debt,” this kind of “just-in-time” management results in a rabbit hole of low morale, high turnover, and limited trust.
See my previous post for the exposition.
Check out this HBR article on the psychology behind meeting overload.
How should you decide if a meeting is worth your time? Here’s one solution.
To prevent burnout, I believe we should remove bullshit work and the futile systems and people that enable it.
The late David Graeber describes this in his book “Bullshit Jobs.”