How to Build (and Keep) a World-Class Data Science Team

Every team needs talented people. In data science, talented people need not only to be good at what they do individually but also able to challenge their colleagues to create effective new solutions to very hard problems.

How do you build a data science team to attract and retain this type of world-class talent?

Over the past twenty-five years, I’ve been lucky enough to lead a number of such teams, ranging in size from three to almost a hundred people. High-performing quant teams are characterized by high levels of output and extremely low levels of staff turnover.

In my experience, there are three main “jobs” that leaders need to take on to manage a first-rate data science or quant group: 1) Build an engaging environment; 2) Make sure the team has access to the resources it needs; and 3) Get their hands dirty — but stay out of the way.

1. Build an engaging environment.

Much of what motivates high-performing data scientists and quants, and what shapes a work environment for them, comes down to the outlook of their colleagues and the dynamics of their daily interactions. High-performing team members need to be comfortable giving colleagues their best ideas and collaborating to shape the most promising of these inchoate thoughts into real-world solutions, while willingly abandoning those that turn out to be less well-founded. The culture of world-class analytics teams is one in which team members (and their managers) are excited by what their teammates can do.

In such groups, team members err on the side of giving their colleagues too much credit for their work and ideas, rather than worrying whether their own contributions will be recognized. They are able to revel in discord and respect differences — which serve to test and refine ideas — while maintaining a sense of positive forward momentum.

An engaging environment is important not only for the staff but also for the company as a whole. For analytics teams to be effective, they must have continuity. The longer team members work with each other, the more they get to know the ways in which different colleagues approach problems and communicate results, and the more they learn to trust colleagues in key areas. While it is possible to hire new analysts with excellent skills, there is no quick way to infuse new members with this kind of team “meta knowledge.” (This is one reason why the same teams often work together in different settings again and again.) It is far cheaper to invest in a stimulating and high-performing environment than it is to replace a seasoned analytics professional.

Analytics groups also need built-in slack time — in part because flexibility and adaptability are important parts of what such teams bring to the enterprise, and in part to allow team members to pursue new research that may not deliver near-term benefits, but that offers an option of high value in the longer term. Having extra capacity allows team members to do the kind of innovative work that originally attracted them to the field — projects that sometimes end up leading to breakthrough products or insights. It also allows them to invest in developing analytic infrastructure, such as standardized data preprocessing tools or model validation suites, that improve team productivity and may be more generally applicable across projects.

2. Make sure the team has access to the resources it needs.

All business units need resources, and generally there are not enough to go around. For analytics teams, resources include the obvious hardware, software, and support staff. But analytics teams also need access to things like time with senior people or academics in the larger organization and beyond or to high-performance computing platforms or specialized data sets for modeling projects.

It can be hard for nontechnical mangers to evaluate the importance of, say, access to an expensive database or a subscription to an online research library. For this reason, the research team manager must be prepared to advocate for the team’s needs with the rest of the organization’s business leaders. Since such departments are often not profit centers, other executives must be reminded of the importance of allocating resources to research. One way to promote understanding is to recruit people from across the organization (not just from the business unit using the research) for project-specific working groups.

3. Get your hands dirty — but stay out of the way.

A lesson often learned the hard way is that the best analytics managers need to be able to give their staff the freedom and trust to take control of the research agenda for solving problems. It is perfectly reasonable for managers to review proposals before green-lighting them, but managers should have enough confidence in their analytics teams to let them take the lead. In the analytics domain, there are often several approaches to solving problems, and researchers often have unique takes on new problems. They need to have the freedom to implement solutions, provided the approaches they select are in line with broader group goals.

This doesn’t mean that managers should only get involved in oversight — quite the opposite. Analytics and data science teams are most successful when managers are “hands on” with the research process and have direct experience with the nitty-gritty challenges of the analytics workflow: data sets that are noisy, infrastructure that is balky, and business units that struggle to articulate their problems in ways that make an analytic response worth having.

In this respect, analytics is different from many other fields. In banking, for example, division managers generally don’t review loan applications. But in analytics, the most successful leaders engage regularly in hands-on research and continue to publish regularly even as they move up the executive ladder. By staying active in line research, analytics managers are able to hone their abilities to judge how difficult projects are and how long they will take — things they need to know to interact effectively with other people inside and outside the company.

Managers who don’t personally do research themselves often have difficulty maintaining trust and morale internally, when asking colleagues to compromise on the quality of their research. For the same reasons, they may also face challenges externally when presenting their teams’ research.

Ongoing experience in doing the work provides empirical evidence to both the team and, importantly, to the market, that a manager is both credible and able to lead. This means that in addition to vision and communication skills, the most successful analytics leaders are the ones that are also very good statisticians and programmers and, importantly, who continue to practice and publish interesting research.

I started this post by noting that data scientists are most effective when surrounded by talented colleagues. In the same way, data scientists seek and recognize strong, visionary leaders. Steve Jobs once observed that A players really like working with other A players — and they don’t want to work with B or C players. Jobs could easily have been speaking of highly talented data science teams and their managers.

This is good news for data scientists. It means that advancing in the organization does not mean giving up what they value so highly — the intellectual and psychic satisfaction that comes from solving really hard problems. It is also good news for organizations that struggle sometimes to sort out the reality from the hype about what is feasible, advisable, and valuable in data science and analytics.