Talent Versus Teamwork

Stop us if you’ve heard this one before: The ragtag group of underdogs overcomes the more skilled favorite thanks to nothing more than their belief in one another. That popular sports movie cliché may feel unrealistic at times, but when it comes to building a team in real life, the value of cohesiveness and chemistry is increasingly measurable and provable. Whether it’s the NBA’s 2004 Detroit Pistons, the 2016 Leicester City Foxes soccer club, or the miracle 2003 Penrith Panthers of Australian rugby, there are many examples of the right players in the right system doing something seemingly impossible. But is it actually possible to quantify team chemistry — and if so, can such assessments really make a difference on the field? We speak with Simon Strachan of Gain Line Analytics to find out.

Transcript

Ben Shields: After finishing dead last during Australia’s National Rugby League season in 2001, and 12th out of 15 teams in 2002, the Penrith Panthers were certainly not expected to do much of anything in 2003. In a sport that demands such a high level of physical and technical skill, the pundits felt that Penrith came up a bit short, and three early-season losses did nothing to disprove that notion. But despite the lack of victories, the Panthers were building something under coach John Lang, who had taken over the year before. The core of the team had not been hastily assembled in a desperate grab for wins but had been built steadily, allowing for important partnerships to form.

Paul Michelman: Players like Luke Lewis, Luke Rooney, and Joel Clinton had emerged as key contributors with time to develop together. Even the players shooting up for Penrith for the first time were carefully selected. For instance, Preston Campbell had been mentored by Lang at their previous stop. But the best example was Rhys Wesser, affectionately known as Rhys-Lightning, a fullback whose playing time had increased every year and who would set the single-season record for tries by a fullback in 2003. The Panthers had put the puzzle pieces together one by one, and the finished product exceeded even their wildest imaginations. Penrith shocked the sport by winning the NRL premiership, and the success of their underdog group, built with teamwork in mind, still resonates to this day. I’m Paul Michelman.

Ben Shields: I’m Ben Shields, and this is Counterpoints, the sports analytics podcast from MIT Sloan Management Review. In this episode, we dive into the world of cohesion analytics and try to answer: Is it the talent, or is it the team that leads to titles?

Paul Michelman: Stop me if you’ve heard this one before. The ragtag group of underdogs overcomes the more skilled favorite thanks to nothing more than their belief in one another. That popular sports movie cliché may feel unrealistic at times, but when it comes to building a team in real life, the value of cohesiveness is increasingly measurable and provable.

Ben Shields: Whether it’s those Penrith Panthers, the 2004 Detroit Pistons, or the 2016 Leicester City Foxes, there are many examples of the right players in the right system doing something seemingly impossible or at least highly improbable. And on the flip side, the cautionary tales are also plentiful. The dream team 2011 Philadelphia Eagles, or this past season’s Boston Celtics show how a talent-laden group can go off the rails.

Paul Michelman: But is it actually possible to quantify what we still refer to as team chemistry? Ben spoke with Simon Strachen of Gain Line Analytics to find out.

Ben Shields: Well, it is a pleasure to welcome onto our show Simon Strachen, who is the cofounder and GM of sport for Gain Line Analytics. Simon, great to have you on the show. And here you are calling us from Melbourne, Australia. Is that correct?

Simon Strachen: Ah, that is, yes. It’s a lovely cool morning here in Melbourne.

Ben Shields: Well, the opposite is true here in Boston, and it gives us great pleasure to say it. All right, we have you here to discuss the thesis: Talent is overrated when it comes to team success. So basically you’re coming on here to tell us that the holy grail of measuring team chemistry is possible. Prove it to us, Simon.

Simon Strachen: Ah, yes, that’s right. And that’s really sort of the basis of the work we’re doing at Gain Line Analytics. And really sort of the fundamental background to that is that when people look at talent, and when people look at skill, there is a lot of baggage that comes with that skill and talent, and that’s sort of the hidden side to it that’s not necessarily picked up in traditional analytics. So a way to look at this is: From a traditional standpoint, most analytics look at skill and output and the score in a way. So we don’t do that ourselves. We focus on a level of understanding between teammates. We call it cohesion. And what we’ve done is we’ve developed a set of metrics that measure the objective level of understanding between players. So when we use the term cohesion, it’s not like social cohesion (whether individuals like each other or how they get on), it’s more around the objective understanding that players have.

So a way to sort of visualize this is: If you have two teams of equal skill, one team that’s never played together, and one team that, say, played a season together — if those two teams play against each other, it’s hard to argue that the team that’s played for a long period of time together is going perform better than the team that’s never played together, because they have a level of understanding between each other. Now, that level of understanding — it’s what we actually call the “3 Ps.” So they have a level of understanding between their teammates in the way they play. They have a level of understanding in their particular role or position they play within their team. And they have a level of understanding of the program such as the game plan — the way the coach wants them to play.

So that sort of idea is not necessarily revolutionary, and it’s a version of shared experience or human capital. But just purely from a skill model, if you were to look at those two teams, and you measured it on skill, those two teams would actually look identical because they have the same amount of skill. If the team, for example, that had the high levels of understanding, if you removed one of those players and replaced the player with the same amount of skill — effectively, from a skill perspective, again, it makes no difference. But from an understanding perspective, you’ve actually not just removed skill, you’ve actually removed a level of understanding from the teammates, and you’ve replaced it with the same amount of skill. But what you haven’t done is you have not replaced a level of understanding; so you’ve actually reduced the amount of understanding in that particular group.

And this is similar to some of the work that Boris Groysberg out of Harvard Business School did in his book Chasing Stars, where he looked at stockbroking analysts and their movement around Wall Street in the transition of talent between stockbroking firms. Basically, what we’ve found is that when players move between different environments, it actually impacts their performance in that way. So skill in itself is actually a function of the relationships that they have in the team. And this is specific to the type of position — the type of role they have with their team. So, in fact, this has a bit of similarity to a previous episode that you did with Cade Massey on trying to measure skill in the NFL. When we look at, say, Groysberg’s work, we look at that in a similar fashion. So a lot of that work that we’ve taken — Groysberg’s work, work out of the military, work out of NASA, work out of surgical crews, theater crews — in understanding how the teams work in that way, to understand how the individual reflects their performance within that group environment, [and] to understand that the individual skill itself has a certain amount of output, but when that output is in an environment where there is a level of understanding, the output is actually different.

Ben Shields: Simon, I’m with you conceptually, and I think a lot of our listeners are with you conceptually. I know when I was 10 years old, and my best friend and I were playing a 2-on-2 basketball game against similarly skilled people, we were going to win more often than not. And I think a lot of our listeners know that when you play with someone you know and you have a strong objective level of understanding with their teammates, you are going to be in a better position to win the game.

I want to dig into how specifically you’re measuring this, because you mentioned at the top that you’ve developed a set of metrics to understand cohesion quantitatively. So can you talk us through a little bit of your methodology, so we can understand a little bit more about how you measure this?

Simon Strachen: When I said right at the beginning that talent comes with baggage, and there’s a hidden side to talent, that is the difference [in] the way we look at this compared to traditionally the way other people have looked at the idea of shared experience of human capital. And that’s one of the critical sides of what our methodology is. So when we look at our metrics, what we try to understand within our metrics is when we look at talent, we also look at their experience as a player. And there are two types of experience: There’s experience within the environment that they’re in, and there’s experience outside the environment. And both of those levels of experience have different impacts on performance. And so we actually take those into account within the metrics, because they actually influence the performance of the individual and the teammates around them.

So we actually have a metric called TWI or teamwork index, which is a way of measuring a team squad, and it’s a reflection of the recruiting philosophy of an organization. In the very simplest way, it’s the ability to measure a team’s philosophy — for example, if they are a building team or if they are a buying team. Do they develop their players internally? Or do they go out to the market and buy them and bring them in? And so the way we do that is to actually understand whether or not the amount of experience they have — whether the experience is external, whether the experience is internal — and then use that within the algorithm to actually come up with our metric for the sport itself, TWI.

So TWI itself is a really good long-term measure for [the] performance of the squad. If your TWI’s high, you’ve got a high capacity to produce in-season cohesion — a game by game cohesion, what happens weekend by weekend. It doesn’t necessarily mean you always will have high cohesion. If you’ve got injuries, if you’ve got a certain selection criteria, you’re in-season cohesion may not necessarily be high, but you’ve got the capacity to produce high in-season cohesion. And I’ll explain the inputs of those metrics. If your TWI is low, if your recruitment philosophy is in a certain way that you don’t have high TWI, you don’t have the capacity to produce high in-season cohesion in that way. So when we talk about in-season cohesion, ultimately it comes back down to selection criteria: How are you selecting those particular players? And that is a version of shared experience in itself. It’s a version of measuring human capital — a version of measuring shared experience. But the way we look at it, it takes into account those 3 Ps. So it’s not just a case of measuring all the people together, it’s measuring the 3 Ps: people, position, program, and also the concept of diminishing returns. So the game you played last weekend has a different weight [than] the game you played two years ago, which is really important as well. There are a number of different factors that go into creating the metrics itself, and so this has been about six years in the making. There’s been a lot of trial and error to produce the metrics and over the sort of different sports. And different sports have different dynamics.

Ben Shields: Yeah, I think that’s a really interesting point. So am I understanding correctly that each team that does the TWI has its own TWI score? That is, they have to have a benchmark score for their own recruiting philosophy? And then once you have that score, you’re able to understand whether or not a player would be a good fit for cohesion? Am I understanding that correctly? That each team has a different TWI score?

Simon Strachen: Yeah, that’s right. So within a particular league, each team will have their own specific TWI. And each league is different. It’s often controlled by, say, the trade policies within the league. Some leagues have very draconian trade policies. Some leagues have very open trade policies. The natural instinct of sports teams is that they will go to the market. Some leagues choose to try and restrict that. Some leagues are quite open with it. Some leagues have salary caps; some leagues have soft caps; some leagues even have operational caps where the actual teams’ operations can only spend to a certain amount. So all those different factors actually influence how teams can actually create TWI. So when we talk about the internal-external experience, the real critical factor here is what happens with that experience. If you basically took virtually any sports league around the world and measured performance by [the] experience of the team, you actually find that there’s very little relationship between experience and sort of the amount of games that someone’s played and actual results.

But the critical thing is to understand where that experience [is] from. And that’s one of the keys to it, because when you’ve got experience, you learn your experience; you learn particular habits; you learn particular playing styles; you learn particular ways of doing things. If that experience was actually learned in a different environment…. Say you’ve got a very, very highly skilled, highly talented player, and you’ve recruited them into your environment. And they are essentially a product of another environment. So they are a product of the relationships around them, which is [consistent with] the findings of Groysberg, which is similar to what Cade Massey said around understanding that skill. You bring them into your environment as well. Generally what we found in our work: They will underperform for a period of time. Some players never actually get back to their previous level of performance. And one of the reasons for this is that they’ve got a set of ingrained habits that they have basically learned their skill set with, and it’s very hard for them to break those particular habits. So you can train with your new team; you can practice with your new team, but come game day when the pressure is on, generally you will fall back to what you know. And there are lots of studies around short-term [and] long-term memory. What happens to someone when they’re under pressure and stress [is] that basically it’s very difficult to access short-term memory, and basically you fall back to what you know. And some of that work is sort of explained in some of Daniel Kahneman’s work, especially in his book Thinking Fast and Slow, and sort of System 1 and 2 — those concepts about how people think.

And there’s a really good video: I recommend this video on YouTube, and it’s called the “Backward Brain Bicycle,” by Destin Sandlin. It’s a really good example of the difference between knowledge and understanding. He basically makes this bike where if you turn the handlebars right, it steers left. If you turn the handlebars left, it steers right. And it goes through this whole process of him learning how to ride this bike. And as he explains in it, he knows how to ride it, but he doesn’t understand how to ride it. And there are some really good points in here about the difference between knowledge and understanding. And this is a similar concept to what these external knowledge and external habits — how they influence a person, especially when they’re under pressure in that way.

The other critical thing here is that when people look at: OK, we brought this new player into our environment, how will that player adapt to the players around them? What’s rarely ever looked at is what will happen to all the other players with that new player as well. And so there’s a period of adaption with the other players as well. So this is where, say, with Groysberg’s work, he looked at how [when] an individual [transferred] into another environment, that affected the individual. But there’s also an influence on the group that that person moved into, because they have to adapt to suit that individual as well. So if you’ve got a super-duper star who’s moving into a new environment, the people around them have to adapt to that person as well. That has an influence on overall performance. There’s always a period of adaption with everybody else as well as that individual. And so all those particular factors — that’s what we have attempted to try and introduce into our algorithms to actually identify how cohesion (as we describe it, that measurement of understanding) actually influences performance.

Ben Shields: So let’s talk about a concrete example, because we have a good sense of your approach. We now understand more about what you’re measuring, specifically with TWI. Talk about a team that has benefited from this metric to improve the performance of a team.

Simon Strachen: So a good example for us is Tottenham Hotspur in the English Premier League. If anyone knows Tottenham’s history over the last couple of years, they’ve actually made what we describe as the big six — the top six in the Premier League. They’re the teams that have the most money. Pretty well over the last dozen years, you could line the Premier League up with how much money they’ve spent. You know, there’s a pretty good correlation between salary and performance in the Premier League. If you’ve got enough money, e.g., Manchester City, you can win the league, or you can be pretty successful. So Tottenham Hotspur has a reasonable amount of money, but they don’t have as much as the Liverpools, Manchester City, Manchester United, Chelsea. While everyone else was signing players, they hadn’t signed anybody over the last couple of years. So their TWI and their in-season cohesion benefited from this significantly compared to the other teams. So they did OK in the Premier League over the last couple of seasons, but they actually made the Champions League final this particular season, based on what we believe is because of their high levels of cohesion. So if you look at skill purely on salary, they don’t have a higher skill base [than] a lot of the other teams that they were playing against, but they had much, much higher cohesion. So they benefited in that respect.

Ben Shields: How does a coach use this information? So they get data about the team as it’s currently constructed. How does a coach, for instance, use this information to improve team chemistry?

Simon Strachen: So ultimately, the coach is the last person who gets the information. Because TWI is actually a governance tool, it’s really what the organization uses to develop their squad. So it’s not necessarily about the coach, it’s about the organization saying: What do we want to be? Do we want to develop our team? Do we want to buy in skill? So it’s around the organization — the board, the owner, the director of football — saying that this is what we want to do in regards to developing our squad. And there needs to be alignment in the organization, and then the coach is just one of those people within the environment itself. So board owner, GM, recruitment manager, list manager, and then the coach is there as well. So everybody knows what the plan is. And [these are], unfortunately, just the vagaries of professional sport. The coach has a contract, and he might have 18 months left in his contract. He knows he has to win, and so he’ll be making short-term, or she’ll be making short-term, decisions around performance instead of the sort of long-term alignment decision.

I just wanted to make that point that when we talk about cohesion, when we talk about TWI, we really talk about ideally having alignment through the whole organization. So from a coaching perspective, really what TWI and the rest of the metrics allow you to do is actually understand [the] context of performance. If you’ve got low TWI, if you’ve got low in-season cohesion, just basically because the way the organization is recruited, even sometimes the best coach in the world cannot necessarily get a high level of performance, because the team has a certain amount of performance capacity. So it’s not necessarily a reflection of the coach itself in the level of performance. It’s basically [that] the team only has a certain amount of capacity. So it allows people actually to make decisions around performance. And likewise with players as well.

So from a short-term perspective: These are our metrics versus the opposition. This is the estimated outcome based on the markers. This gives us a good understanding [of] whether or not we should have won or lost. And this gives context [to] that performance — the opposition were actually very, very good. So the fact that we lost is not necessarily a reflection on our players, just that we don’t have as much understanding as the opposition. So this is the good thing about our cohesion analytics markers: That in some sports, we actually have the ability to predict outcomes better than the market. In some sports, we’re on about the market, and in some sports, we’re actually worse than the market.

We use the market, sort of the betting market, as a way of saying what do people think the outcome of the game is going to be, because it allows us to identify, if we get it wrong, what are the outliers, what are the areas that we’re missing ourselves in games. And the fact that sometimes in sports our analytics allow us to predict better than the betting market is irrelevant for us. It’s more the fact that just using pure cohesion markets — the fact that we can even predict close to sophisticated analytical markers like the betting markets — it sort of shows how strong cohesion analytics is in that way.

Ben Shields: Right. So that’s an interesting point. So you’re taking a look at the relationship between cohesion and whether or not it can predict team success. And that makes total sense.

Simon Strachen: Yeah.

Ben Shields: And I like your point too about the fact that you’re aligning the entire organization around the TWI. That makes sense as well. What I’m still struggling to understand a little bit is in what is being measured and what is being generated. Is it a number on a 100-point scale? Is this survey data? Is it personality-based data? Because this is such a huge topic, Simon, so I don’t mean to be digging in too much here, but when we’re talking about measuring team chemistry, I want to make sure our listeners fully understand how exactly you’re going about it so that we can advance more as a field.

Simon Strachen: We don’t use any psychological data. We don’t use any skill or performance-based data. All we use, in the very simplest form — we obviously throw it in our big, bubbling cauldron of algorithms, and give it a big stir — but in the basic, simplest form, we use play history and team selection, team lists. Because it really comes down to the level of relationships between players. It is an objective level of data. Where has a player played? Has he played with these other players? How long for? Where else has he played? Have they played with other people? What positions have they played? What’s the level of relationship of those particular positions? So in its very simplest form, it is just basically player history.

Ben Shields: That’s very interesting. That’s very helpful. So that leads me to this question: What are you not measuring today that you wish you could measure?

Simon Strachen: Yeah, that’s a really good question. I mean, ultimately, we can only measure what’s available to us. And luckily for us — and this is the great thing about the work we do —the majority of our data is available in the public domain. People in the States are actually quite lucky because a lot of analytics data is available in the public domain. There are a number of sports here in Australia where a lot of the really good meaty data isn’t available because it’s held up in sort of the sports analytics providers. In, say, the NBA, we know the complete playing history of an NBA player, and we know where they played at college, but we don’t necessarily have a complete set of data where they played at high school. And some of the college stuff is not necessarily as good as what we want with the amount of time played and things like that. So if we wanted to get a complete picture of the way we have it, we’d basically have a player’s playing history down to the minute. If a player has played together with somebody else, how long have they played? In what teams? For how long? For how many minutes? So we’d basically have it down to the finest decimal, finest granular point.

Ben Shields: Do you find that you’re missing anything with not having personality information or information on strength of relationships? I mean you bring up the NBA, for instance, and this summer has been amazing from the standpoint of players like Kevin Durant, Kyrie Irving, and DeAndre Jordan, who are all friends, wanting to play together on the Brooklyn Nets. The fact that they have a strong relationship — does that mean that the team is going to be strong on cohesion and play well together and ultimately improve their performance on the court? So are you missing anything from not having personality data?

Simon Strachen: Probably, but you can only take it so far. If we go to personality data — for us, at this point in time — we’re not sure how we can incorporate that in to make it objective. [The] good thing about what we do at the moment is it’s thoroughly objective, because we basically are using player history. When you start putting personality data in, it starts to border on that side of being slightly subjective. Because, basically, myself and my other cofounder, Ben Darwin, we’ve both been involved in professional sport. We know that there are environments where [on] successful teams, the players have actually not liked each other. On field, they’ve actually performed very well, but off field they wouldn’t necessarily care less for each other. So we know that the personality side — if everyone gets on, it’s a wonderful environment, and isn’t that lovely. But we know that’s not necessarily the key to every sporting success in the history of the planet. So we’re not necessarily focused on that. You know, everyone talks about culture. Isn’t that wonderful, you know, if everyone gets on? And that’s beautiful, but we know that’s not necessarily the driver, because there are plenty of examples of teams where players don’t get on, but they’re still successful because they’ve had a high level of understanding.

Ben Shields: And to that point about drivers, is there one more-important factor in team cohesion than another?

Simon Strachen: We work a lot with a lot of professional coaches, obviously at sort of the point end of professional sport in Rugby Union, Rugby League, soccer, a bit of cricket… And the more we talk to coaches around this, the more coaches want to incorporate what we do, not necessarily in the recruiting, but on the training paddock.

What we find is that it’s very hard to replicate the pressure and stress of actual game time on the training paddock, because training is when you really get put under the true pressure of a game situation. In a game, that’s when you understand what the person next to you is going to do. It’s very hard to replicate that at training. But if you can replicate that at training, then that’s beneficial. So during preseason, if you’ve got a choice to play a preseason game or go on an army training camp, you know, we would always say do a preseason game, because that’s going to assist you in developing understanding of what you need to do in the defensive line or how people are going to play together. Army training camps are all very well and going to help your fitness and help social cohesion, but it’s not necessarily going to help you on the defensive line. It’s not necessarily going to help two guys understand if one kicks the ball to the other one, where they need to be, in that way. So actual real game time is very, very important. But then simulating that at training is also very important.

Ben Shields: Yeah, that’s a great point, because it does seem like so much of team cohesion is context dependent. You’ve got to be able to perform in the context of a high-pressure situation. So it makes total sense.

All right, you’re clearly doing some very interesting work in this field. I know your work is not done. So when you look ahead at the next two or three years, where do you see the research on team cohesion going? What advancements, especially for some of the researchers that are listening to this show, do you think need to be made in this field of team cohesion research?

Simon Strachen: When we first started our work, we started with the TWI metric, the one metric. We thought this is fantastic — one metric describes the world. And then we went, “Oh wait a minute, it doesn’t.” So we started, basically, we had TWI — from that, we developed our in-season cohesion marker, and that allowed us to understand: Well this is the in-season for the team. We could break it down to a granular, granular, granular level. So this is actually the attack element of the team. This is actually the defensive element of the team. This is actually some metrics for the individual relationships that are actually associated with defense. These are the teams within teams of that. Then, we found that, OK, understanding the relationships of those two. So in the National Rugby League competition here in Australia, what we found is that — you know, we thought defense was the most critical part, but then a couple of seasons ago, everybody attacked. Everybody changed what we call their “attacking spine,” the people who control the attack of teams. And we found that even the worst defensive teams were actually defending quite well. What we determined was that even if you’re a really bad defensive team — ultimately, if you have nothing coming against you, if your attack is no good — then it doesn’t matter. So we thought, well, it’s actually the spine, the attacking side, that’s actually the controlling feature. So what we did: We brought a machine learning guy on to say, OK, let’s actually work out what is the most important feature here to try and understand the dynamic of: If a team is weak in defense but strong in attack versus a team that’s strong in defense and weak in attack, what’s going to determine the outcome? So the more granular we got with our metrics, the more we couldn’t just look at it and go, “OK, this is going to be the important factor.”

So machine learning, artificial intelligence, for us, is the next step, because of the amount of features we now have. The amount of granular features we now have are so many that it’s really [necessary] to understand what is the important factor. … [For instance,] I’m going to develop my squad, do I develop really strong defensively and [not] worry about attack? Or am I going to develop a really strong attacking team and not necessarily worry so much about defense? Or can I go 50/50? What is the cohesion of the overall competition looking like this season? And then, do I optimize my particular squad to suit where the competition is heading? And so that’s why the machine learning side is going to be so important, because there’s just so much going on.

Ben Shields: Well, that’s an exciting way to end. I think we have come away from this conversation a little bit smarter about ways to measure team cohesion. And Simon, we appreciate you sharing your research and insights with us, and we wish you the best as you continue to tackle this exciting field.

Simon Strachen: Thank you very much. It’s my pleasure, and it’s really great to be on your show. I’m an avid listener, and I’m hoping that, yeah, it will get some people excited around the idea of team cohesion and the work we’re doing, and get some other people thinking about it as well.

Ben Shields: This has been Counterpoints, the sports analytics podcast from MIT Sloan Management Review.

Paul Michelman: You can find us on Apple Podcasts, Google Play, Stitcher, Spotify, and wherever fine podcasts are streamed. And if you have an idea for a topic we should cover or a guest we should invite, please drop us a line at counterpoints@mit.edu.

Ben Shields: Counterpoints is produced by Mary Dooe. Our theme music was composed by Matt Reed. Our coordinating producer is Mackenzie Wise. Our crack researcher is Jake Manashi, and our maven of marketing is Desiree Barry.