Managing Innovation in Small Worlds

The lone scientist who makes breakthrough discoveries is more myth than reality. Thomas Alva Edison, for instance, did not invent the electric light bulb, phonograph and motion pictures all on his own. In truth, those products were the results of years of hard work by teams of researchers. At one point, Edison’s labs employed hundreds of people in a veritable invention factory. Indeed, innovation is typically a group effort, not an individual undertaking. But how exactly do researchers collaborate with one another to innovate?

To answer that, we compiled a dataset identifying all coauthorship relationships of U.S. patent inventors from 1975 through 1999. More than two million unique individual inventors and their patent coauthors were identified for that time period, and the data revealed trends that spilled across time and across company and geographic boundaries. To investigate those trends, we interviewed a representative sample of inventors about their social and collaborative networks, career moves and job changes. (The complete study is contained in “Managing Creativity in Small Worlds,” which was published in the summer 2006 issue of the California Management Review.)

From plots of the data, we discovered that the social network of innovators is a small world, with various clusters of people interconnected by different gatekeepers, individuals who bridge one group with another. The unique features of this type of social network — characterized by intense clustering with occasional bridging ties — confer important benefits in the innovation process. The tight clustering of scientists and engineers engenders trust and the free sharing of information and diffusion of ideas, thus facilitating creativity. But the downside is that the creative process of an isolated team of inventors can go stale as the individuals become vulnerable to group-think, thus decreasing the likelihood that they will come up with novel ideas. That’s why bridges between clusters are crucial, because they bring in fresh information that can help spark the creativity of an insular group of inventors.

Inventors on the Move

Historically, engineers and scientists tended to work within local clusters of collaboration that were isolated within a company (or within a division of a large corporation). Recently, though, people have become increasingly mobile, changing jobs with greater frequency, and these formerly isolated clusters have begun to interconnect into larger networks through which information flows more freely among companies. Over the time period of our study, the degree of clustering remained relatively unchanged but the number of bridging ties increased substantially. The bottom line is that firms no longer invent in isolation — they now innovate in a small world.

Such environments provide both strategic opportunity and potential threat: They can increase creativity within a company, but they also hasten the diffusion of creative knowledge to other firms through personnel and knowledge transfer. The trick, then, is to manage innovation in ways that exploit the opportunities while minimizing the risks.

Unfortunately, small worlds create anxiety for many executives. Given the expense of hiring and training scientists, inventors and other creative professionals, managers understandably resent their diminished ability to appropriate returns on those investments due to frequent job turnovers and other information spillovers. Framing the flow of information as an expected loss of people and ideas, the instinctive response is to plug the dike. One CEO in our study admitted that he intentionally kept his inventors in silos to minimize their awareness and understanding of outside opportunities. But employees who feel suffocated are more likely to become frustrated, and unhappy inventors tend to be uncreative and unproductive, eventually looking for employment elsewhere.

Instead, executives should not assume the worst when managing innovation in a small world. Rather than concentrating on staunching the outflow of talent and information, they should focus on maximizing the inflow. And one key to accomplishing that is to identify, hire, retain and properly manage gatekeepers, those technical professionals who span organizational boundaries and accelerate the process of invention by contributing to and capitalizing on interfirm spillovers of important knowledge.

Identifying Gatekeepers

In our experience, managers’ intuition in identifying gatekeepers tends to be reasonably accurate. Gatekeepers often hold advanced degrees, enjoy deep technical respect from their peers and work closely with other gatekeepers (both within and outside the organization). The best gatekeepers invent, communicate and exploit their boundary-spanning positions to keep abreast of current developments, problems and breakthroughs. They both consume, and contribute to, the scientific literature; they translate important external results for their colleagues; and they identify trends, threats and opportunities for their companies. Managers should be wary of gatekeepers who aggressively control information, because those individuals should instead be using their awareness and brokerage of different clusters to join disconnected people who have the potential for fruitful collaboration. The best gatekeepers are those who continuously make new connections and solidify the most promising introductions with cohesive clusters.

Gatekeepers might be gifted and unique individuals, but they pose several managerial challenges. Because they are creative and aware of technological opportunities, they can sometimes paralyze an organization with pie-in-the-sky possibilities. Rather than focusing on, and explaining, the most important threats or opportunities, unskilled gatekeepers generate unfiltered, unprioritized lists of ideas that can become unwieldy distractions, ultimately making the gatekeeper less effective because he or she is more likely to be ignored in the future.

Furthermore, gatekeepers are often better at generating ideas than at bringing them to fruition. Consider the common challenge of transferring a gatekeeper’s idea into manufacturing. This is often difficult because of the unique bridging position that made the gatekeeper effective in the first place; it also means that he or she is usually the only person who understands all aspects and nuances of his or her idea. In contrast, when an invention arises within a cohesive cluster, almost everyone within the group understands the idea and can explain it to outsiders. In other words, the very position that makes gatekeepers effective makes them less able to diffuse their ideas to others. Embedding gatekeepers within cohesive clusters will ameliorate some of the problems but will also make those individuals less likely to generate new ideas. An alternative solution is to establish strong incentives, including financial bonuses, to reward gatekeepers who fully transfer their ideas and complete projects.

Managers should remember that gatekeepers create the greatest value when they can contribute to corporate strategy. Specifically, senior gatekeepers are in the best position to assess the threat or opportunity posed by an external breakthrough. Such individuals can foresee a disruptive change from a much greater distance, and their early warnings and technical insights can greatly increase a company’s ability to deal with, and even profit from, the disruption. For that to happen, though, they must have some input on corporate strategy, perhaps through a chief technology officer or other senior-level executive.

Gatekeepers as Star Athletes

Even so, managers need to be realistic about their abilities to retain gatekeepers. Like star athletes, gatekeepers are usually adept at exploiting opportunities, both within and outside of their current companies. Thus executives should be open to less conventional means of retaining valuable gatekeepers. For example, a disruptive technology might best be pursued initially outside of an incumbent company by spinning off that operation along with the gatekeeper who identified the opportunity. If that venture proves successful, the original firm is then in a much better position to take advantage of the success through return on the investment, ease of acquisition or purchase of the resulting technology or product.

Perhaps the most important thing to remember about small worlds is that they are like fishbowls. Information travels quickly, and everyone soon learns about what others are doing. That holds true not just for technical know-how but also for management practices. Companies that constrain their gatekeepers too aggressively will quickly gain a bad reputation, impairing their ability to hire top talent. In contrast, a firm that becomes known for its enlightened management of technical talent can benefit enormously. And individuals, too, build widespread reputations in a small-world fishbowl, making it easier for managers to identify and assess prospective hires to find the technical expertise they need. Such are the new dynamics of innovation in a small world, in which knowledge flows freely and talent seeks opportunity.