Implementing a Learning Plan to Counter Project Uncertainty

In new-product development, most management approaches presume a high ratio of knowns to unknowns, and most planning defines prescribed pathways through developmental stages and decision gates. In fact, a by-product of the focus on quality and operational excellence is that companies tend to avoid uncertain situations and resist market experimentation. However, such approaches are counterproductive for any project that has the potential to produce real breakthrough innovations,1 which, by definition, are fraught with a high degree of uncertainty. Indeed, when asked about how they managed breakthrough innovation projects, respondents from a variety of industries expressed difficulty articulating and defining the uncertainties that confronted them and the approaches they could use to attack those uncertainties. (See “About the Research.”) In this article, we offer a framework, called the Learning Plan, that enables companies to manage breakthrough innovation by explicitly recognizing that project teams are proceeding on the basis of assumptions, rather than known facts.

Approaches to Planning in the Face of Uncertainty

In established companies, breakthrough innovation projects often survive for a long time below management’s radar screen, feeding off of resources gained through informal networks and volunteers to the cause. Yet, ultimately, these projects must come out into the open, undergo scrutiny and compete for resources. It makes sense that such projects should receive management attention earlier rather than later so as to reduce time to market and make the most efficient use of developmental resources. So why is that not typically the case? Because funding and organizational commitment require a business plan that reflects a level of certainty that simply cannot be achieved throughout much of the life cycle of a breakthrough innovation project. Companies try to cope with the multidimensional uncertainties associated with breakthrough innovation by imposing managerial discipline and an array of planning approaches. The approach used usually depends upon the level of uncertainty encountered. Projects with lower levels of uncertainty, typically incremental innovations, require one sort of planning tool, but breakthroughs, which are fraught with high degrees of uncertainty on many dimensions, require a different approach. (See “Project Planning Approaches.”)

An operating plan is particularly useful in the context of daily operations in which routines have been utilized many times and outcomes can be predicted with a high degree of certainty. Operating plans assume zero uncertainty, and successful implementation results in zero defects and “meeting the plan.” The Stage-Gate approach,2 which has been widely implemented, is well-suited to manage research projects that move forward in a fairly predictable way with their major uncertainty being technical performance of the product or process. Milestone planning3 and discovery-driven planning4 are targeted at providing tools for managing projects with somewhat greater uncertainty. Each of these approaches defines a disciplined planning and project management approach to innovation development, allowing for mistakes, discovery of false assumptions and unexpected outcomes through actions described as recycle or redirect. However, milestone planning and discovery-driven planning assume a level of clarity about the end game that does not exist for breakthrough innovation projects.

Through our research, we have developed a framework — the Learning Plan — that specifically addresses the managerial challenges associated with high levels of uncertainty along multiple dimensions in breakthrough innovation projects.5 In breakthrough projects where the shape of the ultimate market is unclear, which applications will gain market acceptance most quickly and fully are unknown and the path forward is difficult to visualize, the severity and number of uncertainties make it difficult to define milestones and the pathways to achieving them. In such scenarios, it is more reasonable and useful to identify and prioritize uncertainties that must be resolved, to define alternative approaches to exploring them and to continually assess the value of cumulative learning compared to the costs incurred. This iterative learning loop approach allows managers to decide on an ongoing basis whether the cumulative learning is of sufficient value to warrant continuing the project.

Understanding the Uncertainty Matrix6

The Learning Plan template — presented later in this article — was derived from our observations of the 12 breakthrough innovation projects in our seven-year study. We observed that project teams dealt with four categories of uncertainty: technical, market, organizational and resource. (See “Project Planning Approaches.”)

Technical Uncertainties These relate to the completeness and correctness of the underlying scientific knowledge, the extent to which the technical specifications of the product can be implemented, the reliability of the manufacturing processes, maintainability and so forth.

Market Uncertainties These include the degree to which customer needs and wants are clear and well understood, the extent to which conventional forms of interaction between the customer and the product can be used, the appropriateness of conventional methods of sales/distribution and revenue models and the project team’s understanding of the breakthrough innovation’s relationship to competitors’ products.

Organizational Uncertainties Given the length of the breakthrough innovation life cycle — often 10 years or more — organizational dynamism creates another category of uncertainty. All of the studied projects had to contend with uncertainties related to organizational issues — both within the project and between the project and its various internal and external constituencies. The latter issues included organizational resistance, lack of continuity and persistence, inconsistency in expectations and metrics, changes in internal and external partners and changes in strategic commitment. The uncertainties related to organizational context stemmed from a fundamental conflict between the mainstream organization and the unit engaged in breakthrough innovation, the difficulty of managing the relationship between them and the challenge of managing the transition from breakthrough innovation project to operating entity.7

Resource Uncertainties This emerged as the fourth category of uncertainty, as project teams continually struggled to attract the resources they required. For nine of the twelve projects in our original study, external financing made the difference between project continuation and cancellation. “Resource” in this conceptualization includes not only financial resources but also competencies. In all projects save one, the companies we studied lacked one or more competencies critical to the successful pursuit of their respective opportunities. As a result, project teams — and especially their champions — spent extraordinary amounts of time dealing with resource and competency acquisition through a variety of internal and external partners.

Even when a breakthrough project is formally established, its funding is generally unstable over time. Interest in the project waxes and wanes as decision makers and sponsors come and go. Because the breakthrough innovation life cycle typically lasts a decade or longer, a project can expect to see its supporters and sources of funds change multiple times. Consequently, project champions must be prepared to continually pursue funding from a variety of potential sources.

Implementing the Learning Plan

The Learning Plan allows a team to deal in a proactive way with the ongoing evaluation and redirection that characterizes any breakthrough innovation project, where the specific objectives are often unclear or highly malleable, or where the ultimate goal is clear but the path to it is highly uncertain.

The Learning Plan template encourages project teams to systematically examine each of the four categories of uncertainty. (See “The Learning Plan Template.”) Within that context, it helps managers to uncover gaps in knowledge and create a record of what is known, to prioritize which uncertainties are most critical and propose alternative assumptions about the reality behind each uncertainty and to find ways to test the assumptions and resolve the uncertainties as quickly and inexpensively as possible. (See “The Uncertainty Management Checklist.”) It thereby enables innovation teams to manage in such a way that deviation from the project plan is a function of systematic, disciplined learning and uncertainty reduction.

With any project, the team first must consider multiple approaches for testing each assumption and select the tests to conduct on the basis of efficiency of learning — that is, how much learning is gained per dollar spent during the test period. It is also important to select assumptions testing approaches that will satisfy managers with whom the project team must communicate.

Next, the team and those who will evaluate the team’s progress need to reach agreement on the objectives for each test and how success of each test will be gauged. Once clarity about the testing process and outcomes assessment has been established, the team should proceed with the tests. With active and ongoing monitoring, the team will be able to assess the degree to which each uncertainty has been reduced and reprioritize its uncertainty-reduction activities.

The second part of the Learning Plan template is aimed at evaluating what is being learned. In some cases, the learning derived from any single assumption test may set the project back because unexpected insights emerge and additional latent issues are uncovered. The team can choose to redirect based on the new information (which could reveal a still bigger opportunity) or, if the issue is a showstopper, the project may be closed down.

One pass through the Learning Plan — testing one set of critical assumptions — is a learning loop. At the completion of each learning loop, an evaluation with the team’s oversight board or project evaluation board should occur, during which the results are reviewed, new assumptions are clarified and the next tests identified. If the outcome of the assessment warrants further exploration, funding and resources should be tied to the plan for executing the next learning loop. All members of the team and the oversight committee need to be clear and in agreement regarding the assumptions to be tested and the testing approaches to be deployed in the next learning loop. In our experience, teams need three to four learning loops before the path forward is clear enough on any particular application space to move to a discovery-driven planning mode.

Effective evaluation is critically important for the success of the Learning Plan methodology. If the members of the oversight board are inexperienced in evaluating high-uncertainty projects, they may lack the judgment required for adequately assessing the value derived through execution of a learning loop. The easy answer is to kill the project prematurely. Further, there is a natural tendency to confront the uncertainties with which the team is more comfortable and to ignore others. This is a dangerous problem for teams composed mostly or solely of technical personnel, who generally prefer to focus on technical challenges. Failing to also recognize and confront market, organizational and resource uncertainties increases the likelihood that one of these uncertainties will turn out to be a project killer.

To counteract such tendencies, it is important for oversight boards to be staffed with veterans of high-uncertainty projects. If the company is new to this process, management should consider providing a training program for managers and technical/market specialists who may be assigned to the oversight board. It is often helpful to recruit outsiders to serve as reviewers. Insiders can accelerate their development as effective project reviewers by participating on an oversight board with outsiders who have long and deep experience with decision making under uncertainty, such as retired venture capitalists with experience in the relevant technology or market domain.

The Learning Plan in Action: Two Case Studies

Although the Learning Plan framework and process was originally developed based on insights gained from the longitudinal study of 12 breakthrough innovation projects in 10 large research and development-intensive companies, it has since been refined through direct application in several companies, two of which illustrate its benefits for managing projects involving the development of new technologies for existing markets.

Eaton Truck Group’s Hybrid Electric Vehicle (HEV) Project8

In 2002, Cleveland, Ohio-based global industrial manufacturer Eaton Corp. became an early adopter of the Learning Plan for its hybrid electric vehicle (HEV) project, housed in Eaton’s Truck Group.9 In early 2004, the project was successfully transitioned into the Light/ Medium Duty Truck Division, which is now selling hybrid electric transmissions to FedEx Corp. and other customers.

The project team was challenged by technology vice president Timothy J. Morscheck early on to examine the technical, market, organizational and resource uncertainties that needed to be resolved in order to achieve commercial success. For example, to explore market uncertainties, Kevin Beaty, HEV project manager arranged for the team to experience “a day in the life of a user” — riding along with a package delivery driver from 5 a.m. to 5 p.m. The team experienced the whole delivery route and typical set of driver activities — that is, loading packages, dealing with customers and experiencing the short hops from one destination to the next. The team soon realized that the more transparent the HEV functionality could be the better. These drivers were concentrating very hard on their deliveries and just needed a responsive and reliable delivery vehicle. The “day in the life” team recommended a concentrated effort on functionality and reliability for the real user, the driver. The fuel economy benefits of HEV and the emissions reductions were vital ingredients as well, but special attention had to be paid to functionality and reliability. The team began to work with a reliability consultant, who helped them think through a detailed set of failure mode assessments to root out unreliability and to harden the design for real-world use.

With respect to technical uncertainties, Beaty realized the team wasn’t uncovering system bugs quickly enough, so he established a “bug bounty” to pay rewards to team members who identified design issues that were causing problems or that might cause problems in the future. The bugs were tallied each week and graphically displayed in the war room of the hybrid electric vehicle project. The end result of this attention to detail has been a rapid development process for the early prototype HEV trucks.

With respect to resource uncertainty, the HEV project required technologies and technical competences that did not exist at that time within Eaton, and the Learning Plan became a useful tool for identifying missing resources that had to be acquired from external sources or developed internally. Software system engineers were contracted. Electrical power engineers were hired. Project managers were hired who had managed projects in the automotive industry where driveability assessments were practiced.

With respect to organizational uncertainties, the project ran into organizational resistance as it evolved, surviving due to the senior management team protection and the championing provided by Tim Morscheck. As Morscheck and the project team gained experience with the Learning Plan methodology, Eaton began building a competency in managing high-uncertainty projects and embedded this competency in a newly created organizational support system — a radical innovation hub. “We believe that the success of our innovation hub hinges on the progress we make on uncertainty identification and reduction among our innovation projects,” says Morscheck.

During spring 2005, Morscheck asked a project leader to use the Learning Plan — line by line — to conduct a project assessment. He then asked the project team to compare the assessment outcomes of the Learning Plan approach with the company’s Stage-Gate project management approach, looking for overlaps. Although they identified some overlaps, the team members remained convinced of the value of continuing to use the Learning Plan. In fact, when reports are presented to the project oversight committees, Eaton’s project teams currently use the Learning Plan’s framework, processes and terminology. A lot of attention during the oversight process at Eaton in the past typically had been focused on recognizing and discussing technical uncertainties. Though the Learning Plan methodology does ensure that technical uncertainties are identified, it also has helped Eaton catalogue marketing, organizational and resource uncertainties.

The Learning Plan also enables the project teams at Eaton to account for learning progress toward uncertainty resolution and enables their project evaluators to ask more appropriate questions. As noted by Vishal Singh, one of the hub’s project leaders: “It’s giving them a sense of progress on the program, in terms of what we’re doing. Our reviews are going great, because they see that we’re finding answers to our explicitly stated uncertainties. This allows for a more strategic discussion.”

In the case of Singh’s project, the team recognized as a result of undertaking the first learning loop assessment that their complete focus on technical issues was obfuscating two important facts. One fact was that they did not have some of the required skills in-house to complete much of the technical work. The identification of these gaps allowed the team to direct their scarce resources more effectively than they might otherwise have. The second was that their presumed customer was actually their potential competitor. The team quickly adjusted their priorities to develop external partnerships for needed resources and to make potential customer visits much earlier than the project team had anticipated.

New Business Development at Air Products and Chemicals Inc.

In January 2003, the New Business Development group at Air Products and Chemicals Inc., an industrial gas and chemical manufacturer headquartered in Allentown, Pennsylvania, ran a training session for all business development managers, team leaders and coaches associated with breakthrough innovation activity in the company. The NBD group acts as an incubator for and facilitator of innovation projects. The NBD leader’s network extends deeply into the business units, which have set up receiving organizations to accept and continue to grow innovation projects. Approximately 30 people attended the day-long session. Session participants were introduced to the fundamentals of breakthrough innovation, the challenges of managing high-uncertainty projects and the categories of uncertainty. They were also introduced to the Learning Plan template. In the afternoon, four project teams met with the course instructors to begin to identify the knowns and unknowns associated with their specific projects. Each team left their session with a list of uncertainties and a path forward.

The NBD leader worked with his staff to help them deepen their understanding of the Learning Plan methodology and directed them to begin to use it as they coached and worked with project teams. One project team that had participated in the training workshop allowed us to monitor team progress over the course of the next six months and to assess the usefulness of the Learning Plan project management approach.

The process of implementing the Learning Plan methodology accelerated development of market insights among technical people and involved them in market definition early in the process. It also allowed the team to take ownership for the entire project — that is, marketing and technology — rather than having each functional group owning its own piece. Instead of focusing solely on a single technical solution, the team saw the potential for creating a technology platform that could address the extended market, revealing new dimensions of opportunity beyond the initial vision. Using the template also enabled the project team to look beyond technical and market-related uncertainties and to attend to resource and organizational uncertainties. Early in the project, the team recognized the need for a different skill set than had been assigned to the project: Rather than polymer scientists, the project team needed synthesis people. The team was able to change the resource mix to meet the changing needs of the project. This decision was made by the team after verification that the existing material would not meet one of the major customer’s performance criteria.

The Learning Plan methodology allowed the team to direct the attention of senior management toward pending resource issues and to potential organizational issues related to the eventual home for the project — even to novel organizational approaches. Organizational fit was addressed by the team, which explicitly recognized that the project potentially could fit multiple organizations within the company. This catalyzed a discussion with management about the potential alternative organizational outcomes. The team also recognized the potential for contributions from internal and external partners. For example, the product envisioned in the early stages by the project team had no clear fit within the existing organization. The project team recommended to the oversight board that the new business that might evolve from the project should be set up as a separate business that, in turn, would outsource manufacturing to an internal partner — another business unit. This approach would allow the new, separate business to develop appropriate sales, marketing, distribution and service approaches — that were not aligned with the business unit that would be doing the manufacturing.

The Learning Plan template turned out to be a useful reporting mechanism, serving as a basis for monthly reports to management. Though the outputs were less quantitative than management was accustomed to reviewing, reporting the assumption-testing results within the framework of the Learning Plan template helped explain the value of the accumulating learning to senior management, which in turn helped build trust and facilitate an open discussion.

Implementation of a new project management tool, especially one so foreign to the culture of most large, established companies, typically is not a smooth process, particularly given the inexperience of the managers with the tool. At Air Products, this was the project leader’s first assignment and, according to the NBD leader, the Learning Plan was an effective development tool. The experience of being trained in and then implementing the Learning Plan was also especially useful for technical personnel new to commercialization projects.

The Learning Plan Improves Project Management — and Managers

Just as uncertainty identification and tracking must be a continual process, the Learning Plan must be revised on an ongoing basis to reflect what has already been learned and what remains to be discovered. In this sense, it is more useful as a methodology for monitoring and guiding progress rather than for rigidly controlling and directing the completion of tasks. Progress against the plan needs to be monitored by checking off assumptions that have been tested. The project team records the learning that has resulted, the decisions that have been made and the redirection that has occurred as a consequence of that learning.

Effective use of the Learning Plan and its associated processes enables rapid reforming of the project team’s mental model as rapid, low-cost experiments reveal new understanding — especially in the fuzzy front-end period of the breakthrough innovation life cycle, when the team is typically small. Inevitably, significant fluidity and fuzziness is encompassed within the Learning Plan, which in turn reflects the fluidity and fuzziness of understanding of the uncertainty matrix. In the hands of a skillful innovation manager, the Learning Plan is a tool for dealing efficiently and effectively with the uncertainty matrix. (See “Implications and Benefits for Innovation Managers.”)

The idea behind a dynamic Learning Plan is reflected in the comment of the innovation program manager for one of our case studies: “Using the Learning Plan enabled us to come to a go/no-go decision about this particular innovation project in six months. Before we adopted the Learning Plan methodology, it would have taken us two years to reach this conclusion. The Learning Plan methodology has become a valued approach to improving the performance of our portfolio of innovation projects.”