Learning the Art of Business Improvisation

The ability to innovate and rapidly respond to changes in the business environment is critical to competitiveness and success. Creativity and problem-solving skills are key elements of improving the outcomes in projects that require innovation. Iterative development, improvisation, and experimentation combined with focus and flexibility are needed to identify new business opportunities and effectively execute projects. But business demands also require efficiency, and many organizations pursue disciplined project management and product development practices that emphasize standardization and consistency to help drive down costs.

Is a disciplined approach to product development at odds with creativity and improvisation? Can managers develop skills around building improvisation and creativity, especially for innovative projects? If so, what are the right conditions for improvisation to flourish? In this article, we discuss findings from our study on improvisation in product development projects and how managers can create a team environment conducive to improvisation.

Improvisation Fundamentals

In general terms, improvisation is the ability to create and implement a new or an unplanned solution in the face of an unexpected problem or change. It is often seen as a spontaneous, intuitive, creative problem-solving behavior that mostly happens “on the fly.”

Improvisation has been studied for some time in the fields of music, psychology, and education. In the theater, it is often seen as a “pure” state of creativity, in which a team or individual may rely on intuition and spontaneity to come up with an action or arrangement. In music, researchers have identified different skills that when combined form the core competence for improvisation, including problem solving, communication and expression, proper use of language, creativity, and visualization abilities. Some have suggested that improvisation should itself be a discipline and be officially taught in music schools.

Improvisation has also been studied in organizational strategy and product development. Studies have found positive correlations between improvisation in product development and team performance. It is considered a spontaneous behavior (collectively or individually), and therefore dependent on team members’ attitudes, experience, motivation, intuition, and individual skills. Despite a number of studies on improvisation in the management context, there is no consensus on the most effective approach to develop this competence in project teams.

As a part of an ongoing research program to understand the principles of agility and adaptability in project teams, we examined project and team characteristics related to improvisation practices in product development, software development, and the implementation of software projects. We surveyed project managers, program managers, portfolio managers, and project team members from 17 different industry sectors and 76 countries — 856 professionals overall. One of our primary objectives was to understand the key factors associated with improvisation capabilities in project teams and their potential correlation with agility performance.

In the survey, we measured “improvisation practices” without explicitly mentioning or using the term “improvisation.” We examined three different situations: (1) The project tasks were not assigned to the team members at a detailed level, but the team was provided with a macro vision of the project and had to figure out what tasks and activities had to be executed; (2) the project team applied an iterative learning-execution approach to deal with unexpected changes and uncertainty during the project life cycle; and (3) the team combined new approaches (practices, tools, and techniques) to address different types of problems, changes, and opportunities during the project life cycle. In addition, we measured a number of team and organization characteristics that are related to agility performance.

We also asked respondents to self-declare the type of management approach (agile or traditional) that represented the majority of practices and tools applied to manage the project. The traditional or “waterfall” project management approach is mainly focused on long-term planning, requirements, stability, and well-defined and phase-oriented deliverables. Agile project management, by contrast, relies on iterative and incremental planning and execution processes in which requirements evolve and changes are quickly absorbed.

Improvisation in Innovation Projects

Innovation projects are inherently challenging to execute because of uncertainty and evolving opportunities to improve the ultimate solution. In general, these projects may have significant unknown requirements, volatile scope, and unidentified risks; the technology may be evolving continuously, and therefore the team may be unable to anticipate all tasks and outcomes. In order to succeed, the project team must be able to respond rapidly to changes, recognize opportunities to improve the product, and deliver results under time and cost pressures. They need to come up with solutions that may not have been planned or previously identified, adapt the project, plan accordingly, and get the work done.

We found that projects with extreme changes in requirements (90% or more changes) employed 41% more improvisation practices than projects that had relatively stable requirements (10% or fewer changes). This suggests that higher levels of improvisation, deliberate or not, are more likely to happen in projects that have fluid and unstable requirements. Furthermore, we observed that teams working on projects in which team members don’t possess significant prior experience or technical knowledge were more likely to improvise than were experienced teams working on more familiar projects or technology. That effect appears to be consistent across different types of projects. For example, projects with greater levels of innovation (for example, a product or software totally new to the market) are more likely to result in greater levels of improvisation.

Overall, we observed that demands for more innovation, more changes in requirements, the degree of team experience in similar projects, and the degree of knowledge about the technology all were predictors of the level of improvisation displayed by the team. Each of these factors is a threat to project plans and could therefore trigger the need for improvisation by the team.

Improvisation might be simply a natural response to unexpected problems or new challenges. Or it might be part of a deliberate strategy for a team that understands that its product and project objectives will likely change during the course of the project. In either case, a team that is more skilled at improvisation will likely have a performance advantage in such a dynamic environment. Can the improvisation capabilities of the team be enhanced so that it is better able to more creatively adapt to unexpected changes in a project and turn a challenge into a source of competitive advantage? How might managers promote the right conditions to enable improvisation to occur when needed?

Creating Conditions That Foster Improvisation

Based on our research findings, we believe that the capacity of a team to improvise can be developed and enhanced. Focused effort in three areas can help develop these improvisation competencies.

1. Build a culture that recognizes and views changes positively. We found that greater levels of improvisation came from teams that displayed a positive attitude toward dealing with and accepting ambiguity and project changes. They also displayed a higher level of autonomy in making decisions.

In fast-paced and innovative project environments, teams should be empowered to make decisions locally, be informed about and willing to take risks, and not overly fear potential failure. This will help them to reduce uncertainty more quickly and effectively learn from their experiences. Teams equipped with a broad array of tools and techniques can use them to respond to different types of challenges. The focus should be on helping teams anticipate and recognize changing circumstances and make more rapid and accurate decisions.

2. Create the right team structure and project environment. Project teams with greater improvisation had more frequent meetings and interactions with project leaders and key stakeholders to discuss project-related issues. These meetings occurred at least once a week — and in some cases, every day. The meetings allowed more frequent face-to-face interactions focused on the project and improved communication quality. That, in turn, enabled teams to respond more quickly to changes.

We also saw greater levels of improvisation in smaller teams that displayed more self-directing and self-organizing characteristics, such as being responsible for monitoring and updating the status of their activities and deliverables. This allowed the project manager to stay focused on the aggregated information and on more strategic issues related to the project.

3. Provide management practices and tools that facilitate improvisation. Not surprisingly, teams with greater improvisation characteristics were more likely to use agile management approaches, techniques, and tools. In fact, teams that embraced an agile approach were nine times more likely to have high levels of improvisation compared with teams that used a more traditional (waterfall) approach.

The agile methods we observed in the teams with higher levels of improvisation included iterative development, supported by recurring delivery of higher-value deliverables; constant interactions between stakeholders and the project team; the use of visual tools to collaboratively manage the project with team members; and active involvement with the client and/or user in the development process.

Deliberately or not, project teams are improvising. Improvisation can foster problem solving, creativity, and innovation, and it is becoming a requirement for many organizations. Although improvisation might seem to be spontaneous and intuitive, to do it well requires the development of disciplined and deliberate processes and capabilities. Managers working in dynamic, fast-paced, and highly innovative project environments should develop and refine capabilities in these three areas to create a project environment that will enhance a team’s improvisation competencies — ultimately with an eye toward improving project results and innovation.