Saturday, February 23, 2013

Measuring Innovation

One of the most frequent questions that comes up in conversations I have with corporate leaders is on whether innovation can be measured. This question is often followed by a second question: “Should it be measured?” and a third one: “If so, how?”


Most professionally-managed companies have a strong belief in data-driven management. Yesterday, I was at a meeting where several current and ex-Infosys employees were present, and they all said that one of the distinguishing features of a successful company like Infosys is the fact that collection and analysis of appropriate data precedes all decision-making. There is also a strong belief, reinforced by successful global initiatives like the quality movement, that what gets measured gets acted upon. So, clearly, for an activity that is as critical to the organization as innovation, measurement is important.

Arguments for and against measurement

The reason that some people are skeptical about measuring innovation is that they believe that an activity that is dependent on creative output is inherently difficult to measure. Such people believe that measurement could be counter-productive as it could result in mechanical goal-setting and playing the number games that people within organizations understand so well! And such game-playing could be inimical to the very practice of innovation.

On the other hand, I recently had a very interesting conversation with Kishore Ramisetty who heads several of Intel’s innovation initiatives in India, and who focuses on building an innovation culture within Intel’s 3,500 workforce in India. He has a simple reason for seeking relevant metrics (in his mind, preferably a single metric) to measure innovation – how else would he know if he is making progress?

My own thinking about “measuring innovation” has changed over time. I was ambivalent about measurement earlier, but have become more convinced about the importance of measurement after seeing examples of how it can work, and thanks to the influence of my collaborator, Vinay Dabholkar. However, I am equally sure that it is critical to measure innovation the right way.

The Right Way

What is the right way? In thinking about this, I am influenced by Kaplan and Norton, the originators of the Balanced Scorecard. This popular management tool was proposed by them to prevent excessive focus on financial metrics that could only be acted upon after the event. Instead, Kaplan and Norton suggested using a wider set of metrics that covered processes as they take place, thus providing leading indicators of how the organization is progressing. Ideally, Kaplan and Norton believe that a good balanced scorecard represents and reflects an organization’s theory of business: the key input-process-outcome transformations that result in value addition and appropriation by the firm.


Effective measurement of innovation needs to incorporate similar thinking. It should be based on a model of how innovation happens in the organization. Such a way of thinking has the added advantage of being aligned with the touch points in the innovation process where managerial intervention can help.

Single Metric or Multi-dimensional?

Excessive focus on a single metric can result in distortions in innovation activity that show up after some time. I am aware of one multinational R&D centre in India that underlined the importance of patents as a predominant measure of innovation happening in the centre. Employee incentives were tied to patent filing and patent grants so as to boost patent output. Not surprisingly, filing of invention disclosures grew at a rapid pace, and so did patent filings and grants. All this looked very impressive till the post-Lehman economic downturn when the corporate headquarters started looking at where costs could be cut. The large outflow on patent renewal fees attracted attention, leading to a review of the business benefit of the patents granted and the discovery that very few of the patents were critical to business. Not surprisingly, the company decided to cut back strongly on patent renewals as well as step back from support for more such patenting activity, and this led to a decline in morale within the company and setbacks to the innovation programme.

Similarly, when Procter & Gamble embraced open innovation (Connect & Develop) about ten years ago, the company set a target of half their innovations originating in ideas from outside the company. While the company was very effective in meeting this goal, this hasn’t re-vitalized the innovation engine to the extent that P&G desired (see my earlier blogpost on Open Innovation at P&G for more details).

When companies use a dominant metric effectively, it is often a market-related one (like 3M’s New Product Vitality Index or Cognizant’s measure of value added for the customer). While such a metric can work because the ultimate objective of innovation is growth, revenue and profitability, I tend to be skeptical of a single composite innovation index that mixes input, process and outcome measures together. Such indices are sensitive to the weightage given to each of these components and often end up trying to mix apples, oranges and bananas together.

Measurement of Innovation should take a Process View

In our new book, 8 Steps to Innovation: Going from Jugaad to Excellence, Vinay Dabholkar and I take a process view of innovation. We see innovation as centered around building a pipeline, increasing idea velocity and enhancing batting average. Though building a pipeline of good and relevant ideas is important, that is far from enough; ideas need to move forward, be tested and refined (or rejected), integrated and developed further, so that they move towards achieving business results. Idea velocity represents this movement of ideas through the organization, while batting average indicates how ideas and innovation projects are resulting in business impact. We believe that measurement needs to be aligned to such a process view.


Our thinking in this regard was influenced by companies like Toyota. Between 1951 and 1986, the number of suggestions (ideas) logged with the company went up from 789 to 2.6 million, i.e. from 0.1 suggestions per person per year to 48 suggestions per year. While in 1951, only 8% of employees were participating in the suggestion scheme, by 1986 this number had gone up to 95%. Most impressively, the adoption rate of ideas had gone up from 23 to 96%. The high adoption rate reflects both the ability of the company to execute ideas as the fact that the employees had learnt what kind of ideas are valuable to the company!

Care needs to be taken to measure innovation separately for different kinds of innovation processes. The ideas in a kaizen (continuous improvement) context may be quite different in size, investment and impact from the ideas in a new product development process. Mixing up the data from both these processes may confuse the issue.

How to Use Innovation Measures

Benchmarking is useful for certain measures. When your company is generating only 1 or 2 ideas per person per year, it is safe to conclude that you are not tapping into the innovation potential of your employees compared to Toyota’s 48 ideas per person per year. But, particularly for companies at an early stage of innovation, it’s more important to do better than last year, and to move the innovation process along from the pipeline phase to the validation phase, and finally to business impact. Measuring your own progress vis-à-vis the past may be more helpful than benchmarking at this stage.

Some typical measures that help chart how you are making progress on your innovation journey are shown in the chart below.


8 Steps to Innovation has an extensive discussion of measurement and metrics across the book.

No comments:

Post a Comment